Covers: theory of Nonlinear Functions / Maps

- How to solve eigendecomposition on a whiteboard?
- What is the relevance of nonlinearities for deep learning?

In this video you will learn about:

- Eigendecomposition (contined)
- Non-linearities (Affine Maps)
- Non-linear Functions: Step, Sigmoid, Tanh, ReLU

**It will help you to apply your learning in excercising notebooks in thi**s asset (i.**e. the next asset of** Foundations Of Algebra For Deep Learning **RECIPE** )

Besides other properties, Deep learning is also the powerful technique because it to accounts for **non-linearities** in data. A fully-conmnected NN is only a bunch of regression equations. One neuron would be a sum of all neurons in previous layer wired via this equation: , where is the weight learnd in training . **By introducing a non-linearity, you make the neuron either to FIRE or NOT in a small interval (e.g 0,1 or -1,1) !**

**Non-linearities (Affine Maps)**

Non-linearity introduced by the **Activation function**.
Source: towards.ai

Fail to play? Open the link directly: https://youtu.be/GZYdwXqjOR0

Amir Hajian

0 comment

Contributors

- Objectives
- You will learn fundamental PyTorch operations with tensors for future DL/NN applications.
- Potential Use Cases
- Building NN from scratch
- Who is This For ?
- INTERMEDIATE

Click on each of the following **annotated items** to see details.

Resources4/6

VIDEO 1. Tensors, Matrices, Dot Product

- What are the most basic matrix manipulation techniques I need to know?
- How easy does PyTorch make it to perform these operations?

19 minutes

VIDEO 2. Matrices and Eigen-decomposition

- What is matrix determinant?
- What is matrix Eigendecomposition and what it is similar to (hint: PCA and SVD)?
- What are some special properties of positive-definite matrices?
- Do I need to know and understand all these operations to be a DL practitioner?

23 minutes

VIDEO 3. Mathematical Non-linearities

- How to solve eigendecomposition on a whiteboard?
- What is the relevance of nonlinearities for deep learning?

23 minutes

REPO 4. Hands-on Linear Algebra for Deep Learning

- How to carry out linear algebraic tasks for deep learning in PyTorch?

30 minutes

RECIPE 5. Matrix Algebra

10 minutes

BOOK_CHAPTER 6. Linear Algebra for Deep Learning

- How is linear algebra used in deep learning?

20 minutes

0 comment