Covers: theory of Nonlinear Functions / Maps
Estimated time needed to finish: 23 minutes
• How to solve eigendecomposition on a whiteboard?
• What is the relevance of nonlinearities for deep learning?
##### How to use this item?

In this video you will learn about:

• Eigendecomposition (contined)
• Non-linearities (Affine Maps)
• Non-linear Functions: Step, Sigmoid, Tanh, ReLU

It will help you to apply your learning in excercising notebooks in this asset (i.e. the next asset of Foundations Of Algebra For Deep Learning RECIPE )

Besides other properties, Deep learning is also the powerful technique because it to accounts for non-linearities in data. A fully-conmnected NN is only a bunch of regression equations. One neuron would be a sum of all neurons in previous layer wired via this equation: , where is the weight learnd in training . By introducing a non-linearity, you make the neuron either to FIRE or NOT in a small interval (e.g 0,1 or -1,1) !

• Non-linearities (Affine Maps) Non-linearity introduced by the Activation function. Source: towards.ai

Fail to play? Open the link directly: https://youtu.be/GZYdwXqjOR0
Amir Hajian
0 comment
Recipe
publicShare
Star(0)

### Foundations Of Algebra For Deep Learning

Contributors    Total time needed: ~2 hours
Objectives
You will learn fundamental PyTorch operations with tensors for future DL/NN applications.
Potential Use Cases
Building NN from scratch
Who is This For ?
INTERMEDIATE
Click on each of the following annotated items to see details.
Resources4/6
VIDEO 1. Tensors, Matrices, Dot Product
• What are the most basic matrix manipulation techniques I need to know?
• How easy does PyTorch make it to perform these operations?
19 minutes
VIDEO 2. Matrices and Eigen-decomposition
• What is matrix determinant?
• What is matrix Eigendecomposition and what it is similar to (hint: PCA and SVD)?
• What are some special properties of positive-definite matrices?
• Do I need to know and understand all these operations to be a DL practitioner?
23 minutes
VIDEO 3. Mathematical Non-linearities
• How to solve eigendecomposition on a whiteboard?
• What is the relevance of nonlinearities for deep learning?
23 minutes
REPO 4. Hands-on Linear Algebra for Deep Learning
• How to carry out linear algebraic tasks for deep learning in PyTorch?
30 minutes
RECIPE 5. Matrix Algebra
10 minutes
BOOK_CHAPTER 6. Linear Algebra for Deep Learning
• How is linear algebra used in deep learning?
20 minutes

0 comment