Covers: theory of Nonlinear Functions / Maps
Estimated time needed to finish: 23 minutes
Questions this item addresses:
  • How to solve eigendecomposition on a whiteboard?
  • What is the relevance of nonlinearities for deep learning?
How to use this item?

In this video you will learn about:

  • Eigendecomposition (contined)
  • Non-linearities (Affine Maps)
  • Non-linear Functions: Step, Sigmoid, Tanh, ReLU

It will help you to apply your learning in excercising notebooks in this asset (i.e. the next asset of Foundations Of Algebra For Deep Learning RECIPE )


Besides other properties, Deep learning is also the powerful technique because it to accounts for non-linearities in data. A fully-conmnected NN is only a bunch of regression equations. One neuron would be a sum of all neurons in previous layer wired via this equation: , where is the weight learnd in training . By introducing a non-linearity, you make the neuron either to FIRE or NOT in a small interval (e.g 0,1 or -1,1) !


  • Non-linearities (Affine Maps) Alt text

Non-linearity introduced by the Activation function. Source: towards.ai

Fail to play? Open the link directly: https://youtu.be/GZYdwXqjOR0
Author(s) / creator(s) / reference(s)
Amir Hajian
0 comment
Recipe
publicShare
Star(0)

Foundations Of Algebra For Deep Learning

Contributors
Total time needed: ~2 hours
Objectives
You will learn fundamental PyTorch operations with tensors for future DL/NN applications.
Potential Use Cases
Building NN from scratch
Who is This For ?
INTERMEDIATE
Click on each of the following annotated items to see details.
Resources4/6
VIDEO 1. Tensors, Matrices, Dot Product
  • What are the most basic matrix manipulation techniques I need to know?
  • How easy does PyTorch make it to perform these operations?
19 minutes
VIDEO 2. Matrices and Eigen-decomposition
  • What is matrix determinant?
  • What is matrix Eigendecomposition and what it is similar to (hint: PCA and SVD)?
  • What are some special properties of positive-definite matrices?
  • Do I need to know and understand all these operations to be a DL practitioner?
23 minutes
VIDEO 3. Mathematical Non-linearities
  • How to solve eigendecomposition on a whiteboard?
  • What is the relevance of nonlinearities for deep learning?
23 minutes
REPO 4. Hands-on Linear Algebra for Deep Learning
  • How to carry out linear algebraic tasks for deep learning in PyTorch?
30 minutes
RECIPE 5. Matrix Algebra
10 minutes
BOOK_CHAPTER 6. Linear Algebra for Deep Learning
  • How is linear algebra used in deep learning?
20 minutes

Concepts Covered

0 comment