Covers: implementation of Linear Algebra

- How to carry out linear algebraic tasks for deep learning in PyTorch?

This sub-repo has 2 Notebooks with challenges you can go through to practice the concepts in hands with Python. Also, you can see the answers directly in Solution's File. The recommended use of this is:

```
1.- Take your time to solve the Notebooks by yourself
2.- Contrast your notebook with their respective Notebook’s in ./Solutions File.
```

There are questions inside Quiz.md where you can evaluate yourself and explore more topics that are related with the recipe’s content.

Note: In Notebook1 there is a “Hands-On Challenge” subpart, the structure of this challenge is going to be the same throughout the repository so take a careful look. If you don't have much experience programming with Python this article can help you go faster: https://realpython.com/python-super/

These Notebooks has the following concepts:

- Pytorch Introduction
- Linear Algebra: Tensors, Matrices, Vectors & Scalars
- Dot Product, Matrix Multiplication
- Special Matrices: Diagonal, Indetity, Null
- Matrix Transpose, Matrix Determinant, Inversion, Trace
- Eigendecomposition. Eigenvectors & Eigenvalues
- Positive Definite Matrices, Orthogonal Matrices, Symmetric Matrices
- Non-linear functions: Step, Sigmoid, ReLu, Tanh, Softmax
- Loss functions: MSE, Cross Entropy Loss

Amir Hajian

0 comment

Contributors

- Objectives
- You will learn fundamental PyTorch operations with tensors for future DL/NN applications.
- Potential Use Cases
- Building NN from scratch
- Who is This For ?
- INTERMEDIATE

Click on each of the following **annotated items** to see details.

Resources4/6

VIDEO 1. Tensors, Matrices, Dot Product

- What are the most basic matrix manipulation techniques I need to know?
- How easy does PyTorch make it to perform these operations?

19 minutes

VIDEO 2. Matrices and Eigen-decomposition

- What is matrix determinant?
- What is matrix Eigendecomposition and what it is similar to (hint: PCA and SVD)?
- What are some special properties of positive-definite matrices?
- Do I need to know and understand all these operations to be a DL practitioner?

23 minutes

VIDEO 3. Mathematical Non-linearities

- How to solve eigendecomposition on a whiteboard?
- What is the relevance of nonlinearities for deep learning?

23 minutes

REPO 4. Hands-on Linear Algebra for Deep Learning

- How to carry out linear algebraic tasks for deep learning in PyTorch?

30 minutes

RECIPE 5. Matrix Algebra

10 minutes

BOOK_CHAPTER 6. Linear Algebra for Deep Learning

- How is linear algebra used in deep learning?

20 minutes

0 comment