Covers: theory of Parameter norm penalty

0- How does penalizing weights lead to regularization?
- Why penalty is applied only to weight matrices and not biases?

Read the section 7.1

Deep learning book

0 comment

Contributors

- Objectives
- This list will help you get an intuitive idea of L2 regularization for weights in deep learning.
- Potential Use Cases
- Mathematical foundations behind Deep Learning
- Who is This For ?
- INTERMEDIATEPeople interested in knowing how Deep Learning model training works.

Click on each of the following **annotated items** to see details.

ARTICLE 1. Conceptual understanding of applying L2 penalty on weights.

- Gives a visual understanding of what L2 penalty does for a simple 2-d case.

20 minutes

ARTICLE 2. How is L2 regularization implemented practically?

- How is L2 regularization different from its conceptual understanding?

15 minutes

BOOK_CHAPTER 3. What does L2 penalty actually does to the weights in comparison to unregularized cost function?

- How to mathematically associate the weights learnt using regularized cost function with weights learnt for unregularized cost function?

30 minutes

ARTICLE 4. [Example] A case of L2 regularization for Linear regression.

- How L2 penalty applied to Linear regression problem help to handle strong multicollinearity?

30 minutes

BOOK_CHAPTER 5. What does it mean to penalize weights using norm ?

- How does penalizing weights lead to regularization?
- Why penalty is applied only to weight matrices and not biases?

10 minutes

BOOK_CHAPTER 6. What is regularization?

- What is regularization?
- What is the need of preventing model from overfitting?

20 minutes

0 comment