Covers: theory of L2 regularization

- Gives a visual understanding of what L2 penalty does for a simple 2-d case.

Read the section on L2 penalty.

Terence Parr

Hardik Sahi**Total time needed: **~2 hours

- Learning Objectives
- This list will help you get an intuitive idea of L2 regularization for weights in deep learning.
- Potential Use Cases
- Mathematical foundations behind Deep Learning
- Target Audience
- INTERMEDIATEPeople interested in knowing how Deep Learning model training works.

Go through the following **annotated items** *in order*:

ARTICLE 1. Conceptual understanding of applying L2 penalty on weights.

- Gives a visual understanding of what L2 penalty does for a simple 2-d case.

20 minutes

ARTICLE 2. How is L2 regularization implemented practically?

- How is L2 regularization different from its conceptual understanding?

15 minutes

BOOK_CHAPTER 3. What does L2 penalty actually does to the weights in comparison to unregularized cost function?

- How to mathematically associate the weights learnt using regularized cost function with weights learnt for unregularized cost function?

30 minutes

ARTICLE 4. [Example] A case of L2 regularization for Linear regression.

- How L2 penalty applied to Linear regression problem help to handle strong multicollinearity?

30 minutes

BOOK_CHAPTER 5. What does it mean to penalize weights using norm ?

- How does penalizing weights lead to regularization?
- Why penalty is applied only to weight matrices and not biases?

10 minutes

BOOK_CHAPTER 6. What is regularization?

- What is regularization?
- What is the need of preventing model from overfitting?

20 minutes

Previewing stream ** Math and Foundations**

Upcoming Live Sessions

Videos

Learning Packages

Past Capstones

People

Search for a tag: