L2 norm weight penalty regularization

Total time needed: ~2 hours
This list will help you get an intuitive idea of L2 regularization for weights in deep learning.
Potential Use Cases
Mathematical foundations behind Deep Learning
Who is This For ?
INTERMEDIATEPeople interested in knowing how Deep Learning model training works.
Click on each of the following annotated items to see details.
Resource Asset5/6
ARTICLE 1. Conceptual understanding of applying L2 penalty on weights.
  • Gives a visual understanding of what L2 penalty does for a simple 2-d case.
20 minutes
ARTICLE 2. How is L2 regularization implemented practically?
  • How is L2 regularization different from its conceptual understanding?
15 minutes
BOOK_CHAPTER 3. What does L2 penalty actually does to the weights in comparison to unregularized cost function?
  • How to mathematically associate the weights learnt using regularized cost function with weights learnt for unregularized cost function?
30 minutes
ARTICLE 4. [Example] A case of L2 regularization for Linear regression.
  • How L2 penalty applied to Linear regression problem help to handle strong multicollinearity?
30 minutes
BOOK_CHAPTER 5. What does it mean to penalize weights using norm ?
  • How does penalizing weights lead to regularization?
  • Why penalty is applied only to weight matrices and not biases?
10 minutes
BOOK_CHAPTER 6. What is regularization?
  • What is regularization?
  • What is the need of preventing model from overfitting?
20 minutes

Concepts Covered

0 comment