BOOK_CHAPTERWhat is regularization?
Star
Covers: theory of Regularization
Estimated time needed to finish: 20 minutes
Questions this item addresses:
  • What is regularization?
  • What is the need of preventing model from overfitting?
How to use this item?

Read the introduction part of the chapter

URL: https://www.deeplearningbook.org/contents/regularization.html
Author(s) / creator(s) / reference(s)
Deep learning book
0 comment
Recipe
publicShareStar

L2 norm weight penalty regularization

Collaborators
Total time needed: ~2 hours
Objectives
This list will help you get an intuitive idea of L2 regularization for weights in deep learning.
Potential Use Cases
Mathematical foundations behind Deep Learning
Who is this for ?
INTERMEDIATEPeople interested in knowing how Deep Learning model training works.
Click on each of the following annotated items to see details.
ARTICLE 1. Conceptual understanding of applying L2 penalty on weights.
  • Gives a visual understanding of what L2 penalty does for a simple 2-d case.
20 minutes
ARTICLE 2. How is L2 regularization implemented practically?
  • How is L2 regularization different from its conceptual understanding?
15 minutes
BOOK_CHAPTER 3. What does L2 penalty actually does to the weights in comparison to unregularized cost function?
  • How to mathematically associate the weights learnt using regularized cost function with weights learnt for unregularized cost function?
30 minutes
ARTICLE 4. [Example] A case of L2 regularization for Linear regression.
  • How L2 penalty applied to Linear regression problem help to handle strong multicollinearity?
30 minutes
BOOK_CHAPTER 5. What does it mean to penalize weights using norm ?
  • How does penalizing weights lead to regularization?
  • Why penalty is applied only to weight matrices and not biases?
10 minutes
BOOK_CHAPTER 6. What is regularization?
  • What is regularization?
  • What is the need of preventing model from overfitting?
20 minutes

Concepts Covered

0 comment