Recipe
publicShareStar

L1 norm weight penalty

Collaborators
Reviewers
Total time needed: ~2 hours
Learning Objectives
Mathematical foundations behind Deep Learning
Potential Use Cases
Mathematical foundations behind Deep Learning
Target Audience
INTERMEDIATEPeople interested in knowing how Deep Learning model training works.
Go through the following annotated items in order:
ARTICLE 1. Conceptual understanding of L1 norm weight penalty
  • Gives a visual understanding of what L1 penalty does for a simple 2-d case.
20 minutes
ARTICLE 2. Practical implementation of L1 penalty
  • How is L1 regularization implementation different from its conceptual understanding?
20 minutes
BOOK_CHAPTER 3. Mathematical understanding of what L1 penalty does to weights learnt as compared to weights of unregularized cost function.
  • Mathematical understanding of what L1 penalty does to weights
30 minutes
ARTICLE 4. Difference between restrictions imposed by L1 and L2 penalties on weights
  • Difference between restrictions imposed by L1 and L2 penalties on weights
15 minutes
BOOK_CHAPTER 5. What is regularization for neural networks?
  • What is regularization?
15 minutes
BOOK_CHAPTER 6. How does penalizing parameters lead to regularization?
  • Why is penalty applied only to weights and not biases
10 minutes

Concepts Covered