Covers: theory of L1 norm weight penalty
Estimated time needed to finish: 30 minutes
Questions this item addresses:
  • Mathematical understanding of what L1 penalty does to weights
How to use this item?

Read the section 7.1.2

Author(s) / creator(s) / reference(s)
Terence Parr
0 comment
Recipe
publicShare
Star0

L1 norm weight penalty

Contributors
Total time needed: ~2 hours
Objectives
Mathematical foundations behind Deep Learning
Potential Use Cases
Mathematical foundations behind Deep Learning
Who is This For ?
INTERMEDIATEPeople interested in knowing how Deep Learning model training works.
Click on each of the following annotated items to see details.
ARTICLE 1. Conceptual understanding of L1 norm weight penalty
  • Gives a visual understanding of what L1 penalty does for a simple 2-d case.
20 minutes
ARTICLE 2. Practical implementation of L1 penalty
  • How is L1 regularization implementation different from its conceptual understanding?
20 minutes
BOOK_CHAPTER 3. Mathematical understanding of what L1 penalty does to weights learnt as compared to weights of unregularized cost function.
  • Mathematical understanding of what L1 penalty does to weights
30 minutes
ARTICLE 4. Difference between restrictions imposed by L1 and L2 penalties on weights
  • Difference between restrictions imposed by L1 and L2 penalties on weights
15 minutes
BOOK_CHAPTER 5. What is regularization for neural networks?
  • What is regularization?
15 minutes
BOOK_CHAPTER 6. How does penalizing parameters lead to regularization?
  • Why is penalty applied only to weights and not biases
10 minutes

Concepts Covered

0 comment