ARTICLEDifference between restrictions imposed by L1 and L2 penalties on weights
Star
Covers: theory of L1 norm weight penalty
Estimated time needed to finish: 15 minutes
Questions this item addresses:
  • Difference between restrictions imposed by L1 and L2 penalties on weights
How to use this item?

Read the entire article

URL: https://explained.ai/regularization/L1vsL2.html#sec:3.1
Author(s) / creator(s) / reference(s)
Terence Parr
0 comment
Recipe
publicShare
Star0

L1 norm weight penalty

Contributors
Total time needed: ~2 hours
Objectives
Mathematical foundations behind Deep Learning
Potential Use Cases
Mathematical foundations behind Deep Learning
Who is This For ?
INTERMEDIATEPeople interested in knowing how Deep Learning model training works.
Click on each of the following annotated items to see details.
ARTICLE 1. Conceptual understanding of L1 norm weight penalty
  • Gives a visual understanding of what L1 penalty does for a simple 2-d case.
20 minutes
ARTICLE 2. Practical implementation of L1 penalty
  • How is L1 regularization implementation different from its conceptual understanding?
20 minutes
BOOK_CHAPTER 3. Mathematical understanding of what L1 penalty does to weights learnt as compared to weights of unregularized cost function.
  • Mathematical understanding of what L1 penalty does to weights
30 minutes
ARTICLE 4. Difference between restrictions imposed by L1 and L2 penalties on weights
  • Difference between restrictions imposed by L1 and L2 penalties on weights
15 minutes
BOOK_CHAPTER 5. What is regularization for neural networks?
  • What is regularization?
15 minutes
BOOK_CHAPTER 6. How does penalizing parameters lead to regularization?
  • Why is penalty applied only to weights and not biases
10 minutes

Concepts Covered

0 comment