Covers: theory of Parameter norm penalty

0- Why is penalty applied only to weights and not biases

Read the section 7.1

Deep learning book

0 comment

Contributors

- Objectives
- Mathematical foundations behind Deep Learning
- Potential Use Cases
- Mathematical foundations behind Deep Learning
- Who is This For ?
- INTERMEDIATEPeople interested in knowing how Deep Learning model training works.

Click on each of the following **annotated items** to see details.

Resources5/6

ARTICLE 1. Conceptual understanding of L1 norm weight penalty

- Gives a visual understanding of what L1 penalty does for a simple 2-d case.

20 minutes

ARTICLE 2. Practical implementation of L1 penalty

- How is L1 regularization implementation different from its conceptual understanding?

20 minutes

BOOK_CHAPTER 3. Mathematical understanding of what L1 penalty does to weights learnt as compared to weights of unregularized cost function.

- Mathematical understanding of what L1 penalty does to weights

30 minutes

ARTICLE 4. Difference between restrictions imposed by L1 and L2 penalties on weights

- Difference between restrictions imposed by L1 and L2 penalties on weights

15 minutes

BOOK_CHAPTER 5. What is regularization for neural networks?

- What is regularization?

15 minutes

BOOK_CHAPTER 6. How does penalizing parameters lead to regularization?

- Why is penalty applied only to weights and not biases

10 minutes

0 comment