Covers: theory of Parameter norm penalty

- Why is penalty applied only to weights and not biases

Read the section 7.1

Deep learning book

Hardik Sahi**Total time needed: **~2 hours

- Learning Objectives
- Mathematical foundations behind Deep Learning
- Potential Use Cases
- Mathematical foundations behind Deep Learning
- Target Audience
- INTERMEDIATEPeople interested in knowing how Deep Learning model training works.

Go through the following **annotated items** *in order*:

ARTICLE 1. Conceptual understanding of L1 norm weight penalty

- Gives a visual understanding of what L1 penalty does for a simple 2-d case.

20 minutes

ARTICLE 2. Practical implementation of L1 penalty

- How is L1 regularization implementation different from its conceptual understanding?

20 minutes

BOOK_CHAPTER 3. Mathematical understanding of what L1 penalty does to weights learnt as compared to weights of unregularized cost function.

- Mathematical understanding of what L1 penalty does to weights

30 minutes

ARTICLE 4. Difference between restrictions imposed by L1 and L2 penalties on weights

- Difference between restrictions imposed by L1 and L2 penalties on weights

15 minutes

BOOK_CHAPTER 5. What is regularization for neural networks?

- What is regularization?

15 minutes

BOOK_CHAPTER 6. How does penalizing parameters lead to regularization?

- Why is penalty applied only to weights and not biases

10 minutes

Previewing stream ** Math and Foundations**

Upcoming Live Sessions

Videos

Learning Packages

Past Capstones

People

Search for a tag: