Recipe
publicShareStar

Gradient Descent

Collaborators
Reviewers
Total time needed: ~4 hours
Learning Objectives
This list will help you understand gradient descent for optimization of functions
Potential Use Cases
Mathematical foundations behind Gradient descent
Target Audience
INTERMEDIATE
Go through the following annotated items in order:
ARTICLE 1. Need for optimization in Deep Learning
It explains why Deep learning algorithms use optimization.
5 minutes
ARTICLE 2. Gradient Descent Explained
Provides intuitive understanding of gradient descent as first order optimization algorithm
20 minutes
VIDEO 3. Multivariate calculus
It provides intuitive understanding of multivariate calculus
60 minutes
VIDEO 4. Optimization
Application of multivariable calculus to optimization problem
60 minutes
ARTICLE 5. Jacobian matrix
Explains what Jacobian captures for multivariable functions.
20 minutes
BOOK_CHAPTER 6. Second derivative as indication of curvature
Explains how to understand the impact of curvature on performance of gradient descent
30 minutes

Concepts Covered