Momentum Optimizer in Deep Learning | Explained in Detail

In this video, we will understand in detail what is Momentum Optimizer in Deep Learning.

Momentum Optimizer in Deep Learning is a technique that reduces the time taken to train a model.

The path of learning in mini-batch gradient descent is zig-zag, and not straight. Thus, some time gets wasted in moving in a zig-zag direction. Momentum Optimizer in Deep Learning smooth out the zig-zag path and make it much more straighter, thus reducing the time taken to train the model.

Momentum Optimizer uses Exponentially Weighted Moving Average, which averages out the vertical movement and the net movement is mostly in the horizontal direction. Thus zig-zag path becomes straighter.

In this video, we will also understand what Exponentially Weighted Moving Average is, and thus this video is a full in-depth explanation of Momentum Optimizer in Deep Learning.

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖

▶ Mini Batch Gradient Descent:    • Mini Batch Gradient Descent | Deep Le...  

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖

▶ Watch Next Video on RMSprop Optimizer:    • RMSprop Optimizer Explained in Detail...  

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖

✔ Improving Neural Network Playlist:    • Overfitting and Underfitting in Machi...  

✔ Complete Neural Network Playlist:    • Neural Network python from scratch | ...  

✔ Complete Logistic Regression Playlist:    • Logistic Regression Machine Learning ...  

✔ Complete Linear Regression Playlist: https://www.youtube.com/watch?v=mlk0r...

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖

Timestamp:
0:00 Agenda
1:00 Why do we need Momentum?
2:53 Exponentially Weighted Moving Average
8:29 Momentum in Mini Batch Gradient Descent
9:50 Why Momentum works?

➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖

Subscribe to my channel, because I upload a new Machine Learning video every week: https://www.youtube.com/channel/UCJFA...