Layer Normalization - EXPLAINED (in Transformer Neural Networks)

Lets talk about Layer Normalization in Transformer Neural Networks!

ABOUT ME
⭕ Subscribe: https://www.youtube.com/c/CodeEmporiu...
📚 Medium Blog:   / dataemporium  
💻 Github: https://github.com/ajhalthor
👔 LinkedIn:   / ajay-halthor-477974bb  

RESOURCES
[ 1🔎] Code for video: https://github.com/ajhalthor/Transfor...
[2 🔎 ] The paper that introduced the concept: https://arxiv.org/pdf/1607.06450.pdf
[3 🔎 ]Layer normalization in transformer architecture: https://arxiv.org/pdf/2002.04745.pdf
[4 🔎 ]Batch Normalization underperforms with NLP tasks. Reasons are empirical : https://arxiv.org/pdf/2003.07845.pdf
[5 ] Residual Connections minimize vanishing gradients: https://stats.stackexchange.com/quest...
[6 🔎] Transformer Main Paper: https://arxiv.org/abs/1706.03762

PLAYLISTS FROM MY CHANNEL
⭕ ChatGPT Playlist of all other videos:    • ChatGPT  
⭕ Transformer Neural Networks:    • Natural Language Processing 101  
⭕ Convolutional Neural Networks:    • Convolution Neural Networks  
⭕ The Math You Should Know :    • The Math You Should Know  
⭕ Probability Theory for Machine Learning:    • Probability Theory for Machine Learning  
⭕ Coding Machine Learning:    • Code Machine Learning  


MATH COURSES (7 day free trial)
📕 Mathematics for Machine Learning: https://imp.i384100.net/MathML
📕 Calculus: https://imp.i384100.net/Calculus
📕 Statistics for Data Science: https://imp.i384100.net/AdvancedStati...
📕 Bayesian Statistics: https://imp.i384100.net/BayesianStati...
📕 Linear Algebra: https://imp.i384100.net/LinearAlgebra
📕 Probability: https://imp.i384100.net/Probability

OTHER RELATED COURSES (7 day free trial)
📕 ⭐ Deep Learning Specialization: https://imp.i384100.net/Deep-Learning
📕 Python for Everybody: https://imp.i384100.net/python
📕 MLOps Course: https://imp.i384100.net/MLOps
📕 Natural Language Processing (NLP): https://imp.i384100.net/NLP
📕 Machine Learning in Production: https://imp.i384100.net/MLProduction
📕 Data Science Specialization: https://imp.i384100.net/DataScience
📕 Tensorflow: https://imp.i384100.net/Tensorflow

TIMSTAMPS
0:00 Transformer Encoder Overview
0:56 "Add & Norm": Transformer Encoder Deep Dive
5:13 Layer Normalization: What & why
7:33 Layer Normalization: Working out the math by hand
12:10 Final Coded Class