The animated Transformer: the Transformer model explained the fun way!

In this video, I'll be taking you through the amazing yet mysterious Transformer model (sorry no Autobots or Decepticons), which is the unsung hero behind ChatGPT and other LLMs. You'll learn end-to-end how a Transformer works.

Papers
Original Transfomer: https://arxiv.org/pdf/1706.03762.pdf
GPT-3: https://arxiv.org/pdf/2005.14165.pdf
Batch normalization: https://arxiv.org/pdf/1502.03167.pdf
Layer normalization: https://arxiv.org/pdf/1607.06450.pdf
Fast Transformer: https://arxiv.org/pdf/1911.02150.pdf
Flash Attention: https://arxiv.org/pdf/2205.14135.pdf

00:00 - Introduction
1:40 - Input to the model
2:00 - Tokenization in Transformer
2:25 - Special tokens used by Transformer
3:01 - The input processor
3:51 - Some important notation and hyperparameters
4:40 - The importance of the context window size
5:24 - Basics of the Transformer
5:54 - RNNs vs Transformers
7:41 - Two types of attention: bidirectional vs causal
9:44 - Batch normalization vs Layer normalization
10:57 - Continuing on the Transformer
11:23 - Predictions with the Transformer
11:41 - Softmax for sequences
12:33 - Inference with the Transformer
13:14 - Sampling strategies for tokens
13:45 - Continuing on the Transformer
15:05 - Computing token embeddings
15:34 - Positional embeddings
17:00 - Self-attention layer
19:05 - Self-attention computations
21:31 - Multi-head attention
23:47 - Conclusion