Coding a ChatGPT Like Transformer From Scratch in PyTorch

In this StatQuest we walk through the code required to code your own ChatGPT like Transformer in PyTorch and we do it one step at a time, with every little detail clearly explained.

NOTE: This StatQuest assumes that you are already familiar with the concepts behind...
Decoder-Only Transformers:    • Decoder-Only Transformers, ChatGPTs s...  
The Essential Matrix Algebra for Neural Networks:    • Essential Matrix Algebra for Neural N...  
The Matrix Math Behind Transformers:    • The matrix math behind transformer ne...  

You can get the code here: https://github.com/StatQuest/decoder_...

The full Neural Networks playlist, from the basics to AI, is here:    • The Essential Main Ideas of Neural Ne...  

Learn more about GiveInternet.org: https://giveinternet.org/StatQuest NOTE: Donations up to $30 will be matched by an Angel Investor - so a $30 donation would give $60 to the organization. DOUBLE BAM!!!

If you'd like to support StatQuest, please consider...
Patreon:   / statquest  
...or...
YouTube Membership:    / @statquest  

...buying my book, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
https://statquest.org/statquest-store/

...or just donating to StatQuest!
paypal: https://www.paypal.me/statquest
venmo: @JoshStarmer

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
  / joshuastarmer  

0:00 Awesome song and introduction
1:12 Loading the modules
2:04 Creating the training dataset
6:17 Coding Position Encoding
14:09 Coding Attention
21:04 Coding a Decoder-Only Transformer
26:39 Running the model (untrained)
29:18 Training and using the model

#StatQuest #PyTorch #chatgpt