Скачать или смотреть 🔒 Implementing a Compact Causal Self-Attention Class – Live Coding w/ Sebastian Raschka (Ch. 3.5.3)

In this concise live-coding session, ML authority ‪@SebastianRaschka‬ shows you how to wrap up everything you’ve learned into Chapter 3.5.3: Implementing a Compact Causal Self-Attention Class from his book Build a Large Language Model (From Scratch). Discover how to build a single, reusable PyTorch module that enforces autoregressive (causal) masking within your self-attention layer—perfect for next-token prediction.

0:00 - Integrating Masks into Self-Attention
0:42 - Transition to Batch Input Handling
1:46 - Implementing Causal and Dropout Masks
4:07 - In-Place Operations and Mask Application
5:43 - Final Self-Attention Pipeline and Testing
7:50 - Conclusion

📘 About the Book
Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself!

🔗 Get the Book: https://hubs.la/Q03l0mSf0

🔔 Subscribe for more chapter walkthroughs, deep-dive tutorials, and expert insights from Manning Publications.

#SebastianRaschka #CausalSelfAttention #Transformer #DeepLearning #PyTorch #LLM #MachineLearning #NLP #ManningPublications #LiveCoding