Direct Preference Optimization (DPO) - How to fine-tune LLMs directly without reinforcement learning

Direct Preference Optimization (DPO) is a method used for training Large Language Models (LLMs). DPO is a direct way to train the LLM without the need for reinforcement learning, which makes it more effective and more efficient.
Learn about it in this simple video!

This is the third one in a series of 4 videos dedicated to the reinforcement learning methods used for training LLMs.

Full Playlist:    • RLHF for training Language Models  

Video 0 (Optional): Introduction to deep reinforcement learning    • A friendly introduction to deep reinf...  
Video 1: Proximal Policy Optimization    • Proximal Policy Optimization (PPO) - ...  
Video 2: Reinforcement Learning with Human Feedback    • Reinforcement Learning with Human Fee...  
Video 3 (This one!): Deterministic Policy Optimization

00:00 Introduction
01:08 RLHF vs DPO
07:19 The Bradley-Terry Model
11:25 KL Divergence
16:32 The Loss Function
14:36 Conclusion

Get the Grokking Machine Learning book!
https://manning.com/books/grokking-ma...
Discount code (40%): serranoyt
(Use the discount code on checkout)