GPT-1 | Paper Explained & PyTorch Implementation

Improving Language Understanding by Generative Pre-Training(GPT) is the first model by OpenAI which leverages self-supervised learning and uses a transformer architecture.

Paper:

https://s3-us-west-2.amazonaws.com/op...

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

GitHub Repos:

https://github.com/maciejbalawejder/D...

https://github.com/lyeoni/gpt-pytorch - training

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Connect with me on:

Linkedin -   / maciej-balawejder-rt8015  

GitHub - https://github.com/maciejbalawejder

Medium -   / maciejbalawejder  

Buy Me a Coffee - [https://www.buymeacoffee.com/mbalawejder](https://www.buymeacoffee.com/mbalawejder)

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Timestamps:
0:00 Introduction
1:00 GPT
2:15 Self-Supervised Learning
3:35 Loss functions
4:30 Architecture
5:17 Textual Entailment
6:17 Question Answering
7:12 Semantic Similarity
8:13 Classification
9:30 Model Specifications
11:55 Conclusions
12:30 PyTorch Implementation
13:30 Decoder Layer
15:30 GPT Architecture
16:42 Language Modelling Head
17:11 Classification Head