Deep dive in transformer positional encodings

Описание к видео Deep dive in transformer positional encodings

- Detailed review of the positional encodings from "Attention Is All You Need" paper
- Time codes
0:00 Intro
2:10 Agenda
4:19 Encodings vs embeddings
7:03 Why are positional encodings needed
15:02 Explain & visualise encodings formula
33:32 How are encodings added to the input
40:06 Impact on transformer inner pieces
45:00 Summary
- Notebook used to create visualisations
https://drive.google.com/file/d/1IG-H...
- Original "Attention Is All You Need" paper
https://arxiv.org/pdf/1706.03762
- Annotated transformer
https://nlp.seas.harvard.edu/2018/04/...
- Visualisation of matrix multiplication
http://matrixmultiplication.xyz

Комментарии

Информация по комментариям в разработке

Похожие видео