Скачать или смотреть Let's Recreate Google Translate! | Neural Machine Translation

In this first part video we talk about how Google Translate probably works, and a little bit of some general theory behind Neural Machine Translation (NMT). Specifically, we touch on transformers, a common NLP model for sequential data, and how they rely on a specific type of attention called self-attention. We load up our pretrained model, mt5, using the Hugging Face transformer library and model repository.

Relevant Links:
GitHub repo: https://github.com/ejmejm/multilingua...
Colab code: https://colab.research.google.com/dri...
Transformer paper: https://arxiv.org/abs/1706.03762
mT5 paper: https://arxiv.org/abs/2010.11934
Article on how transformers work: https://towardsdatascience.com/transf...