FNet: Mixing Tokens with Fourier Transforms – Paper Explained

Ms. Coffee Bean explains and visualizes the paper "FNet: Mixing Tokens with Fourier Transforms" and what it all has to do with 🍰. Get some coffee and join Ms. Coffee Bean for this video!
➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring....
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕
Patreon:   / aicoffeebreak  
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Discussed paper:
📄 Lee-Thorp, J., Ainslie, J., Eckstein, I., & Ontanon, S. (2021). FNet: Mixing Tokens with Fourier Transforms. https://arxiv.org/pdf/2105.03824.pdf

Referenced videos:
📺 Ms. Coffee Bean explains the Transformer:    • The Transformer neural network archit...  
📺 ViLBERT (Vision and Language with Transformers):    • Transformer combining Vision and Lang...  
📺 ViT: An image is worth 16x16 words:    • An image is worth 16x16 words: ViT | ...  
📺 Multimodality:    • Multimodal Machine Learning models do...  

Outline:
00:00 Why attention is "meh"
02:46 How does FNet work?
05:28 How does Fourier help?
08:15 FNet performance

Other references:
📄 Tolstikhin, I., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., ... & Dosovitskiy, A. (2021). MLP-Mixer: An all-MLP Architecture for Vision. arXiv preprint arXiv:2105.01601. https://arxiv.org/abs/2105.01601
📄 Liu, Hanxiao; Dai, Zihang; So, David R.; Le, Quoc V. "Pay Attention to MLPs" (2021) https://arxiv.org/abs/2105.08050

Music 🎵 : Cuttin It Close by DJ Freedem

--------------------------------------------------
🔗 Links:
YouTube:    / aicoffeebreak  
Twitter:   / aicoffeebreak  
Reddit:   / aicoffeebreak  

#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​