Mistral 7b - the best 7B model to date (paper explained)

Mistral 7b - the best 7B model to date!

Mistral 7b is the first open-source 7b model that surpasses all other models in this category both in terms of speed and efficiency. The main contributing factors to the success are Grouped-query attention (GQA), Sliding Window Attention (SWA), Rolling Buffer Cache, Pre-fill and Chunking.

In this video let's have a look at all three contributions of the Mistral 7 billion model in detail followed by the result comparing LLAMA 2 and code LLAMA with Mistral 7b from Mistral AI.

⌚️ ⌚️ ⌚️ TIMESTAMPS ⌚️ ⌚️ ⌚️
0:00 - Intro
0:55 - Sliding Window Attention (SWA)
3:57 - Rolling Buffer Cache
5:05 - Pre-fill and Chunking
6:36 - Results
7:50 - Instruction Finetuning
8:28 - LLM boxing
10:14 - Conclusion

🛠 🛠 🛠 RELATED LINKS 🛠 🛠 🛠
Mistral AI official announcement: https://mistral.ai/news/announcing-mi...
Mistral 7b paper: https://arxiv.org/pdf/2310.06825.pdf
Github code: https://github.com/mistralai/mistral-src
Documentation: https://docs.mistral.ai

RELATED VIDEOS
LLAMA 2:    • LLAMA 2 paper explained - first free ...  


🛠 🛠 🛠 MY SOFTWARE TOOLS 🛠 🛠 🛠
✍️ Notion - https://affiliate.notion.so/aibites-yt
✍️ Notion AI - https://affiliate.notion.so/ys9rqzv2vdd8
📹 OBS Studio for video editing - https://obsproject.com
📼 Manim for some animations - https://www.manim.community
🎵 My music - https://www.bensound.com and


📚 📚 📚 BOOKS I HAVE READ, REFER AND RECOMMEND 📚 📚 📚
📖 Deep Learning by Ian Goodfellow - https://amzn.to/3Wnyixv
📙 Pattern Recognition and Machine Learning by Christopher M. Bishop - https://amzn.to/3ZVnQQA
📗 Machine Learning: A Probabilistic Perspective by Kevin Murphy - https://amzn.to/3kAqThb
📘 Multiple View Geometry in Computer Vision by R Hartley and A Zisserman - https://amzn.to/3XKVOWi

MY KEY LINKS

YouTube:    / @aibites  
Twitter:   / ai_bites​  
Patreon:   / ai_bites​  
Github: https://github.com/ai-bites​

WHO AM I?
I am a Machine Learning Researcher / Practioner who has seen the grind of academia and start-ups equally. I started my career as a software engineer 15 years back. Because of my love for Mathematics (coupled with a glimmer of luck), I graduated with a Master's in Computer Vision and Robotics in 2016 when the now happening AI revolution just started. Life has changed for the better ever since.

#machinelearning #deeplearning #aibites