LLaVA - the first instruction following multi-modal model (paper explained)

There is a lot of emerging interest in developing multimodal foundation models similar to foundation models for language which are LLMs. LLAVA which stands for Large Language and Vision Assistant is the first paper to apply instruction tuning to visual data thereby pushing the possibilities of Large Multimodal Models (LMMs). This video explains the first paper in the LLaVA series of papers such as LLaVA, LLaVA-RLFH, LLaVA-Med and the latest LLaVA 1.5

RELATED LINKS
LLaVA project page: https://llava-vl.github.io
LLaVA code: https://github.com/haotian-liu/LLaVA
LLaVA demo: https://llava.hliu.cc
LLaVA dataset: https://github.com/haotian-liu/LLaVA/...
LLaVA 1 paper: https://arxiv.org/abs/2304.08485
LLaVA 1.5 paper: https://arxiv.org/abs/2310.03744
LLAVA RLHF: https://llava-rlhf.github.io/
LLAVA Med: https://arxiv.org/pdf/2306.00890.pdf


🛠 🛠 🛠 MY SOFTWARE TOOLS 🛠 🛠 🛠
✍️ Notion - https://affiliate.notion.so/aibites-yt
✍️ Notion AI - https://affiliate.notion.so/ys9rqzv2vdd8
📹 OBS Studio for video editing - https://obsproject.com
📼 Manim for some animations - https://www.manim.community
🎵 My music - https://www.bensound.com and


📚 📚 📚 BOOKS I HAVE READ, REFER AND RECOMMEND 📚 📚 📚
📖 Deep Learning by Ian Goodfellow - https://amzn.to/3Wnyixv
📙 Pattern Recognition and Machine Learning by Christopher M. Bishop - https://amzn.to/3ZVnQQA
📗 Machine Learning: A Probabilistic Perspective by Kevin Murphy - https://amzn.to/3kAqThb
📘 Multiple View Geometry in Computer Vision by R Hartley and A Zisserman - https://amzn.to/3XKVOWi


MY KEY LINKS
YouTube:    / @aibites  
Twitter:   / ai_bites​  
Patreon:   / ai_bites​  
Github: https://github.com/ai-bites​


WHO AM I?
I am a Machine Learning Researcher / Practioner who has seen the grind of academia and start-ups equally. I started my career as a software engineer 15 years back. Because of my love for Mathematics (coupled with a glimmer of luck), I graduated with a Master's in Computer Vision and Robotics in 2016 when the now happening AI revolution just started. Life has changed for the better ever since.

#machinelearning #deeplearning #aibites