Qwen SmallThinker (3B) : This NEW Small Reasoning MODEL IS AMAZING! (Opensource & Local)

Check out PhotoGenius AI here: https://photogenius.ai/

USE CODE "KING25" for 25% OFF on ALL MEMBERSHIPS ON PhotoGenius AI

In this video, I'll be telling you about Qwen Small Thinker which is a new Small 3B Model that can reason through questions like the OpenAI's O1 / O3 model and can be on par with OpenAI's O1 & O3 model. This is a super small model that can be run locally.

----
Key Takeaways:

➡️ SmallThinker is a revolutionary 3B parameter reasoning model that punches way above its weight, offering a powerful alternative to larger language models. Its small size makes it perfect for running inference on edge devices which will change AI accessibility for everyone.

🧠 This open-source AI model, fine-tuned from Qwen 2.5 3B, demonstrates impressive performance in STEM and general reasoning tasks, even outperforming larger models in certain benchmarks, showcasing its potential in many different AI applications.

🚀 Experience lightning-fast inference speeds by utilizing SmallThinker as a draft model with VLLM for speculative decoding alongside models like QwQ. This innovative technique can boost QwQ inference speeds significantly, making your workflow much more efficient in any large language model task.

💻 With its compact size, SmallThinker is ideal for local deployment and offers remarkable accessibility, even on devices with limited resources, like a base M1 Mac or computers with as little as 8GB of RAM. On-device AI is the future.

🤖 SmallThinker excels in complex reasoning challenges, including math problems and logic puzzles, making it a standout amongst small AI models. It has even mastered challenges that stump other models like the apple pie test and the diagonal test. It's one of the most advanced generative AI for its size.

🧪 SmallThinker is built on an openly available synthetic dataset from QwQ responses, making it a readily accessible tool. This model has great potential for further fine-tuning on specific tasks to achieve even greater performance. This is one of the best options for building your AI model and training AI.

-----
Timestamps:

00:00 - Introduction
01:03 - PhotoGenius AI (Sponsor)
02:12 - About SmallThinker 3B & Speculative Decoding
04:36 - Testing
08:28 - Final Chart & Thoughts
09:24 - Using SmallThinker as a Draft model