SOTA LLM for Measuring Hallucinations in LLMs|

In this video, we are going to test Bespoke-MiniCheck which is the SOTA fact-checking model despite its small size.
We have the answers from RAG but how do you know that the responses of the LLMs are accurate and to the point. We use another LLM to get the checking done.
Simple !!

Watch this video and see the magic unfolding !!

These are the things that you will learn:
Bespoke-MiniCheck Overview:

A state-of-the-art model designed to combat AI hallucinations.
Focuses on "grounded factuality," ensuring AI-generated content is supported by given context.
Performance on LLM-AggreFact Benchmark:

Ranks first with 77.4% grounded factuality accuracy.
Outperforms larger models like Vectara’s HHEM 2.1 and Claude 3.5 Sonnet.
Grounded Factuality Explained:

Measures how well AI claims are supported by a provided context.
Critical for applications like Retrieval-Augmented Generation (RAG) systems.
Model Capabilities:

Trained using a proprietary platform for highly accurate results.
Can provide a factuality score for any AI-generated claim with respect to its context.
Small Model with High Efficiency:

Despite its small size (7B parameters), it performs better than much larger models.
Fast response times (200ms on GPUs) and can run on consumer-grade hardware.
API Availability:

Access the model via a self-serve API platform.
Available on Bespoke Console with a simple client library for easy integration.
Try Before You Buy:

Experience the model’s performance at the Bespoke Playground for free.
Integration Support:

Integrated with platforms like Ollama and GuardrailsAI.
Supports “yes/no” responses, with logit support coming soon.
Real-World Application:

Especially useful for improving the factuality of AI responses in legal, research, and other critical domains.

#bespoke #minicheck #sota #ai #llm #factcheckmodel #benchmarks

Links:
Model in Ollama: https://ollama.com/library/bespoke-mi...
Bespoke: https://bespokelabs.ai/bespoke-minicheck
HuggingFace: https://huggingface.co/bespokelabs/Be...
PlayGround: https://playground.bespokelabs.ai/
Arxiv paper Link: https://arxiv.org/pdf/2404.10774
LLM AggreFact Leaderboard: https://llm-aggrefact.github.io/

CHANNEL LINKS:
🕵️‍♀️ Join my Patreon for keeping up with the updates:   / promptengineer975  
☕ Buy me a coffee: https://ko-fi.com/promptengineer
📞 Get on a Call with me at $125 Calendly: https://calendly.com/prompt-engineer4...
💀 GitHub Profile: https://github.com/PromptEngineer48
🔖 Twitter Profile:   / prompt48  

0:00 Intro
0:48 Intro to BeSpoke
1:50 Technical Depths
4:10 Ollama Implementation
7:30 Grounded Factuality
9:18 Playground
10:45 Conclusion