Offline Speech Recognition on Meta Quest: Testing Unity Sentis + Whisper AI

In this video, I walk you through my Meta Quest project featuring offline speech recognition using Unity Sentis and OpenAI’s Whisper model. Learn how I integrated Whisper for local transcription and optimized the app for performance.

📀 Download my Unity project for FREE from here:
  / project-source-107788463  

← Previous Video • VR Meets AI: Introducing Unity Sentis:    • VR Meets AI: Introducing Unity Sentis  


▬ Support My Work ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

❤️ Support me on Patreon:   / ludicworlds  
☕ Buy me a coffee: https://ko-fi.com/ludicworlds

The 'Unity Asset Store' links provided below are affiliate links. If you click on them and make a purchase, I may receive a small commission at no extra cost to you. This support helps me continue creating content for this channel.

🏷️ 50% Off: https://assetstore.unity.com/?on_sale...

Thank you for your support!


▬ Timestamps ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

0:00 - Intro
0:19 - Demo
1:25 - Project Overview
2:15 - Unity Sentis
2:49 - Installing Sentis
3:17 - Downloading Whisper
4:05 - Importing Whisper
4:37 - Whisper Overview
6:53 - Get Project Free!
7:05 - Project & Scene Setup
7:33 - Scripts
11:38 - Final Points
12:12 - Outro


▬ Useful Links ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

► Introducing Whisper: https://openai.com/index/whisper/
► Unity Sentis - product page: https://unity.com/products/sentis
► Drive better gameplay experiences on user devices with AI | GDC 2024:
   • Drive better gameplay experiences on ...  

► Hugging Face - Unity Models: https://huggingface.co/unity
► Sentis 'Whisper Tiny' Model: https://huggingface.co/unity/sentis-w...
► Whisper paper: https://cdn.openai.com/papers/whisper...
► Mel Spectrograms Explained Easily:
   • Mel Spectrograms Explained Easily  

► https://github.com/LudicWorlds/unity-...
► 'Unity Sentis' Docs: https://docs.unity3d.com/Packages/com...
► 'Unity Sentis' Docs - Split inference over multiple frames:
https://docs.unity3d.com/Packages/com...


▬ Credits ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

► Music by Scott Buckley - released under CC-BY 4.0 - https://www.scottbuckley.com.au

#VR #AI #Unity #Sentis