🎮 Hello XR Developers! Today’s tutorial is all about using Meta’s Voice SDK to define and understand intents, create a wake word, and capture the transcript of what we said after the wake word was heard by our system. Let’s break down the process into manageable sections to get you started quickly!

Setting Up wit.ai

1. Login to wit.ai:
   • Head over to wit.ai and log in with your Meta account.
   • Create a new app and give it any name you prefer.
2. Define Intent:
   • Go to the “Understanding” tab.
   • Under “Management,” define a new Intent called wake_word.
3. Add Entity:
   • In the “Understanding” tab, type your desired wake word, e.g., “Hey Quest.”
   • Mark the words, create a new entity called wake_word, and save it.
   • Train the model by adding various utterances like “hey west” or “hi quest” to handle different accents and speech variations.
   • Alternatively, use synonyms in the Entities section to cover variations.

Installing Meta SDKs in Unity

1. Install Meta XR Core SDK and Meta Voice SDK:
   • Links to the SDKs:
     • Meta XR Core SDK
     • Meta Voice SDK – Immersive Voice Commands
2. Unity Project Setup:
   • Open Unity and create a new project.
   • Go to Meta > Voice SDK > Get Started and paste the Server Access Token from your wit app (found under Management > Settings).
   • Save the generated configuration file in your project.
3. Add App Voice Experience:
   • Go to Assets > Create > Voice SDK > add App Voice Experience to scene.
   • Ensure your custom wit app configuration file is assigned in the wit runtime configuration.

Configuring Voice Experience in Unity

1. Understanding Viewer:
   • Go to Meta > Voice SDK > Understanding Viewer.
   • Test your wake word to ensure correct detection.
   • Add the response matcher to the App Voice Experience.
2. Voice Manager Script:
   • Create a script to manage voice commands:
     • Reference the App Voice Experience and response matcher.
     • Add a Text field for displaying the transcript.
     • Add Unity Events for wake word detection and full transcription.
     • Use a Boolean (voiceCommandReady) to manage when to listen for commands.
3. Event Listeners:
   • In the Awake method, add listeners for:
     • Voice experience stopped (to restart listening).
     • Partial and full transcription events.
     • On multi-value event from response matcher (to detect the wake word).
   • Start the voice experience in the Awake method.
   • Remove listeners in the OnDestroy method to prevent memory leaks.
4. Handle Transcriptions:
   • Update the text component in real-time for partial transcriptions.
   • Set the Boolean to false after full transcription to listen for the wake word again.
   • Invoke custom logic upon wake word detection and full transcription.
5. Adding Components:
   • Add the script to your App Voice Experience game object.
   • Reference the voice experience and response matcher.
   • Optionally, set up a canvas with a TextMeshPro component for displaying text.
   • Add an audio source to signal events with sounds.
6. Adjust Keep-Alive Time:
   • If the voice experience cuts off too early, adjust the keep-alive time under the Wit Runtime Configuration.

Testing and Final Adjustments

• Deploy and test your setup to ensure everything works as expected.
• Adjust settings as needed to fine-tune performance.
• This works in the Editor as well as on the Meta Quest device.

Support Black Whale🐋

Thank you for following this article. Stay tuned for more in-depth and insightful content in the realm of XR development! 🌐

Did this article help you out? Consider supporting me on Patreon, where you can find all the source code, or simply subscribe to my YouTube channel!