How to create a Hand Tracking App using Unity’s XRI

  • 27 November 2023
  • 28 November 2023
  • 4 min read

Introduction

Hello, XR developers! ๐ŸŒŸ The world of Unity’s XR Interaction Toolkit is buzzing with the arrival of the new XR Hands package ๐Ÿ–๏ธ. This package is a game-changer, offering an API for hand tracking data on devices like the Meta Quest and, potentially, Appleโ€™s Vision Pro. Our focus today will be on using the OpenXR provider with the Meta Quest device. Let’s embark on this exciting journey!

Setting Up Your Unity Project

  1. Creating a New Project: Launch Unity 2022 LTS and start a new project. Choose the 3D URP template for the Universal Render Pipeline, name your project, and select a suitable location on your computer ๐Ÿ–ฅ๏ธ.
  2. Configuring Build Settings: Navigate to ‘Build Settings’ and switch to the Android platform, which is necessary for running apps on the Meta Quest. Select ‘ASTC’ for texture compression, a format that offers superior visual quality and versatility ๐Ÿ“ฑ.
  3. Installing XR Plugin Management: This tool is essential for configuring settings for various XR providers. Install it from the Unity package manager, followed by the Open XR plugin. This step is crucial for accessing hand data later ๐Ÿ› ๏ธ.

Preparing for Hand Tracking

  1. Updating the Input System: Unity 2022 automatically updates the input system. If you’re using an older version, you might need to do this manually. This step ensures compatibility with the new hand tracking features ๐Ÿ‘.
  2. Setting Interaction Profiles: Add interaction profiles for each platform you intend to use. For the Meta Quest 2, select the Oculus Touch Controller Profile under the Open XR settings ๐ŸŽฎ.

Installing Necessary Packages

  1. XR Interaction Toolkit: Find and install this toolkit from the Unity Registry in the package manager. It’s the backbone of our hand tracking setup ๐Ÿ“ฆ.
  2. XR Hands Package: Install this package next. It includes the ‘Hand Visualiser’ prefab and other essential samples for hand tracking ๐Ÿ‘€.

Setting Up the Sample Scene

  1. Fixing Shader Issues: If you encounter pink shaders, convert the affected materials to URP. This step ensures that your visuals appear correctly in the Unity environment ๐ŸŽจ.
  2. Disabling Hand Menu Setup: Currently, the Hand Menu Setup feature might not work as expected. Temporarily disable it to avoid complications โš ๏ธ.
  3. Exploring XR Interaction Hands Setup: Familiarize yourself with the new components in the XR Interaction Toolkit, such as the XR Input Modality Manager, which helps switch between hands and controllers. Pay attention to components for gaze interaction and the stabilizing elements added to controllers and hand interactors ๐Ÿ‘€๐Ÿ”ง.

Simplifying the Scene for Hand Interaction

  1. Creating a New XR Origin: Start by setting up a new ‘XR Origin’ with only the essential components for hand interaction. Focus on the ‘direct interactor’ for object manipulation ๐Ÿ› ๏ธ.
  2. Configuring Input Actions: Define and bind input actions for both hands and controllers. Ensure that the actions are intuitive and correspond to the expected interactions in your app ๐ŸŽ›๏ธ.

Final Touches and Testing

  1. Environment Setup: Add a plane to serve as the floor and a cube as an interactable object. Customize their appearance for a more engaging visual experience ๐ŸŒˆ.
  2. Testing Hand Tracking: Experiment with the hand visualizer in the Unity editor. Interact with the cube using hand tracking to ensure everything is functioning correctly โœ….

Conclusion and Next Steps

You’ve now set up a basic hand-tracking application in Unity! This tutorial is just the beginning. We’ll explore more advanced topics like reading hand data, implementing locomotion with hand tracking, and dissecting the new features of the XR Interaction Toolkit in upcoming posts ๐Ÿš€.

Join Our Community

Have questions or want to discuss XR development? Join our XR-developer community on Discord and engage with peers in this exciting field ๐Ÿ’ฌ.

Support Our Efforts

If you find our content helpful, consider supporting us on Patreon. Your contributions enable us to continue creating free, high-quality educational material ๐Ÿ‘.

Thank you for following this tutorial. Stay tuned for more in-depth and insightful content in the realm of XR development! ๐ŸŒ

Did this article help you out? Consider supporting me on Patreon or simply subscribe to my YouTube channel!

Leave a Reply

Your email address will not be published. Required fields are marked *