Leonardo XR

Interaction Package Guides — LeonardoXR SDK

Interaction Package Guides — LeonardoXR SDK

Interaction Package Guides — LeonardoXR SDK

Version: 1.0.2 Target Device: Youbiquo Leonardo XR Base SDK: Snapdragon Spaces 1.0.4

This guide describes how to configure the interaction layer for LeonardoXR applications using standard Unity packages:

  • XR Interaction Toolkit (XRIT) 3.3.0 — high-level interaction system for controllers, ray interactors, and direct interactors.
  • XR Hands 1.7.2 — access to hand joint data and gesture recognition.

Dependencies and Versions

Package Version Source
XR Interaction Toolkit 3.3.0 Unity Registry
XR Hands 1.7.2 Unity Registry
AR Foundation 6.2.1 Unity Registry
Snapdragon Spaces SDK 1.0.4 Tarball (Qualcomm)
LeonardoXR SDK 1.0.2 Tarball (Youbiquo)
UniTask 2.5.10 Unity Registry

Note: All dependencies must be installed following the procedure described in Getting Started before proceeding with the interaction configuration.

1. XR Interaction Toolkit Configuration

Installation

The package is installed via Window > Package Manager > Unity Registry > XR Interaction Toolkit > Install (version 3.3.0).

If the prompt “Enable the new input system?” appears during installation, click Yes. If the prompt “XR Interaction Layer Mask Update Required” appears, click I Made a Backup, Go Ahead! (for new projects) or No Thanks if a previous version of XRIT was already present.

Import Starter Assets

  1. Open Window > Package Manager, select In Project, and then select the XR Interaction Toolkit package.
  2. In the Samples section, click Import on Starter Assets.
  3. Also click Import on Hand Interaction Demo (required for hand tracking samples).

The Starter Assets include:

  • Pre-configured Input Action Asset for XR controllers and hands.
  • Reference Prefab for XR Origin with Ray Interactor and Direct Interactor.
  • Basic materials and shaders for interaction visual feedback.

Input Actions Configuration

After import, an XRI Default Input Actions asset is added to Assets/Samples/XR Interaction Toolkit/[version]/Starter Assets/. This asset is already configured for standard controller profiles (including the Companion Controller via the Microsoft MMRC Profile) and for hands.

To assign the asset to the system:

  1. Go to Edit > Project Settings > XR Plug-in Management > Input System.
  2. Ensure that Active Input Handling is set to Both or Input System Package (New).

2. Scene Configuration — XR Origin

Basic Scene Structure

The minimum hierarchy for an interactive scene on LeonardoXR is:

AR Session
XR Origin (Action-based)
  └── Camera Offset
        ├── Main Camera (tagged "MainCamera")
        ├── LeftHand Controller
        │     ├── XR Ray Interactor (Left)
        │     └── XR Direct Interactor (Left)
        └── RightHand Controller
              ├── XR Ray Interactor (Right)
              └── XR Direct Interactor (Right)

Create the base structure by selecting GameObject > XR > XR Origin (Action-based) from the main menu.

Warning: Disable the AR Camera Background component — it is not supported on LeonardoXR and will be automatically disabled at runtime. Also disable AR Camera Manager if you are not retrieving RGB frames from the camera.

Input Action Manager

Add the Input Action Manager component to the XR Origin (or to a persistent GameObject). In the Action Assets field, assign:

  1. XRI Default Input Actions (from the imported Starter Assets).
  2. Snapdragon Spaces Input Actions (from Samples/Snapdragon Spaces/[version]/Core Samples/Shared Assets/Input Actions).

3. Controller Configuration (Companion Controller)

Leonardo XR’s Companion Controller uses the Microsoft Mixed Reality Motion Controller Profile to send pointing and button inputs from the companion app on your smartphone.

Adding the Controller Profile

  1. Go to Edit > Project Settings > XR Plug-in Management > OpenXR > Enabled Interaction Profiles.
  2. Click + and add Microsoft Mixed Reality Motion Controller Profile.

Note: Do not add the Hand Interaction Profile alongside the Companion Controller — it takes precedence and will interfere with controller pointing.

Device Pointer Prefab

  1. Locate the Device Pointer prefab in Samples/Snapdragon Spaces/[version]/Core Samples/Shared Assets/Prefabs/Interaction.
  2. Add it as a child of the Controller (left or right) within the XR Origin.
  3. In the XR Interactor Line Visual component, set Line Width to 0.005.

4. XR Hands Configuration

Installation

Installed via Window > Package Manager > Unity Registry > XR Hands > Install (version 1.7.2).

Importing the Hand Samples

  1. Open Window > Package Manager > In Project > XR Hands > Samples.
  2. Import:
    • Gestures — gesture recognition system.
    • Hand Visualizer — hand mesh visualization.

Enable Hand Tracking Subsystem

  1. Navigate to Edit > Project Settings > XR Plug-in Management > OpenXR (Android tab).
  2. Under All Features, enable Hand Tracking Subsystem.
  3. Also enable Hand Tracking in the Snapdragon Spaces feature group.

Warning: The Hand Tracking Subsystem must be enabled in All Features — not only in the Snapdragon Spaces section — to function correctly on LeonardoXR.

Enable Hand Interaction Profile

  1. Navigate to Edit > Project Settings > XR Plug-in Management > OpenXR > Enabled Interaction Profiles.
  2. Click + and add Hand Interaction Profile.

Note: Do not add Hand Interaction Profile if you are using the Companion Controller — the two profiles conflict. See the “Combining Controllers and Hand Tracking” section below.

XR Hand Subsystem and XR Origin

The XR Hands package integrates with the XR Interaction Toolkit via the XR Hand Tracking Events and XR Hand Skeleton Driver components. These components are automatically configured on the XR Origin Controller when using the Starter Assets prefabs from the Hand Interaction Demo.

Recommended hierarchy with XR Hands:

XR Origin (Action-based)
  └── Camera Offset
        ├── Left Hand
        │     ├── XR Hand Skeleton Driver
        │     ├── XR Hand Mesh Renderer (optional)
        │     └── XR Direct Interactor
        └── Right Hand
              ├── XR Hand Skeleton Driver
              ├── XR Hand Mesh Renderer (optional)
              └── XR Ray Interactor (for distance pointing)

Access to Joint Data via XR Hands API

using UnityEngine.XR.Hands;

XRHandSubsystem handSubsystem = XRHandSubsystemUtility.GetSubsystem();

if (handSubsystem != null)
{
    XRHand leftHand = handSubsystem.leftHand;
    if (leftHand.isTracked)
    {
        if (leftHand.GetJoint(XRHandJointID.IndexTip).TryGetPose(out Pose indexTipPose))
        {
            // indexTipPose.position = left index tip position
        }
    }
}

Gesture Recognition

Use the XR Hands Gestures system to detect static or dynamic hand poses:

using UnityEngine.XR.Hands;

// On the scene: add the XRHandGestures component to the XR Origin
// Then subscribe to the events:

[SerializeField] XRHandGestures gestures;

void OnEnable()
{
    gestures.gesturePerformed += OnGesturePerformed;
}

void OnGesturePerformed(XRHandGesture gesture, Handedness handedness)
{
    Debug.Log($"Gesture: {gesture.gestureName} performed with {handedness}");
}

5. Configuring Interactables

Adding an Interactable to an Object

To make an object selectable by a ray or direct interactor:

  1. Add a Collider to the object (e.g., Box Collider).
  2. Add the XR Grab Interactable component for grabbable objects, or XR Simple Interactable for objects that receive events without being moved.
// Example: reacting to a select with XR Simple Interactable
using UnityEngine.XR.Interaction.Toolkit;

public class MyButton : MonoBehaviour
{
    [SerializeField] XRSimpleInteractable interactable;

    void OnEnable()
    {
        interactable.selectEntered.AddListener(OnSelected);
    }

    void OnSelected(SelectEnterEventArgs args)
    {
        Debug.Log("Selected by: " + args.interactorObject);
    }
}

Interaction Layer Mask

XRIT 3.x uses Interaction Layer Masks to filter which interactors can interact with which interactables. Configure the layers via the Inspector or by using:

interactable.interactionLayers = InteractionLayerMask.GetMask("Default", "UI");

6. UI Canvas and Interaction

Canvas in World Space

For interactive 3D UI on LeonardoXR:

  1. Create a Canvas and set Render Mode to World Space.
  2. Add the Tracked Device Graphic Raycaster component to the Canvas (replacing the standard Canvas Graphic Raycaster).
  3. Add the XR UI Input Module to the scene’s EventSystem GameObject (replacing the default UI Input Module).

Recommended Canvas Settings

Parameter Recommended Value
Render Mode World Space
Dynamic Pixels Per Unit 1
Reference Pixels Per Unit 100
Scale 0.001 (for a ~1m canvas = 1000 pixel units)

7. Combining Controllers and Hand Tracking

On LeonardoXR, it is not possible to simultaneously use the Companion Controller (with raycast) and full Hand Tracking (with raycast from both hands) due to OpenXR profile conflicts.

Option A — Controllers Only (Companion Controller)

  • Enable the Microsoft MMRC Profile among the available profiles.
  • Do not add the Hand Interaction Profile.
  • The Companion Controller handles pointing; hands can be used as colliders but not as raycast interactors.

Option B — Hands Only (Dual Hand Mode)

  • Enable the Hand Interaction Profile among the available profiles.
  • Do not add the Microsoft MMRC Profile.
  • Remove the Device Pointer Prefab from the Controller.
  • Both hands are tracked as full interactors; no input from the mobile device.

Option C — Controller + One Hand

  • Add the Microsoft MMRC Profile to the enabled profiles.
  • Keep only the Right Hand Controller active in the XR Origin.
  • The left hand can still function as a collider and trigger for poke interactions, but not as a ray interactor.

8. UniTask — Asynchronous Operations

UniTask 2.5.10 is included as a dependency to handle high-performance asynchronous operations (asset loading, waiting for subsystems, etc.) without GC allocations.

Installation

Installed via Window > Package Manager > Unity Registry > UniTask > Install (version 2.5.10), or as a git URL.

Basic Usage

using Cysharp.Threading.Tasks;

public class HandSubsystemWaiter : MonoBehaviour
{
    async UniTaskVoid Start()
    {
        // Wait for the XR Hands subsystem to be ready
        await UniTask.WaitUntil(() =>
            XRHandSubsystemUtility.GetSubsystem()?.running ?? false);

        Debug.Log("XR Hands subsystem ready.");
    }
}

UniTask is used internally by the LeonardoXR SDK for asynchronous operations. It is recommended to use it in your application code as well to maintain consistency with the SDK lifecycle.

OpenXR Interaction Profile References

Profile Usage
Microsoft Mixed Reality Motion Controller Profile Companion Controller (smartphone as controller)
Hand Interaction Profile Hand tracking with poke/pinch interactors

Note: The two profiles cannot coexist with raycasting enabled on both. Select one of the options described in the “Combining Controllers and Hand Tracking” section.