# Hand tracking or controllers in WebXR, and how to support both

> How WebXR describes hands and controllers, what each can and cannot do, and how to write one React Three Fiber scene that works with either.

Michal Takáč, 2 October 2026 · Guides
Source: https://graspable.dev/blog/webxr-hand-tracking-vs-controllers

A headset user can pick up the controllers or put them down at any moment. A WebXR app that handles only one of the two stops working halfway through a session. This guide explains how WebXR sees hands and controllers, where they differ, and how to support both with @react-three/xr.

## One model for every input

WebXR describes every way of pointing as an [`XRInputSource`](https://developer.mozilla.org/en-US/docs/Web/API/XRInputSource). A controller is one. A tracked hand is one. A tap on a phone screen is one. Each has:

- `handedness`: `'left'`, `'right'` or `'none'`.
- `targetRaySpace`: where it points. Use this for aiming.
- `gripSpace`: where it is held. Use this for showing a model or holding an object.
- `targetRayMode`: how it aims, for example `'tracked-pointer'` for a controller or hand.
- `profiles`: names that identify the hardware, listed in the [WebXR input profiles registry](https://github.com/immersive-web/webxr-input-profiles).

Two properties tell hands and controllers apart:

- A controller has a [`gamepad`](https://developer.mozilla.org/en-US/docs/Web/API/XRInputSource/gamepad) with buttons and thumbstick axes, defined by the [WebXR Gamepads Module](https://www.w3.org/TR/webxr-gamepads-module-1/).
- A tracked hand has a [`hand`](https://developer.mozilla.org/en-US/docs/Web/API/XRInputSource/hand): an [`XRHand`](https://developer.mozilla.org/en-US/docs/Web/API/XRHand) with 25 joints, defined by the [WebXR Hand Input Module](https://www.w3.org/TR/webxr-hand-input-1/). It is only there if the session was started with the `hand-tracking` feature.

Both fire the same events. `select` is the main action: a trigger press on a controller, a pinch of thumb and index finger with hands on Meta Quest. `squeeze` is the grip button. If your app listens for [`select`](https://developer.mozilla.org/en-US/docs/Web/API/XRSession/select_event) and aims with `targetRaySpace`, it already works with both.

## What differs

**Controllers have:**

- Buttons, a trigger, a grip and a thumbstick, each with an exact value.
- Haptics. The controller can vibrate.
- Motion sensors inside, so tracking can carry on for a moment when the cameras cannot see the controller.

**Hands have:**

- Nothing to pick up. People can start at once.
- 25 joint poses per hand, so you can detect poses and touch things with a fingertip.
- No buttons and no thumbstick. Moving around needs another design, such as teleporting by pinching.
- No haptics.
- Tracking that is lost when the cameras cannot see the hand, for example when one hand covers the other.

Do not guess which one your users prefer. Support both, and if you need a number, measure it with your own app and your own users.

## Reacting when the input changes

Input sources come and go during a session. The session fires [`inputsourceschange`](https://developer.mozilla.org/en-US/docs/Web/API/XRSession/inputsourceschange_event) when a controller is put down and a hand appears in its place. Never store an input source at session start and keep using it.

With @react-three/xr you do not handle that event yourself. The hooks below return the current state and re-render when it changes.

## One scene for both

@react-three/xr gives hands and controllers the same pointers: a ray for things far away, a grab pointer for things within reach, and, for hands, a touch pointer on the index finger. They all send ordinary React Three Fiber pointer events. So the main interaction needs no branching:

```jsx
import { useState } from 'react'
import { Canvas } from '@react-three/fiber'
import { XR, createXRStore } from '@react-three/xr'

// Hand tracking is requested by default. It is written out here to be clear.
const store = createXRStore({ handTracking: true })

function Button3D() {
  const [on, setOn] = useState(false)
  return (
    // Trigger, pinch and fingertip touch all arrive as onClick.
    <mesh position={[0, 1.3, -0.5]} onClick={() => setOn(!on)}>
      <boxGeometry args={[0.15, 0.15, 0.05]} />
      <meshStandardMaterial color={on ? 'orange' : 'gray'} />
    </mesh>
  )
}

export default function App() {
  return (
    <>
      <button onClick={() => store.enterVR()}>Enter VR</button>
      <Canvas>
        <XR store={store}>
          <ambientLight intensity={2} />
          <Button3D />
        </XR>
      </Canvas>
    </>
  )
}
```

See the [interactions tutorial](https://pmndrs.github.io/xr/docs/tutorials/interactions) for the full list of pointer events.

## Reading a thumbstick

When you need something only a controller has, ask for the controller's state and handle its absence. `useXRInputSourceState` returns `undefined` when that controller is not there, for example because the user switched to hands.

```jsx
import { useRef } from 'react'
import { useFrame } from '@react-three/fiber'
import { XROrigin, useXRInputSourceState } from '@react-three/xr'

function ThumbstickMove() {
  const origin = useRef(null)
  const left = useXRInputSourceState('controller', 'left')

  useFrame((_, delta) => {
    const stick = left?.gamepad['xr-standard-thumbstick']
    if (!stick || !origin.current) return
    // Moves along the world axes. A real app would move relative to where the head faces.
    origin.current.position.x += (stick.xAxis ?? 0) * delta
    origin.current.position.z += (stick.yAxis ?? 0) * delta
  })

  return <XROrigin ref={origin} />
}
```

The names such as `xr-standard-thumbstick` and `xr-standard-trigger` come from the input profiles registry. The [gamepad tutorial](https://pmndrs.github.io/xr/docs/tutorials/gamepad) lists them.

## Reading a hand joint

When you need something only a hand has, ask for the hand's state. This puts a small sphere on the tip of the right index finger:

```jsx
import { XRSpace, useXRInputSourceState } from '@react-three/xr'

function IndexTip() {
  const hand = useXRInputSourceState('hand', 'right')
  const tip = hand?.inputSource.hand.get('index-finger-tip')
  if (!tip) return null
  return (
    <XRSpace space={tip}>
      <mesh>
        <sphereGeometry args={[0.008]} />
        <meshBasicMaterial color="orange" />
      </mesh>
    </XRSpace>
  )
}
```

`XRSpace` keeps its children at the pose of that joint on every frame. In the plain API the same thing is `frame.getJointPose(joint, referenceSpace)`, which also returns the joint's radius.

## Listening for a pinch or a trigger

For the main action, listen for `select` and you get both:

```jsx
import { useXRInputSourceEvent } from '@react-three/xr'

function LogSelect() {
  useXRInputSourceEvent('all', 'select', (event) => {
    const kind = event.inputSource.hand ? 'hand' : 'controller'
    console.log(`select from the ${event.inputSource.handedness} ${kind}`)
  }, [])
  return null
}
```

For your own gestures, such as a fist or a thumbs up, compare joint positions yourself. Pick the threshold by trying it on a headset with several people. Hands differ in size.

## A short checklist

- Aim with `targetRaySpace` and act on `select`. That covers both.
- Make every target reachable by a ray, because a hand has no thumbstick to move closer with.
- Offer a way to move that needs no thumbstick.
- Treat controller state and hand state as things that can be `undefined` on any frame.
- Do not depend on haptics to tell the user something happened. Show it or play a sound as well.
- Test the switch: start with controllers, put them down, carry on with hands.

## Doing this in Graspable

Projects made in Graspable use @react-three/xr, so the code in this guide fits them as it is. You can ask the agent for the behaviour instead, for example: "Make the button work with hand tracking as well as controllers, and add teleport by pinching."

The emulated headset in the preview has both controllers and hands, so you can try both kinds of input before you put a headset on. See [Preview and headset](https://graspable.dev/docs/preview-and-headset.md). Emulated hands are simple poses. How well a gesture works with real fingers is something to test on a real headset.

## Sources

- [XRInputSource](https://developer.mozilla.org/en-US/docs/Web/API/XRInputSource) and [XRHand](https://developer.mozilla.org/en-US/docs/Web/API/XRHand), MDN
- [WebXR Hand Input Module](https://www.w3.org/TR/webxr-hand-input-1/), W3C
- [WebXR Gamepads Module](https://www.w3.org/TR/webxr-gamepads-module-1/), W3C
- [WebXR input profiles registry](https://github.com/immersive-web/webxr-input-profiles), Immersive Web Working Group
- [Interactions](https://pmndrs.github.io/xr/docs/tutorials/interactions), [custom inputs](https://pmndrs.github.io/xr/docs/tutorials/custom-inputs) and [gamepad](https://pmndrs.github.io/xr/docs/tutorials/gamepad) tutorials, @react-three/xr