# Graspable 0.19.8: gestures, the real room, and AI inside your apps

> Graspable 0.19.8 gives web projects hand and head gestures and physics that meets your real furniture, and lets a published app ask an AI model about what it sees, with no key anywhere in the app.

Michal Takáč, 31 October 2026 · Product
Source: https://graspable.dev/blog/gestures-real-room-and-ai-in-your-apps

![The React XR starting scene in the emulated headset: a workbench with balls, a basket and buttons](https://graspable.dev/site/templates/react-xr.webp)

Graspable 0.19.8 is about apps that notice you. A new React XR project can tell a thumbs-up from a pinch, a ball you throw in AR lands on your real table, and once you publish, the app can ask an AI model what is in front of it and answer out loud. The parts are small and sit in the project as ordinary files, so the agent uses them when you ask for something that needs them, and you can read every line.

## Hands that mean something

Until now, hands in our web templates could point, grab and press. That covers a lot, but people also make shapes with their hands and expect something to happen. The React XR and XR Game templates now recognise six of them: pinch, point, fist, thumbs-up, open palm and spread. They are worked out from the joints the headset already reports, every frame, with a short hold before a gesture counts and a lower threshold before it ends, so a hand halfway between two shapes does not flicker.

Heads count too. A nod and a shake are recognised from the headset's own motion, while looking around and slow turns are ignored. That gives you a yes and a no without a single button.

In a new React XR project, a thumbs-up adds a ball to the bench. In XR Game, a thumbs-up after a round starts the next one. In your own code it is one hook:

```js
import { useHandGestures, useHeadGestures } from './lib/use-gestures'

useHandGestures((e) => {
  if (e.name === 'thumbs-up' && e.phase === 'start') addBall()
}, { gestures: ['thumbs-up'] })

useHeadGestures((name) => {
  if (name === 'nod') confirm()
})
```

Or skip the code and ask: "open the menu when I show an open palm, close it with a fist."

## Physics that meets your room

In AR, a Quest 3 or an Android XR headset knows where your floor, table and walls are. The React XR template now turns those surfaces and the room's scanned shape into solid ground for the physics, and updates them when the headset refines its picture of the room. Drop a ball over your desk and it lands on the desk.

Where the headset's browser offers depth, the scene can also hide behind real things: your hand passes in front of a virtual cube instead of being drawn over by it. It is one line when the store is created, `createXRStore(withOcclusion({}))`, and devices without depth simply skip it.

## AI inside a published app

This is the one I have wanted for a while. An app made in Graspable can now ask an AI model a question, with a picture of what it is showing, and hear the answer.

The usual way to do this is to put an API key in the app, which means anyone who opens it can read the key and spend on it. Graspable apps hold no key. The app sends its request to its own address, and Graspable asks the model on your behalf, with the model and the fixed instruction you chose for that app. Visitors sign in to nothing. You pay from your Credits, at what the model costs, under a daily limit you set, and the app stops asking when the limit is reached.

Every web template has the helper:

```js
import { ask, snapshot, listen, speak } from './lib/ai.js'

// What is on the table? Asked with a picture of the scene.
const answer = await ask('What is on the table?', { image: snapshot(gl, scene, camera) })

// A spoken question and a spoken answer, using the browser's own speech.
speak(await ask(await listen()))
```

A chemistry set that explains the reaction you just started, a museum piece that answers questions about itself, a puzzle that gives hints when you nod at it: these are a request away now. Tell the agent what the app should ask and when, and it uses the helper. Then switch AI on for the app on graspable.dev, under **Account → Published apps**, and pick the model. It comes with Builder and higher, and works in the preview on your computer as soon as the project is published.

What we keep is counts: how many requests an app made and what they cost. Never the questions, pictures or answers.

## Where to start

Create a React XR project and try the thumbs-up, or open one you already have and ask the agent to add gestures or the real room. The details are in the docs: [gestures and the real room](https://graspable.dev/docs/templates.md) and [AI in your app](https://graspable.dev/docs/publishing.md).

The full list of changes is in the [changelog](https://graspable.dev/changelog#v0.19.8). The app updates itself, and new installs are on the [download page](https://graspable.dev/download).
