# Speech and gaze you can edit frame by frame

> Import timed mouth cues, key a character’s named bones, and let an agent make changes to the same saved take. What works today, and what our native character study taught us.

Michal Takáč, 19 October 2026 · Product
Source: https://graspable.dev/blog/editable-character-performances

A character's mouth can follow the recording while its eyes look somewhere else. You should be able to fix either without rebuilding the whole performance.

Graspable now imports timed mouth cues as editable facial keyframes. You can also animate a character's named bones, and ask an agent to add speech, blinks or gaze to the same saved take. The character needs those controls in its source model.

## Put speech on the timeline

Open a web 3D project, choose **Animate**, and select the character. Use **Animate this object** if it does not have an animation identity yet.

Under **Speech performance**, import a [Rhubarb mouth-cue JSON file](https://github.com/DanielSWolf/rhubarb-lip-sync). Map its spoken shapes to the character's actual facial controls, check the amount and start frame, then choose **Check speech keys**. Graspable shows how many channels and keys it will add before you apply them.

Choose **Add speech keys** and save the take. Scrub to a word, adjust a mouth weight, or retime a key. Undo reverses the import. Add the recording as an audio clip at the matching start frame; the cue importer does not analyze audio or generate a voice.

Existing mapped mouth channels are protected by default. Choosing replacement replaces those entire channels, including manual keys outside the new speech range. Other expressions and body channels stay as they were.

In our desktop check, a 6.56-second recording supplied 40 cues covering seven spoken shapes. The importer produced seven editable tracks with 259 keys. We checked replacement and undo in the running interface as well as testing the underlying data.

## Direct a glance yourself, or ask the agent

Choose **Property → Rig rotations**, select a uniquely named bone and set its local angles. The saved keys use quaternion rotation. A keyed bone overrides an imported animation clip after the clip updates; other bones keep their animation. Duplicate bone names cannot be bound ambiguously.

An agent can work on the same named channels. For example:

> Keep my mouth keys. Add two blinks and have the character look toward the doorway at frame 120, then back at frame 180. Read the existing take first and leave the body animation alone.

The agent can apply blink and gaze edits independently of speech. It must use the model's actual controls and respect the rig's local axes. Saved edits have revision checks and history, so a stale agent edit is refused. Your recent saved manual changes also enter the next request's context. This workflow coordinates human and agent edits; it does not add remote participant presence or simultaneous conflict merging.

## A good source rig still needs a careful export

We tested the import path with Rain, a stylized Blender character. Early exports lost an eyebrow, broke the mouth during head turns, and produced dark patches on closed eyelids. Some of those files passed format validation. Looking at the rendered face caught problems a valid file could not.

The native rig uses constraints and corrective deformation that a single exported bone rotation does not reproduce. For this study we evaluated the native performance, preserved the missing mirrored anatomy, and fitted corrective positions and normals for the runtime model. We also baked the native eye maps with explicit UVs and retained the vertex colours needed for its pupils.

![Rain, a stylized character with blue eyes, turning her head in the desktop browser acting study](https://graspable.dev/blog/editable-character-performances/rain-runtime-turn.webp)

*An actual desktop browser capture from the native-export study, at 8.5 seconds. Rain Rig © Blender Foundation | cloud.blender.org, CC-BY. Derived performance and runtime preparation by Graspable.*

The corrected mouth, blink and head-turn poses passed our desktop visual review. This is a baked technical study of a stylized character, not an automatic converter for arbitrary rigs or a photorealistic replacement cast. Changing its skeletal performance requires rebaking the fitted corrections. The editable native source remains separate.

[Rain's original rig and license](https://studio.blender.org/characters/rain/v2/) are available from Blender Studio.

## What still needs a headset

The authoring changes have desktop tests and browser checks. On a physical Quest 3, a 35-second test of the new two-character study delivered about 7.2 frames per second in a 72 Hz session. It used full stereo resolution, without audio or the studio lighting used in the desktop screenshot. This study still needs substantial performance work. Final sustained iPhone testing and Quest performance acceptance remain pending; desktop visual review does not establish headset frame rate, controller comfort or readability.

You can already open the experimental XR authoring panel from the animation workspace, but its controller workflow still needs physical Quest checks. Hand manipulation and headset audio monitoring remain unfinished.

[Read the animation workflow and its current limits](https://graspable.dev/docs/animation.md).