Graspable is a desktop AI app builder. Its agent edits ordinary project files: React Three Fiber and WebXR for browser experiences, Unity or Unreal Engine for native projects, and Swift with RealityKit for its Apple Vision Pro template. You can open those files in the corresponding editor and keep working by hand.
Choose a native project when you need a device SDK, installed-app distribution or features your target browser cannot provide. WebXR projects open from a link in supported browsers; immersive modes and input depend on the device. A native template still needs its engine, build modules, device permissions and signing where required. Graspable uses those tools on your computer to build and inspect the result.
The native templates (Unity, Unreal Engine, Apple Vision Pro) and the plugins that drive those engines come with the Pro plan and higher; the Expo panel app comes with Builder. They run on your computer only: a Cloud Agent cannot build them, because the engine is on your disk and signed in as you.
Can one project target every headset?
Each template creates a project for its chosen stack. The Unity headset templates share a starting scaffold, but use different device packages and settings. The Apple Vision Pro template creates a separate SwiftUI and RealityKit project. Graspable does not automatically convert a WebXR or Unity project into that Swift project.
For example, a native Quest app and a native Vision Pro app made from these templates are two projects. You may reuse compatible models, textures and parts of the design, but interaction code, build configuration and device tests belong to each target. For a shared web project, check whether every intended browser supports the immersive features you use.
Should I use Graspable or Unity?
You can use both: choose a Unity template in Graspable to have its agent work on a Unity project. Unity remains the engine that compiles and runs it. Its installation, licensing, target modules and device setup still apply. Use the Unity Editor for manual scene editing and its own development tools; use Graspable for agent edits and the checks described below.
If you want a browser experience instead, start with React XR. That creates a web project, with a different rendering and interaction stack. See the Unity comparison for the workflow choices, and AR films and tabletop games for phone-browser tracking limits.
What you need
For the Unity templates
- Unity 6 installed with Unity Hub, with the Android module (Unity Hub → the Editor's settings → Add modules → Android Build Support, with SDK, NDK and OpenJDK).
- The Unity CLI, Unity's own command-line tool. It is what Graspable uses to drive the Editor without a window.
- A Unity account. Unity Personal is free below Unity's revenue threshold; the Editor checks its licence through Unity Hub.
For the Apple Vision Pro template
- A Mac with Xcode and its visionOS platform (Xcode → Settings → Components). The visionOS Simulator comes with it.
For the Unreal Engine template
- Unreal Engine 5, installed with the Epic Games Launcher. On a Mac also Xcode with its Metal Toolchain component. With 5.8 or later the agent also writes Blueprints (experimental).
- For C++ in an Unreal project, the compiler your engine asks for: Xcode on a Mac, Visual Studio 2022 with the workload “Game development with C++” on Windows; on Linux the engine brings its own. Settings → Native engines shows what your engine needs and whether it is there, and has a button that opens the App Store's Xcode page on a Mac. On Windows the button starts Visual Studio's installer through Windows' own package manager; that path is new and has been tried less than the Mac one, so if it does not finish, install Visual Studio 2022 with that workload yourself and Graspable finds it.
Settings → Native engines shows what Graspable found: the Unity Editors and their modules, the Unity CLI, Xcode and its visionOS simulators, and what is missing. Graspable installs none of it and never signs in to Unity or Apple for you; it uses what is on your computer.
The Unity templates
Three headset templates share a Unity scaffold and differ in their target packages and settings: Unity for Quest (Meta Quest 3, Quest 3S, Meta VR Glasses, with Unity's Meta OpenXR package), Unity for Android XR (Android XR headsets and glasses, with Unity's Android XR package) and Unity for XREAL (XREAL glasses, through Android XR). Each creates its own project with Unity 6, OpenXR, AR Foundation for the real room, the XR Interaction Toolkit 3 with XR Hands, the Universal Render Pipeline, and Unity's Pipeline package, which lets Graspable talk to the Editor.
The start scene is mixed reality. The real room shows through, a small table holds a ball you can grab and a button you can press, and you place a marker on any real surface by pointing and pinching. Hands, gaze and controllers all work; Meta VR Glasses have no controllers, so the scene is designed for hands first.
The scene's rig is Unity's own for hands and controllers together, and it switches by itself: bare hands are shown, poke with a fingertip and pinch near or far; controllers point, grab, select, move, turn and teleport; gaze works where the headset tracks eyes. Everything the agent adds works with all of them.
The kit under Assets/Graspable is what the agent builds with:
- Materials that look like what they are. Photographed concrete, plaster, oak, walnut, steel plate, painted metal and rubber, with their roughness and relief, and plain metals, paints and plastics. Boxes keep the texture at its real size, so a long board does not stretch its grain.
- Models. Shelves, a tool chest, a cardboard box and a drill are included, and any glTF model you add is imported with its materials.
- Behaviour. Labels sized in metres so they stay readable, scene buttons, grabbable things, placing on real surfaces with anchors, a hint panel that follows you, mechanisms that a hand moves (levers, doors, handwheels on hinges; drawers and faders on rails), and panels of controls in the scene (buttons, sliders, switches). The toolkit's own pieces are used as they are: sockets that hold a tool, two-handed scaling, teleport areas and anchors, climbing.
The textures and models are from Poly Haven and are in the public domain (CC0).
A room in the Editor
On a headset the app is seen in your real room. In the Editor there is no room, so the templates switch on AR Foundation's XR Simulation with a workshop of Graspable's own: a concrete floor, plaster walls, an oak workbench, shelves. In play mode the camera shows that workshop where the headset would show your room, and its floor, walls and bench are found as surfaces, so placing things on surfaces, anchors and raycasts can be tried without a headset. The workshop is not part of your app and is never in a build.
The agent works in the Editor
The agent does not write scene files by hand. It reads the scene as the Editor sees it and changes it there: it creates and arranges objects, adds components and sets their properties, makes prefabs, sets materials, and can run a few lines of C# in the Editor for what the commands cannot do. These are Unity's own commands, answered by the windowless Editor that also takes the pictures. Every change is saved to the project at once and asks for your approval the same way a file edit does.
Phone AR and tabletop films
The Unity AR tabletop template starts with Signal Garden, an editable rescue film and relay puzzle. It uses AR Foundation for native iPhone and Android surface placement, and a separate OpenXR configuration for Quest passthrough.
Choose Signal Garden → Target → Phone AR or Quest in Unity before building. Graspable's Build Android app action exports the selected configuration without automatically installing it on a connected device. iOS needs Unity's iOS build module and Xcode signing. See AR films and tabletop games for the browser version and tracking requirements.
The Apple Vision Pro template
A native visionOS app in Swift: SwiftUI for a window with controls, RealityKit for an immersive space around you. The start app is a window with a finish picker (oak, painted metal, steel plate) and a button that opens the space, where a block of that material spins and can be looked at, pinched and dragged. The template has the same photographed surfaces as the Unity kit, as physically based materials that the room you are in lights. The Xcode project is set up so that a new Swift file is part of the build the moment it exists; nobody edits the project file.
The simulator shows the window in a virtual room. Real surfaces and hand tracking work on the device only, and the template says so to the agent, so what it builds stays usable in the simulator.
The Unreal Engine template
An Unreal Engine 5 project for OpenXR headsets, with one difference from an ordinary one: its level is a Python script. Unreal keeps levels and materials in binary files that nobody can read, review or have an agent edit as text. In this template Scripts/build.py describes the scene with a small kit (photographed materials, boxes and cylinders placed in metres, labels, light), and your own Unreal Editor builds the level from it without a window. The agent edits the script; you can read every change.
What it can and cannot do today:
A scene that looks like visualisation work: scanned furniture, props and plants at their real size, photographed surfaces that keep their scale on any shape, glass, daylight in real units with a sky seen through windows, spot and area lights, and a camera-style exposure. The models, surfaces and skies come from Poly Haven (free for any use) and are fetched by name the first time a scene uses them, then kept in the project.
Unreal's high renderer on the desktop: the picture, and PC VR, use deferred shading with light that bounces (Lumen) and its reflections. A standalone headset always uses Unreal's mobile renderer, without bounce; the agent keeps a little fill light so the scene reads there too.
A budget with every build: triangles, meshes, materials and lights, against what a standalone headset draws comfortably. The agent reads it and tells you.
Conventions a studio would recognise: Blueprints, materials and folders are named and placed the way most Unreal teams do it, so the project is readable when you open it in the Unreal Editor. The skills that carry these rules come with the template; see Skills.
The scene: materials, lights, rooms, layout and text are built from the script, and the agent sees a picture of the game from where your eyes would start.
The person: Unreal's own VR rig is the player, with a tracked head, two controllers or bare hands, grabbing with the grip or a pinch, teleporting and turning. Graspable copies the rig and the engine's headset settings from your Unreal Engine installation into the project on the first build.
Things to do: the script makes shapes that can be picked up and thrown, and parts held by real joints that you grab and move: doors, lids, levers and handwheels on hinges, drawers and faders on rails. It marks the floor you may teleport on.
Behaviour, in Blueprints: logic that reacts, such as a lever that switches a lamp or a button that starts something, lives in Blueprints. With Unreal Engine 5.8 or later the agent writes them itself; see The Unreal Editor, live. With an older engine a script cannot write that logic, and the agent says which reaction is missing instead of pretending.
Systems, in C++: when a project needs a system of its own, such as a component many actors share or work that must be fast, the agent writes it as a C++ module in
Source/, following the template's skill for it. The check compiles it with the engine's own build tool before it builds the level, and a compiler error comes back with its file and line. A project without C++ stays as it is and needs no compiler.Your own models: a model file you added to the project (glb, glTF, FBX, OBJ) is placed in the level by its path, like a scanned one by its name.
Build for headset packages the Android app with Unreal's own build tool and installs it over USB. It needs Unreal's Android set-up (the SDK, NDK and JDK versions of your engine, installed with the engine's SetupAndroid script); without it the button says so in a few seconds.
You need Unreal Engine 5 on your computer. On a Mac, Unreal also needs Xcode's Metal Toolchain to draw (Xcode → Settings → Components). The first picture on a computer takes many minutes while Unreal prepares its shaders; a game window opens for a moment each time. Close the project in the Unreal Editor before a picture is taken: while an Editor has the project open, the game shows grey checkers instead of materials.
The Unreal Editor, live
This is experimental: it needs Unreal Engine 5.8 and has been tried on a Mac only.
Unreal Engine 5.8 and later carry Epic's own server for AI agents inside the Editor. Graspable starts your Editor without a window, with that server on a port of this computer only, and gives the agent three tools:
| Tool | What it does | Asks first |
|---|---|---|
unreal_look |
Lists the Editor's toolsets and their tools, and reads: a Blueprint's graph as text, its variables, the assets in a folder | No |
unreal_edit |
Makes and changes Blueprints, materials, UI widgets and data assets. A Blueprint's graph is written as a short text script and compiled | As a file edit |
unreal_run |
Runs a small script that calls several of those tools in one go | As a command |
The level stays the Python script. A Blueprint holds the behaviour, is kept under Content/Blueprints, and the script places it in the level. The first call starts the Editor, which takes about half a minute once the engine's caches exist; it closes by itself after ten idle minutes. Nothing is switched on in your project file: the server is enabled for that start only.
UE-MCP, an optional plugin
Experimental, like the tools above.
UE-MCP is an open-source set of Editor tools made by its own community: Blueprints, materials, levels, animation, audio, effects, UI and more. Switch it on under Settings → Plugins and connectors. UE-MCP reaches the Editor through a bridge that must be compiled for your engine. Graspable does that once for each Unreal Engine version, in the background, on the first request on an Unreal project: it takes several minutes, the request carries on without the tools and says so, and a later request has them. After that the compiled bridge, about 20 MB, is copied into each Unreal project the agent works on.
- It needs Unreal Engine 5.6 or later (5.4 on Windows) and a C++ compiler: Xcode on a Mac, Visual Studio with the game development workload on Windows.
- With Unreal Engine 5.8 the tools work in the Editor Graspable runs without a window. With an older engine they work while the project is open in the Unreal Editor.
- The bridge lives in the project's
Pluginsfolder. It is left out of the project's history and of what you publish. - Graspable uses a fixed version of UE-MCP, which changes only with a Graspable update, and leaves out its tool for filing reports with its authors.
- The agent asks before it uses one of its tools, as with every plugin. See Plugins and connectors.
The Panel app template
A flat app that opens as a floating window in a headset's home space. Meta Quest, Meta VR Glasses and Android XR run Android apps this way, and a panel is often the right shape for a tool. It is an Expo project (React Native): it runs as a web page in the preview, a Cloud Agent can work on it, and Expo's own tools build it as an Android app for the headsets.
How the agent works in a native project
The agent edits files: C# under Assets/Graspable/Scripts and scenes under Assets/Scenes in a Unity project; Swift under App/ in a Vision Pro project. It never writes Unity's .meta files or the Xcode project file.
When the agent wants to see its work, it calls verify_app, as it does for a web project. For a native project that means:
- Build. Unity: the Editor opens the project in batch mode and compiles every script; errors come back as
File.cs(line): CS1234: message. Vision Pro: Xcode builds the app for the simulator; errors come back asApp/File.swift:line:col: message. The agent fixes them first. - Run. Unity: an Editor stays open for the project without a window, enters play mode in the simulated workshop, lets the app run for a moment (the agent can ask for a later moment) and renders the main camera to a picture, with the console. Vision Pro: the app is installed and launched in the visionOS Simulator with its immersive space opened, and after a moment the simulator's screen is saved.
- Report. The agent gets the picture and the errors.
A run ends only when the project builds and runs without errors; failures go back to the agent for repair, as always.
The preview
A native project has no dev server. The preview pane shows the picture the engine took; Capture a frame (Unity) or Run in the simulator (Vision Pro) takes a new one. Open in Unity and Open in Xcode open the project in your own editor window. One Unity Editor runs per project: opening the window takes over from the windowless one, which is also closed after ten minutes without use and when Graspable quits. It holds a Unity licence seat while it runs, like any open Editor.
The previous picture stays visible while you capture another frame, including if the new capture fails. Replacing the picture or closing the preview releases the old image from memory.
Getting the app onto a headset
Unity. Build for headset in the preview pane builds the Android app and, when a headset is plugged in over USB with developer mode on, installs it there; the app then appears under Unknown Sources in the headset's library. Without a headset the app lands in the project's Builds/ folder, for Meta Quest Developer Hub or adb install. The first build takes many minutes; later ones are shorter. Building needs the Android module in Unity; installing needs the Android platform tools (adb) on your computer.
Apple Vision Pro. Open the project in Xcode, choose your developer account under Signing, and run it on your Vision Pro.
Panel app. npm run android in the project builds and installs the Android app with Expo.
Publishing and examples
Publish works for native projects too: the address shows the picture from the project's check, others can continue from its source, and a Unity or Unreal Engine project can offer its headset app as a download and link to its pages in the stores. See Publishing. Made in Graspable has an example for each native template, each built from one request that you can read and continue from.
What is not there yet
- Unreal Engine on a headset: the template builds and pictures the level on your computer; its rig, grabbing and joints are Unreal's own and are set up by the script, but have not been tried on a device. Passthrough in Unreal needs the headset maker's plugin, which the template does not include.
- Meta's own simulator and Meta XR Operator: Graspable's play-mode check uses AR Foundation's XR Simulation. You can use Meta's tools in the Unity window as you would without Graspable.
- Emulators for Android XR and XREAL: the check runs play mode in the Editor's simulated room; the headset is where you try the real thing.
- Hands in the picture: the simulated room has surfaces and a camera, but nobody pinching. What needs a pinch is tried on the headset, or shown by a demo mode you ask the agent for.
- Recorded headset sessions as tests (fixtures) exist for web projects only.
Where things are
| Web project | Unity project | Vision Pro project | |
|---|---|---|---|
| Code | App.jsx, components/ |
Assets/Graspable/Scripts/*.cs, Assets/Scenes/*.unity |
App/*.swift, App/Views/ |
| Packages | package.json |
Packages/manifest.json |
none (Apple's frameworks) |
| First install | npm install |
the Editor's first open (resolves packages, compiles) | the first Xcode build |
| Check | build, load in a headless browser, replay fixtures | compile, a frame of play mode | build, run in the simulator |
| Preview | the dev server in the app, with an emulated headset | a rendered frame; or the Unity window | the simulator's screen; or Xcode |
| Hidden from the file list | node_modules, dist |
Library, Temp, Logs, Builds, .meta files |
the .xcodeproj, derived data |