Native XR

Most Graspable projects are WebXR: a web page that runs on any headset's browser. Some things need a native app: the full headset SDKs, store distribution, passthrough and room understanding on every device, or performance the web cannot carry. For those, Graspable builds Unity projects for the headsets that run Android (Meta Quest, Meta VR Glasses, Android XR, XREAL) and Swift projects for Apple Vision Pro. The agent edits C# or Swift and scenes the way it edits a web project, and an engine on your computer, running without a window, builds the project and takes a picture of it so the agent and you can see it.

Native templates come with the Builder plan and higher. They run on your computer only: a Cloud Agent cannot build them, because the engine is on your disk and signed in as you.

What you need

For the Unity templates

• Unity 6 installed with Unity Hub, with the Android module (Unity Hub → the Editor's settings → Add modules → Android Build Support, with SDK, NDK and OpenJDK).
• The Unity CLI, Unity's own command-line tool. It is what Graspable uses to drive the Editor without a window.
• A Unity account. Unity Personal is free below Unity's revenue threshold; the Editor checks its licence through Unity Hub.

For the Apple Vision Pro template

• A Mac with Xcode and its visionOS platform (Xcode → Settings → Components). The visionOS Simulator comes with it.

Settings → Native engines shows what Graspable found: the Unity Editors and their modules, the Unity CLI, Xcode and its visionOS simulators, and what is missing. Graspable installs none of it and never signs in to Unity or Apple for you; it uses what is on your computer.

The Unity templates

Three templates share one project and differ in the headset they target: Unity for Quest (Meta Quest 3, Quest 3S, Meta VR Glasses, with Unity's Meta OpenXR package), Unity for Android XR (Android XR headsets and glasses, with Unity's Android XR package) and Unity for XREAL (XREAL glasses, through Android XR). They are Graspable's own project, not Unity's sample: Unity 6, OpenXR, AR Foundation for the real room, the XR Interaction Toolkit 3 with XR Hands, the Universal Render Pipeline, and Unity's Pipeline package, which lets Graspable talk to the Editor.

The start scene is mixed reality. The real room shows through, a small table holds a ball you can grab and a button you can press, and you place a marker on any real surface by pointing and pinching. Hands, gaze and controllers all work; Meta VR Glasses have no controllers, so the scene is designed for hands first.

The kit under Assets/Graspable is what the agent builds with: labels sized in metres so they stay readable, scene buttons, grabbable things, placing on real surfaces with anchors, a hint panel that follows you, and a flat-screen fallback for play mode, where there is no passthrough.

The Apple Vision Pro template

A native visionOS app in Swift: SwiftUI for a window with controls, RealityKit for an immersive space around you. The start app is a window with a colour picker and a button that opens the space, where a shape spins and can be looked at, pinched and dragged. The Xcode project is set up so that a new Swift file is part of the build the moment it exists; nobody edits the project file.

The simulator shows the window in a virtual room. Real surfaces and hand tracking work on the device only, and the template says so to the agent, so what it builds stays usable in the simulator.

The Panel app template

A flat app that opens as a floating window in a headset's home space. Meta Quest, Meta VR Glasses and Android XR run Android apps this way, and a panel is often the right shape for a tool. It is an Expo project (React Native): it runs as a web page in the preview, a Cloud Agent can work on it, and Expo's own tools build it as an Android app for the headsets.

How the agent works in a native project

The agent edits files: C# under Assets/Graspable/Scripts and scenes under Assets/Scenes in a Unity project; Swift under App/ in a Vision Pro project. It never writes Unity's .meta files or the Xcode project file.

When the agent wants to see its work, it calls verify_app, as it does for a web project. For a native project that means:

1. Build. Unity: the Editor opens the project in batch mode and compiles every script; errors come back as File.cs(line): CS1234: message. Vision Pro: Xcode builds the app for the simulator; errors come back as App/File.swift:line:col: message. The agent fixes them first.
2. Run. Unity: an Editor stays open for the project without a window, enters play mode, lets a few frames pass and renders the main camera to a picture, with the console. Vision Pro: the app is installed and launched in the visionOS Simulator, and after a moment the simulator's screen is saved.
3. Report. The agent gets the picture and the errors.

A run ends only when the project builds and runs without errors; failures go back to the agent for repair, as always.

The preview

A native project has no dev server. The preview pane shows the picture the engine took; Capture a frame (Unity) or Run in the simulator (Vision Pro) takes a new one. Open in Unity and Open in Xcode open the project in your own editor window. One Unity Editor runs per project: opening the window takes over from the windowless one, which is also closed after ten minutes without use and when Graspable quits. It holds a Unity licence seat while it runs, like any open Editor.

Getting the app onto a headset

Unity. Build for headset in the preview pane builds the Android app and, when a headset is plugged in over USB with developer mode on, installs it there; the app then appears under Unknown Sources in the headset's library. Without a headset the app lands in the project's Builds/ folder, for Meta Quest Developer Hub or adb install. The first build takes many minutes; later ones are shorter. Building needs the Android module in Unity; installing needs the Android platform tools (adb) on your computer.

Apple Vision Pro. Open the project in Xcode, choose your developer account under Signing, and run it on your Vision Pro.

Panel app. npm run android in the project builds and installs the Android app with Expo.

What is not there yet

• Unreal Engine. Settings → Native engines already lists an installed Unreal Engine, so you can see what the next engine will find, but there is no template or check for it.
• Meta's own simulator and Meta XR Operator: Graspable's play-mode check uses Unity's XR Interaction Simulator. You can use Meta's tools in the Unity window as you would without Graspable.
• Emulators for Android XR and XREAL: the check runs play mode in the Editor; the headset is where you try the real thing.
• Recorded headset sessions as tests (fixtures) exist for web projects only.

Where things are