Meta glasses

Meta's AI glasses have no app store and run no apps of their own. There are two ways in. Graspable has templates for both: two for the phone app (one that shows every capability, one finished showcase) and one for the web.

Both come with the Builder plan and higher. The glasses and your phone are yours; Graspable installs nothing on either.

Before you start

• Install the Meta AI app on your phone and connect the glasses in it.
• Turn on Developer Mode in the Meta AI app: Settings, App Info, tap the version number five times, then confirm. With it on, an unpublished app may register with your glasses.
• For the phone app: an iPhone on iOS 17.2 or later with Xcode 16 or later on your Mac, or an Android phone on Android 12 or later with Android Studio. The toolkit needs Meta AI 290 or later and glasses software 128 or later; the app tells you when either is behind.

The phone app

Create a project from Meta glasses app or Glasses field notes and describe what you want. The preview runs the app against the glasses emulator, described below, so you can build and try everything before it goes near a phone. To run it on your iPhone with your glasses, press Run on my phone above the preview.

Run on my phone

Plug the iPhone into the Mac with a cable the first time, unlock it and tap Trust. Graspable builds the app with your Xcode, installs it and opens it. The first build takes several minutes. After that the app on the phone loads its code from your computer, so each change the agent makes shows on the phone in a few seconds, and pressing the button again takes about ten seconds. A new build is made only when the app's settings or its native packages change.

What it needs, and the bar tells you which is missing:

• A Mac with Xcode from the App Store, opened once, and CocoaPods.
• Your Apple ID added in Xcode under Settings, Accounts. Meta's toolkit needs Wi-Fi access that Apple lets only a paid Apple Developer membership sign, so a free account cannot build these apps.
• Developer Mode on the iPhone: Settings, Privacy & Security, Developer Mode.
• The phone awake and on the same Wi-Fi as the computer. The first time, the phone asks whether the app may find devices on your network. Allow it.

While it is on, the project's code is offered to devices on your network, as with any Expo development server. Stop ends that. The first time you run a project, Graspable gives it an app identifier of its own in app.json, because Apple signs each identifier for one developer.

The same build works without glasses: the Mock screen, and "Try with pretend ones" in Field Notes, pair Meta's pretend glasses inside the app on the phone.

An Android phone is not built from Graspable yet. With Android Studio installed, npx expo run:android --device in the project folder does it.

On the phone, the Glasses screen registers the app with Meta AI (the Meta AI app opens and asks you), asks for the camera and microphone, lists your glasses with their battery, wear and thermal state, and opens a session. The other screens use it: Camera streams the glasses' camera into the phone and takes photos; Display sends a screen to Meta Ray-Ban Display glasses and gets the taps back; Senses shows inputs, motion and speech; Extras lets the glasses open the app by voice and take a full-quality photo; Mock pairs pretend glasses of every model so you can try all of it without hardware.

What the toolkit calls experimental (inputs, motion, speech, voice launch, standalone photos, audio in the stream) runs with Developer Mode on, and an app that uses it cannot be published to other people until Meta opens it. The camera stream, photos from it and the display can be published through Meta's own review and release channels, from the Wearables Developer Center.

Field Notes, a showcase

Glasses field notes is the same toolkit as one finished app for glasses without a display. Press Look and the phone shows what the glasses see. The shutter, or a tap on the temple, saves a moment. Say a note listens through the glasses and writes the first sentence under the photo. The phone answers "saved" and "noted" through the glasses' speakers, which are your phone's audio output while you wear them. Temple taps, the horizon line and spoken notes use the experimental parts of the toolkit; where the glasses refuse one, the app carries on without it. In the preview, the emulator stands in for the glasses.

The glasses emulator

The preview of a phone app shows it in a phone-sized frame with a panel beside it: pretend glasses and a pretend Meta AI app. Nothing needs installing, and it works on any computer.

• The Meta AI app. When the app registers or asks for the camera or the microphone, a sheet slides over the phone as the Meta AI app would, and you answer it. A switch turns Developer Mode off, and registering then fails the way it does on a real phone.
• The glasses. A pair of Ray-Ban Meta to start with; add any model. Switch a pair off, take it out of range, take it off, fold it, drain its battery, overheat it, or make it need an update, and watch what your app does.
• What the glasses see. A test card, a camera of this computer (a phone used as a webcam works too), the screen, or a picture or video file. A slider tilts the wearer's head.
• On the glasses. Tap the temple, swipe, press the capture or action button.
• Voice. Type a sentence and the glasses "hear" it, or speak if the computer has a microphone. The panel shows what your app said back into the wearer's ear.
• What the glasses hear. For an app that asks for sound with the camera stream: silence, a steady tone, a microphone of this computer, or a sound file. Your app gets it in the form the glasses send it, and a meter shows the level.
• The display. With a Meta Ray-Ban Display pair added, the panel draws the screen your app sends, and its buttons can be pressed.
• Break something. Make registering, a permission, the stream or a photo fail on purpose.

The agent uses the same emulator to check its work. After a change it can pair the glasses, answer the Meta AI app, start the camera, tap the temple, say a sentence, fold the glasses or make a step fail, and read what your app showed and said. Ask it to "try what happens when the glasses overheat" and it does that.

The first time a project uses a camera or a microphone of your computer, your system asks whether Graspable may use it.

The emulator follows Meta's toolkit closely, including the pause before glasses become ready, but it is not the glasses. Timing, picture quality, the microphones and what Meta's servers allow can differ. Try on a real pair before you rely on something.

The web app for Display glasses

Create a project from Display glasses web app. The preview shows the app in a frame of the glasses' size: the arrow keys move, Enter selects, Escape goes back, which is what the Neural Band and the touchpad send. Publish the app as you publish any Graspable app. Its page then shows an install code beside the frame; scan it with the glasses or the Meta AI app, and the app appears at the end of the app grid on the glasses. Without the code: Meta AI app, Devices, the Display glasses, App connections, Web apps, Add a web app, then the address.

The display is additive, so black is transparent and bright colours show. Keep text large, screens short and one thing per screen. The agent knows these rules. The Meta Ray-Ban Display Simulator, a Chrome extension from Meta, shows a page the way the glasses do.

What Graspable does not do

• It does not publish to Meta for you. Meta's review, release channels and testers are in the Wearables Developer Center, with your own Meta developer account.
• It does not hide the capture LED, record people without their knowledge, or send what the camera sees anywhere you did not ask for, and the agent is told not to build such things.
• A published Display web app is an ordinary web page at your app's address; it holds no key and knows nothing of your Meta account.