Meta's glasses SDK,
minus the boilerplate and most of the suffering.

Why Spectra exists

Meta's Wearables Device Access Toolkit is genuinely capable — it gives your app the wearer's-eye camera, open-ear audio, and on Ray-Ban Display glasses a declarative display API. It's also two separate native SDKs (Swift and Kotlin), shipped through GitHub Packages behind a token, with callbacks, result builders, and a fair few ways to hold it wrong.

Spectra collapses all of that into one small, opinionated, multiplatform surface:

Install

Spectra targets Android (minSdk 29) and iOS (15.2+), matching the Meta AI app's floor. Current release: com.umain.spectra:spectra:0.2.1, published to GitHub Packages (Maven) and as a Swift Package.

Android / KMP — GitHub Packages (Maven)

GitHub Packages requires a token even to read public packages — create a PAT with read:packages. Spectra also pulls Meta's SDK transitively on Android, so add that repository too.

// settings.gradle.kts
dependencyResolutionManagement {
    repositories {
        google(); mavenCentral()
        // Spectra
        maven {
            url = uri("https://maven.pkg.github.com/jacksonmafra-umain/spectra")
            credentials {
                username = "<your-github-user>"
                password = providers.gradleProperty("github_token").orNull
                    ?: System.getenv("GH_TOKEN") ?: ""
            }
            content { includeGroup("com.umain.spectra") }
        }
        // Meta SDK (transitive, Android only)
        maven {
            url = uri("https://maven.pkg.github.com/facebook/meta-wearables-dat-android")
            credentials {
                username = ""
                password = providers.gradleProperty("github_token").orNull
                    ?: System.getenv("GH_TOKEN") ?: ""
            }
            content { includeGroup("com.meta.wearable") }
        }
    }
}

// shared/build.gradle.kts
kotlin {
    sourceSets {
        commonMain.dependencies {
            implementation("com.umain.spectra:spectra:0.2.1")
        }
    }
}

iOS — Swift Package Manager

The release ships a prebuilt Spectra.xcframework as a binary Swift package. In Xcode: File → Add Package Dependencies… → the repo URL → version 0.2.1. No token needed (public release asset).

// Package.swift
.package(url: "https://github.com/jacksonmafra-umain/spectra", from: "0.2.1")
// then: import Spectra

Build, test & publish locally

Use the Gradle wrapper. The project runs on the current toolchain — Gradle 9.1, AGP 9, Kotlin 2.4, Compose Multiplatform 1.11 (the AGP 9 KMP plugin, com.android.kotlin.multiplatform.library, with an androidLibrary {} block). If the wrapper is missing, bootstrap it once with any Gradle 9.x on your PATH: gradle wrapper --gradle-version 9.1.

Run the tests

The mock-driven test suite needs no glasses and no token. It's the fastest way to confirm everything works.

./gradlew :spectra:allTests                # mock-driven tests, no token needed
./gradlew :spectra:compileKotlinMetadata   # compile common code + mock
./gradlew check                            # compile + tests + lint, the lot

Build & run the demo

./gradlew :demo:assembleDebug    # build the Android Playground apk
./gradlew :demo:installDebug     # install on a device/emulator, then launch "Spectra Playground"

Publish to Maven Local

Consume Spectra from another project on your machine, no remote repository required. This publishes the multiplatform metadata plus the Android and iOS variants to ~/.m2/repository under com.umain.spectra:spectra:0.2.1.

./gradlew :spectra:publishToMavenLocal
ls ~/.m2/repository/com/umain/spectra      # verify it landed

Then point the consuming project at Maven Local and depend on it:

// settings.gradle.kts of the OTHER project — mavenLocal() first so it wins
dependencyResolutionManagement {
    repositories {
        mavenLocal()
        google()
        mavenCentral()
    }
}

// shared/build.gradle.kts of the OTHER project
kotlin {
    sourceSets {
        commonMain.dependencies {
            implementation("com.umain.spectra:spectra:0.2.1")
        }
    }
}

Quickstart

The whole integration is a fixed sequence: initialize, register, permit, session, stream. Skip a step and the SDK tells you off via a typed error. Here it is end to end against the mock — identical shape on real hardware.

import com.umain.spectra.Spectra
import com.umain.spectra.core.*
import com.umain.spectra.camera.*

suspend fun run() {
    val spectra = Spectra.mock()          // swap for Spectra.create(context, bridge) on Android

    spectra.initialize()
    spectra.startRegistration()           // deeplinks to the Meta AI app on device

    // Wait for the platform's auto-selector to report an active pair of glasses.
    // This is the honest "is there a device?" signal — see "MockDeviceKit & devices".
    spectra.hasActiveDevice.first { it }

    spectra.requestPermission(Permission.CAMERA)   // gates the stream's contents

    val session = spectra.createSession(DeviceSelector.Auto).getOrThrow()
    session.start()
    session.state.first { it == SessionState.RUNNING }

    val stream = session.openCameraStream(
        StreamConfiguration(quality = VideoQuality.LOW, frameRate = FrameRate.FPS_15)
    ).getOrThrow()

    stream.frames
        .onEach { frame -> render(frame) }   // ByteArray RGBA; convert at the edge
        .launchIn(scope)
    stream.start()

    val photo = stream.capturePhoto().getOrThrow()
}

Run the demo (Android & iOS)

The Spectra Playground is one shared Compose Multiplatform screen, the same code on both platforms. It walks the exact flow your app would, and it runs with or without real glasses.

1. Build it. Android: cd demo && ./gradlew :androidApp:installDebug. iOS: open demo/iosApp/Spectra.xcodeproj and Run (simulator for the mock, a real device for actual glasses).
2. Connect. The white welcome screen has a single blue Connect my glasses button — it kicks off registration (deeplinking to the Meta AI app on a real device).
3. Stream. Once registered you land on the black Stream Your Glasses Camera screen. When a device is active the blue Start streaming button enables; tap it to grant camera, open a session, and start the camera stream in one go. The circular button takes a still mid-stream.
4. Disconnect. The gear (top-right) reveals a Disconnect button — that's startUnregistration().
5. No glasses? Fake one. The floating ladybug button (bottom-right) opens a debug sheet with the Mock Device Kit: Enable MockDeviceKit → Pair Ray-Ban Meta, and a device goes active immediately. The same sheet has the Backend toggle (Mock vs. real glasses).

Test with the mock

Spectra.mock() is a complete in-memory simulation. It enforces the same preconditions as the real SDK — initialize before use, register before sessions — so passing against the mock means you actually learned the steps. Drive the failure paths on purpose:

// Rehearse "user said no" without misconfiguring anything.
val denied = Spectra.mock(MockConfig(autoGrantPermissions = false))

// Rehearse a failed registration and your error UI.
val broken = Spectra.mock(MockConfig(failRegistration = true))

It runs on the JVM, in CI, and inside a Compose or SwiftUI preview. The Spectra Playground demo runs on it by default, and on the real backends when you flip the switch.

MockDeviceKit & the empty-devices trap

Hard-won lesson, now baked into the API. With real glasses connected and registered, Wearables.devices (the raw device list) can still be empty — so code that waits for devices.isNotEmpty() hangs forever, even though everything is fine. Meta's own CameraAccess sample sidesteps this: it never reads that list. It listens to the auto-selector's active device stream instead.

Spectra does the same. The iOS backend folds AutoDeviceSelector.activeDeviceStream() into devices, and every client exposes the honest signal directly:

// Wait for a usable device the reliable way.
spectra.hasActiveDevice.first { it }      // Flow<Boolean>, fed by the auto-selector
val session = spectra.createSession(DeviceSelector.Auto).getOrThrow()

Camera permission gates the stream's contents, not the device's existence — which is why hasActiveDevice can be true before you've granted anything.

Running with no hardware: MockDeviceKit

To exercise the whole register → session → stream pipeline without real glasses, use the developer-only MockDeviceKit. It's null on backends that can't fake a device (a real Android build); the mock client and the real iOS backend (via Meta's MockDeviceKit.shared) both provide one.

spectra.mockDeviceKit?.let { kit ->
    kit.enable()        // turn the simulator on
    kit.pairGlasses()   // pair (and power on) a simulated Ray-Ban Meta -> hasActiveDevice flips true
    kit.state.collect { s -> /* s.enabled, s.pairedCount */ }
}

Session lifecycle

The single most important rule: the device drives session state and tells you asynchronously. You react.
You never assume the cause — the SDK deliberately won't tell you why a session paused, only that it did.

session.state.collect { state ->
    when (state) {
        SessionState.RUNNING -> showLive()
        SessionState.PAUSED  -> holdAndWait()   // it may resume or stop; both are fine
        SessionState.STOPPED -> releaseAndOfferRestart()
        SessionState.STARTING -> showConnecting()
    }
}

Registration & permissions

Registration is the one-time handshake that links your app to the user's glasses through the Meta AI app.
Camera permission is then granted per app, checked across all of the user's linked glasses — if any pair has said yes, you're granted.
Spectra hides that bookkeeping.

spectra.registrationState.collect { state ->
    when (state) {
        RegistrationState.Registered   -> enableFeatures()
        RegistrationState.Registering  -> showProgress()
        is RegistrationState.Failed    -> showError(state.reason)
        RegistrationState.NotRegistered -> showConnectCta()
    }
}

when (spectra.checkPermission(Permission.CAMERA)) {
    PermissionStatus.GRANTED -> startStreaming()
    else -> spectra.requestPermission(Permission.CAMERA)
}

Camera & photos

Frames arrive as a Flow<VideoFrame> of raw bytes. Collect on a background dispatcher and convert to a platform image at the very last moment, on the thread that draws.

val stream = session.openCameraStream(
    StreamConfiguration(quality = VideoQuality.MEDIUM, frameRate = FrameRate.FPS_24)
).getOrThrow()

stream.frames.onEach { frame ->
    // frame.bytes is RGBA, frame.width x frame.height
}.launchIn(scope)

stream.start()
val photo: Photo = stream.capturePhoto().getOrThrow()  // a still, mid-stream

The display (Ray-Ban Display)

Build screens with a declarative DSL that will feel like Compose or SwiftUI, because that's deliberate.
Each sendContent replaces the entire display — there's no partial update, and the glasses retain no state.
Your phone is the single source of truth.

val display = session.attachDisplay().getOrThrow()
display.state.first { it == DisplayState.STARTED }

display.sendContent {
    flexBox(direction = Direction.COLUMN, gap = 12, paddingAll = 16) {
        text("Oil Change Guide", style = TextStyle.HEADING)
        text("Six easy steps. Roughly.", style = TextStyle.BODY)
        text("Duration: 30 min", style = TextStyle.META, color = TextColor.SECONDARY)
        button("Start", iconName = IconName.ARROW_RIGHT, onClick = ::showFirstStep)
    }
}

Audio: microphones & speakers

Audio is not part of the Device Access Toolkit. On real hardware the Ray-Ban speakers and microphone are plain Bluetooth,
shared with the system audio stack, so Spectra wraps the platform audio session — not the SDK — behind its own SpectraClient.audio capability (null where a backend can't route audio).

val audio = spectra.audio ?: return   // null when the backend can't route audio

audio.state.onEach { /* it.profile, it.micLevel, it.isPlaying, it.routedToGlasses */ }.launchIn(scope)

audio.playToGlasses("Hello from your glasses")   // A2DP, high quality
audio.startMicCapture()                           // HFP, emits a live level
audio.stopMicCapture()

Android

Get the real client from Spectra.create(context, bridge). Registration and the camera permission both deeplink to the Meta AI app and return via an ActivityResult contract
that Android insists you register before the host is resumed — so those three calls live in an ActivityBridge you implement against a real Activity. Spectra handles everything else.

val spectra = Spectra.create(
    context = applicationContext,
    bridge = object : ActivityBridge {
        override fun launchRegistration() = Wearables.startRegistration(activity)
        override fun launchUnregistration() = Wearables.startUnregistration(activity)
        override suspend fun requestCameraPermission(): PermissionStatus =
            permissionLauncher.requestAndAwait(Permission.CAMERA)
    },
)

Manifest entries (Bluetooth permissions, the Meta callback scheme, the DAM flag) are listed in the Android integration guide.

iOS

Get the client from Spectra.create(bridge). Meta's iOS toolkit is a Swift package and Kotlin/Native speaks Objective-C interop, not Swift
so Spectra crosses the gap with a small protocol, SpectraNativeBridge, that Swift implements against Wearables.shared.
The demo ships a complete implementation (SpectraBridge.swift): registration, the camera-permission flow, the active-device stream, camera streaming, photo capture, and MockDeviceKit.
It's wired and works on real Ray-Ban Meta glasses today; Spectra.mock() still runs on the simulator and in SwiftUI previews.

Go live: wiring it to Meta AI and the glasses

Everything above runs on the mock. This is what it takes to point Spectra at actual hardware — which means dealing with Meta's Wearables Developer Center, a Managed Meta Account, and a permissions review.
None of it is hard. All of it is bureaucratic. Budget an afternoon and a strong opinion about acronyms.

Step 1 — One organization, one MMA, and the colleague who already made it

Your company gets exactly one Managed Meta Account (MMA) organization in Admin Center.
Before you create one, ask IT / your lead whether it already exists, because creating a second one is the kind of mistake that generates a meeting.
One admin (IT, lead, PM) signs up to the Wearables Developer Center, gets redirected to set up the MMA
org at work.meta.com under the company's real name, and then invites everyone else. Only members of that MMA org can join your Developer Center team.

Reference: Onboarding & organization management.

Step 2 — Create the project and harvest your credentials

In the Developer Center, New project → name + description. Open Configuration and define your mobile app. This is where you get the two strings the SDK actually cares about: APPLICATION_ID and CLIENT_TOKEN.

Step 3 — Product listing and the permissions essay

Provide an app name and icon (PNG/JPEG, up to 200×200, dark + light if you're keen). Then, under Permissions, justify every capability you request — camera, microphone, voice invocation. This text is for Meta's internal reviewers, not end users; they use it to decide whether your camera request is reasonable or just nosy. Write it like a human who has a legitimate reason, because someone reads it.

Reference: Manage projects.

Step 4 — Put the credentials in the app

Outside Developer Mode, the SDK uses these for attestation — proving your app is the app it claims to be. Get them wrong and it simply won't connect, with an error that won't hold your hand.

Android — in AndroidManifest.xml, plus the Bluetooth permissions and the callback URL scheme:

<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.INTERNET" />

<application>
  <meta-data android:name="com.meta.wearable.mwdat.APPLICATION_ID" android:value="${mwdat_application_id}" />
  <meta-data android:name="com.meta.wearable.mwdat.CLIENT_TOKEN"   android:value="${mwdat_client_token}" />
  <meta-data android:name="com.meta.wearable.mwdat.DAM_ENABLED"    android:value="true" />

  <activity android:name=".MainActivity" ...>
    <intent-filter>
      <action android:name="android.intent.action.VIEW" />
      <category android:name="android.intent.category.DEFAULT" />
      <category android:name="android.intent.category.BROWSABLE" />
      <data android:scheme="yourappscheme" />
    </intent-filter>
  </activity>
</application>

iOS — in Info.plist, the MWDAT dictionary plus the external-accessory and Bluetooth keys:

<key>MWDAT</key>
<dict>
  <key>AppLinkURLScheme</key><string>yourappscheme://</string>
  <key>MetaAppID</key><string>$(META_APP_ID)</string>
  <key>ClientToken</key><string>$(CLIENT_TOKEN)</string>
  <key>TeamID</key><string>$(DEVELOPMENT_TEAM)</string>
  <key>DAMEnabled</key><true/>
</dict>
<key>UISupportedExternalAccessoryProtocols</key>
<array><string>com.meta.ar.wearable</string></array>

Step 5 — Versions and release channels (a.k.a. how testers get in)

Cut a version (Distribute → Versions → Create new version, pick Major/Minor/Patch like a responsible adult), then create a release channel and invite testers by email.
Each channel holds one version at a time; channels are invite-only. The emails must be real Meta Accounts — note, not Managed Meta Accounts; the two sound identical and are
not, which is exactly the sort of thing that eats a morning. Testers accept at wearables.meta.com/invites.

Reference: Set up release channels.

Step 6 — Prepare the actual glasses

On the test device: Meta AI app v254+, glasses firmware v20+ (Ray-Ban Meta) or v21+ (Ray-Ban Display).
Pair the glasses, then enable Developer Mode: Settings → App Info → tap the app version five times, toggle it on.
For the display feature you also install DAT onto the glasses themselves: Meta AI v272+, firmware v125+, put the glasses on, then trigger the install from Dev Mode (keep the app open; it takes 5–10 seconds, and it'll refuse below 10% battery because of course it will).

Reference: Getting started / setup and Display overview.

Step 7 — The tester's side, in the Meta AI app

Once invited, a tester picks the right Release Channel (device menu → Settings → Release Channel) and manages what your app may touch under Settings → Connected Apps → your app → camera set to Always allow / Always ask / Don't allow. The device shows up via the auto-selector once it's connected and registered (see MockDeviceKit & devices); the camera permission then governs whether the stream may run.

Step 8 — Swap the mock for the real backend

Both real backends are already wired in the demo. On Android, Spectra.create(context, bridge) delegates to mwdat — you just supply the GitHub Packages token and the manifest credentials. On iOS, Spectra.create(bridge) uses the shipped SpectraBridge.swift against Wearables.shared — set your Meta App ID / Client Token / Team ID in Info.plist and a custom URL scheme for the registration callback. The flow your code calls — initialize, register, wait for an active device, permit, session, stream, display — does not change one line between mock and hardware. That was the whole point.

For AI agents

Spectra is built agentic-first. Point your coding assistant at these and it can write correct integrations without guessing:

For the underlying SDK, Meta also serves a full API reference at wearables.developer.meta.com/llms.txt?full=true and ships Claude Code / Cursor / Copilot integrations from its repos.

Known issues (upstream)

Inherited from the toolkit's developer preview, so worth knowing before you file a bug against Spectra:

• Installation to the glasses fails below 10% battery, or silently if Wi-Fi is off (Android).
• You can't reconfigure streaming within one session — stop and start a new one.
• Android may crash on rapid repeated captures during long (>1 min) streams.
• App Store submission isn't supported yet; distribute via release channels. iOS uses ExternalAccessory, which trips MFi/privacy-manifest review.
• Only one third-party app stays registered at a time in Developer Mode.

Full list: Meta — Known issues.

Meta references

Spectra wraps the toolkit; these are the primary sources it's built against.