VModal for Smart Glasses: an Android and Rust SDK for first-person live video

What the VModal smartglass SDK actually solves

The problem is not capturing video. It is keeping a first-person stream alive from a wearable camera to a live ingest service while the network misbehaves. The README frames the split plainly: "Your app owns the product experience; the SDK owns the demanding path between a wearable camera and your live ingest service." That path includes capture coordination, H.264 encoding, MPEG-TS muxing, SRT transport, reconnects and transport metrics.

The intended reader is an Android developer building a glasses app for remote expert support, hands-free field work, live training or inspection. The README lists those four scenarios in a table. It is not a search product despite the repository description mentioning visual memory search. A note in the README states that the SDK streams live camera video and that video upload, indexing and multimodal search belong to the separate VModal Android SDK. Anyone arriving expecting image search has the wrong repository.

Kotlin on the device, Rust on the media path

The architecture is a deliberate two-language split. A Kotlin client manages the public API, Meta DAT integration, capture, MediaCodec, coroutine state, events and lifecycle. A Rust core manages bounded encoded-media queues, H.264 normalization, MPEG-TS muxing, SRT transport, reconnects and transport metrics.

The data flow in the README diagram runs from smart glasses through Meta DAT, into Android H.264 encoding, into the Rust media core, out as encrypted SRT to a live ingest. A separate branch shows your Android app calling SmartGlassClient, and an authenticated backend returning a short-lived target that the client passes down into the capture and transport layers.

That boundary has a cost. The README claims the Activity never needs to understand a JNI handle, SRT socket or encoded GOP, and that the backend never manages an Android camera session. The trade is that you inherit a native library, a build flavor system and an opaque transport layer. When a stream fails, the visible surface is metrics and typed failures, not a stack you can step through in Kotlin. The README does not document a debug mode for the Rust side.

Backpressure, reconnects and metrics on the transport

Two mechanisms carry the reliability story. The first is GOP-aware backpressure with bounded queues. When transport falls behind, the README says the SDK can signal the Kotlin layer to drop safely or request a new keyframe. This is a latency-versus-completeness decision: dropping frames keeps the stream current instead of letting delay accumulate. The README does not state the default queue size or the drop policy thresholds.

The second is SRT reconnect behaviour with configurable delays, jitter and timeouts. When a target is expired or rejected, LiveStreamTargetProvider can fetch a fresh destination while the app keeps presenting one stream lifecycle. That is the interesting design choice: the app sees a continuous stream even though the underlying destination changed.

Metrics are exposed through client.metrics for connection health, queue pressure and transport behaviour. The README suggests a developer panel, field diagnostic screen or support workflow. It does not enumerate the metric names or units.

Building and running your first stream

The README does not inline build commands. It points to docs_sdk/readme.md as the quickstart, described as covering prerequisites, local builds, SmartGlassClient creation, short-lived stream targets, state and metrics collection, and deterministic cleanup. The repository layout confirms that directory exists alongside kotlin_client/ and system_core/.

The stated prerequisites are Android API 31+, JDK 17, and Android compile SDK 36. Treat those as the gate before anything else.

There are two build flavors. The offline flavor exists for local lint and JVM tests without Meta artifacts or device credentials. The meta flavor pulls in the Meta Wearables DAT dependencies for real-device integration. The README does not print the Gradle task names for either flavor, so read docs_sdk/readme.md and use the task names it gives there rather than assuming one. The repository does ship the Gradle wrapper, so the entry point is the wrapper script:

./gradlew

For a first run without physical glasses, the repository ships examples/00_simulator/, which the README describes as installing an Android 14 AVD and exercising the Meta Mock Device Kit. Its own README is the entry point.

On the integration side, the README's checklist is short and specific: exactly one SmartGlassClient per active capture owner, no competing camera session while it is streaming, a user-facing camera permission and Meta registration flow in the host app, and a foreground service if streaming may continue outside the foreground. The client is created by your app, and the backend returns a LiveStreamTarget that the client consumes. The README does not publish the constructor signature or the target fields.

Where this SDK is the wrong choice

The clearest limitation is scope. If your product needs recorded video uploaded, indexed and searched, this repository does not do it. The README says so directly and redirects you to the VModal Android SDK. Building search on top of a live SRT stream is not what the transport layer is for.

The second limitation is platform. This is Android only, with Meta DAT integration and Meta Wearables dependencies. The topics list mentions google-glass and google-glasses, but nothing in the README describes a Google Glass capture path, so do not read those tags as support.

The third is operational. The README warns that the app may briefly receive SRT fields but must never hold account-level Cloudflare or live-platform credentials, and that the stream ID and passphrase should not be logged or persisted. That means you need an authenticated backend endpoint before the SDK is useful at all. A prototype without one can only run against the offline flavor.

Finally, the licence is not stated in the repository description or the README. Without a licence file, the terms under which you may ship this in a commercial app are unknown.

How it differs from a plain MediaCodec plus RTMP pipeline

The common alternative is assembling the path yourself: Camera2 or CameraX for capture, MediaCodec for H.264, and a library such as LFLiveKit or a plain RTMP or SRT client for transport. That route keeps everything in Kotlin and gives you full visibility into every stage.

The difference in approach is where the failure handling lives. A hand-rolled pipeline typically leaves reconnect logic, queue bounding and keyframe requests to the application, and each app reinvents them slightly differently. VModal pushes those into a Rust core with GOP-aware backpressure and SRT reconnect parameters, and exposes the result as metrics and typed failures through a Kotlin API using Flow and StateFlow.

You pay for that with a native dependency and a less transparent failure path. If your team already has a working streaming pipeline and wants to tune muxing or transport internals directly, this SDK's abstraction works against you. If your team is small and the reconnection behaviour is the part you keep getting wrong, the abstraction is the point.

Maintenance, licence and upgrade cost

The repository is not archived, and the last push was on 2026-09-13. That is a recent push, but a single push date says nothing about release cadence, and no releases were retrieved for this repository. There is no version number to pin against in the repository description or the README.

Upgrade cost is shaped by the native core. A Rust media core compiled into an Android app means ABI targets, NDK toolchain versions and Gradle wiring that a pure-Kotlin dependency does not have. The offline and meta flavor split also means your CI needs a path that builds without Meta artifacts, or every build requires device credentials.

The licence is unknown here. Before shipping, locate the licence file in the repository or ask in the VModal Discord linked from the README. Do not assume permissive terms from the presence of a public repository. This is a factual gap, not a legal opinion.

Editorial conclusion

Adopt it if you are shipping an Android app for Meta smart glasses and you want capture, encoding, muxing and SRT transport handled behind a Kotlin boundary, with a backend that mints short-lived targets. Do not adopt it if you need video upload, indexing or multimodal search; the README points those use cases at the separate VModal Android SDK. Before committing, verify the licence, confirm the docs_sdk quickstart against your Android API 31+ and JDK 17 toolchain, and check that your backend can issue a fresh LiveStreamTarget on demand.

Frequently asked questions

Official sources

1. Issues
2. README
3. v-modal/vmodal_sdk_smartglass on GitHub