Library / SDK
v-modal/vmodal_sdk_smartglass avatar
v-modal/vmodal_sdk_smartglass

The VModal smart glasses SDK hands your app an SRT stream and keeps the camera plumbing out of it

SDK for Visual Memory Search with Smart Glasses

351 stars1 forksRustLicense varies

At a glance

What is it?
A Kotlin client over a Rust media core for live first-person video from Meta smart glasses: H.264 encoding on Android, MPEG-TS muxing in Rust, encrypted SRT to your ingest, with GOP-aware backpressure that drops frames or asks for a keyframe instead of accumulating latency. Credentials stay on your backend, which mints short-lived targets.
Who is it for?
Adopt this SDK if you are building a live first-person video feature on Meta smart glasses and want the transport handled for you: hardware H.264 encoding, MPEG-TS muxing, SRT reconnects, queue pressure and metrics, without writing a camera session or a streaming protocol. Do not adopt it for the other half of the product, because video upload, indexing and multimodal search belong to the VModal Android SDK, and this one streams only.
Can I use it commercially?
Not without permission. GitHub finds no licence file in the repository, and without a licence all rights are reserved by default: you may read the code but not reuse it. Check the README, or ask the authors, before using it.
Is it still maintained?
Yes. The repository last received commits 26 days ago.
What is it written in?
Mainly Rust, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • v-modal/vmodal_sdk_smartglass351 ★ · -24
Star history
Repository2026-09-172026-11-15
v-modal/vmodal_sdk_smartglass375351

-24 Stars in the last 59 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

It streams, and upload, indexing and search are somebody else's SDK

The first thing to establish is what this repository is not. The note at the top of the documentation says it streams live camera video from Android smart glasses, and that for video upload, indexing and multimodal search you should use the VModal Android SDK instead.

That boundary matters because the name is misleading. The project description calls it an SDK for visual memory search with smart glasses, and a search pipeline implies storage, retrieval and multimodal work. None of that is here. What is here is the narrow, hard part of a product: getting frames off a wearable, across a weak network, into somebody's ingest service without the user seeing a stall.

The division of labour is the selling point. Your app owns the product experience, and the SDK owns the demanding path between a wearable camera and your live ingest service.

What the SDK takes over is concrete: Meta DAT capture lifecycle and Android camera coordination, hardware H.264 encoding, MPEG-TS muxing, reconnect behaviour, bounded queues, backpressure signals, metrics and typed failures. What stays yours is screens, permissions, Meta registration, foreground-service policy, authentication and stream controls.

Two plausible uses are named in the mapping table: remote expert support where a technician needs a reliable first-person video link with reconnects, and hands-free field work where the capture lifecycle has to survive the user moving around.

Glasses to ingest in six hops, and a backend that never touches a camera

The path is drawn as one diagram, and it is the clearest description of the design.

mermaid
flowchart LR
    G[Smart glasses] --> D[Meta DAT]
    D --> K[Android H.264]
    K --> R[Rust media core]
    R --> S[Encrypted SRT]
    S --> I[Live ingest]

    A[Your Android app] --> C[SmartGlassClient]
    B[Authenticated backend] --> T[Short-lived target]
    T --> C
    C --> K
    C --> R

Two flows meet in SmartGlassClient. One is media: glasses, the Meta DAT layer, hardware encoding on Android, the Rust core, encrypted SRT, live ingest. The other is authority: your app creates the client, and your authenticated backend hands it a short-lived target that the client then uses.

That second flow is why credentials work the way they do. Your backend validates the user and returns a short-lived ingest target, which keeps account-level provider credentials away from the device entirely.

The four-way ownership split underneath the diagram is the reason the code is small. The Kotlin client manages the public API, Meta DAT integration, capture, MediaCodec, coroutine state, events and lifecycle. The Rust core manages bounded encoded-media queues, H.264 normalization, MPEG-TS muxing, SRT transport, reconnects and transport metrics. Your app owns the screens and the policy. Your backend owns validation.

The summary the documentation gives is worth keeping in mind when reading any of it: the Activity never needs to understand a JNI handle, an SRT socket or an encoded GOP, and the backend never needs to manage an Android camera session.

The public API hides Meta DAT types and Rust handles behind Flow and StateFlow

The Kotlin boundary is the part you will actually touch, and its discipline is in what it refuses to expose.

The public API does not expose Meta DAT types or Rust handles. That is the whole design goal in one sentence, and it is what lets the internals change without breaking your app.

Ownership is also split by time rather than by file. Your host app owns the camera permission and the registration experience, and it does so before streaming starts. The SDK owns its single capture session only while streaming is happening. So there is a window during which the camera belongs to the client and a window during which it belongs to your activity, and the checklist later in the documentation turns that into a hard rule: no competing camera session while SmartGlassClient is streaming.

State is expressed with coroutines rather than callbacks. The Kotlin side is described as a compact API using Flow and StateFlow, and the client owns coroutine state, events and lifecycle. For a capture session that starts, reconnects and stops on conditions, that choice means the app composes state instead of reconciling it.

Exactly one SmartGlassClient is expected per active capture owner, which is the other half of that rule and the reason two screens cannot both believe they own the stream.

GOP-aware backpressure drops frames or requests a keyframe instead of queueing them

This is the design decision that makes the difference on wearable hardware, and it is stated as a choice rather than an optimisation.

Bounded latency, not unbounded memory. GOP-aware backpressure and bounded queues keep a weak network from turning into ever-growing latency. When transport falls behind, the SDK can signal the Kotlin layer to drop safely or to request a new keyframe.

Both signals are honest about video. Dropping safely means discarding something the decoder can survive losing. Requesting a new keyframe means accepting a cost now to avoid a stream the receiver cannot decode until the next one. Neither is possible without understanding the encoded stream, which is why the Rust core owns H.264 normalization and why the Kotlin layer only receives a signal.

The consequence for a weak network is that latency stays bounded and quality degrades. A queued-forever design produces a stream that is minutes behind and therefore useless for anything first-person.

Observability comes with it. client.metrics exposes connection health, queue pressure and transport behaviour, and the documentation names three consumers: a developer panel, a field diagnostic screen, or a support workflow. Queue pressure in particular is the number that tells a support engineer whether a user is seeing a slow link or a stalled sender.

An expired target is replaced behind one coherent stream lifecycle

Recovery is described as two layers, one for the transport and one for the destination.

SRT reconnects use configurable delays, jitter and timeouts, which is the ordinary answer to a broken socket. The interesting case is a target that has expired or been rejected rather than a dropped connection.

When that happens LiveStreamTargetProvider can fetch a fresh destination while the app continues to present one coherent stream lifecycle. Your app therefore does not have to model target expiry as an error state and restart the session; it asks the provider for another one and the stream continues.

That design has a credential requirement, and the documentation is unusually explicit about it. The app may briefly receive the SRT fields required to contribute live video. It should never contain account-level Cloudflare or live-platform credentials. After validating the app user, your backend mints or leases a short-lived target. Do not log or persist its stream ID or passphrase.

Those three sentences are the security boundary of the whole integration. The device holds something scoped enough to contribute video and nothing scoped enough to act as your account, which is also why your backend has to implement a mint-or-lease endpoint and why the integration checklist lists an authenticated backend endpoint returning a fresh LiveStreamTarget as a requirement rather than a suggestion.

Two build flavors, one of which exists so you can test without Meta credentials

The project ships two build flavors and the difference is about what you can do without hardware.

The offline flavor is for local lint and JVM tests without Meta artifacts or device credentials. The meta flavor is for real-device integration with the Meta Wearables DAT dependencies. Nothing else changes between them.

That split is the reason continuous integration is possible at all for a project whose real target is a headset. A team can lint, run unit tests and check API usage on every commit without a Meta account or a device, and only switch to the meta flavor when somebody is actually holding glasses.

The simulator is the companion to that. examples/00_simulator covers installing an Android 14 AVD and exercising the Meta Mock Device Kit, so the capture path can be driven without physical glasses.

The production checklist is short enough to be a real gate rather than a wish list: Android API 31 or newer, JDK 17 and Android compile SDK 36; a compatible Meta smart-glasses registration and Meta Wearables DAT setup; a user-facing camera permission and Meta registration flow in the host app; a foreground service when streaming may continue outside the foreground; an authenticated backend endpoint returning a fresh LiveStreamTarget; exactly one SmartGlassClient per active capture owner; and no competing camera session while streaming.

The foreground-service line is the one that surprises Android developers, because streaming that outlives the activity needs a declared service, not a background thread.

kotlin_client/ and system_core/ are the two halves, and there is no licence file

The repository layout matches the architecture exactly. kotlin_client/ is the Kotlin side, system_core/ is the Rust media core, and the Gradle files at the root tie them together, with a gradlew wrapper and gradlew.bat for the command line. docs_sdk/ holds the implementation guide, examples/ carries the simulator, and readme_assets/ holds the images used in the documentation.

There is also a file named .vmodal-public-source at the root, which suggests the repository is published as a public snapshot rather than developed in the open, and that snapshot posture shows in what the documentation does and does not contain.

What it does not contain is the build and integration detail. The README points to the SDK quickstart at docs_sdk/readme.md for prerequisites, local builds, SmartGlassClient creation, short-lived stream targets, state and metrics collection, and deterministic cleanup. In other words, the commands that get you from a clone to a running stream are in a linked guide rather than in the file you are reading, which is a real constraint on evaluating this from the repository alone.

On upkeep: there are no published releases, no licence file sits at the repository root, and the last push was on 2026-09-13. Resolve the licence before shipping anything built on this, and read the quickstart rather than assuming the README is the whole integration.

Editorial conclusion

Adopt this SDK if you are building a live first-person video feature on Meta smart glasses and want the transport handled for you: hardware H.264 encoding, MPEG-TS muxing, SRT reconnects, queue pressure and metrics, without writing a camera session or a streaming protocol. Do not adopt it for the other half of the product, because video upload, indexing and multimodal search belong to the VModal Android SDK, and this one streams only. Verify three things first: that you can satisfy API 31 or newer with JDK 17 and compile SDK 36 plus Meta Wearables DAT registration, that your backend can mint and lease a short-lived LiveStreamTarget rather than embedding an account credential in the app, and which build flavor you need, since the offline flavor exists precisely so you can lint and run JVM tests without Meta artifacts or device credentials. There is no licence file at the repository root and no published release, and the last push was on 2026-09-13.

Frequently asked questions

What does the VModal smart glasses SDK actually do?

It streams live camera video from Android smart glasses. Frames go through Meta DAT, hardware H.264 encoding on Android, the Rust media core with MPEG-TS muxing, and encrypted SRT to your live ingest service.

Does the VModal SDK handle video upload and search?

No. The documentation states that this SDK streams live camera video, and that video upload, indexing and multimodal search are handled by the VModal Android SDK instead.

What are the Android requirements for the VModal smart glasses SDK?

Android API 31 or newer, JDK 17 and Android compile SDK 36, plus a compatible Meta smart-glasses registration and Meta Wearables DAT setup. The host app needs a user-facing camera permission and Meta registration flow, and a foreground service if streaming continues outside the foreground.

How should credentials be handled with the VModal SDK?

Your backend validates the user and mints or leases a short-lived ingest target. The app may briefly receive the SRT fields needed to contribute video but should never carry account-level Cloudflare or live-platform credentials, and you should not log or persist the stream ID or passphrase.

Official sources

  1. Issues
  2. README
  3. v-modal/vmodal_sdk_smartglass on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/v-modal-vmodal-sdk-smartglass.svg)](https://hysenlabs.com/projects/v-modal-vmodal-sdk-smartglass)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/v-modal-vmodal-sdk-smartglass.svg)](https://hysenlabs.com/projects/v-modal-vmodal-sdk-smartglass)

Was this useful?