CosmoEdge: a C++ edge AI engine that runs the same video pipeline on Sophon, Rockchip and x86

The gap CosmoEdge is trying to fill: model serving stops where deployment starts

Getting a detection or VLM model to run on an NPU is the easy half. The half that consumes engineering time is everything around it: pulling frames off a camera, resizing and colour-converting them for the accelerator, drawing boxes and labels back onto the stream, deciding when a detection becomes an alarm, and getting that alarm somewhere useful. Teams usually rebuild that layer per board, and the per-board code drifts.

CosmoEdge positions itself as that layer. The README describes it as going "beyond model serving with a complete application layer for model import, visual orchestration, alarms, and event delivery." The intended audience is an engineer who already has a model and a target chip, and wants the surrounding application rather than another inference wrapper.

The repository is C++17 (the runtime badge says C++17, the topics list cpp17) and the primary language GitHub reports is C. The core engine and console are Apache-2.0. The README is explicit that certified hardware, commercial preset models and Model Guard distribution protection sit outside that boundary, so the open licence covers the software, not the model supply chain.

One engine, one backend per build: how the platform split actually works

The architecture is not a runtime abstraction that dispatches to whichever accelerator is present. Each build selects one inference backend, and models are generated for that target. The README states this plainly: "each build selects one inference backend and uses models generated for that target platform."

That decision shapes everything downstream. Sophon BM1688 and CV186X use BMRT with .nn artifacts, and the README notes chip-aware validation for target-specific .nn files. Rockchip RK3576 and RV1126B use RKNN with target-specific .rknn artifacts, with RKLLM on RK3576 for VLM. x86 Linux and Windows use ONNX Runtime with .onnx. There is no portable model format that moves between them.

The RKNN data path gets its own bullet in the release notes: targeted DMA-BUF-to-RGA input, persistent bound-input, native quantized output, and direct YOLOv8 tensor decoding paths, each with explicit fallbacks. Those are the mechanisms that keep copies off the CPU, and the fallbacks matter because DMA-BUF and RGA availability depends on the board and driver stack. The release notes do not say what triggers each fallback or what it costs in throughput.

VLM support is uneven by design. BM1688, CV186X and RK3576 are listed as VLM supported; the README says RV1126B VLM is outside the v1.1 release claim. BM1684X is listed as planned and not part of the current scope.

Installing CosmoEdge on x86 and running a first detection task

The lowest-friction entry point needs no edge hardware. The README's x86 quick start clones the repository and brings up a Compose stack. It notes that x86 mode uses the same UI and workflow with lower throughput than an NPU deployment, which is the honest framing: this is for evaluating the workflow, not for capacity planning.

git clone https://github.com/cosmo-wander-ai/cosmo-edge.git
cd cosmo-edge
sudo docker compose -f docker-compose.x86.yml up -d --build

On Windows the README points at a different file, docker-compose.x86.windows.yml, with the same command shape. Docker Compose V1 users can substitute docker-compose for docker compose. After the stack comes up, the README says to open http://localhost:8080, which is where the console is served.

Apple Silicon takes a separate route through a shell script rather than the x86 Compose file, and the README tells you to read the macOS Docker Preview scope document before relying on it.

./scripts/macos-docker-preview.sh doctor
./scripts/macos-docker-preview.sh up
./scripts/macos-docker-preview.sh status

The three subcommands are a check, a start and a status query, and the console is then at http://127.0.0.1:8080 rather than localhost. The README describes this as a local, single-video workflow under linux/amd64 emulation on Apple Silicon, validated through multiple end-to-end lab rounds, and explicitly not native performance evidence.

For Sophon hardware the build is a packaging run through the Sophon Compose file, with the chip model passed as an argument. The README gives bm1688 as the default when the chip model is omitted, and shows the same command with --chip for CV186X. The README excerpt is truncated mid-flag for the CV186X case, so read the repository's own Quick Start rather than reconstructing that argument.

./scripts/docker-compose.sh -f docker-compose.sophon.yml run --rm cosmo-sophon-package --chip bm1688

Once the console is reachable, the README sends you to the Scenario Configuration tutorial at https://www.cosmowander.ai/docs/tutorials/02-scenario-config/scenario-config to create the first AI detection task. The repository does not document that configuration flow inline, so the tutorial is the actual source for it.

What the benchmark pack does and does not tell you

CosmoEdge ships a public benchmark report under docs/benchmarks/scenario-bench/v1.1/README.md. According to the release notes it covers single workloads, concurrent mixed workloads, a controlled 72-hour dual-CV profile, and validated VLM performance, with sanitized artifacts.

The VLM numbers are the part to read carefully. The README says VLM figures are "exact short-run gate boundaries for the recorded protocol," and the platform table gives those boundaries as 0.1 FPS per channel through 6 channels on BM1688 and CV186X, and through 4 channels on RK3576. That is a low frame rate by design: a VLM looking at a scene every ten seconds is a different product from a detector running at 25 FPS, and the channel counts are the ceiling of the recorded protocol, not a scaling curve.

The 72-hour test is separate and uses controlled local-loop input over fixed-channel profiles. The README states the conditions are documented in the benchmark report. Nothing in the repository describes a real camera feed, a network video recorder, or a variable-bitrate stream, so treat the long-run figure as a stability claim about the software under a fixed input, not as a field deployment measurement.

The package.json shows the benchmark pipeline is scripted and verifiable rather than hand-assembled: benchmarks:v1.1:validate, benchmarks:v1.1:checksums, and models:sophon:verify all exist as npm scripts, and there are separate private verification scripts for the long-run and VLM runs. That is a stronger setup than a PDF of numbers, because the checksums and validation steps are reproducible from the repository. It still does not tell you what your camera and your scene will do.

Where CosmoEdge is the wrong tool

The platform table is the first filter, and it is a hard one. If your target is BM1684X, the README lists it as planned with no runtime and no artifact type. There is no partial support to work around.

The second filter is the model artifact. Because each build binds to one backend and one model format, a .onnx model running under ONNX Runtime on x86 does not carry over to an RKNN board. You need a .rknn or .nn build of that model for the target, produced by the corresponding toolchain. If your model has operators the target toolchain cannot lower, CosmoEdge does not solve that for you; the README's model-import language describes the application-layer path, not a converter.

The third filter is the emulation paths. The macOS Preview is scoped to a single local video under linux/amd64 emulation. It is a developer convenience, and the README says so directly. If your only development machine is an Apple Silicon laptop and you need to measure anything, the Preview is not the instrument.

Finally, the licence boundary is not the whole product. Apache-2.0 covers the core engine and console in the repository. Commercial preset models, certified hardware and Model Guard distribution protection are separate. Model Guard 2.3 protects commercial preset-model distribution in Sophon Protected packages; the README states that Open and Protected expose the same application features with no SKU-gated software functionality, and differ in model encryption and device-provisioning tooling. So the open build is not feature-crippled by design, but the models you may want are not necessarily in the open package.

CosmoEdge against a general inference runtime

The natural comparison is a general-purpose inference runtime such as ONNX Runtime or a vendor SDK used directly. The difference is where the boundary sits.

A runtime gives you a session: load a graph, feed tensors, read tensors. Everything above that, frame acquisition, colour conversion, scaling, drawing, alarm rules, event transport, is your code, and you write it once per platform. CosmoEdge puts that layer in the repository and keeps it constant across targets, which is why the README can claim the same video-ingest, orchestration, event and observability workflow on BM1688, CV186X, RK3576 and RV1126B.

The trade is control. A runtime lets you shape the graph and the memory layout yourself. CosmoEdge selects one backend per build and ships its own media and decode paths, so the RKNN data path is fixed to the DMA-BUF-to-RGA, persistent bound-input and native quantized output design described in the release notes, with fallbacks you do not choose. If your pipeline needs a preprocessing step that design does not accommodate, you are working against the engine rather than with it.

ONNX Runtime also runs on x86 without an NPU, which CosmoEdge does too, but the x86 build is positioned as a workflow preview with lower throughput rather than a deployment target. The README's own framing of x86 is "Linux / Windows supported; macOS Preview," so x86 is supported, just not the reason the project exists.

Maintenance, releases and what the licence leaves open

The release cadence visible in the repository is two stable tags: v1.0.0 on 2026-07-03 and v1.1.0 on 2026-08-24, plus a test-video tag in June. The last push to the repository was on 2026-09-15. The repository is not archived.

Upgrade cost is dominated by the model artifacts, not the engine. Moving from v1.0.0 to v1.1.0 adds Rockchip release platforms and the macOS Docker Preview, but because each build binds to one backend and one model format, an engine upgrade and a model re-generation are usually the same maintenance event. The release notes for 1.1 also mention chip-aware validation for target-specific .nn artifacts, which implies the model and the chip model are validated together rather than independently. Budget for re-running the toolchain per target, not just for pulling a new tag.

On licensing: the LICENSE file is Apache-2.0, and the README states the core engine and console in this repository are released under it. The same paragraph carves out certified hardware, commercial preset models and Model Guard distribution protection as separate boundaries. Apache-2.0 is a permissive licence with a patent grant and notice requirements, but it says nothing about the model weights you load into the engine, and this repository does not document the terms of the commercial preset models. That is a question for the vendor, not something the repository answers. This is a description of what the files say, not legal advice; read LICENSE and NOTICE before you ship.

Editorial conclusion

Adopt CosmoEdge if you are shipping video analytics or on-device VLM on BM1688, CV186X, RK3576 or RV1126B and want the orchestration, OSD and event layer already written in C++17 instead of stitched together per board. Skip it if your target is BM1684X, which the README lists as planned and outside the current release scope, or if Apple Silicon is your only machine, because the macOS path is an amd64 emulation Preview for one local-video workflow. Before you commit, verify three things: that your chip model appears in the platform table for v1.1, that your model exists as a .nn or .rknn artifact for that target, and that the FPS your scenario needs sits inside the benchmark report's stated channel counts and gate boundaries.

Frequently asked questions

Official sources

1. cosmo-wander-ai/cosmo-edge on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases