wkvm treats a model state like a virtual machine

A state slot is the allocation object, not a KV block

The framing is a virtual machine hypervisor applied to recurrent model state. The primary allocation object is a fixed-size per-request state slot, not a paged key-value block chain, and the target family is RWKV-7, GDN, Mamba2, and hybrid-linear models. The analogy is stated plainly in the project's own shorthand: states are the VMs, so they can be created, snapshotted, forked, hibernated, resumed, and live-migrated, with the engine playing the role of the hypervisor.

That choice only makes sense for models whose memory is a fixed-size recurrent state rather than a growing cache, which is exactly the family wkvm targets. A paged block chain is the right primitive when attention memory expands with context, and the wrong one when the state is the same size regardless of how much you have read.

The management vocabulary follows directly. Snapshot, fork, and resume are operations you perform on a state handle, so a session can be duplicated or parked and restored rather than recomputed from a prompt.

parent-token-v1 binds state to an exact token history

The chat path rests on one contract called `parent-token-v1`, and it is the most carefully specified part of the repository. Each parked state is bound to the model, the user, the chat, the current assistant message, the previous assistant parent, the exact visible history, and an internal raw-token digest.

The reason is stated without hedging: wkvm preserves the exact provider-generated token history, including hidden or noncanonical tokens, instead of pretending that a decode to text and back to tokens round trip is identical. That is a claim about correctness rather than speed, and it is what makes state reuse safe when the same provider and the same exact token sequence are involved.

When the binding cannot be honoured, the failure mode is a fresh start rather than a wrong answer. Edits, branches, stale parents, or missing metadata restart safely. wkvm also mirrors the persisted outer-whitespace normalisation of Open WebUI 0.10.2, and the helper disables reasoning-tag extraction, because either would change the visible history and break the check that guards the digest.

The chat surface is deliberately narrow

What is exposed is small. Blocking and streaming requests to `/v1/chat/completions`, plus `/v1/models`, `/health`, and `/metrics`. Requests must be text messages with `temperature=0`, `top_p=1`, and `n=1`. Tools, images, logprobs, and custom stop sequences are not supported.

That is not an oversight; it is the scope. The Open WebUI path is described as a compatibility demo rather than a same-semantics replacement for full-KV Gemma serving, because routed-span keeps a bounded approximate memory. The native durable-state engine and API for RWKV-7 are a separate path from the one a browser chat exercises.

Greedy defaults and a single sample are the right choices for a demo whose point is state continuity, since sampling makes exact history reuse harder to reason about. The cost is that this endpoint cannot answer questions about the model that depend on logprobs, tools, or multimodal input.

Loopback only, and no API key

Both services bind to loopback, and the README states plainly that wkvm currently has no API-key enforcement, so port 8000 should not be exposed to a LAN or the internet. That is an unusual thing for a README to volunteer, and it is the single most important operational fact here.

The binding also shapes deployment. A pinned Docker Compose alternative exists, and it uses Linux host networking precisely because the wkvm CLI binds to `127.0.0.1` and therefore cannot be reached through ordinary container port publishing.

The helper starts the engine with `--enable-openai-chat`, four conservative state slots, normal EOS handling, and the checkpoint-native production profile. It pins greedy UI defaults and disables unsupported background and tool features, and it deliberately does not reuse the high-memory benchmark recipe or the `--ignore-eos` flag. Those two omissions are the difference between a demo you can run on one GPU and a benchmark configuration you cannot.

The demo helper is a script with a doctor step

Getting to a browser chat is a helper script with five verbs: install, doctor, start, smoke, and stop.

# Install isolated WKVM and Open WebUI 0.10.2 environments.
./scripts/open_webui_demo.sh install

# Accept google/gemma-4-E4B-it on Hugging Face, then authenticate and download.
hf auth login
hf download google/gemma-4-E4B-it \
  --local-dir "$HOME/models/gemma-4-E4B-it"

# Check the machine, start both loopback services, and verify a real chat.
WKVM_MODEL_DIR="$HOME/models/gemma-4-E4B-it" \
  ./scripts/open_webui_demo.sh doctor
WKVM_MODEL_DIR="$HOME/models/gemma-4-E4B-it" \
  ./scripts/open_webui_demo.sh start
./scripts/open_webui_demo.sh smoke

The model directory is passed in through an environment variable rather than a flag, which keeps the script's argument parsing out of the way, and the doctor step runs before anything starts so a wrong machine fails early. After startup you open the local address, create the first local account, and select the model, then stop both services with the stop verb.

The tested target is stated as Linux, an NVIDIA GPU with 24 GB of VRAM, Python 3.12, and roughly 40 GB of free disk for the checkpoint, environments, and caches. A separate installation and troubleshooting guide is the document to read before adapting this to other hardware.

A 12,000 token needle test with a pinned source

The long-context recall lane uses natural prose rather than repeated filler, which matters because filler can be predicted without reading anything. The passage is a contiguous excerpt of Lewis Carroll's Alice's Adventures in Wonderland, taken from the Hugging Face dataset `common-pile/project_gutenberg`, document 11, whose row metadata declares it public domain.

The prompt contained exactly 12,000 rendered tokens, with the needle beginning at zero-based rendered token index 253. The run returned `BLUE-742`, `Samarkand`, and `lantern`, at a browser-observed time to first token of 2.162 seconds and end to end of 3.705 seconds.

What makes this reproducible is the pinning. The source is fixed to a dataset revision, a converted-Parquet revision, and a document-text SHA-256 hash, and a helper script fetches and verifies it using `hf datasets sql`, which requires the Hugging Face CLI's DuckDB support. A recall result is only worth reading if the input can be reconstructed exactly, and most published needle tests cannot make that promise.

The speed claims are scoped and the caveats stay in public

The performance section opens by refusing the headline it could have claimed: there is no honest workload-independent claim that the engine is ten times faster. The evidence supports scoped, workload-specific speedups, and it also contains workloads where vLLM remains faster. A later exact-trace vLLM optimization audit superseded an earlier ten-times result from the same hardware.

The current checkpoint is quoted with its conditions. A strict 2026-07-23 run of 32 sessions against 8 concurrent streams passed all 256 requests and reused all 224 eligible continuations, with 32 sessions opened, none closed, and no parent-history rejections. The final repetition completed in 94.953 seconds at 345.097 output tokens per second, which is 2.153 times the tested vLLM mode-0 profile and 5.305 times the tested SGLang profile.

Even that is labelled as one controlled cross-run comparison on an active RTX 4090 desktop, with repeated rotated runs still required before the ratios count as a publication-grade envelope. An A800 scout run reports 12.107 times against vLLM, but on a dirty worktree, without frozen incumbent optimisation profiles, and with no matching SGLang run, so it is explicitly not a publication-grade claim yet. Publishing your own weaker number beside your own caveats is rarer than it should be.

Version 0.0.1 with a GPU-free core

The manifest says version 0.0.1, requires Python 3.11 or newer, declares an Apache 2.0 license text, and lists one console script, a Gemma server entry point. The optional dependency groups are split by purpose: a GPU group with numpy and torch, a Gemma server group adding transformers, safetensors, and accelerate, and an open WebUI benchmark group adding a Socket.IO client and a websocket client.

The comment on the first group explains the design: the M0 core is dependency-free on purpose, because the scheduler and arena are pure bookkeeping and must stay unit-testable without a GPU or torch. Scheduling logic that cannot be tested without hardware is scheduling logic that gets tested late.

The rest of the tree matches a research project with a demo attached. There is a deployment directory, a docs directory, an experiments directory holding results, a paper directory, a scripts directory, tests, a roadmap, and the package itself, plus a `.keel/` directory. There are no GitHub releases, and the last push to the main branch is dated 2026-09-10.

Editorial conclusion

wkvm suits an engineer who wants state-level primitives, snapshotting, forking, and resumption for linear attention models, and who will read the benchmark caveats before repeating any speed figure. It is not a drop-in server for general chat: the surface is text-only and greedy, and the Gemma path is approximate rather than semantically equivalent. Before exposing anything, note that the engine has no API key enforcement and binds to loopback. Before trusting a benchmark number, check which profile was tested and whether the comparison was frozen.

Frequently asked questions

Official sources

1. Issues
2. README
3. xiaol/wkvm on GitHub