Beta9: a self-hostable runtime for GPU endpoints, sandboxes and task queues
Ultrafast serverless GPU inference, sandboxes, and background jobs
At a glance
- What is it?
- Beta9 packages a Python decorator API on top of a Go scheduler and container runtime. The pitch is fast cold starts and scale-to-zero for AI workloads, but the AGPL licence and the split between the open engine and the managed cloud are the parts to weigh before adopting it.
- Who is it for?
- Adopt Beta9 if you already run Kubernetes or bare-metal GPU nodes and want endpoint, sandbox and task-queue primitives driven from Python decorators rather than YAML. Do not adopt it if you need a permissive licence for a closed product, or if you have no GPU capacity to point it at, since the README frames GPU supply as either Beam's cloud or your own hardware.
- Can I use it commercially?
- Yes, with strict conditions. AGPL-3.0 is a network copyleft licence: if people use a modified version over a network, for example as a hosted service, you must offer them its source code under the same licence.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Go, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap Beta9 targets: GPU work that arrives in bursts
Most teams running model inference hit the same wall. A GPU node costs money whether or not a request is in flight, and the workloads that need it are rarely steady. A fine-tuning job runs for two hours, then nothing. A sandbox that executes LLM-generated code is needed for a few seconds per user action. A queue of image jobs drains overnight. Provisioning a cluster for that pattern means paying for idle capacity, and writing the autoscaling and queueing layer yourself means owning a scheduler.
Beta9 is aimed at that gap. The README describes it as "a fast, open-source runtime for serverless AI workloads" that gives "a Pythonic interface to deploy and scale AI applications with zero infrastructure overhead." The target user is a Python developer who wants to describe a workload with a decorator and let something else decide how many containers to start. The topics list on the repository confirms the framing: faas, serverless-containers, autoscaler, gpu, llm-inference. This is not a training framework and not a model server. It is the layer underneath, the thing that decides where your container runs and when it stops.
Three primitives, three different lifetimes
The README exposes three distinct entry points, and the differences between them matter more than the shared decorator syntax.
The first is the endpoint. An @endpoint decorator takes an Image, a gpu string such as "A10G", cpu and memory values, and an autoscaler object. In the README example that autoscaler is QueueDepthAutoscaler(max_containers=5, tasks_per_container=30), which tells you the scaling signal is queue depth rather than CPU or request latency. Each container is expected to handle 30 tasks before another is added, up to a ceiling of five containers. That is a batching-oriented model, and it implies the runtime is holding requests somewhere while it waits for capacity.
The second is the sandbox. Sandbox(image=Image()).create() returns an object whose process.run_code method executes a string remotely. The README's stated use case is running LLM-generated code inside an isolated container. This is a different lifecycle from an endpoint: it is created on demand and presumably discarded, rather than kept warm for traffic.
The third is the task queue. A @task_queue decorator takes a name, an image, resource values, an inputs schema, and a TaskPolicy(max_retries=3). Inputs are declared with the schema module, and the handler receives a typed input plus a keyword-only context. The README shows the function being invoked directly with my_background_task.put(image_url=...), which the comment notes works "without deploying it." That local-invocation path is the interesting detail: the same function object is both a callable and a queue producer.
The Go runtime behind the Python decorators
The repository's primary language is Go, while every code sample in the README is Python. That split tells you where the work happens. The pip package is a client; the scheduler, container runtime and caching layer are the Go services, and the README claims a "custom container runtime, scheduler, and embedded caching" as the mechanism behind sub-second container launches. It does not describe how the cache is keyed or what is cached (image layers, filesystem state, or both), so treat the cold-start figure as a design goal rather than something you can verify from the README alone.
The scaling model is fan-out. The feature list says workloads can be spread across "100s of containers," and the autoscaler parameters in the endpoint example are the only concrete control surface shown. Storage is handled through what the README calls "distributed storage volumes" that can be mounted, though no mount syntax appears in the supplied material, so how a volume is declared and attached is something you would need to check in the docs.
GPU supply is deliberately pluggable. The README says you can run on Beam's cloud with 4090s and H100s "or bring your own GPUs." That single sentence is doing a lot of architectural work: it means the runtime is meant to be pointed at hardware you already control, which is consistent with the self-hosting section further down.
Getting it running: one pip install and a decorator
Installation is a single command:
pip install beam-client
The README's quickstart then sends you to create an account at beam.cloud and follow an onboarding guide at platform.beam.cloud. That path is the managed cloud. The self-hosting option is mentioned separately, in a block that says Beta9 is "the open-source engine powering Beam" and that you can "self-host Beta9 for free or choose managed cloud hosting." The README does not include self-hosting instructions, a docker-compose file, a Helm chart, or environment variables. If you intend to run it yourself, the README is not the document you need.
The workload-side config lives in the decorators. From the supplied examples the keys are: image (with python_version), gpu, cpu, memory, autoscaler, name, inputs, and task_policy. Deployment of a task queue function uses the CLI form shown in a comment: beam deploy app.py:my_background_task --name image-processor. Note the file:function syntax, which mirrors how the local invocation works. The task queue example also gates the .put() call behind if __name__ == "__main__", so the producer runs as a normal script.
Where this design will frustrate you
The autoscaler is the clearest constraint. QueueDepthAutoscaler in the README example caps at max_containers=5. That is a small ceiling for a production inference service, and it is presented without any discussion of how the runtime behaves when the queue keeps growing past the cap. Whether excess tasks are rejected, queued indefinitely, or shed is not stated. If you are sizing for a traffic spike, that is the first question to answer.
The cold-start claim is similarly unqualified. "Launch containers in under a second" appears as a feature bullet with no stated conditions: no image size, no cache-warm versus cache-cold distinction, no GPU model. Container start time on a GPU node depends heavily on whether the driver and CUDA libraries are already resident. The README does not say.
There is also a real mismatch between the open engine and the documented experience. Every code sample works locally against the client library, but the quickstart routes through a hosted account. Someone evaluating Beta9 for an air-gapped or on-premise deployment gets the decorator API from the README and nothing about the control plane they would have to operate. The task queue's local .put() invocation softens this, since you can exercise the queue semantics before deploying, but it does not tell you what the self-hosted scheduler looks like in production.
How it compares to Ray Serve and KServe
The closest comparison is Ray Serve. Ray also gives you a Python-first API for scaling model workloads across a cluster, and it also handles fan-out to many workers. The difference in approach is the unit of deployment. Ray Serve composes deployments inside a long-running Ray cluster that you manage as a persistent resource; the cluster is the thing that exists, and deployments live inside it. Beta9 inverts that. The decorator describes a workload, and containers appear and disappear around it, with scale-to-zero as the default per the README's feature list. If you already run Ray for training, adding Beta9 means running a second control plane. If you want nothing long-lived, Beta9's model matches the problem better.
KServe is the other reference point, and it sits on Kubernetes with InferenceService custom resources. It assumes you are already a Kubernetes shop and expresses configuration as YAML. Beta9's README pitches the opposite: no YAML, a decorator, and a runtime that manages containers itself. The trade-off is that KServe inherits the Kubernetes ecosystem for networking, secrets and observability, while Beta9's self-hosted story is not documented in the README at all. A team with an existing Kubernetes platform team may find KServe's familiarity cheaper than Beta9's ergonomics.
Licence and release cadence
Beta9 is licensed AGPL-3.0. That is a copyleft licence with a network clause: if you modify the software and let users interact with it over a network, the AGPL's terms reach that interaction. For an internal inference service that you do not distribute, this is usually workable. For a product where you would want to modify the runtime and keep your changes proprietary, it is a constraint worth reading carefully with your own counsel. The README links the licence badge directly to the AGPL-3.0 file in the repository, and the self-hosting note makes no mention of a separate commercial licence. Whether the client library and the Go services carry the same terms is not stated in the supplied material.
On maintenance, the release history shows a worker component versioned independently from the rest of the project, with worker-0.1.752 published on 2026-09-08 and two more worker releases in the same week. That cadence suggests active development on the worker specifically, and it also means the version number you pin is not a single project version. If you self-host, upgrading means tracking a component that ships several times a week. The repository is not archived, and the last push recorded is 2026-09-10.
Editorial conclusion
Adopt Beta9 if you already run Kubernetes or bare-metal GPU nodes and want endpoint, sandbox and task-queue primitives driven from Python decorators rather than YAML. Do not adopt it if you need a permissive licence for a closed product, or if you have no GPU capacity to point it at, since the README frames GPU supply as either Beam's cloud or your own hardware. Before committing, verify three things against the repository: which components the AGPL-3.0 licence actually covers, whether the self-hosted path exposes the same autoscaler and volume features as the managed cloud, and whether worker-0.1.x release cadence fits your upgrade window.
Community notes