ARahim3/kaggle-tpu-lab: serving open models on Kaggle's free TPU v5e-8

What kaggle-tpu-lab actually gives you

The project targets a narrow problem: you want a capable open model behind an API for a coding agent, and you do not want to buy a GPU or rent one. Kaggle hands out TPU v5e-8 sessions on a free quota, which the README puts at around 20 TPU hours a week. The repository turns that session into a server. Two models are wired up. Qwen3.8-27B runs in bf16 with no quantization at a 262k context, and the README reports about 130 tokens per second on one stream and about 540 across eight. GLM-5.3-Flash is a 320B mixture-of-experts model stored at 3-bit for the experts and int8 for the rest, reported at roughly 64 tokens per second on one stream, with prefill around 1,600 tokens per second.

The audience is the terminal-first developer. The README explicitly names Claude Code, Codex CLI, opencode and pi as clients, and the whole point of the endpoint is that it speaks the OpenAI and Anthropic APIs, so the agent does not need to know it is talking to a TPU. The repository is Python, MIT licensed, and the last push was on 2026-09-15.

How a session builds an engine across eight chips

The architecture is a notebook plus a kernel script per model. The notebook attaches public Kaggle datasets that hold the weights, and where it helps a pre-built compile cache, then builds the engine across the eight chips, opens a Cloudflare tunnel, and prints a URL and an API key. The final cell is the server itself; leaving it running is what keeps the endpoint alive. A keepalive holds the session up to Kaggle's limit, which the README describes as a little under nine hours.

The two recipes take different routes. Qwen3.8-27B runs on vllm-tpu with one patch, so the serving path is the vLLM one you may already know. GLM-5.3-Flash runs on an engine the authors wrote in JAX, and the README states that as far as they know it is the first time that model has run on a TPU. That is the more interesting half of the repository: a 320B MoE at 3-bit expert weights is not something vLLM would have handled out of the box here, so the JAX engine is doing the quantization-aware loading and the MoE routing itself. The repository also lists prefix caching and speculative decoding among its topics, which fits the prefill and decode numbers quoted.

The config block is shared between the launcher and the notebook, so a setting changed in one place means the same thing in the other. That is a small design decision with real consequences: you cannot silently diverge the terminal route from the notebook route.

Installing it and getting a first endpoint

There are two routes. The notebook route needs nothing installed beyond a Kaggle account, and the README says to phone-verify it under Settings to get TPU access. The terminal route needs Python 3.9+ and the Kaggle CLI. Start by cloning and launching:

git clone https://github.com/ARahim3/kaggle-tpu-lab
cd kaggle-tpu-lab
python launch.py serve

According to the README, `launch.py serve` pushes the kernel with the Kaggle CLI and follows its progress. Qwen3.8-27B is the default recipe today, and `--model` picks another once one exists. The same file provides `status` and `stop`. Expect the run to take on the order of twenty minutes before a URL appears; the README quotes about 22 minutes for Qwen and about 16 for GLM-5.3-Flash.

Once the notebook prints the URL and key, wire a coding agent with one line. The README gives this example for Claude Code:

ANTHROPIC_BASE_URL=<url> ANTHROPIC_AUTH_TOKEN=<key> ANTHROPIC_MODEL=<model> claude

The folder READMEs carry the exact lines for Claude Code, Codex CLI and opencode, so use those rather than guessing at environment variable names for other clients. What you should see is the agent answering from the hosted model, with the request going out through the Cloudflare tunnel to the TPU session.

The nine-hour session is the design constraint

The honest limitation is not speed, it is lifetime. A session ends at Kaggle's limit, a little under nine hours, and the README says plainly that after that you run it again and get a new URL. Every client you wired up now points at a dead host. Nothing in the README documents rollback, a stable hostname, or a way to migrate a running session, so any workflow that assumes a fixed base URL will break on a schedule you do not control.

The quota compounds it. Around 20 TPU hours a week, against sessions of up to nine hours, means a handful of runs. This is a lab and a demo harness, not a deployment target. If you need a model endpoint that survives a restart, or that several people share, this repository is the wrong tool, and its own framing (the name, the per-model notebooks, the write-ups of what was measured) suggests the authors know that.

There is a second, quieter constraint: only two models are wired up. Adding one means a new top-level folder named after the model, with a README, a `kernel/` directory holding the serving script, a `notebook/` directory holding the run-all notebook generated from it, and whatever the recipe needs, which for GLM-5.3-Flash meant an entire JAX engine. That is not a config change.

How it differs from running vLLM yourself

The obvious alternative is standing up vLLM on a rented GPU and pointing the same agents at it. The difference is where the work sits. With vLLM on a GPU you own the machine, the process supervision, and the cost, and in exchange you get a stable endpoint and a wide model menu. kaggle-tpu-lab inverts that: someone else owns the machine and the quota, the endpoint is ephemeral, and the model menu is two entries.

The interesting comparison is inside the repository. Qwen3.8-27B runs on vllm-tpu plus one patch, so it is the familiar path ported to TPU. GLM-5.3-Flash does not, because a 320B MoE at 3-bit expert weights needed a purpose-built JAX engine. If you are choosing between the two recipes, that is the trade: the vLLM-backed one is closer to a known serving stack, and the JAX one is the only way the repository gets a 320B model onto eight chips at all.

Licence and the cost of keeping up

The code is MIT. The model weights are not covered by that licence, and the README says each folder states which licence applies to its weights, so check the folder before you assume anything about redistribution or commercial use. Nothing here is legal advice; the point is simply that the MIT header on the repository does not travel with the weights.

Upgrade cost is mostly the cost of the upstream moving. Qwen's recipe depends on vllm-tpu and one patch, and a patch against a moving target is the kind of thing that rots quietly. The GLM recipe depends on the authors' own JAX engine, so upgrades are theirs to make. The README does not describe a versioning or release process, and no releases were retrieved. The last push was on 2026-09-15, two days before this was written, so the repository is current, but that says nothing about whether the patch still applies to the next vllm-tpu.

Who should pick this up

Take it if you are experimenting with coding agents and want a large-context open model behind an API without a GPU bill, and you are comfortable re-running a notebook when the session dies. The 262k context on both models is the part that is genuinely hard to get for free elsewhere, and the one-line agent wiring is the part that makes it usable in an afternoon.

Leave it if you need uptime, a fixed URL, multi-user access, or a model outside the two recipes. Leave it too if you cannot get TPU access on your Kaggle account, since everything here depends on that. The first thing to verify is your quota and TPU eligibility; the second is the folder README for your model, because that is where the exact launch lines and the measured numbers live.

Editorial conclusion

Adopt it if you want a coding agent pointed at a 262k-context open model without a GPU or a cloud bill, and you can live with a session that ends a little under nine hours in. Do not adopt it for anything that needs a stable URL, an uptime guarantee, or a model not in the two recipe folders. Before you commit, verify that your Kaggle account has TPU access and quota, and read the folder README for the exact launch lines for your agent.

Frequently asked questions

Official sources

1. ARahim3/kaggle-tpu-lab on GitHub
2. Issues
3. License: MIT
4. README