Judge0: self-hosted sandboxed code execution for untrusted programs

The problem Judge0 solves: running code you did not write

Most applications that accept code from a user eventually have to run it. The README lists the audience directly: AI agents, competitive programming platforms, e-learning platforms, candidate assessment and recruitment platforms, online code editors and IDEs. What those have in common is that the program being executed is untrusted. A submission can loop forever, allocate until the machine swaps, fork until the process table fills, or read files it should not see.

Judge0's answer is to make sandboxed compilation and execution the product rather than a detail you bolt on. The README claims support for 90+ languages, multi-file programs, additional files alongside the single-file program, custom compiler options, command-line arguments, and user-defined time and memory limits. Those are the knobs an assessment platform needs when a candidate submits a project rather than a snippet.

The project has been around since August 2016 and is the subject of a paper, Robust and Scalable Online Code Execution System, presented at MIPRO 2020. That matters less as a quality signal than as a hint about intent: this is infrastructure that was designed to be deployed and scaled, not a weekend script.

How a submission flows through the server, worker, Postgres and Redis

The repository ships a docker-compose.yml with four services. server is the API, worker runs the queue consumer via ./scripts/workers, db is postgres:16.2, and redis is redis:7.2.4 started with --appendonly no and a password taken from $$REDIS_PASSWORD. Both server and worker mount ./judge0.conf read-only and both are marked privileged: true.

That last flag is the architecture in one line. The sandbox needs capabilities that an ordinary container does not get, so the compose file grants them. It is also the reason Judge0 is a poor fit for a shared Kubernetes cluster where you do not control the node.

The API itself is a plain HTTP JSON surface. A submission carries a language_id, source_code and optional stdin. The README's example posts to https://ce.judge0.com/submissions?wait=true, and the wait=true query parameter is what turns an asynchronous queue submission into a blocking call that returns the result. Redis carries the queue between server and worker; Postgres holds submissions and their results. The README also lists webhooks, so a caller that does not want to poll can receive an HTTP callback instead.

One detail worth noticing in the Dockerfile: the image is built FROM judge0/compilers:1.4.0. The language toolchains live in a separate base image, which is why the Judge0 image itself is small and why the compiler set is versioned independently of the API.

Installing Judge0 with Docker and running a first submission

The README points self-hosters at the deployment procedure in CHANGELOG.md rather than reproducing the steps, so treat that file as the authoritative install guide and not this section. What the repository does show is the shape of the deployment: a docker-compose.yml that expects a judge0.conf next to it, mounted read-only into both the server and the worker.

The compose file defines the port mapping for the API:

services:
  server:
    image: judge0/judge0:latest
    volumes:
      - ./judge0.conf:/judge0.conf:ro
    ports:
      - "2358:2358"
    privileged: true

So a local instance answers on port 2358. The Dockerfile confirms this with EXPOSE 2358 and ENTRYPOINT ["/api/docker-entrypoint.sh"], CMD ["/api/scripts/server"].

Once something is listening, the README's own example is the fastest way to see whether the pipeline works end to end. It submits a Python program with language_id 109 and passes Alice on stdin:

curl \
  -H "Content-Type: application/json" \
  -d '{
      "language_id": 109,
      "source_code": "print(f\"hello, {input()}\")",
      "stdin": "Alice"
  }' \
  "https://ce.judge0.com/submissions?wait=true"

Against your own instance you would replace the host with your server and keep the same body. The response carries the execution result, including stdout. The README does not document the exact JSON field names of the response in the section reproduced here, so check the API documentation at ce.judge0.com before writing a parser.

There is also an official Python SDK. The README gives this example:

# pip install judge0
import judge0
result = judge0.run(source_code="print(f'hello, {input()}')", stdin="Alice", language=judge0.PYTHON)
print(result.stdout)

The SDK wraps the same HTTP call, so it is a convenience rather than a second protocol. If you are integrating from Python, start there; if you are not, the curl shape above is the whole contract.

Where Judge0 is the wrong tool

The privileged containers are the first real limitation. Any deployment that gives a process the ability to run arbitrary untrusted code with elevated container privileges deserves its own host. If your platform team will not approve privileged workloads, Judge0's default compose file will not run as written, and the README does not offer a non-privileged configuration.

Resource limits are per submission, not global. The README advertises user-defined time and memory limits, which is exactly what you want for fairness between candidates, but it also means a caller who can set those limits can set them generously. Rate limiting and authentication are not described in the README at all. The public ce.judge0.com endpoint is a shared service; a self-hosted instance behind no auth is an open compute resource.

The release history is the second thing to weigh. v1.13.1 and v1.13.1-extra were both tagged on 2024-04-18. There is no newer tagged release in the list. The repository's last push was on 2026-08-17, so work continues on master, but anyone who deploys a pinned release is deploying code that was tagged more than two years before that push. If your policy is to run only tagged releases, plan for that gap.

Finally, Judge0 executes code. It does not grade it. There is no notion of test cases, scoring, or a problem statement in the README. If you want an online judge in the competitive programming sense, you are building the judging layer yourself on top of the execution layer.

Judge0 compared with Piston

Piston is the comparison people search for, and the difference is mostly about scope. Piston is a code execution engine: you send it a language and some code, it runs the code in an isolated environment and returns the output. It is a smaller surface with fewer moving parts.

Judge0 puts more between the caller and the sandbox. There is a Postgres database for submissions, Redis for the queue, a separate worker process, webhooks for callbacks, and an API that models a submission as a durable object with an id you can revisit. That is the difference in approach: Piston is a function call, Judge0 is a job system. If you need to accept a submission, return immediately, and notify the client later, Judge0's webhooks and queue exist for that. If you just want stdout, the queue and the database are overhead you have to operate.

The language story differs too. The README describes two flavors, Judge0 CE on the master branch and Judge0 Extra CE on the extra branch, which "differ mostly in the supported languages." Piston's language set is its own; the README does not compare the two, so verify coverage for the specific languages you need rather than trusting a count. The README's 90+ figure applies to Judge0, and the full list lives at ide.judge0.com.

Licence, deployment cost and the upgrade path

Judge0 is GPL-3.0. Your application talks to it over HTTP or through the Python SDK, which is the ordinary way to use a network service, but the licence question is one your own counsel should answer rather than one this article can settle. The repository also carries SECURITY.md, CODE_OF_CONDUCT.md and TELEMETRY.md at the top level, so there are documented policies for security reports and telemetry; read TELEMETRY.md before deploying if data leaving your host is a concern.

Operationally, the compose file is the cost model. Four services, two of them privileged, one Postgres volume, one Redis with persistence deliberately disabled (--appendonly no). Logging is capped at 100M per container through the json-file driver, and every service has restart: always. That is a small footprint to run and a nontrivial one to secure.

Upgrades are the sharp edge. The compose file references judge0/judge0:latest, which means a docker compose pull can move you to an untagged build with no version boundary. The Dockerfile sets ENV JUDGE0_VERSION "1.13.1", but the image tag in compose is not pinned to it. The CHANGELOG.md holds the deployment procedure, so it is also where migration notes for a version bump would live. Pin the image to a digest if you want upgrades to be a decision rather than an accident.

Editorial conclusion

Adopt Judge0 if you need to run code you do not trust and you can give the worker containers privileged access on a host you control; the docker-compose.yml sets privileged: true on both server and worker, so a shared or locked-down cluster is the wrong place for it. Do not adopt it if you only need a browser scratchpad, or if a managed API with an SLA is what you actually want to buy. Before committing, read the deployment procedure in CHANGELOG.md, pin judge0/judge0:latest to a digest, and check the licence question that matters to you: GPL-3.0 governs the Judge0 code itself, and your application talks to it over HTTP.

Frequently asked questions

Official sources

1. judge0/judge0 on GitHub
2. License: GPL-3.0
3. Project website
4. README
5. Releases