# Backend.AI: a container cluster for GPU and NPU workloads > Backend.AI is an open source, container-based computing cluster platform from Lablup that allocates isolated compute sessions on demand, with pluggable accelerator backends for CUDA, ROCm, Gaudi, TPU and IPU hardware. Setting it up is not a pip install: it needs PostgreSQL, Valkey, etcd and a container engine, and the main branch pins CPython 3.13.7. **lablup/backend.ai** — Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs. - Repository: https://github.com/lablup/backend.ai - Website: https://www.backend.ai - Stars: 677 · Forks: 186 - Language: Python - License: LGPL-3.0 - Published: 2026-10-21 · Updated: 2026-10-21 · Language: en - Canonical page: https://hysenlabs.com/projects/lablup-backend-ai ## Sessions, not containers, are the unit of allocation Backend.AI describes itself as a streamlined, container-based computing cluster platform that hosts popular computing and ML frameworks and diverse programming languages. What it allocates is a compute session on demand or in batches, isolated per tenant, with customisable job schedulers driven by its own orchestrator, named Sokovan. The framing matters for the reader: this is not a notebook server with a queue in front of it, it is a multi-tenant cluster with a control plane, per-node controllers and a storage abstraction. The audience is a research group or platform team that wants to hand out GPU or NPU capacity to many users and keep them apart, and the repository is explicitly the open source server side of that product, with the Python client SDK included as a reference implementation of API clients. Everything is exposed through REST and GraphQL APIs, so the platform is usable from a script, a notebook or a service. ## Accelerator support is the pluggable part, not the GPU The list of hardware backends is the most distinguishing feature. The README names CUDA GPU, ROCm GPU, Rebellions, FuriosaAI, HyperAccel, Intel Gaudi, Tenstorrent, Google TPU, Graphcore IPU and other NPUs. That is a much broader claim than a single-vendor CUDA cluster, and it is the reason the platform is worth looking at for teams whose hardware is not an NVIDIA card. The agent side of the code shows how that is structured: there is a manager directory for the cluster control plane, an agent directory for the per-node controller, and under the agent a docker backend, a kubernetes backend and a dummy backend. The architecture is therefore not tied to a single runtime, and the kernel and runner directories are where the in-container side lives. The dependency list also carries vendor specific packages, including habana-pyhlml for Intel Gaudi hardware and tt-smi for Tenstorrent, so accelerator support is visible in the requirements rather than hidden behind an interface. ## The infrastructure floor is higher than a typical service This is where an evaluation can go wrong. Backend.AI does not start from a laptop with a Python virtualenv. The required stack is Docker Compose v2, PostgreSQL 16 or newer, Valkey 9.1 or newer, etcd 3.5 or newer, and Prometheus 3.x, with Grafana, Loki, Tempo and the OpenTelemetry Collector listed as recommended for observability. The container engine is either Docker 20.10 or newer or Podman 5.4.0 or newer, and the accelerator setup differs between them: Docker needs the NVIDIA runtime registered with the daemon, while Podman needs a CDI spec for the accelerator. The README also names specific tested versions, PostgreSQL tested with 16.3, Valkey with 9.1.0, etcd with 3.5.14 and Prometheus with 3.1.0, which tells you these are floors and reference points rather than loose suggestions. System requirements are Linux on a Debian or RHEL base, or macOS, with sudo access for installation, and 4 CPU cores plus 8GB of RAM recommended for development. ## Container engine versions decide your distribution The distribution table is unusual enough to be worth reading line by line, because the Podman floor is the constraint that bites. RHEL 9.x ships 5.4.0 or newer with the caveat that older minors under EUS may ship an older Podman, so you have to check what the OS repository actually offers. Rocky Linux, AlmaLinux and Oracle Linux at 9.6 package 5.4.0. CentOS Stream 9 packages 5.8.5. Ubuntu 26.04 LTS packages 5.7.0 and Debian 13 packages 5.4.2. The two cases the README calls out explicitly are the ones below the floor: Ubuntu 24.04 LTS packages Podman 4.9.3 and Debian 12 packages 4.3.1, both of which are below the floor, and the instruction is to use Docker there. That is a small paragraph with a large practical effect, because Ubuntu 24.04 is a common server base, and the consequence is that your choice of container engine is decided by the distribution rather than by preference. ## The dev install script and the six processes it leaves you running The getting started path is scripted, and it is worth knowing exactly what the script does before you run it. Cloning and installing takes three commands. ```bash git clone https://github.com/lablup/backend.ai.git cd backend.ai ./scripts/install-dev.sh ``` The README says the script checks required dependencies such as Docker and Python, sets up a Python virtual environment with Pantsbuild, starts the halfstack infrastructure including PostgreSQL, Valkey, etcd and Apollo Router, initialises database schemas, and creates default API keypairs and user accounts. The observability and object storage parts are opt-in through Compose profiles, enabled with the --profile observability and --profile storage flags, so a default run stays smaller. After that, each component runs in its own terminal. The manager is the control plane and the agent is the per-node controller, and both are started the same way. ```bash ./backend.ai mgr start-server --debug ``` ```bash ./backend.ai ag start-server --debug ``` The storage proxy, the web server, the account manager for SSO and unified user profile, and the app proxy coordinator and worker for in-container service access follow the same pattern. ```bash ./backend.ai storage start-server --debug ``` ```bash ./backend.ai web start-server --debug ``` ```bash ./backend.ai am start-server --debug ``` ```bash ./backend.ai app-proxy-coordinator start-server --debug ./backend.ai app-proxy-worker start-server --debug ``` ## Running your first session from the CLI The client needs credentials before it can create a session, and the environment script is what supplies them. ```bash source env-local-user-session.sh # This script prints your default User ID and Password; ./backend.ai login # When prompted, enter the User ID and Password shown above. ``` With that in place, a session is one command. ```bash ./backend.ai run python -c "print('Hello Backend.AI!')" ``` The web interface is served on port 8090 locally, with credentials from the env-local-*.sh files. The pattern in these commands is the whole point of the design: you never SSH into a shared machine, you ask the platform for a session and it hands you an isolated container with your environment already in it. The repository root also carries a backend.ai entry script and a bai directory alongside the pants.toml build configuration, and the pyproject.toml requires Python ~=3.13.7, so the client and server are built together with Pantsbuild rather than installed from a wheel. ## Websocket tunnelling, vfolders and the in-container access paths How a user actually reaches a running session is described in a section on compute sessions, also called kernels. Backend.AI provides websocket tunnelling into individual computation sessions, so that users can reach in-container applications through their browser and the client CLI in a secure way. The README enumerates what is available inside a session: Jupyter and JupyterLab in most container images, a web-based terminal since all container sessions have intrinsic ttyd support, SSH with SFTP and SCP with an auto-generated per-user SSH keypair, and web-based VSCode in most sessions. PyCharm and other IDEs can use on-demand sessions through SSH remote interpreters. Storage is handled by vfolders, an abstraction layer over existing network-based storage such as NFS or SMB, where each vfolder behaves like cloud storage that can be mounted into any session and shared between users and groups with differentiated privileges. The per-user auto-generated SSH keypair is the detail that makes IDE integration practical rather than theoretical. ## Licence, versions and what the documentation does not cover The project is LGPL-3.0, with the LICENSE file at the root of the repository. The versioning scheme is date based, with the recent releases including 26.10.0a3 on 2026-10-20, 26.10.0a2 on 2026-10-19 and 26.4.15 on 2026-10-16, and the last push to the repository was on 2026-10-20, so the branch is moving. The build and release process uses towncrier for news fragments, with the fragments collected in a changes directory and the template in changes/template.md, and the release notes are written to a per-version file under CHANGELOG. For multi-node testing and production, the README does something unusual for an open source project: it points you to the documentation for community-supported deployment guides, and asks you to contact the sales team for professional paid support and deployment options. The package also publishes a backend.ai-manager distribution on PyPI, and the README states that this repository contains all open source server-side components and the client SDK. ## Conclusion Backend.AI fits a lab or platform team that wants a self-hosted multi-tenant compute service with per-session isolation, GPU or NPU support, and both REST and GraphQL APIs, and that is prepared to run a stateful infrastructure stack. It does not fit a single developer who wants a notebook on demand, since the dependency list alone is several services plus sudo access, and it does not fit a team pinned to older Python or distribution versions, because the main branch requires CPython 3.13.7 and Podman 5.4.0 or newer. Before you commit, check your container engine version against the table in the README, because Ubuntu 24.04 and Debian 12 ship Podman versions below the floor and the README tells you to use Docker there instead. ## FAQ ### What is Backend.AI and who is it for? It is a container-based computing cluster platform from Lablup that allocates and isolates computing resources for multi-tenant computation sessions on demand or in batches, with customisable job schedulers driven by its orchestrator named Sokovan. It is aimed at teams that want to hand out GPU or NPU capacity to many users, and it exposes everything through REST and GraphQL APIs. ### What infrastructure does Backend.AI need before it will start? The required stack is Docker Compose v2, PostgreSQL 16 or newer, Valkey 9.1 or newer, etcd 3.5 or newer and Prometheus 3.x, with Grafana, Loki, Tempo and the OpenTelemetry Collector optional for observability. The container engine is Docker 20.10 or newer, or Podman 5.4.0 or newer, and the system needs Linux or macOS with sudo access for installation. ### Which accelerators does Backend.AI support? The README lists CUDA GPU, ROCm GPU, Rebellions, FuriosaAI, HyperAccel, Intel Gaudi, Tenstorrent, Google TPU, Graphcore IPU and other NPUs. The agent side of the code has a docker backend, a kubernetes backend and a dummy backend, and the requirements include vendor packages such as habana-pyhlml and tt-smi. ### How do I run my first Backend.AI session? Run the development install script, start the components with the ./backend.ai start-server commands, then source env-local-user-session.sh and run ./backend.ai login with the User ID and Password the script prints. After that a session is one command, for example ./backend.ai run python -c, and the web interface is served on port 8090. ### Can I use Podman on Ubuntu 24.04 with Backend.AI? Not according to the README. It says Ubuntu 24.04 LTS packages Podman 4.9.3 and Debian 12 packages 4.3.1, both below the 5.4.0 floor, and tells you to use Docker on those systems. The distribution table lists the packaged Podman version for RHEL, Rocky Linux, AlmaLinux, Oracle Linux, CentOS Stream, Ubuntu 26.04 LTS and Debian 13. ## Sources - [lablup/backend.ai on GitHub](https://github.com/lablup/backend.ai) - [License: LGPL-3.0](https://github.com/lablup/backend.ai/blob/main/LICENSE) - [Project website](https://www.backend.ai) - [README](https://github.com/lablup/backend.ai/blob/main/README.md) - [Releases](https://github.com/lablup/backend.ai/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/lablup-backend-ai