AIOS: Operating System Kernel for LLM-Based Tools

What AIOS manages for language-model tools

AIOS is an operating system kernel that treats language models as a compute layer to be scheduled and managed. Instead of a traditional OS kernel managing CPU, memory, and I/O, AIOS manages access to language models, tool execution, memory buffers, and persistent storage. The kernel receives requests from the Cerebrum SDK (the application layer) and dispatches them through a chain of system calls to modules that handle scheduling, context switching, memory allocation, and tool invocation. AIOS is designed to support multiple language models running concurrently, each with its own resource budget and task queue. It runs as a Python application and exposes both a web UI and a terminal UI.

Architecture: kernel, modules, and SDK layers

AIOS consists of the kernel (this repository) and the Cerebrum SDK (a separate repository). The kernel manages four core resources: LLM cores, the context manager, the memory manager, and the tool manager. Each module handles one concern. The LLM core abstracts access to different model providers: OpenAI, Anthropic, Groq, Hugging Face, vLLM, Ollama, and others. The context manager tracks the execution state of each tool or task. The memory manager allocates and deallocates buffers for intermediate results. The tool manager dispatches tool calls (Google search, WolframAlpha, external APIs) and manages the execution sandbox. Computer-use tools extend this with a VM controller and MCP server for safe interaction with desktop environments. The Cerebrum SDK lets developers build tools by calling the kernel through a standardized interface.

Setting up AIOS with Python dependencies and uvicorn

AIOS requires Python and its dependencies, listed in requirements.txt. Key packages include litellm (for model abstraction), pydantic, fastapi and uvicorn (for the web server), transformers and accelerate (for local model loading), and chromadb and qdrant-client (for vector storage). Install with pip:

pip install -r requirements.txt

For Docker deployment, use the included Dockerfile:

FROM python:3.11.8-slim-bullseye
WORKDIR /app
COPY . ./
RUN pip install --no-cache-dir -r requirements.txt
CMD exec uvicorn server:app --host 0.0.0.0 --port 8000 --workers 1

Set up API keys in a .env file, following the template in .env.example. Then run the kernel with uvicorn, which starts the web service. The terminal UI can also start independently for CLI-based workflows.

Deployment modes: development, distribution, and discovery

AIOS defines three machine roles in a distributed setup. An ADM (Agent Development Machine) is where developers build and test tools using Cerebrum SDK. An AUM (Agent UI Machine) is a client that provides a user interface for interacting with running tools. An AHM (Agent Hub Machine) is a central server that hosts a repository of tools, letting users publish, download, and share them. In a single-machine setup, all three roles can run on one computer. For scaling, you can separate development machines from UI clients and run the hub on dedicated infrastructure. The kernel supports remote kernel mode, allowing distributed tool deployment across multiple machines.

Model and tool support

AIOS does not hard-code a single language model. The litellm abstraction layer provides access to GPT-4, Claude, Gemini, Groq, Deepseek, and open-source models via vLLM and Ollama. You can mix models in one system: some tools use GPT-4, others use a local Deepseek instance. Function calling works with both closed-source APIs and open-source models (Hugging Face, vLLM, Ollama). External tool integration includes Google Search, WolframAlpha, Rapid API, and arbitrary REST endpoints. Computer-use agents can control a VM and issue commands, with the MCP server providing semantically consistent mapping between tool intentions and system operations. The tool manager maintains a registry of available tools and handles their execution lifecycle.

Memory and storage systems

The memory manager allocates buffers for intermediate results during tool execution. Tools can request memory with a specified size and duration. The context manager tracks execution state: which tool is running, what data flows between tools, and where breakpoints occur. AIOS integrates ChromaDB and Qdrant for vector storage, supporting tools that need semantic search or retrieval-augmented generation. You can also bring your own memory provider: the system supports Mem0ai and Zep (pinned to version 2.0.0 for API stability). Persistent storage for tool outputs and logs goes to the file system by default, configurable per deployment.

Not suitable for large-scale distributed data processing

AIOS is fundamentally designed for tool scheduling and resource management, not for data processing at scale. If your workload requires handling large datasets or distributed compute across thousands of machines, traditional distributed systems (Spark, Kubernetes) are more appropriate. The kernel assumes that each tool is a callable service that returns within a reasonable time; long-running batch jobs need to be wrapped as asynchronous tasks with callbacks. AIOS does not replace a traditional database or file system; it manages the orchestration layer above them. The scheduler is centralized, so horizontal scaling requires running multiple independent kernel instances and managing tool discovery yourself.

Comparison with agent frameworks

Libraries like AutoGen, MetaGPT, and Open Interpreter provide ways to build multi-tool systems from Python. AIOS differs in scope: these frameworks handle the task logic, while AIOS handles resource allocation and scheduling beneath the frameworks. AIOS makes it possible to run multiple instances of the same tool in parallel, manage context across long tool chains, and persist memory for stateful tools. You can onboard a tool built with ReAct, Reflexion, or OpenAGI directly into AIOS; the framework layer and the OS kernel layer solve different problems. AIOS adds infrastructure; the frameworks add orchestration logic.

Active development with research publications at academic conferences

The last push was on 2026-07-20, showing active development. Recent releases include v0.3.0 (January 2026) and v0.2.2 (March 2025), with regular updates adding support for new models and deployment modes. The project is published as research at academic conferences: the foundational paper was accepted at COLM 2025, and related work on memory and computer use has been published at NAACL and ICLR. The license status is unlisted (NOASSERTION); check the LICENSE file in the repository for reuse terms before deploying in production.

Editorial conclusion

Adopt AIOS if you are building software that relies on language models as a compute substrate and need centralized resource scheduling across multiple tools. It is not for you if your workflow is simpler than agent-based task management or if you do not need fine-grained control over memory and scheduling. Before committing, verify that the Cerebrum SDK matches your framework of choice and that the kernel's scheduling model aligns with your tool topology.

Frequently asked questions

Official sources

1. agiresearch/AIOS on GitHub
2. Issues
3. Project website
4. README
5. Releases