imp

What the project is

imp is a Cuda project hosted on GitHub. From-scratch C++23/CUDA inference engine for the NVIDIA RTX 5090 (sm_120a). The best single-GPU backend for agentic AI: tool calling, long-context loops, reasoning and concurrent sub-agents. Decode beats llama.cpp b9976 by 42-48% on dense GGUF (measured 2026-07-12), at-or-ahead of vLLM on NVFP4. 100% written by Claude Code. imp is an LLM inference engine that targets exactly one chip: the NVIDIA RTX 5090. The repository is maintained by kekzl and tracks activity through its public issue tracker and commit history. Readers who want a working example rather than a bare library will find the documentation and the linked resources useful for getting a first build running.

What it offers

The imp repository documents several concrete capabilities that shape how people use it. A from-scratch C++23/CUDA engine for consumer Blackwell (sm 120a), with its own GGUF and SafeTensors loaders, tokenizer, paged KV cache and kernels. A server that speaks both the OpenAI and the Anthropic APIs natively, so an agent stack written against either runs without a shim. Also a C library and a CLI, a server. Portable. There is no CPU path, no other GPU, no fallback. A multi-GPU or datacenter-batching engine. These points are taken from the project README, so they reflect what the maintainers actually ship rather than marketing claims. Checking the file list and the example directories gives a fuller picture of how each piece fits together.

How to set it up

Getting imp running starts with cloning the source from https://github.com/kekzl/imp. The README lists the commands needed to fetch the code and build or launch it. Representative steps from the documentation include: docker run --gpus all -v ~/models:/models -v imp-cache:/home/imp/.cache/imp \ curl -s http://localhost:8080/v1/chat/completions \ git clone https://github.com/kekzl/imp.git && cd imp docker compose build imp-server. After the code is on disk, the project instructions walk through configuration and any dependencies. Following the order shown in the README avoids the common setup mistakes that come from mixing steps.

License and project status

imp is written primarily in Cuda and is released under the MIT license, which sets the terms for reuse and redistribution. The project reports around 36 stars on GitHub, a signal of how many developers have bookmarked it. The source was last updated on 2026-08-21. The canonical location is https://github.com/kekzl/imp, where the license file, the changelog and the open issues give the most current state of the work. Anyone planning to depend on it should read the license text directly before shipping it inside another product.

Editorial conclusion

The imp source lives at https://github.com/kekzl/imp under the MIT license and is mainly written in Cuda. The README and the linked examples remain the place to confirm the current behaviour before adopting the project.