xLLM: a hardware table with six vendors, one driver pin, and three deployment scripts off the main branch
A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.
At a glance
- What is it?
- The inference engine donated to the OpenAtom Foundation in July 2026 targets Chinese accelerators, and its repository documents six of them. Reading the packaging and build scripts against that page turns up a CUDA path with no hardware row, TileLang kernel compilation gated on a single string, three model deployment documents living on a preview branch, and two different homepages.
- Who is it for?
- xLLM is worth a look if you deploy on Ascend, and the Ascend path is the one the page actually specifies: it is the only row with a driver floor, the only vendor with a TileLang kernel compile path, and the vendor whose deployment scripts are named individually. Everything else needs work before you commit to it.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly C++, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- xLLM-AI/xllm1,622 ★ · +54
| Repository | 2026-09-13 | 2026-11-15 |
|---|---|---|
| xLLM-AI/xllm | 1.6k | 1.6k |
54 Stars in the last 63 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
Three deployment scripts sit on a preview branch and one points at another repository
The news list is where a reader goes for a working launch command, and the links in it do not agree on where those commands live.
The GLM-5.3-Flash entry from 2026-08-27 links to `blob/preview/glm-5.3-flash/testspace/run_glm_53_flash.sh`. The MiniMax-M3 entry from 2026-06-13 links to `blob/preview/minimax-m3/testspace/run_minimax_m3.sh`. The DeepSeek-V4 entry from 2026-04-24, folded under More News, links to `blob/preview/deepseek-v4-mlu/testspace/run_deepseek_v4.sh`. Three of the deployment documents are on a branch named `preview`, not on `main`, which is the default branch.
The rest degrade further. The GLM-5 entry from 2026-02-12 sends you to `zai-org/GLM-5/blob/main/example/ascend.md`, a file in somebody else's repository. The GLM-4.5 and GLM-4.6 entry points at a Chinese README file in `zai-org/GLM-4.5`. The GLM-4.6V entry links the `zai-org/GLM-V` repository root with no file. The GLM-4.7 entry links the `zai-org` organization page and stops there.
So the most recent model gives you a specific shell script on a side branch, and the models a year older give you progressively less: a foreign file, then a foreign README, then a bare repository, then a bare organization. None of the four is in this repository.
The hardware table names six accelerators and pins a driver version for one
Six rows sit under Hardware Support, and the Remark column is populated for exactly one of them.
| Hardware | Abbreviation | Example | Remark | | --- | --- | --- | --- | | Ascend NPU | NPU | A2, A3 | HDK Driver 25.2.0 + | | Cambricon MLU | MLU | MLU | | | Moore Threads GPU | MUSA | S5000 | | | Hygon DCU | DCU | BW1000 | | | MetaX MACA | MACA | MXC500 | | | Iluvatar CoreX GPU | ILU | BI150 | |
Only the Ascend row carries a Remark, and it is a floor rather than a pin: `HDK Driver 25.2.0 +` allows anything from that release upward. For the other five vendors the cell is empty, so nothing on the page states a driver or toolkit version for Cambricon, Moore Threads, Hygon, MetaX or Iluvatar.
The Example column has its own gap. Ascend names two parts, A2 and A3. Moore Threads names S5000, Hygon names BW1000, MetaX names MXC500 and Iluvatar names BI150. Cambricon puts the abbreviation MLU in the Example column, repeating the value from the column next to it rather than naming a part.
The Overview calls the project specifically optimized for Chinese AI accelerators, and the Highlights repeat that under Mainstream Hardware Support. This table is the whole of that claim, and it leaves five of six vendor toolchains unpinned.
The build configures CUDA and six Chinese backends; the table lists none of them
The packaging script pulls in one environment setter per device family:
from scripts.build_support.env import (
get_cxx_abi,
get_torch_root_path,
set_cuda_envs,
set_dcu_envs,
set_ilu_envs,
set_maca_envs,
set_mlu_envs,
set_musa_envs,
set_npu_envs,
)Seven setters. Six of them map onto the table by abbreviation: `set_npu_envs` for Ascend, `set_mlu_envs` for Cambricon, `set_musa_envs` for Moore Threads, `set_dcu_envs` for Hygon, `set_maca_envs` for MetaX, `set_ilu_envs` for Iluvatar. The seventh is `set_cuda_envs`, and CUDA has no row in the hardware table at all. So the build has a device path the support matrix does not mention, while the matrix is otherwise a one to one picture of the setters.
The same script also reaches for `get_device_type` alongside `get_torch_version`, `get_torch_cmake_prefix_path`, `get_cxx_abi`, `get_cpu_arch`, `get_python_version`, `get_cmake_dir`, `get_ascend_platform` and `get_version`. One device string therefore drives both the environment variables and the compiler paths, which is convenient and also means the string has to be spelled consistently everywhere it is compared.
The two module level flags at the top of the script are `BUILD_TEST_FILE: bool = True` and `BUILD_EXPORT: bool = True`, so the test and export surfaces are built by default rather than behind a flag.
TileLang kernel compilation is gated on one literal string
One helper decides whether any kernel compilation happens at all:
def _maybe_compile_tilelang_kernels(device: str, jobs: int | str | None = None) -> None:
if device != "npu":
returnThe comparison is against `npu` and nothing else, so every other device, including the CUDA path the build imports a setter for, returns before any compiler runs. When the string does match, the helper resolves `get_ascend_platform()`, creates an output root at `get_cmake_dir()` joined with `xllm/compiler/tilelang`, and shells out to a launcher script:
cmd = [
sys.executable,
os.path.join(base_dir, "xllm", "compiler", "tilelang_launcher.py"),
"compile-kernels",
"--target",
"ascend",
"--output-root",
output_root,
"--device",
target_platform,
]Two arguments describe the same vendor at different granularity: `--target` is the hard coded literal `ascend`, and `--device` carries the resolved platform string. The `--jobs` argument is only appended when a value was passed, so a build that supplies none hands parallelism to whatever default the launcher applies.
The kernel build therefore runs under the Python interpreter that invoked the package build, not under a separate toolchain step, which is why `sys.executable` leads the command.
The package calls it a large language model system; the repository calls it four model types
The one line description in the Python metadata reads "A high-performance inference system for large language models." The repository description reads "A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation." The Overview on the front page calls it an efficient LLM inference framework.
So three descriptions in one repository cover two different scopes. Vision language models, diffusion transformer models and recognition models appear in the repository metadata and in neither of the other two.
The classifiers narrow it further. The list carries `Programming Language :: C++`, `Operating System :: POSIX`, two Scientific/Engineering topics including Artificial Intelligence, and Python 3.10 through 3.12 under a 3 :: Only marker. No classifier names a model family, and there is no entry for Python 3.13 even though `requires-python` is an open `>=3.10`.
The example directory contradicts the narrow description rather than confirming it. It ships `generate_vlm.py`, `generate_embedding.py`, `generate_beam_search.py`, `generate.py` and `sample.py` alongside `__init__.py`. Vision and embedding entry points are present in the tree, so a reader who installs the package and reads only its metadata would not learn that those two paths exist.
One console script, and a version number read out of a text file at build time
The entry point table has a single row:
[project.scripts]
xllm = "xllm.launch_server:main"Everything the documentation splits across quick start, launch, online service and offline inference is reached through that one command, whose target is the server launcher. There is no separate console script for the offline path.
Neither the version nor the long description is written in the manifest. Both are declared dynamic and resolved from files:
[tool.setuptools.dynamic]
readme = { file = ["README.md"], content-type = "text/markdown" }
version = { file = ["version.txt"] }`version.txt` sits at the repository root, so the number a package index displays is whatever that plain text file contained when the build ran. It has to be bumped in step with the tag, and a build from a tree where it was forgotten reports the old number rather than refusing.
The readme field names `README.md` only. `README_zh.md` sits beside it in the root listing and is not named here, so the package index page renders the English document and the Chinese one stays a repository file.
Two homepages and a license declared twice in two different ways
The project listing shows a homepage of `https://xllm-ai.com/`. The package points somewhere else:
[project.urls]
Homepage = "https://docs.xllm-ai.com/"
Documentation = "https://docs.xllm-ai.com/"
Repository = "https://github.com/xLLM-AI/xllm"Homepage and Documentation carry the same value, so the project has one documentation URL doing the job of both, while the site recorded outside the package is a different address. Nothing on either side reconciles the two.
The license is declared twice. `license = { text = "Apache 2.0" }` uses the free text form rather than an SPDX expression or a pointer to the `LICENSE` file, and the classifier list separately carries `License :: OSI Approved :: Apache Software License`. Two mechanisms describe the same grant, and the classifier is what a validator reads while the free text field is what a human reads.
The license header at the top of the front page reads `Copyright 2022 JD Co.`, dated 2022, on a repository that now sits under the xLLM-AI organization and was donated to the OpenAtom Foundation on 2026-07-06. The `LICENSE` file at the root is Apache-2.0. The header year and the hosting arrangement are two different facts about provenance, and only one of them has moved.
Four agent tool directories at the root, and a build that can install a hook
The root listing carries `.agents/`, `.claude/`, `.cursor/` and `.gemini/` as directories, plus `AGENTS.md` and `CLAUDE.md` as files. Four tool specific configuration trees for coding agents are committed side by side, which is a maintenance surface in its own right since each needs updating when the project's conventions change.
C++ and Python have their own gates at the same level. `.clang-format` and `.clang-tidy` cover the native code, and `.pre-commit-config.yaml` covers the rest. The Python linter is ruff, configured with `line-length = 120` and `third_party`, `build` and `cibuild` excluded. Its rule set selects pycodestyle, pyflakes, pyupgrade, bugbear, simplify, isort and logging format, and its ignore list is annotated: F405 and F403 for star imports, F821 for undefined names under string annotations and dynamic imports, F401 because side effect imports such as torch_npu are common, and F841 because kernel code keeps precomputed constants in named variables.
The packaging script imports `check_and_install_pre_commit` from the repository's own `scripts.build_support.utils`, so a build can install that tooling rather than assume it. The same script imports its device and path helpers from `scripts.build_support.env` and `scripts.build_support.utils`, which means a source distribution has to carry the repository's `scripts/` tree for the build to run at all; `MANIFEST.in` at the root is what governs that, and `THIRDPARTYNOTICES.md` and `RELEASE.md` sit beside it, along with `.gitmodules`, `third_party/` and `vcpkg.json` for the C++ dependencies.
Editorial conclusion
xLLM is worth a look if you deploy on Ascend, and the Ascend path is the one the page actually specifies: it is the only row with a driver floor, the only vendor with a TileLang kernel compile path, and the vendor whose deployment scripts are named individually. Everything else needs work before you commit to it. Confirm the driver and toolkit version for Cambricon, Moore Threads, Hygon, MetaX or Iluvatar from the vendor, because the table leaves those cells empty. If you are on CUDA, note that the build configures it even though no row claims support for it. Fetch any deployment script from the preview branch yourself rather than trusting that link to stay put, and read the vendor's own repository for the older GLM entries. Install from the release tags and check that version.txt matches the tag you meant, since the manifest takes its number from that file. Read LICENSE yourself rather than trusting either of the two declarations in the package metadata, and confirm the 2022 JD copyright header against the current OpenAtom Foundation hosting before redistributing. None of this argues against the engine; it argues for reading the build scripts rather than the badge row.
Frequently asked questions
Which AI accelerators does xLLM support?
The hardware table names six: Ascend NPU, Cambricon MLU, Moore Threads GPU under MUSA, Hygon DCU, MetaX MACA and Iluvatar CoreX GPU. Only the Ascend row carries a Remark, giving HDK Driver 25.2.0 as a floor; the other five rows leave that column empty.
Does xLLM run on NVIDIA GPUs?
The build script imports set_cuda_envs alongside the six setter functions for the Chinese accelerator backends, so a CUDA path exists in the packaging code. The hardware support table on the front page lists no NVIDIA row at all.
What command does pip install xllm provide?
One console script named xllm, mapped to xllm.launch_server:main. The version and long description are declared dynamic and read at build time from version.txt and README.md respectively.
Which models has xLLM added support for recently?
The most recent news entries name GLM-5.3-Flash on 2026-08-27, the donation to the OpenAtom Foundation on 2026-07-06, and MiniMax-M3 on 2026-06-13. Earlier folded entries add DeepSeek-V4, GLM-5, GLM-4.7, GLM-4.6V and the GLM-4.5 and GLM-4.6 series.
Where can I find the launch script for a model xLLM supports?
The GLM-5.3-Flash, MiniMax-M3 and DeepSeek-V4 entries link to shell scripts under a branch named preview rather than main. The GLM-5 entry links a file in the zai-org/GLM-5 repository, and the GLM-4.7 entry links only the zai-org organization page.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/xllm-ai-xllm)Star history chart for your README
[](https://hysenlabs.com/projects/xllm-ai-xllm)Was this useful?