# veScale: the default branch still installs the tree the README calls old

> veScale is ByteDance's open cut of an internal PyTorch Distributed library for hyperscale LLM and RL training. Its packaging, not its ambition, is the story: torch 2.7.1 pinned to cu121 wheels, a Dockerfile that holds one word, and a legacy tree the open package still ships.

**volcengine/veScale** —  Byted PyTorch Distributed for Hyperscale Training of LLMs and RLs

- Repository: https://github.com/volcengine/veScale
- Stars: 1,048 · Forks: 65
- Language: Python
- License: Apache-2.0
- Published: 2026-10-14 · Updated: 2026-10-14 · Language: en
- Canonical page: https://hysenlabs.com/projects/volcengine-vescale

## A small open cut of ByteDance's internal PyTorch Distributed stack

The repository describes veScale as an internal PyTorch Distributed library whose job is hyperscale distributed training of LLMs and RLs, and it publishes what it calls a small piece of that stack for the wider community. The audience is narrow: teams sharding model state across many hosts, not people running one accelerator experiment. Topics are llm-training and pytorch, the primary language is Python, and work happens on the main branch. Two arXiv entries sit under Publication, one labelled veScale 2025-2026 at arxiv.org/abs/2602.22437 and one labelled veScale 2023-2025 at arxiv.org/abs/2509.07003, so the public code trails research that has already shipped a generation. The code is offered under the Apache License v2.0, which is the part most teams check first when they plan to wrap the package inside a larger training stack.

## Old veScale sits in legacy while setup.py still installs the package named vescale

The first line of the README announces that old veScale has been moved to legacy/ and that a new veScale is coming. The packaging in the same repository has not caught up with that announcement. setup.py hardcodes NAME = "vescale" and pulls the version out of vescale/__init__.py, so an ordinary install of this branch still delivers the tree the README calls old. Both legacy/ and vescale/ appear in the top-level entries, next to docs/, scripts/, test/, pyproject.toml, requirements.txt and .github/. Read only the headline, then import the package, and you land on the pre-move code with nothing in the tree saying so. That mismatch is the first thing to settle before writing a single training loop against this branch.

## requirements.txt pins torch 2.7.1 and holds numpy below 2.0.0

requirements.txt is the only dependency statement in the repository, and it is unusually specific:

```
--extra-index-url https://download.pytorch.org/whl/cu121
torch==2.7.1
expecttest
hypothesis
pytest
numpy<2.0.0
```

The extra index URL points at the cu121 wheel channel, so the default path expects those CUDA builds, and torch is pinned to one exact patch number rather than a range. numpy is held below 2.0.0, which keeps the project on the 1.x array interface. expecttest, hypothesis and pytest are test-time dependencies, so the test suite leans on property-based cases rather than only fixed examples. What is absent counts as much: no CPU-only variant, no second requirements file for another CUDA version, no matrix of supported combinations. A cluster already running a different torch build has to install its own torch first and expect this file to be edited.

## The legacy setuptools backend exists so setup.py can import local packages

The build section of pyproject.toml is unusual, and a comment above it explains why:

```
[build-system]
requires = [
    "setuptools",
    "wheel",
    "typing-extensions",
]
```

The backend is set to setuptools.build_meta:__legacy__, chosen so setup.py can import local packages while the build runs, which modern backend handling no longer guarantees. The version is then read out of the package by hand:

```
    with open(version_file_path) as f:
        (version,) = re.findall('__version__ = "(.*)"', f.read())
```

If that regex matches nothing, the build stops with a ValueError naming the file it could not read. Repackage that line with different quoting or spacing and nothing installs, with an error that points at the file rather than the pattern. A package_files() helper walks directories to gather data files into the distribution, which is how non-Python assets travel with the package.

## The root Dockerfile holds a TODO, so the quick-start page is the only route

The top-level tree lists a Dockerfile, and that Dockerfile contains a single line reading TODO. No image can be built from it, so any container workflow has to be written from scratch. The Content list instead sends a new user to two pages inside the repository: ./docs/texts/quick-start.md for the first run and ./docs/texts/raggedshard.md for the second documented component. Neither page body is reproduced in the repository README, and no install command line appears there either. With the packaging files as the whole of the recorded route, the practical sequence is short: make an environment, install torch==2.7.1 from the cu121 index, install the package from the checkout, then open the quick-start page for the first real call. Everything past that point lives outside the files listed at the root.

## black targets py38 through py311 while ruff targets py39

Two linters are configured in one file, and they disagree about the oldest interpreter:

```
[tool.black]
line-length = 120
target-version = ["py38", "py39", "py310", "py311"]

[tool.ruff]
target-version = "py39"
line-length = 120
```

The formatter will accept syntax that the linter's target-version does not model, so the two settings can drift. lint.select turns on B, C4, G, E, F, SIM1, W, UP and PERF, plus PGH004, PIE807, PIE810, PLE and TRY302, while lint.ignore carries codes annotated please fix for B007, B008, B017, G101, G201 and G202. Those comments are a debt ledger kept inside configuration. One line points at a file the root does not contain: the note "Synchoronize the ignores with .flake8" sits above lint.ignore, and no .flake8 appears among the top-level entries. A contributor told to sync lint settings has nowhere to read from.

## The cited papers put veScale-FSDP against stock FSDP at scale

The real alternative is the FSDP implementation that ships inside PyTorch itself, which the pinned torch==2.7.1 already contains. The citation block frames veScale against that baseline in two papers, and the titles carry the claim. The 2026 entry is titled veScale-FSDP: Flexible and High-Performance FSDP at Scale under primaryClass cs.DC, so the difference of approach is claimed on the scaling axis of the sharding scheme itself. The 2025 entry is titled veScale: Consistent and Efficient Tensor Programming with Eager-Mode SPMD under cs.PL, which moves the argument to the programming model instead, naming eager mode under SPMD as the consistency mechanism. No measurement from either paper is reproduced in the repository, so the head-to-head numbers live in the arXiv text rather than in the code a reader can open here.

## Where this default branch is the wrong thing to adopt

Three things push some teams away. Anyone on a torch build other than 2.7.1, or outside the cu121 wheel channel, edits a pinned file before anything runs. Anyone who needs a container gets a Dockerfile that says TODO. And anyone who wants published versions to pin gets nothing, since the repository has no GitHub releases and the last push landed on 2026-03-03 while the README promises a new veScale on the way. Scope narrows the use case further: what is published is a small piece of an internal stack, and the topics stop at llm-training and pytorch with nothing aimed at inference serving. Before committing anyone to this branch, open legacy/ and confirm which half of the split the installed package came from.

## Conclusion

This fits teams already training across many hosts on torch 2.7.1 and cu121 wheels, who want the FSDP and RaggedShard DTensor pieces named in the README and can read both arXiv papers first. Single-box researchers, teams on another torch build, and anyone deploying from containers get more out of the papers than out of this tree. Verify one thing before writing a training script: which half of the legacy/ split the installed package came from, because setup.py still names vescale and nothing in the tree marks that boundary.

## FAQ

### What is veScale and who is it for?

veScale is ByteDance's PyTorch Distributed library for hyperscale distributed training of LLMs and RLs, and this repository open-sources a small piece of it. The audience is teams sharding model state across many hosts, and the repository topics are llm-training and pytorch.

### How do I install veScale?

requirements.txt pins torch==2.7.1 behind the cu121 extra index and holds numpy below 2.0.0, and setup.py builds a package named vescale. The Content list sends you to docs/texts/quick-start.md for the first run, and the top-level Dockerfile holds only a TODO marker, so there is no image to build.

### Does the veScale repository publish tagged versions?

The repository has no GitHub releases, and setup.py reads the version string out of vescale/__init__.py rather than from a tag. The last push was on 2026-03-03.

### What is RaggedShard DTensor in veScale?

It is one of the two pieces the README links, the other being the Quick Start page, and it has its own document at docs/texts/raggedshard.md. The repository README gives no detail on its internals, so that page is the place to look.

### Which licence does veScale use?

Apache License v2.0, stated in the repository README and in the header of setup.py. The top-level entries show a single LICENSE file, and the README points at it.

## Sources

- [Issues](https://github.com/volcengine/veScale/issues)
- [License: Apache-2.0](https://github.com/volcengine/veScale/blob/main/LICENSE)
- [README](https://github.com/volcengine/veScale/blob/main/README.md)
- [volcengine/veScale on GitHub](https://github.com/volcengine/veScale)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/volcengine-vescale