vLLM 2080 Ti Definitive: a hardware fork for dual Turing cards running Qwen 27B

The bet: modified Turing cards instead of one modern GPU

Two RTX 2080 Ti cards with the 22GB memory modification and an NVLink bridge give 44GB of VRAM and 1,232 GB/s of aggregate bandwidth, against 24GB and 1,008 GB/s for a single 3090 Ti in the comparison table the README publishes. The same table lists 8,704 physical CUDA cores and 136 SMs for the dual-Turing setup, versus 5,376 cores and 84 SMs for the 3090 Ti, and puts the secondary-market price anchor at roughly $550 against roughly $1,100. Those are physical-resource figures, not measured token rates, and the README presents them as the reason the project exists rather than as a benchmark result.

The problem the fork addresses is narrower than "run LLMs on old GPUs". SM75 is not a target that upstream vLLM keeps in working order, so the patches, environment pins and launch flags needed to make a 2080 Ti stack start and generate have to live somewhere reproducible. This repository is that somewhere: a hardware-focused fork that preserves the patched source alongside the launch profiles. The audience is someone who already has the cards, or can buy them cheaply, and wants one serious model served locally rather than a cluster.

Single-concurrency by design, and what that rules out

The README is unusually direct about the serving shape. The project optimizes for extreme single-concurrency performance: one personal-agent style workload, one 27B, 31B or 35B model, and the largest context window the hardware can hold. It states plainly that this is not a multi-tenant serving stack, and that multi-agent use works best as queued workspace isolation rather than parallel long-prefill throughput. Long prefill is described as capacity-safe when tuned but effectively serialized by the runtime scheduler on the TP=2 profile.

That is a real constraint, not a disclaimer. If your workload is several concurrent users each pulling a long document into context, the scheduler serializes the expensive part and your aggregate throughput collapses toward the single-request rate. The fork's headline number, 100+ tok/s single-request decode on the Qwen3.6 27B FP8 baseline, is a per-stream figure. Nothing in the material claims a concurrent-request throughput number, and you should not read one into it.

How the runtime turns Turing silicon into tokens

The mechanism is a stack of named components rather than one trick. Marlin handles the weight route: FP8, INT4 and NVFP4 are all listed as validated for the 27B lane. FlashQLA and FlashInfer provide the fast prefill path. TurboQuant and INT8 KV cache formats extend context beyond what FP16 KV would allow on 44GB. MTP (multi-token prediction) decoding is supported across the 27B matrix, and CUDAGraph integration appears as a no-eager option, marked supported on FP16 and TurboQuant KV but only partial on INT8 KV.

The 27B support matrix is the clearest artifact in the repository. It crosses three KV formats against Marlin weight routes, MTP, native 256K context, YaRN extension, CUDAGraph, fast prefill and multimodal image serving. YaRN is only listed for INT8 KV, and only there is it a target route. The shipped presets are normal, fast and safe for FP16 KV, normal and safe for INT8 KV, and fast for TurboQuant KV. That grid is the practical decision surface: pick the KV format first, then the preset, and the rest follows.

Validated checkpoints are narrower than "vLLM can load it"

The repository separates supported from recommended. A supported checkpoint can start and generate; a recommended checkpoint also has a useful throughput and context tradeoff on dual 2080 Ti. That distinction matters because the generic 27B FP8 route is documented as covering the official Qwen3.6 and Qwen3.8 checkpoints specifically, while other quantized rows stay checkpoint-specific. The validated table lists Qwen/Qwen3.8-27B-FP8, Qwen/Qwen3.6-27B-FP8 and Jackrong/Qwopus3.6-27B-v2-FP8 under the 27B FP8 route, with AWQ-INT4 and GPTQ-INT4 checkpoints also marked recommended.

For the 35B MoE lane, the shipped presets cover FP16-KV 256K text-only in normal and aggressive variants, FP16-KV 136K text plus image in normal and aggressive variants, and a 178K fast MTP3 preset. The README also reports a 256K text-only result for that lane. Note what is absent: no multimodal preset is validated for Gemma4 31B, and the 27B YaRN route is not a target preset for FP16 or TurboQuant KV. If your model is not in the table, treat it as unsupported until you have reproduced a preset yourself.

Getting a stack running from the repository

The fork is released as v0.1.17 on base vLLM 0.21.0, with the default branch named vllm-2080ti-definitive-0.1.x. The README frames the repository as preserving the patched source, launch profiles and runtime notes needed to reproduce the working stack, so the intended path is to check out that branch and build the patched vLLM rather than pip-installing upstream and hoping SM75 survives. The material I have does not include the exact build commands, the launch script filenames or the full set of config keys, so I cannot give you a copy-pasteable invocation without inventing one. What the README does give is the vocabulary you will be matching against once you open the launch profiles: KV format (FP16, INT8, TurboQuant), preset name (normal, fast, safe, aggressive), weight route (Marlin with FP8, INT4 or NVFP4), MTP depth such as MTP3, tensor parallelism fixed at TP=2, and the prefill backend choice between FlashQLA and FlashInfer.

That is the practical first task after cloning: read the shipped profiles for your target model lane and confirm which of those axes each preset sets, because the preset names are not self-explanatory across lanes. A "fast" preset on the 27B TurboQuant route and a "fast" preset on the 35B lane are not the same configuration.

Where the fork breaks down: the Gemma4 31B lane

The Gemma4 31B route is the clearest evidence that this is a working fork rather than a finished product. It is labelled experimental, and the support table is mostly yellow and red. FP16 KV with GPTQ or QAT weights is marked green for the Marlin route and CUDAGraph, with FlashInfer prefill, and the QAT assistant MTP3 path is partial. The validated context is described as about 170K KV headroom observed, itself a partial status. INT8 KV carries a known init issue, and TurboQuant KV carries a capacity shortfall, both red. Neither has a shipped preset. No multimodal preset is validated for this route at all.

So the honest reading is that Gemma4 31B is a research lane. If you need Gemma, you are on your own for the KV format choice, and the two extended-KV formats are documented as failing rather than merely untested. The 27B and 35B Qwen lanes are where the green marks concentrate, and that is where the fork's claims actually rest.

The alternative: upstream vLLM on newer hardware

The obvious comparison is stock vLLM on a single 24GB card such as a 3090 or 4090. The difference is not just speed. Upstream tracks current architectures as its primary target, so you get a supported install path, current kernels and a community that is testing the same binary you are running. What you give up is the 44GB pool and the NVLink link between the two cards, which is what makes a 27B model at 256K context fit at all on this class of hardware. A single 24GB card cannot hold the same KV budget, and the fork's whole point is that the dual-Turing configuration can.

The second alternative is simply not using a fork: run a smaller quantized model on the 2080 Ti pair through a generic runtime. That avoids maintaining a patched vLLM tree, but you lose the Marlin weight routes, the TurboQuant and INT8 KV options, MTP decoding and the CUDAGraph path that the fork documents as validated. The trade is maintenance burden against capability on hardware that upstream does not prioritize.

Licence and the cost of tracking a fork

The repository is Apache-2.0, the same licence family as vLLM itself, which is the permissive outcome you would want for a fork that ships patched source. That is a statement about the licence identifier in the repository metadata, not legal advice; if you redistribute a modified build, read the Apache-2.0 terms and the upstream notices yourself.

The maintenance cost is the part the material supports least well, but the release cadence is visible. Between late August and early September the project shipped v0.2.1-pre2, v0.1.17 and v0.2.1-pre3, with the default branch sitting on the 0.1.x line while 0.2.1 previews circulate. A pre-release line running alongside a stable line means you have to decide which branch you are tracking and re-validate your preset after each move, because the validated checkpoint table is tied to specific releases. Pinning to v0.1.17 on base vLLM 0.21.0 is the conservative position; the 0.2.1 previews are where new routes land first. There is no homepage listed for the project, and support appears to run through a Discord invite in the README, which tells you something about how formal the release process is.

Editorial conclusion

Adopt this fork if you already own two 22GB RTX 2080 Ti cards with NVLink and want one long-context Qwen3.x 27B or 35B agent workload served fast from local hardware. Do not adopt it for multi-tenant serving, and do not expect the Gemma4 31B lane to behave like a finished route: the README marks INT8 KV there as a known init failure and TurboQuant KV as a capacity shortfall. Before committing, verify that your cards are the 22GB modification rather than stock 11GB, that your checkpoint appears in the validated table for the weight format you plan to use, and that the preset you pick is listed as validated rather than experimental.

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. Releases
5. weicj/vLLM-2080Ti-Definitive on GitHub