Qwen3.8-27B on a Single 16-32 GB Nvidia Card: What the One-Click Installer Actually Does
Qwen3.8-27B on 16-32 GB Nvidia GPUs one-click install for Windows / Linux
At a glance
- What is it?
- MiaAI-Lab's kit picks an EXL3 quant that fits your VRAM, builds its own Python environment, downloads the weights and serves an OpenAI-compatible endpoint on Windows or Linux. The convenience is real; the VRAM check is the part that decides whether it works.
- Who is it for?
- Adopt it if you have a Turing-or-newer Nvidia card with 12 GB of VRAM or more, 64-bit Python 3.11+ and Node 22.19+, and you want an OpenAI-compatible endpoint without touching the CUDA toolkit or a compiler. Skip it if you need a multi-user server, a non-Nvidia accelerator, or a documented rollback path, none of which the README covers.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 8 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 16, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap this fills: a 27B model on one consumer card, without a build toolchain
Running a 27B model locally usually means three separate chores: choosing a quantization that fits your card, getting an inference engine compiled against the right CUDA and PyTorch versions, and wiring up something that speaks HTTP. The kit collapses all three into one launcher. It targets a specific pairing, Qwen/Qwen3.8-27B weights in turboderp's EXL3 quantizations, served by exllamav3, and it decides the quant for you by reading the VRAM on the card it finds. The README describes it as a serving kit rather than a framework, and that is the right frame: it does not add a new runtime, it automates an existing one.
The intended user is someone with one Nvidia card who wants a chat window and an API, not someone building an inference platform. The README states that CUDA Toolkit, Visual Studio Build Tools and Git are not needed, because the engine arrives as a prebuilt wheel. That single decision removes the most common reason these setups fail on a fresh Windows machine.
What the launcher picks for your GPU, and why 2.0 bpw is the floor
The quantization is chosen at setup from your VRAM, and the README says you can change it any time. The 2.0 bpw quant is the interesting one: it is the only weight set here that is not turboderp's own upload, published instead at Mia-AiLab/Qwen3.8-27B-EXL3-2.0bpw under the tag SC_2.00bpw_H3_V3. Everything from 2.5 bpw upward is pulled from turboderp/Qwen3.8-27B-exl3 by revision. So the low end of the range rests on a single non-upstream quant, and if you are evaluating output quality at 16 GB, that is the artifact you are evaluating.
Disk cost scales with the choice: the README gives 9.7 to 22.9 GB per quant, before the Python environment and PyTorch. The hardware floor is broader than the headline suggests. An Nvidia card with 12 GB or more and compute capability 7.5+ (Turing and newer) qualifies, with 16 GB described as the size the kit was built around. Driver 570 or newer is required because the default PyTorch build is cu128.
Installing on Windows: START-HERE.bat, then start.bat
On Windows everything lives in the windows\ folder, and the two entry points answer two different questions. START-HERE.bat installs; start.bat starts. The README is explicit that start.bat never downloads and that START-HERE.bat --no-start never loads, which means fetching a second quant size does not disturb a running model.
The first run opens a page in your browser rather than asking questions in the console. It reports what it found on the card, offers the sizes that fit, then installs and downloads with a progress bar and a live log. When the download finishes it loads that model and hands the page to the chat, so a single double-click goes from nothing to a working chat window.
windows\START-HERE.bat install, then start what was installed
windows\START-HERE.bat --no-start install only - for fetching a second sizeIf you would rather answer console questions than use the web page, the README says to set SETUP=console in .env. The download is resumable: closing the window, losing the connection or rebooting picks up from the byte it stopped at, and a half-downloaded model is labelled as such in the menus. Nothing offers to start a model until every weight file is on disk.
The daily start path and the free-VRAM check that matters most
After the first install, windows\start.bat is the everyday command. It asks which model to run if more than one size is on disk, with Enter taking the last one used and a 45-second timeout so an unattended machine still comes up. Then it checks free VRAM against the GPU_MEM_GB budget in .env plus a margin. If that much is not free, it lists the programs holding VRAM (browsers, games, Discord, other AI tools) and waits, with Enter to re-check, c to continue anyway, q to quit, and an automatic continue after 120 seconds.
The README's warning here is the most useful sentence in the file: on Windows, with too little free VRAM the driver pages the model into system RAM instead of failing, and it then runs many times slower. That is not a crash you can diagnose from an error message. It looks like a working model that has become unusable. Free VRAM, not total VRAM, is the number to check before blaming the quant.
The chat UI opens at http://127.0.0.1:3080/. The console window that start.bat opened is the server, so closing it stops the model. A tray icon offers Open Simplex, Restart the model, Show the Simplex folder, View the log and Quit Simplex, and TRAY=no in .env turns that off. Every launch writes a transcript to logs\, and the first successful launch adds Start-menu and desktop shortcuts unless SHORTCUTS=no is set.
Stopping selectively, and the doctor command when something is wrong
Stopping is more granular than starting, which is unusual and useful. windows\stop.bat stops both the model and the chat UI, but the flags let you keep one alive.
windows\stop.bat stop both the model and the chat UI
windows\stop.bat --harness-only leave the model loaded, close the UI
windows\stop.bat --server-only leave the UI running, unload the modelFor anything that is not covered by the troubleshooting table, the README points at windows\simplex.bat doctor. The table itself covers four cases: Python not found (install 64-bit 3.11+ from python.org and tick Add python.exe to PATH), a model that loads but crawls (free VRAM, or lower CONTEXT_SIZE or GPU_MEM_GB in .env), and "Images: off" in the Ready box, which the README attributes to the vision tower not fitting next to your context and suggests lowering CONTEXT_SIZE for. The Python case is the one hand-installed dependency in the whole kit.
The engine install, the wheel table, and where the fallback compiles
The .env.example is the most informative file in the repository, and it describes a mechanism the README only summarises. On first launch, linux/start.sh creates .venv and installs the engine. Normally that is a wheel install: the engine's own GitHub release publishes a wheel per combination of CUDA line, torch version and Python, and the launcher installs the one matching the venv, roughly a 100 MB download with no compiler. The comment notes that PyPI has no 1.4.4 (it jumps 1.4.2 to 1.4.5) while the release does, which is why the kit does not simply pip-install from the index.
The source build is the fallback, used when no wheel fits: an unbuilt CUDA line, aarch64 (GB10/Spark), or a torch newer than the release. EXL3_REPO forces that source build from a git+https:// URL or a path that exists on the machine, and the file warns not to copy a Linux or Spark path onto Windows. TORCH_INDEX_URL overrides the PyTorch wheel index, and the default cu128 is deliberate: the engine release builds for cu128 and cu132 only, so torch from any other line means there is no prebuilt engine and the kit compiles. cu128 covers Blackwell and needs driver 570+. GB10/Spark uses cu130 and compiles either way.
The torch version is chosen, not just the index, because the newest torch is not always one the engine has a wheel for. tools/wheels.py reads that from the same table used to pick the wheel, so the two cannot drift, and the file gives the command to inspect it:
.venv\Scripts\python.exe tools\wheels.pyThat shared-table design is a real safeguard. A launcher that hardcodes a torch version separately from its wheel-selection logic drifts the moment either side is updated.
Where this kit is the wrong tool, and what to use instead
The README does not document rollback. There is no described path for undoing an install, reverting to a previous quant, or recovering a .venv that a wheel install left in a bad state, beyond deleting the folder the kit created. The isolation is the mitigation: everything stays inside .venv/, models/, logs/, apps/ and .dsh/, nothing enters the system Python, and nothing needs administrator rights. If you want versioned, reversible deployments, that has to come from outside this kit.
Single-user is the other boundary. The endpoint binds to 127.0.0.1:3080, the console window is the server, and the tray controls are per-desktop. Nothing in the README describes authentication, concurrent sessions or multi-tenant serving. If you need several people hitting one model, or a service that survives a logout, this is the wrong shape.
For a different approach, consider llama.cpp's server mode. It runs GGUF quantizations rather than EXL3, and it is not tied to Nvidia hardware at all: CPU, Metal and Vulkan backends mean it runs on machines this kit cannot touch, and its quantization range extends below 2.0 bpw for people with less than 12 GB. The trade-off runs the other way too. EXL3 is an exllamav3 format, and this kit exists because exllamav3's prebuilt wheels and per-quant VRAM selection remove the compile step. Choosing llama.cpp means choosing a wider hardware range and a different quality-per-bit curve. If your card is Nvidia with 7.5+ compute capability, the kit's automatic quant selection is the thing you would be giving up.
Maintenance, licence and what the repository does not tell you
The repository is not archived, and the last push was on 2026-09-10. There are no retrieved releases, so the changelog surface is the commit history and the README itself. The upgrade cost is concentrated in one place: the engine wheel table. When a new CUDA line or torch version appears, the kit either has a matching wheel or it compiles, and the .env.example comment names the exact conditions that force compilation. That is the failure mode to watch on upgrade, and tools/wheels.py is the command that tells you which side of it you are on.
The project is MIT licensed. The weights are a separate matter and the README is clear about provenance: 2.5 bpw and up come from turboderp/Qwen3.8-27B-exl3 by revision, while the 2.0 bpw quant is Mia-AiLab's own upload. If you redistribute anything, check the terms attached to the weight files you actually pulled, not just the licence on the installer. This is not legal advice, and quantized model weights can carry their own terms independent of the serving code.
What the README does not cover: rollback, uninstall, authentication, multi-user serving, and any measured throughput or quality comparison between the quant sizes. The VRAM table tells you what fits, not what each size costs you in output quality.
Editorial conclusion
Adopt it if you have a Turing-or-newer Nvidia card with 12 GB of VRAM or more, 64-bit Python 3.11+ and Node 22.19+, and you want an OpenAI-compatible endpoint without touching the CUDA toolkit or a compiler. Skip it if you need a multi-user server, a non-Nvidia accelerator, or a documented rollback path, none of which the README covers. Verify first that your card reports compute capability 7.5 or higher, that your driver is 570 or newer, and that the quant size the launcher offers actually fits your free VRAM rather than the card's nominal capacity.
Frequently asked questions
What GPU and driver do I need for the Qwen3.8-27B one-click install?
The README requires an Nvidia GPU with 12 GB of VRAM or more and compute capability 7.5+ (Turing and newer), with 16 GB described as the size the kit was built around. The driver must be 570 or newer, because the default PyTorch build is cu128.
Do I need to install the CUDA Toolkit or Visual Studio Build Tools?
No. The README states that CUDA Toolkit, Visual Studio Build Tools and Git are not needed, because the engine arrives as a prebuilt wheel. Compiling is the fallback for platforms no wheel covers, such as an unbuilt CUDA line or aarch64.
Why does Qwen3.8-27B run slowly after it loads on Windows?
The README warns that with too little free VRAM the Windows driver pages the model into system RAM instead of failing, and it then runs many times slower. The fix it suggests is freeing VRAM or lowering CONTEXT_SIZE or GPU_MEM_GB in .env.
Community notes