MOSS: A 16B Bilingual Chat Model With Plugin Calls, and the VRAM Bill That Comes With It
An open-source, tool-augmented conversational language model from Fudan University
At a glance
- What is it?
- OpenMOSS/MOSS is a 16B-parameter Chinese and English dialogue model from Fudan University, shipped alongside its SFT data and fine-tuning code under a split licence. The interesting part is the plugin-tuned variant; the practical part is the memory table.
- Who is it for?
- Adopt MOSS if you need a self-hosted bilingual Chinese and English chat model with a documented plugin path and enough GPU memory to hold it: an A100/A800 at FP16, or a single 3090 at int4, where the README estimates 7.8GB to load and 12GB for a turn. Do not adopt it if you want a small model, a Windows or macOS deployment today, or a single permissive licence covering weights, data and code.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 10 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What MOSS actually ships, and who the 16B size is aimed at
MOSS is a bilingual conversational language model with tool-calling support, released by OpenMOSS at Fudan University. The repository bundles three things that are usually separate: the moss-moon-003 model weights on Hugging Face, the multi-turn dialogue data used for supervised fine-tuning, and the deployment plus fine-tuning code. The README describes the base model as self-supervised pretrained on roughly 700B words of Chinese, English and code, with a compute figure of about 6.67x10^22 floating point operations.
The audience is narrow and specific. The moss-moon family has 16 billion parameters. At FP16 the README says it runs on a single A100 or A800, or on two 3090 cards; at INT4 or INT8 it fits on one 3090. That is not a laptop model and not a CPU-first model, though the example code notes the sft checkpoint can also run on CPU with FP16 taking about 30GB of memory. If you are choosing a model to embed in a product, the deciding question is not benchmark position but whether you can afford the card.
The plugin angle is what separates this repository from a plain chat checkpoint. The moss-moon-003-sft-plugin variant was fine-tuned on roughly 1.1 million dialogue turns plus about 300,000 plugin-augmented multi-turn conversations, and the README lists four plugins: a search engine, text-to-image generation, a calculator, and an equation solver. The screenshots under the examples section show a search call, a maths word problem, equation solving, image generation, Chinese-language conversation, code, and a refusal case. Those are illustrative images in the README, not measurements.
The four-stage training pipeline behind the moss-moon-003 checkpoints
The model list reveals a pipeline rather than a single artifact. moss-moon-003-base is the pretrained base. moss-moon-003-sft is that base fine-tuned on about 1.1 million dialogue turns, which the README credits with instruction following, multi-turn dialogue, and the ability to decline harmful requests. moss-moon-003-sft-plugin is the same base fine-tuned on the dialogue data plus the plugin data, adding the four tool capabilities.
On top of that sits a preference stage. moss-moon-003-pm is a preference model trained on preference feedback collected against moss-moon-003-sft, and moss-moon-003 is the final model obtained by training moss-moon-003-sft with that preference model, described as having better factuality, safety and reply stability. A plugin counterpart, moss-moon-003-plugin, is described the same way with stronger intent understanding and plugin use. Both the pm model, the final model, the plugin final model, and the preference data were listed as forthcoming at the time of the README, not as released artifacts.
That distinction matters more than it looks. If you clone the repository expecting the preference-tuned weights, you will find the sft and sft-plugin checkpoints plus their int4 and int8 quantisations, and a note that the rest is coming. The quantised set is complete for the sft line: moss-moon-003-sft-int4, moss-moon-003-sft-int8, moss-moon-003-sft-plugin-int4, moss-moon-003-sft-plugin-int8. The README does not give a date for the missing pieces, so treat any plan that depends on moss-moon-003-pm as unverified.
Getting a conversation out of the sft checkpoint
The documented path is short. Clone the repository, create a Python 3.8 conda environment, and install the pinned dependencies:
git clone https://github.com/OpenMOSS/MOSS.git cd MOSS conda create --name moss python=3.8 conda activate moss pip install -r requirements.txt
The README warns that torch and transformers should not be older than the recommended versions in that file, and that triton currently supports only Linux and WSL, not Windows or macOS. That single sentence rules out native Windows and Mac deployment for now, regardless of GPU.
The usage example loads the tokenizer and model directly through transformers with remote code enabled, which means you are executing code shipped alongside the checkpoint rather than code in the repository you cloned:
from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("OpenMOSS-Team/moss-moon-003-sft", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("OpenMOSS-Team/moss-moon-003-sft", trust_remote_code=True).half()
That is a real supply-chain consideration, not a formality. trust_remote_code=True executes Python from the Hugging Face repository, so the version you pin is the version you audit. The README does not document an offline or vendored loading path, so if your environment forbids remote code execution you will need to work that out yourself.
The VRAM table is the real specification
The hardware section is the most concrete thing in the repository. For batch size 1, the README gives three rows. FP16 needs 31GB to load the model, an estimated 42GB to complete one dialogue turn, and 81GB to reach the maximum dialogue length of 2048 tokens. Int8 needs 16GB, 24GB and 46GB respectively. Int4 needs 7.8GB, 12GB and 26GB.
Read the third column before you buy anything. The gap between loading a model and holding a long conversation is where deployments fail. An int4 model that loads in under 8GB still wants 26GB at full context, which is beyond a 24GB card. The README labels the middle column as an estimate, and it is the only performance figure the repository offers, so treat the 2048-token column as the number that decides your hardware.
There is also a structural constraint buried under the table: quantised models do not support model parallelism at the time of writing. You cannot split an int8 or int4 checkpoint across two cards to escape the memory ceiling. The FP16 path is the one that scales across devices, which is presumably why the README pairs it with a single A100/A800 or two 3090s. If your plan was to run int4 on two mid-range cards, the documented answer is no.
Three licences, one repository, and what that means for reuse
The badges at the top of the README point to three separate licence files. The code is Apache-2.0. The data is CC BY-NC 4.0, which carries a non-commercial restriction. The model weights are GNU AGPL 3.0. These are not interchangeable, and they attach to different artifacts in the same download.
For an engineer, the practical consequence is that the permissive code licence does not settle what you can do with the outputs or the training data. CC BY-NC 4.0 on moss-003-sft-data and moss-003-sft-plugin-data means commercial use of those datasets is restricted by their terms. AGPL 3.0 on the weights is a copyleft licence with a network-use clause, which is a different obligation from Apache-2.0. I am not a lawyer and this is not legal advice; the point is that a team planning a commercial deployment has three documents to read, not one, and the repository does not reconcile them for you.
On maintenance, the material supports only limited statements. The repository is not archived and the most recent push recorded is 2026-09-06, but no releases were retrieved, so there is no version history to reason about upgrade cost. The dependency surface is the transformers library plus the pinned requirements.txt, and because the model loads with remote code from the checkpoint repository, an upstream change to that remote code is a change to your runtime. Pin the model revision as well as the pip versions.
Where MOSS is the wrong tool, and what to compare it against
The README states the limitation plainly: because of the small parameter count and the autoregressive generation paradigm, MOSS may still produce misleading replies containing factual errors, or harmful content including bias and discrimination. It asks users to judge generated content carefully and not to spread harmful output. That is an honest disclosure, but it also means MOSS is not a knowledge base. If your application needs reliable factual recall, retrieval over your own documents will do more for accuracy than swapping checkpoints.
The plugin design is also narrower than the word suggests. Four tools are documented: search engine, text-to-image, calculator, equation solver. There is no general function-calling schema described in the README, no tool registry, and no example of adding a fifth plugin. The plugin deployment is split into a separate repository, MOSS WebSearchTool, and the broader serving stack lives in MOSS Vortex. If you need arbitrary tool orchestration, the documented surface here will not cover it.
A real alternative in the same weight class is a general instruction-tuned model served through a standard inference stack such as vLLM or TGI, where the serving layer handles batching, paged attention and multi-card sharding without the quantisation restriction MOSS documents. The difference in approach is architectural: MOSS ships a chat checkpoint, a plugin-tuned sibling and a reference deployment split across repositories, while a serving-first stack assumes the model is a swappable component behind an OpenAI-compatible endpoint. If your requirement is Chinese and English conversation with a documented plugin path and you want the training data in hand, MOSS is the more self-contained choice. If your requirement is throughput, concurrency or the freedom to change models without changing your deployment, the serving-first route is the better fit.
Editorial conclusion
Adopt MOSS if you need a self-hosted bilingual Chinese and English chat model with a documented plugin path and enough GPU memory to hold it: an A100/A800 at FP16, or a single 3090 at int4, where the README estimates 7.8GB to load and 12GB for a turn. Do not adopt it if you want a small model, a Windows or macOS deployment today, or a single permissive licence covering weights, data and code. Before committing, verify three things: which of the moss-moon-003 checkpoints you actually need (the preference-tuned moss-moon-003 and moss-moon-003-pm were listed as forthcoming, not released), whether your hardware matches the quantised path, since the README states quantised models do not support model parallelism, and whether your use of the CC BY-NC 4.0 SFT data is non-commercial.
Community notes