Functionary: A Deprecated Tool-Calling Model Snapshot and What It Still Tells You

What Functionary Actually Solves, and Who It Was Built For

Most chat models return text. Functionary returns text or a structured call to a function defined as a JSON Schema object, in the style of OpenAI GPT function calls. The README states that the model determines when to execute functions, whether in parallel or serially, and can understand their outputs. It only triggers functions as needed. That last sentence is the design intent: the model is supposed to decide that no tool is required and answer directly, rather than calling something on every turn. The intended user is an engineer building an agent loop who wants the scheduling decision inside the model rather than in a hand-written router. Function definitions arrive as JSON Schema, which means the same schema you would hand to an OpenAI-compatible endpoint can be handed to this one. The topics listed on the repository (agents, function-calling, llm, ml) match that audience.

The Deprecation Notice Changes the Question Entirely

The first thing in the README is a warning block: the project is deprecated and no longer actively maintained, the repository reflects a very old snapshot, and the code, models, and documentation are significantly out of date. It says the repository remains available for reference only, with no updates, bug fixes, or support, and that issues and pull requests may not be reviewed. That is unusually blunt, and it should govern how you read everything else. A changelog entry dated 2024/12/24 announces meetkai/functionary-v4r-small-preview, described as the first version that can generate reasoning steps before using tools. The repository's last push is dated 2026-06-30, so the file tree was touched after the final model release, but the README does not describe what changed and no releases were retrieved. Treat the changelog as a historical record, not a support commitment.

How the Tool-Calling Loop Is Wired: Servers, Schemas, and Two Engines

The architecture visible in the material is a thin Python server in front of an inference engine. There are two entry points, server_vllm.py and server_sglang.py, and the README frames them as alternatives you pick between rather than components that work together. The model side is a fine-tune of Meta's Llama 3 or Llama 3.1 instruct models: the changelog names meta-llama/Meta-Llama-3.1-70B-Instruct as the base for functionary-medium-v3.1 and meta-llama/Meta-Llama-3.1-8B-Instruct for the small v3.1 and v3.2 models. Tool definitions are JSON Schema objects. The README also notes that v3.2 small uses MeetKai's own prompt template while v3.1 small uses Meta's original template as described in Meta's custom tool-calling documentation, and it claims v3.2 is better than v3.1. That is a claim from the project, not a measurement you can verify from the repository. One concrete capability worth noting is the code interpreter: the changelog says v2.4 models were the first with code-interpreter ability, enabled by passing {type: "code_interpreter"} in the tools list. The v4r preview adds a reasoning step before tool use, which changes the token flow: the model produces reasoning tokens first, then the call.

Getting a Server Up: Real Commands and Their Constraints

Installation is an editable install with an extra that selects the engine: pip install -e .[vllm] for vLLM, or pip install -e .[sglang] --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer-python for SGLang. The SGLang path pins you to a CUDA 12.4 and Torch 2.5 FlashInfer wheel, which is a real constraint on the host you build. Starting the small model looks like this: python3 server_vllm.py --model "meetkai/functionary-v4r-small-preview" --host 0.0.0.0 --port 8000 --max-model-len 8192. The SGLang equivalent swaps the flags: python3 server_sglang.py --model-path "meetkai/functionary-v4r-small-preview" --host 0.0.0.0 --port 8000 --context-length 8192. Note that the context flag is named differently in each server (--max-model-len versus --context-length) and the model flag is --model versus --model-path. The README states the medium models require 4xA6000 or 2xA100 80GB and need tensor parallelism, passed as --tensor-parallel-size 2 for vLLM or --tp 2 for SGLang. For the vLLM medium path the README instructs you to export VLLM_WORKER_MULTIPROC_METHOD=spawn first, citing a vLLM issue. LoRA serving is available only in vLLM, either at startup with --enable-lora --lora-modules {name}={path} or dynamically through the /v1/load_lora_adapter and /v1/unload_lora_adapter endpoints, with chat requests then naming the adapter in the model field. The README also mentions a Text-Generation-Inference service installed via Docker, but the text is truncated mid-command, so the TGI path cannot be followed from what is available here.

Where This Breaks Down, and Why It Is the Wrong Tool for Most New Work

The deprecation notice is the first failure mode and the largest one. The README states plainly that the code, models, and documentation are significantly out of date and that issues and pull requests may not be reviewed. If you hit a bug in server_vllm.py, there is no support path described. The second failure point is version drift against the serving engines. The SGLang install pins FlashInfer to a cu124/torch2.5 wheel, and the vLLM path carries a workaround for a specific vLLM issue. Both engines move quickly, and a snapshot that is not maintained will not track their API changes. The third is hardware. The medium models need 4xA6000 or 2xA100 80GB, which puts them out of reach for a single-GPU workstation, and the README does not give a smaller-footprint path for those weights. Fourth, the documentation is thin in exactly the places that matter for production: there is no description of error handling when a tool returns malformed output, no retry semantics, and no stated behaviour when the model emits a call to a function that was not defined. The README says the model can understand tool outputs, but not what it does when it cannot. If you need a maintained, supported tool-calling stack with a security response process, this is not it.

The Alternative: Let the Serving Engine Own the Template

The clearest alternative is to run the base Llama 3.1 instruct models directly through vLLM or SGLang and use the tool-calling template those engines and Meta already ship. The difference in approach is where the tool-calling logic lives. Functionary bakes it into fine-tuned weights plus a MeetKai prompt template, so the model itself decides when to call, in parallel or serially, and a dedicated server script translates that into an API. The base-model route keeps the weights untouched and relies on the prompt format documented by Meta for user-defined custom tool calling, which the README itself cites for functionary-small-v3.1. That route trades the fine-tune's specialised behaviour for a model you can update independently of the server code. The README's own changelog supports this reading: v3.1 small uses Meta's original template and v3.2 small uses MeetKai's own, and the project says v3.2 is better. The value the project added was the template and the fine-tune, and both are now frozen. A second alternative is an OpenAI-compatible hosted endpoint if your constraint is operational rather than architectural, since the JSON Schema tool format is described as similar to OpenAI GPT function calls, which keeps the client-side schema portable.

Licence, Maintenance Cost, and What to Check Before Touching the Code

The repository is MIT licensed, which is permissive and places few restrictions on reuse, modification, or redistribution. That is the licence on the code in this repository. It does not automatically cover the model weights, which live on Hugging Face under the meetkai namespace and are built on Meta's Llama 3 and Llama 3.1 models; those carry their own terms, and the README does not restate them. If you plan to redistribute weights or a fine-tune, read the model card on Hugging Face rather than assuming the MIT grant extends to them. This is not legal advice; check with your own counsel. On maintenance cost, the honest number is close to zero going forward, because the project states it will not provide updates, bug fixes, or support. The cost you carry is the opposite kind: the cost of owning a frozen fork. You inherit the vLLM multiprocessing workaround, the pinned FlashInfer wheel, and the two divergent server scripts with different flag names. Upgrading vLLM or SGLang later means re-testing server_vllm.py or server_sglang.py yourself, with no upstream to merge from. Before you spend time on it, verify three things: that the specific model name you intend to load still resolves on Hugging Face, that the server script's arguments match the engine version you installed, and that your hardware meets the 4xA6000 or 2xA100 80GB requirement if you want the medium weights. If any of those fail, the snapshot has already outlived its usefulness for you.

Editorial conclusion

Adopt Functionary only as a reference implementation if you are studying how a model can emit parallel or serial tool calls and consume their outputs; do not adopt it for new production work, because the README states the project is deprecated, unmaintained, and that issues and pull requests may not be reviewed. Before doing anything with it, verify the model weights you intend to load still exist on Hugging Face, confirm the vLLM or SGLang server files in the repository match the version of those engines you have installed, and check the MIT licence terms against your own redistribution plans. The safest use is reading server_vllm.py and server_sglang.py to understand the request path, not deploying them.

Official sources

1. Issues
2. License: MIT
3. MeetKai/functionary on GitHub
4. README