Gorilla: A Berkeley Toolkit for Teaching LLMs to Call APIs, and for Testing Who Does It Best
Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
At a glance
- What is it?
- Gorilla is a Berkeley project that trains open models to turn natural language into correct API calls, and it now runs the Berkeley Function Calling Leaderboard (BFCL) that compares dozens of models on tool use. The repo is a mixed bag: it offers real inference and evaluation code, but its sprawling scope means you must pick the component you need.
- Who is it for?
- Adopt Gorilla if you need a research-grade benchmark for function calling (BFCL v4) or if you want to fine-tune a model for API use with Apache-2.0 licensed weights. Skip it if you are looking for a production runtime: the repository does not ship a maintained serving stack for Gorilla OpenFunctions v2, and the GoEx engine is a separate experimental project.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 156 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What Gorilla Actually Solves, and for Whom
Gorilla addresses a specific failure of large language models: they hallucinate API calls. Given a request like "book a flight," a general LLM might invent a function name that does not exist or use the wrong parameters. Gorilla's original contribution, per the README, was to fine-tune models on 1,600+ real APIs from a curated corpus called APIBench, so the model produces semantically and syntactically correct calls. The target user is a developer or researcher building an agent that must invoke external tools, not someone who wants a chatbot. The project has since expanded into evaluation: the Berkeley Function Calling Leaderboard (BFCL) is now the most active part of the repository, with the latest release (v1.3, July 2025) adding BFCL V4 Agentic, which tests tool calling in settings that include web search with multi-hop reasoning and error recovery. So the repo serves two audiences: those who want to train or run a tool-calling model, and those who want to compare models on standardized function-calling tasks.
How the Repository Is Organized: Models, Data, and Benchmarks
The GitHub repository is not one tool but a collection of subprojects. The top-level README lists the original Gorilla paper, the APIBench dataset, inference code under /gorilla/inference, evaluation code under /gorilla/eval, and then a series of later additions: OpenFunctions v2, the Berkeley Function Calling Leaderboard (in /berkeley-function-call-leaderboard), GoEx (a runtime for LLM-generated actions), Agent Arena (a crowdsourced agent comparison), and RAFT (a domain-specific RAG method). Each subproject has its own directory and its own documentation. That means when you clone the repo, you are not getting a single installable package; you are getting a monorepo of research artifacts. The active development is clearly on the leaderboard: the recent releases (v1.1, v1.2, v1.3) are all titled "Berkeley Function Calling Leaderboard Updates." The original Gorilla model code is older, with the first model released in May 2023. This split matters: if you want to reproduce the original paper, you need to navigate to the older directories; if you want to evaluate a model on BFCL, you need to follow the leaderboard's own README.
The Mechanism: How Gorilla Trains Models to Call APIs
The core idea, as stated in the README, is that Gorilla "enables LLMs to use tools by invoking APIs. Given a natural language query, Gorilla comes up with the semantically- and syntactically-correct API to invoke." The original approach was fine-tuning on APIBench, a dataset of APIs that the project describes as "the largest collection of APIs, curated and easy to be trained on." The fine-tuning teaches the model to map a user request to a specific API call, including the domain, the operation, and the parameters. Later, OpenFunctions v2 extended this to a more general function-calling format, presumably allowing users to define their own function schemas. The README does not give the training code or hyperparameters in the snippet, but it points to the paper (arXiv 2305.15334) for details. What is clear is that Gorilla is not a retrieval system; it is a generative model that has memorized API patterns from the training corpus. That is why hallucination reduction is a stated goal: the model must produce a call that exists in the API documentation, not a plausible-looking fake.
How to Get It Running: Commands and Entry Points
The README points to separate READMEs for each component, so the exact commands depend on which part you want. For the original Gorilla model, there is a CLI interface documented in /inference/README.md, and the repo mentions a Colab notebook for the first model. For the leaderboard, the code lives in /berkeley-function-call-leaderboard, and the changelog (CHANGELOG.md) documents dataset and model updates. The README also mentions that models are available on Hugging Face under the gorilla-llm organization, so you can download weights from there. For OpenFunctions v2, the blog post is linked but the actual inference code is not shown in the snippet. To run BFCL evaluation, you would clone the repo, navigate to the leaderboard directory, and follow its own README for installing dependencies and running the evaluation scripts. The README does not provide a single "pip install gorilla" command. That is a friction point: you must self-navigate the subproject structure.
The Berkeley Function Calling Leaderboard: What It Measures and Its Evolution
The BFCL is the project's most visible output. It started as a static leaderboard for function calling, then evolved through versions. V2 added enterprise-contributed data and real-world scenarios. V3 introduced multi-turn and multi-step function calling, using a state-based evaluation system that checks whether a model can handle sequential functions and service states. V4, announced in July 2025, moves to agentic settings: it evaluates web search with multi-hop reasoning and error recovery, agent memory management, and format sensitivity. The leaderboard also tracks cost and latency metrics, which were added in April 2024. This is a concrete tool: if you are choosing between models for an agent, BFCL gives you a standardized comparison across multiple dimensions, not just single-turn accuracy. But note that the leaderboard is a benchmark, not a guarantee of production performance. The README does not disclose the exact evaluation harness details beyond the descriptions in the blogs, so you should read the changelog and the specific BFCL README before trusting a model's rank for your own workload.
Limitations and Where It Is the Wrong Tool
One clear limitation is the lack of a unified, maintained runtime for deploying Gorilla models. The README lists GoEx as a runtime for LLM-generated actions, but it is described as a separate project with its own paper (arXiv 2404.06921), and it is not integrated into the main Gorilla inference path. Another limitation is that the original Gorilla model is from 2023; the field has moved to larger instruction-tuned models with native function calling, so an older fine-tuned 7B model may be obsolete for many tasks. The README claims 500k requests served, but that is a historical stat, not a current SLA. Also, the repository is a research artifact: it includes datasets and evaluation code, but not a production-grade API gateway or error-handling layer. If you need to call thousands of live APIs in a business application, you would be better served by a dedicated function-calling framework or a managed API gateway. The BFCL itself has a limitation: it is continuously updated, so a model's score from six months ago may not be comparable to today's scores, and the README warns to check the changelog for updates.
Alternatives: What Else Does Function Calling, and How Does It Differ?
The most direct alternative is OpenAI's function calling, which is built into the GPT-4 API and is referenced in the repository's topics (openai-functions, gpt-4-api). OpenAI's approach is proprietary and hosted: you send a function schema to the API, and the model returns a structured call. Gorilla's approach is different: it offers open-source models that you can fine-tune and run locally, and it provides a benchmark (BFCL) to compare any model, including OpenAI's, on function calling. Another alternative is open-source models with native tool use, such as those from Mistral or Meta, which are not tied to Gorilla's APIBench training. The key difference is that Gorilla is a research project that gives you the training data and evaluation harness, not a turnkey service. If you want a closed, managed solution, OpenAI is simpler; if you want to control your weights and evaluate fairly, Gorilla's BFCL is the reference point. The README does not compare itself to these alternatives, so this is my reading based on the repository's stated scope.
Maintenance, License, and Upgrade Cost
The repository is under the Apache-2.0 license, which is permissive for commercial use. The README highlights that the Gorilla models are "Commercially usable, Apache 2.0 licensed." That is a concrete advantage over some research models that restrict commercial use. However, the maintenance cost is real: the project is a moving target. The leaderboard updates frequently (v1.1 in August 2024, v1.2 in January 2025, v1.3 in July 2025), so if you build an evaluation pipeline around BFCL, you must track the changelog to understand dataset changes. The original Gorilla model code has not seen a release since the initial versions, so the maintenance focus is on the leaderboard, not the inference stack. Upgrading from one BFCL version to another may require changes to your evaluation scripts, and the README does not promise backward compatibility. The repository is not archived, and the last push was April 2026, so it is actively maintained, but that activity is concentrated on the benchmark, not on a stable API for Gorilla models.
Editorial conclusion
Adopt Gorilla if you need a research-grade benchmark for function calling (BFCL v4) or if you want to fine-tune a model for API use with Apache-2.0 licensed weights. Skip it if you are looking for a production runtime: the repository does not ship a maintained serving stack for Gorilla OpenFunctions v2, and the GoEx engine is a separate experimental project. Before adopting, verify which BFCL version matches your use case (V3 for multi-turn, V4 for agentic web search and memory), check the changelog for dataset updates, and confirm that the model you plan to evaluate is on the leaderboard's supported list.
Community notes