What is Function calling?

What actually happens when a model calls a function

The name is misleading in one respect, and getting past that is the whole idea. The model does not call anything. There is no execution inside the model, and no network connection from the model to your server. What the model does is emit text that has a particular shape, and your application is what gives that text meaning.

The mechanism has four steps, and they repeat as often as the model needs.

First, you send the model a list of the functions it is allowed to ask for. Each entry has a name, a description in natural language, and a schema for the parameters, usually JSON Schema. The description is not decoration: it is the only thing the model has to go on when deciding which function fits a request, and a vague description produces a wrong choice with high confidence.

Second, the model returns either ordinary text or a tool call. A tool call names one of the functions and supplies an object matching that function's parameter schema. At this point nothing has happened yet. The model has written down a request.

Third, your application receives that request, and this is the only place where a decision gets made. You check whether the function name is one you offered, whether the arguments validate against the schema, whether the current user is allowed to make that call, and whether the arguments make sense in your domain. Then you run the function.

Fourth, you put the result back into the conversation, typically as a message tied to the call's identifier, and the model continues with it in context. It may then call another function, or answer the user.

So the loop is: offer tools, receive a proposal, validate and execute, return the result, repeat. Everything that makes function calling safe or unsafe lives in the third step, which is code you write.

Why the pattern exists, and when you do not need it

The reason this exists is that language models are good at producing text and bad at knowing what is true right now. A model cannot read your database, check the current time, send an email or look up a customer's balance, because none of those are inside it. What it can do is decide, from the conversation and from the list of tools you gave it, that one of those operations is now needed.

That capability unlocks a specific class of application: one where the assistant has to take an action, not just describe one. A support agent that looks up an order. A developer assistant that runs a test. A data tool that turns a question into a query. In each case the model is the part that understands the request, and your function is the part that touches the world.

When you do not need it is at least as important, and the reasons are practical rather than philosophical.

If the answer is in the prompt, do not build a tool. If you can paste the information into the context, a tool call adds a round trip, a schema to maintain, and a new failure mode for no gain.

If the operation is deterministic and already exists as code, calling it through the model is the wrong layer. Sending a nightly email is a cron job. A script that reconciles two systems does not become more correct because a language model decided to run it.

If the user can see the answer another way, do not rebuild it. A dashboard exists because a person wants a number; a function that returns the same number on demand is often a worse interface with more moving parts.

The test that separates the two cases is simple. Ask whether the task requires knowledge that only exists outside the model at this moment. If yes, a tool is the right shape. If no, it is an extra layer for nothing.

Where tool calling goes wrong

The failure modes here are well known and worth naming before you build on the pattern, because most of them are predictable and most of them are yours to fix rather than the model's.

Argument errors are the most common. Models will produce well-formed arguments that are wrong: a date in the wrong format, an identifier that does not exist, a string where a number belongs, or a value outside an enum. This is why schema validation in your application is not optional, and why a tool whose parameters cannot be validated is a tool that will eventually do something you did not intend.

Permission confusion is the second. The model has no concept of which user is asking. If the function takes an account identifier as a parameter, the model will happily propose an account identifier the current user cannot see, and it will do so fluently. Identity has to come from the session, not from the arguments.

Side effects are the third, and they are the expensive one. A tool that reads is safe to retry. A tool that charges a card, sends an email or creates a refund is not, and models do call the same function twice when a response looks like an error. Make the function idempotent where you can, and require explicit confirmation in your interface for the ones you cannot make idempotent.

The other two are quieter. A large tool list degrades selection, because every extra entry competes in the same context, and descriptions that overlap produce confident wrong picks. And tool results consume context, so an agent that loops calls will eventually push the useful part of the conversation out of the window. Both are reasons to expose fewer, better-described tools rather than a comprehensive catalogue.

Tool calling and the Model Context Protocol

Function calling solves one half of the problem: how a model asks for something. The other half is how the tools get there, and for a long time every agent framework invented its own answer, which meant the same weather API had to be rewritten for every assistant.

The Model Context Protocol is an attempt to standardise that second half. It describes a way for a server to publish a catalogue of tools, resources and prompts over a standard interface, and for a client to fetch that catalogue and invoke what it needs. A model that speaks function calling at one end and an MCP client at the other can therefore reach a large set of tools without any of them being written for that model.

The practical effect is on the shape of your code. Instead of a function per tool wired into one application, you have a server process that exposes tools, and clients that connect to it. That is a better fit when several assistants need the same tools, and a worse fit when one application has two tools, because it adds a process and a protocol to debug.

It also moves a decision you were already making. Authentication, rate limiting and audit logging are yours either way, and in an MCP setup they usually sit at the server boundary rather than in the agent. That is a better place for them, because it is the one place a tool cannot route around.

How tool calling shows up in open-source projects

The projects below cover different layers of the same loop, from the model-facing library to the tool server.

Frameworks that wrap the loop for a specific language or model family. QwenLM/Qwen-Agent packages function calling, RAG, a code interpreter and MCP support into atomic LLM and Tool classes, and its install is a single pip command, though the framework assumes you already have a Qwen model endpoint. langchain4j/langchain4j is the equivalent for the JVM: one API over more than 20 model providers and more than 30 embedding stores, plus agents and RAG, built for Java rather than ported to it. pguso/ai-agents-from-scratch is the teaching version, walking from a raw local model call to a ReAct agent with tools and memory using node-llama-cpp and no agent framework at all, and its README is honest about the prerequisites.

Local inference, where the tool definitions and the model live on the same machine. oobabooga/textgen is a desktop application for running models locally with a web UI, an OpenAI and Anthropic compatible API, vision and tool-calling support, and no telemetry.

Agents that use tools as their whole purpose. NousResearch/hermes-agent is a self-hosted agent with persistent memory across sessions, scheduled tasks, tool use, and integrations for messaging platforms including Telegram, Discord and Slack, and it is designed to run unattended on a VPS, a container or serverless infrastructure rather than only while a terminal is open.

The tool-serving side. aipotheosis-labs/aci bundles OAuth, per-user credentials and more than 600 tool integrations behind one MCP endpoint or a Python SDK; its repository was last pushed on 2024-12-01, so it is worth treating the self-hosted stack as a beta you maintain. IBM/mcp-context-forge sits in front of MCP, A2A and REST or gRPC services and exposes a single endpoint with centralised discovery, guardrails and management, which is the place where tool calling gets its policy layer rather than only its plumbing.

In practice

The practical shape of function calling is that your application stays in charge: the model names a function and supplies arguments, and your code validates, authorises and executes before anything happens. Start by exposing the fewest tools that cover the real requests, write descriptions that a stranger could act on, take identity from the session rather than from the arguments, and make the side-effecting calls idempotent or explicitly confirmed. To read further, pguso/ai-agents-from-scratch shows the loop without a framework, QwenLM/Qwen-Agent and langchain4j/langchain4j show it in two different ecosystems, and IBM/mcp-context-forge shows what it looks like once there are many tools behind one endpoint.