groq-python is the official client, and it is generated code

One API, one generated client

The Groq Python library provides convenient access to the Groq REST API from any Python 3.10+ application. It includes type definitions for all request parameters and response fields, and offers both synchronous and asynchronous clients powered by httpx.

The last detail of that sentence matters more than it looks. The library is generated with Stainless, which is a code generation service for API clients. That single fact explains almost everything about the project: the shape of the API surface, the consistency between the sync and async clients, the typing story, the error classes, and the release rhythm.

It also tells you what to expect when a new endpoint appears on the Groq side. You do not wait for a hand-written wrapper. You wait for a release of the client, and the new endpoint appears with the same conventions as everything else. The cost of that approach is that the library's code is not hand-tuned, and the repository layout reflects that, with a src directory, a bin directory, an examples directory and a scripts directory, plus a noxfile.py and a devcontainer.

For the audience, the choice is straightforward. If you have decided that Groq is your inference provider, this client removes the ordinary work of writing typed request bodies, parsing responses and handling errors, and it does so in a way that is consistent with how the rest of the industry ships SDKs.

Installing it and making the first call

The package is on PyPI and the install is one line:

# install from PyPI
pip install groq

The client reads the API key from the environment. The README notes that you can provide an api_key keyword argument, but recommends using python-dotenv to add GROQ_API_KEY to a .env file so that the key is not stored in source control. That is a small thing that a lot of SDK readmes do not bother to say.

The first call is a chat completion:

import os
from groq import Groq

client = Groq(
    api_key=os.environ.get("GROQ_API_KEY"),  # This is the default and can be omitted
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Explain the importance of low latency LLMs",
        }
    ],
    model="openai/gpt-oss-20b",
)
print(chat_completion.choices[0].message.content)

Two details in that snippet are worth copying verbatim into your own code. The api_key argument is marked as the default and can be omitted, because the client picks the environment variable up on its own, so passing it explicitly is only for clarity or for an unusual setup. And the model identifier is a full namespaced string such as openai/gpt-oss-20b, which tells you the API is addressed by model name rather than by a model handle you create in advance.

The response is a Pydantic model, so the content is reached through choices, then message, then content, and each step is typed.

The async client, and swapping httpx for aiohttp

The asynchronous client is the same API with await in front of each call. You import AsyncGroq instead of Groq, and the README says the functionality between the synchronous and asynchronous clients is otherwise identical.

import os
import asyncio
from groq import AsyncGroq

client = AsyncGroq(
    api_key=os.environ.get("GROQ_API_KEY"),  # This is the default and can be omitted
)


async def main() -> None:
    chat_completion = await client.chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": "Explain the importance of low latency LLMs",
            }
        ],
        model="openai/gpt-oss-20b",
    )
    print(chat_completion.choices[0].message.content)


asyncio.run(main())

The interesting part is the HTTP backend. By default the async client uses httpx, and the README says that for improved concurrency performance you may also use aiohttp as the HTTP backend. That is enabled by installing the extra:

# install from PyPI
pip install groq[aiohttp]

and then instantiating the client with http_client=DefaultAioHttpClient():

import os
import asyncio
from groq import DefaultAioHttpClient
from groq import AsyncGroq


async def main() -> None:
    async with AsyncGroq(
        api_key=os.environ.get("GROQ_API_KEY"),  # This is the default and can be omitted
        http_client=DefaultAioHttpClient(),
    ) as client:
        chat_completion = await client.chat.completions.create(
            messages=[
                {
                    "role": "user",
                    "content": "Explain the importance of low latency LLMs",
                }
            ],
            model="openai/gpt-oss-20b",
        )
        print(chat_completion.id)

The async with form in that example is worth copying too, since it closes the underlying HTTP client when the block ends. The aiohttp backend is a legitimate choice for a service that keeps many concurrent streams open, where the connection handling behaviour of the two libraries differs.

Streaming with Server Side Events

The library supports streaming responses using Server Side Events, which is the mechanism behind token-by-token output.

The synchronous form sets stream=True and iterates over the returned object:

from groq import Groq

client = Groq()

stream = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Explain the importance of low latency LLMs",
        }
    ],
    model="openai/gpt-oss-20b",
    stream=True,
)
for chat_completion in stream:
    print(chat_completion.id)

The async client uses the exact same interface, with an await on the call and an async for on the loop:

from groq import AsyncGroq

client = AsyncGroq()

stream = await client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Explain the importance of low latency LLMs",
        }
    ],
    model="openai/gpt-oss-20b",
    stream=True,
)
async for chat_completion in stream:
    print(chat_completion.id)

The one difference between the two is a small but consequential detail of style. In the sync version, the call returns a stream object immediately and the loop is a plain for. In the async version, the call itself is awaited, and the loop is async for. If you write both halves of an application against the same interface, that difference is the one you will have to remember, and the README states it plainly enough that you will not discover it at runtime.

Note also that the streamed objects are the same typed response models, with an id on each chunk, so a consumer can log or correlate chunks without parsing raw text.

Typed parameters, Pydantic responses and file uploads

The typing story is the part of a generated client that saves the most time, and the README describes it precisely. Nested request parameters are TypedDicts, and responses are Pydantic models, which also provide helper methods for things like serialising back into JSON with model.to_json() and converting to a dictionary with model.to_dict().

Typed requests and responses give you autocomplete and documentation inside your editor. If you want type errors shown in VS Code to help catch bugs earlier, the README suggests setting python.analysis.typeCheckingMode to basic.

Nested parameters are dictionaries typed with TypedDict, which is how a parameter such as compound_custom is passed:

from groq import Groq

client = Groq()

chat_completion = client.chat.completions.create(
    messages=[
        {
            "content": "string",
            "role": "system",
        }
    ],
    model="meta-llama/llama-4-scout-17b-16e-instruct",
    compound_custom={},
)
print(chat_completion.compound_custom)

File uploads follow the same pattern. Request parameters that correspond to file uploads can be passed as bytes, or a PathLike instance, or a tuple of filename, contents and media type:

from pathlib import Path
from groq import Groq

client = Groq()

client.audio.transcriptions.create(
    model="whisper-large-v3-turbo",
    file=Path("/path/to/file"),
)

The async client uses the exact same interface, and the README adds that if you pass a PathLike instance, the file contents are read asynchronously automatically. That is the kind of detail that a hand-written client rarely gets right and that a generated one inherits from the specification.

The error taxonomy you will actually catch

Error handling is where a client library earns its keep, and the README is specific about the three cases you need to distinguish.

When the library is unable to connect to the API, for example because of network connection problems or a timeout, a subclass of groq.APIConnectionError is raised. When the API returns a non-success status code, that is a 4xx or 5xx response, a subclass of groq.APIStatusError is raised, and it carries status_code and response properties. All errors inherit from groq.APIError.

Rate limiting has its own class, groq.RateLimitError, which is worth catching separately because it is the one condition that is usually worth retrying rather than surfacing:

import groq
from groq import Groq

client = Groq()

try:
    client.chat.completions.create(
        messages=[
            {
                "role": "system",
                "content": "You are a helpful assistant.",
            },
            {
                "role": "user",
                "content": "Explain the importance of low latency LLMs",
            },
        ],
        model="openai/gpt-oss-20b",
    )
except groq.APIConnectionError as e:
    print("The server could not be reached")
    print(e.__cause__)  # an underlying Exception, likely raised within httpx.
except groq.RateLimitError as e:
    print("A 429 status code was received; we should back off a bit.")
except groq.APIStatusError as e:
    print("Another non-200-range status code was received")
    print(e.status_code)
    print(e.response)

The ordering of those except clauses matters, since RateLimitError and APIConnectionError are both subclasses of the broader APIError family, and the specific ones have to be caught first. The APIConnectionError branch also prints e.__cause__, which is the underlying exception raised inside httpx, and that is where the real diagnosis of a network problem lives.

The README then lists the error codes in a table, and that table is cut off in the text available here, so the mapping from status code to exception class beyond the three described above is something to confirm in api.md.

Release train, tooling and what the repository shows

The release history is regular: version 1.7.0 on 2026-08-26, 1.6.0 on 2026-07-24, and 1.5.0 on 2026-06-21. The last push to the repository was on 2026-09-04, and the repository is not archived.

That regularity is a consequence of the code generation model. The library is regenerated from the API specification and released automatically, which is why the metadata points at a release-please manifest and a release-please configuration file, and why a CHANGELOG.md sits at the top of the tree. For a consumer, the practical advice is to pin the version and to read the changelog before upgrading, because regenerated clients can rename response model fields and change default behaviour between minor versions.

The development tooling is visible in the packaging metadata. The project uses rye for environment management, with the version pins in requirements-dev.lock, and the dev dependencies include pyright and mypy for type checking, ruff for linting and formatting, pytest with pytest-asyncio and pytest-xdist for the suite, respx for mocking HTTP, time-machine for time-dependent tests, dirty-equals for structural assertions, and nox for task automation. A requirements.lock file alongside requirements-dev.lock means the runtime dependency set is pinned as well, which is unusual and helpful for reproducible deployments.

The runtime dependencies are equally small: httpx, pydantic, typing-extensions, anyio, distro and sniffio, with aiohttp and httpx_aiohttp in the optional extra. The examples directory shows the shapes a user will actually write, with chat_completion.py, chat_completion_async.py, chat_completion_async_streaming.py, chat_completion_stop.py, chat_completion_streaming.py and embedding.py, and the full API reference is api.md rather than the README.

Editorial conclusion

groq-python fits a Python team that has decided to build on the Groq API and wants the official client rather than hand-rolled HTTP calls, since the type definitions, the streaming helper and the error taxonomy are already solved. It is the wrong choice if you need to support several providers behind one interface, since the surface is specific to one REST API, and it is the wrong choice if you cannot accept a generated codebase. Before adopting it, pin the version, since the library is regenerated on a regular release train and the response model classes change with it, and read api.md, which is the reference for the whole surface rather than the README.

Frequently asked questions

Official sources

1. groq/groq-python on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases