# Triton client libraries: talking to an inference server from Python, C++ and Java

> The Triton client repository holds the client side of NVIDIA's inference server: C++ and Python libraries that speak HTTP/REST or gRPC and can pass tensors through shared memory, a Java library contributed by Alibaba Cloud PAI with a limited feature set, and generated gRPC stubs for other languages. The Python library installs with pip, and the whole set is BSD-3-Clause.

**triton-inference-server/client** — Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.

- Repository: https://github.com/triton-inference-server/client
- Stars: 702 · Forks: 260
- Language: C++
- License: BSD-3-Clause
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/triton-inference-server-client

## The client half of the Triton project

The Triton Inference Server is a server, and this repository is the part you program against. Its stated purpose is to simplify communication with Triton by providing several client libraries and examples of how to use them, and the README is explicit that questions and problems go to the main Triton issues page rather than to this repository, which tells you the client and the server are versioned and maintained as one project.

Three libraries are provided. The C++ and Python APIs are described together as making it easy to communicate with Triton from a C++ or Python application, and they are the pair the project invests in most. The Java API was contributed by the Alibaba Cloud PAI Team and, the README says, makes it easy to communicate with Triton from a Java application using HTTP/REST requests, with only a limited feature subset supported for now.

On top of those three sits a fourth path. The protoc compiler can generate a gRPC API in a large number of programming languages, and the repository keeps the generated output as examples: a Go example under src/grpc_generated/go, Java and Scala examples under src/grpc_generated/java, and a JavaScript example under src/grpc_generated/javascript.

So the choice for an adopter is between an official library and your own generated stub. The libraries give you conveniences, the generated code gives you the protocol and nothing else.

## Two protocols, and the server capabilities behind them

The C++ and Python libraries can send either HTTP/REST or gRPC requests, and the README describes the capability set those requests reach: inferencing, status and health, statistics and metrics, and model repository management.

That list matters when you are deciding whether a client is enough or whether you need to drop to the wire protocol. Inference is the obvious part. Status and health are what a service mesh or a Kubernetes probe needs, and the statistics and metrics surface is what you scrape for operational insight into the server rather than the model. Model repository management is the one that surprises people, because it means the client can load and unload models on a running server, which turns the client into a piece of server administration tooling as well as an inference caller.

The libraries also support using system and CUDA shared memory for passing inputs to and receiving outputs from Triton. That is the performance path for large tensors, and it is why the shared memory utilities exist in the Python package at all. The trade-off is that shared memory requires the client and the server to be on the same machine, so a client library that is easy to use over HTTP and gRPC from anywhere becomes a different piece of code when you want the shared memory path.

The client is therefore not only a request wrapper. Depending on how you use it, it becomes a local transport optimisation, an operations client, or both.

## pip install tritonclient, and the four extras that decide the transport

The easiest route to the Python client is pip, and the README gives the commands with the extras spelled out. Installing everything is:

```bash
$ pip install tritonclient[all]
```

Using all installs both the HTTP/REST and the gRPC client libraries. There are two optional packages, grpc and http, that install support for a specific protocol, so a client that only needs REST installs:

```bash
$ pip install tritonclient[http]
```

A third optional package, cuda, must be installed to use the cuda_shared_memory utilities. The all specification installs it by default, but in other cases cuda has to be named explicitly, which the README demonstrates by combining it with the protocol you actually need:

```bash
$ pip install tritonclient[http, cuda]
```

The components of the install packages are listed as http, grpc with service_pb2, service_pb2_grpc and model_config_pb2, and utils, where the Linux distribution includes shared_memory and cuda_shared_memory.

That list is worth reading closely, because it tells you what the Python package actually ships. The gRPC component includes the generated protobuf modules for the service and for the model configuration, which is why the pip package is self-contained for gRPC use rather than requiring a local protoc run. The utils component is where the shared memory helpers live, and the note that the Linux distribution includes them tells you the shared memory path is a Linux story in this package.

## Where to get the C++ and Java libraries, and how to build them

Beyond pip, the README lists three more routes. The C++, Python and Java client libraries can be downloaded from the Triton GitHub release page corresponding to the release you are interested in, which is the way to match a client to a specific server version. A pre-built Docker image containing the client libraries can be pulled from NVIDIA GPU Cloud, which is the option for teams that do not want to install native dependencies on the host. And the libraries can be built with cmake, which the README mentions as a third possibility alongside the two download routes.

The build route is the one that matters if you need a client that is not packaged. The repository root has a CMakeLists.txt, a .clang-format file for the C++ sources, a .pre-commit-config.yaml and a pyproject.toml, and the source itself lives under src. The JavaScript, Go, Java and Scala clients are the generated gRPC code, so building those is not a cmake job at all but a protoc job.

That is the practical distinction between the two kinds of client in this repository. The C++ and Python libraries are first party code that you compile or install, and the generated clients are protocol bindings produced by protoc from the service definition, with the repository keeping the output as a reference for each language.

A note on the split of responsibility: the README says to ask questions or report problems on the main Triton issues page, so bug reports about the client belong with the server project, not here.

## The examples are organised by transport, and image_client is the full one

The examples are the fastest way to see the API shape, and the README organises them by language and by transport.

There are C++ and Python versions of image_client, an example application that uses the C++ or Python client library to execute image classification models on Triton. That is the most complete example, and the README notes that many of the examples use models from the Triton example model repository, which means you can run them against known models rather than your own.

The simpler examples come in pairs named by transport. The C++ examples that demonstrate the HTTP/REST client are named with a simple_http_ prefix and the ones demonstrating the gRPC client with a simple_grpc_ prefix, and the Python examples follow exactly the same convention, under src/python/examples. The Java examples live under src/java/src/main/java/triton/client/examples.

There is also a separate pair of Python examples that talk to Triton through a generated gRPC API rather than through the tritonclient library. grpc_client.py is a simple example of API usage, and grpc_image_client.py is described as functionally equivalent to image_client but built on a generated gRPC client stub.

That last one is the interesting comparison. image_client and grpc_image_client.py do the same job through two different client stacks, so reading them side by side shows exactly what the official library adds over hand rolled generated stubs.

## BSD-3-Clause, no tagged releases and a recent push

The licence is BSD-3-Clause, and the badge and the licence file in the repository root say so. It is one of the most permissive licences in common use, allowing redistribution in source and binary form with or without modification, provided the copyright notice and the disclaimer are retained, and with the usual clause that the NVIDIA name may not be used to endorse derived products. For a client library that you will link into a commercial service, that is a straightforward answer.

Interestingly, the same BSD-3-Clause text is repeated at the top of the README and again at the top of the pyproject.toml, in both cases inside a copyright block for NVIDIA CORPORATION and its affiliates. That is a common pattern for keeping the notice attached to files that get distributed separately.

On maintenance, the repository is not archived and the last push was on 2026-10-19. It publishes no GitHub releases of its own, and the README points at the Triton server release page for downloads, which means the client is versioned with the server rather than independently. That is the right model for an inference client, since a client talking to a server over a versioned protocol should not drift from it, but it also means you will be pulling a client that matches a specific server release rather than the newest thing on PyPI.

The rest of the tree is small and standard: a src directory, a CMakeLists.txt, a pyproject.toml, a SECURITY.md for reporting vulnerabilities, a .pre-commit-config.yaml, a .clang-format and a .github directory. The primary language is C++, and the last push date means the code is current even though the release history lives elsewhere.

## Conclusion

Adopt the Triton client libraries when your service needs to call models hosted in Triton Inference Server and you want the vendor's own bindings rather than hand written gRPC stubs, especially if you also need to pass large tensors through system or CUDA shared memory. Do not adopt them if you need full feature parity from Java, since the Java library is HTTP/REST only and supports a limited subset, or if you would rather generate your own stubs with protoc and speak the protocol directly. Verify first which optional package your transport needs, http, grpc or cuda, and that the client version you install matches the Triton server release you run, since the client libraries are downloaded from the Triton GitHub release page for the release you are interested in.

## FAQ

### How do I install the Triton Python client library?

Use pip with an extra: pip install tritonclient[all] installs both the HTTP/REST and gRPC client libraries, pip install tritonclient[http] installs only the REST client, and pip install tritonclient[http, cuda] adds the cuda package needed for the cuda_shared_memory utilities. The cuda package must be named explicitly unless you use all.

### Which protocols does the Triton client support?

The C++ and Python libraries can send either HTTP/REST or gRPC requests, reaching inferencing, status and health, statistics and metrics and model repository management. The Java library is HTTP/REST only and supports a limited feature subset, and the protoc compiler can generate a gRPC API in other languages.

### Does the Triton client support shared memory?

Yes. The C++ and Python libraries support using system and CUDA shared memory for passing inputs to and receiving outputs from Triton. The utils component of the Python package includes shared_memory and cuda_shared_memory in the Linux distribution, and the cuda extra must be installed to use the cuda_shared_memory utilities.

### Where can I get the Triton client libraries besides pip?

The README lists the Triton GitHub release page for the release you are interested in, a pre-built Docker image containing the client libraries from NVIDIA GPU Cloud, and building the libraries with cmake. Issues and questions go to the main Triton issues page rather than to the client repository.

## Sources

- [Issues](https://github.com/triton-inference-server/client/issues)
- [License: BSD-3-Clause](https://github.com/triton-inference-server/client/blob/main/LICENSE)
- [README](https://github.com/triton-inference-server/client/blob/main/README.md)
- [triton-inference-server/client on GitHub](https://github.com/triton-inference-server/client)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/triton-inference-server-client