Lepton AI: a Python client and lep CLI for NVIDIA DGX Cloud Lepton

What problem leptonai solves, and for whom

The README states the library is the Python library and lep CLI for NVIDIA DGX Cloud Lepton, and that it lets you operate that platform from Python and the command line. That framing matters. This is a control-plane client, not an inference server and not a training framework. The problems it addresses are operational: creating an endpoint from a container image, listing endpoints, checking status, launching a batch job, starting an interactive dev pod, and calling a deployed endpoint from Python as if it were a local function. The audience is therefore narrow and specific. You need an existing Lepton workspace with credentials. If you have that, the library removes the step of hand-writing HTTP calls against the platform API and gives you a CLI that maps onto the same operations. If you do not have that, the package has little to offer, because there is no documented local serving path that stands on its own.

How the CLI, Client and OpenAPI schema fit together

Two mechanisms are visible in the material. The first is the lep CLI, which the README describes as a command-line tool to create and manage endpoints, batch jobs, dev pods, Ray and Slurm clusters, fine-tuning jobs, storage and secrets. The second is the Client class, which does something more interesting: according to the README, it reads the endpoint's OpenAPI schema and exposes each path as a method. That is the core design decision. Instead of shipping a fixed SDK surface per model type, the client introspects whatever schema the endpoint publishes and generates callable methods from it. The README shows this directly: c.paths() prints the available paths, c.run.__doc__ prints the docstring for a discovered method, and c.run(inputs="hello world") invokes it. The Client constructor takes either a workspace and endpoint name with a token, or a local(port=8080) helper for something running on the same machine. That local form is the one escape hatch from the cloud dependency, and it is limited: it points at a port, it does not provision anything.

Getting it running: install, login, deploy, call

The install is a single pip command, and the README notes it also installs the lep CLI: pip install -U leptonai. Authentication is lep login, which the README says opens a browser to fetch credentials if you do not pass them in. From there the documented flow is three commands: lep endpoint create -n my-endpoint --container-image my-registry/my-app:latest, then lep endpoint list, then lep endpoint status -n my-endpoint. Batch work uses lep job create -n my-job --container-image my-registry/my-trainer:latest --command "python train.py", and an interactive pod uses lep pod create -n my-pod --resource-shape gpu.a10. On the Python side, the documented call pattern is Client("my-workspace", "my-endpoint", token="MY_TOKEN") or Client(local(port=8080)). The README suggests running lep --help or lep <command> --help for anything not shown, and points at the published CLI references for the full guide. Note that every deployment example takes a container image. There is no documented path here for shipping a plain Python function to the platform without building an image first.

Endpoint authentication is the part most likely to surprise you

The README spends more space on token handling than on any other topic, which is a signal about where users get burned. In workspaces with secure endpoint defaults enabled, an endpoint created without --tokens is protected automatically, and the create command prints the generated API token. If you do not save that value, you have a running endpoint you cannot authenticate against. The --public flag only controls IP reachability; it does not disable API-token authentication. lep endpoint status reports the two dimensions separately as IP Access and API Token Authentication, which is the right way to present them but also means a single glance at the status output can mislead you if you only read one line. Getting out of token auth requires --allow-unauthenticated-access, and the CLI displays a warning when you use it. lep endpoint get redacts literal tokens by default, and --show-tokens is documented as something to use only when you need a credential-bearing response or a reusable spec export, with the output handled as a secret.

The create() versus create_with_response() split, and why it exists

There is a genuine API wart documented here. SDK callers that leave endpoint authentication unspecified should use client.deployment.create_with_response(...) and save the token from the returned resource. The older create(...) method keeps its boolean return contract and therefore cannot return a server-generated credential, so it emits a RuntimeWarning for requests that may ask the server to generate one. That is a coherent reason for the warning: a boolean cannot carry a token. The practical consequence is that any existing code calling create(...) against a workspace with secure defaults enabled may create an endpoint whose token it never receives. Updates have a matching trap. The README states that SDK updates may not clear api_tokens by sending an empty list alone; you must set allow_unauthenticated_access=true in the same update, or replace the list with at least one token. The former hidden update --remove-tokens option is rejected outright, so scripts built around it will fail rather than silently do the wrong thing.

The agent skill, and what it does not decide for you

The repository ships an agent skill at plugins/lepton-cli/skills/lepton-cli/SKILL.md that lets Claude Code or Codex drive the lep CLI from natural language, for listing endpoints, inspecting jobs and dev pods, checking workspace status, and managing workloads. Installation differs per agent. Codex uses codex plugin marketplace add leptonai/leptonai followed by codex plugin add lepton-cli@lepton-skills, or the interactive /plugins command. Claude Code uses /plugin marketplace add leptonai/leptonai and /plugin install lepton-cli@lepton-skills. The README also documents a manual fallback: clone the repo and copy the skill directory into ${CODEX_HOME:-$HOME/.codex}/skills/lepton-cli or $HOME/.claude/skills/lepton-cli, then restart the agent. One design point is worth calling out: the README says the skill asks for explicit confirmation before any command that modifies or deletes a workload. That is a read-first posture, and it means the skill will not be a fully unattended operator without you changing how you invoke it. The skill uses the same lep CLI, so it inherits your workspace authentication state entirely.

When a plain web framework is the better answer

If your goal is to expose a model behind an HTTP endpoint on hardware you control, a FastAPI application with your own container and your own deployment pipeline is a shorter path, and it does not require a Lepton workspace or lep login. The difference is not quality, it is where the control plane lives. Lepton AI assumes the platform owns endpoint lifecycle, resource shapes, job scheduling and cluster management; FastAPI assumes you own all of that and only need request routing and schema generation. The OpenAPI-driven Client is genuinely convenient, but it is convenient for calling endpoints on the Lepton platform. Nothing in the README suggests you can point it at an arbitrary third-party service and get the same method discovery, though the Client(local(port=8080)) form indicates it works against anything publishing a compatible schema on a local port. A second alternative worth naming is the platform's own web console or REST API. The CLI and Client are wrappers over those operations, so if you are doing one-off work, going straight to the console avoids installing and authenticating a CLI at all.

Maintenance, versioning and licence

The release history shows a large jump: 0.27.3 in June 2026, then 0.46.0 in August 2026, with a push to main in September 2026. A jump of that size between minor versions is worth reading the release notes for before upgrading, because the README describes behavioural changes around token handling that landed somewhere in that window: the create() RuntimeWarning, the rejection of update --remove-tokens, and the requirement to pair an empty api_tokens list with allow_unauthenticated_access=true. If you pin an older version, expect the old semantics. The repository is not archived and the README states the project intends to use this open source repo as the source of truth going forward, noting that early development happened in a separate mono-repo, which explains commits from leptonai/lepton. The licence is Apache-2.0, which is permissive and includes an explicit patent grant; that is a statement about the licence text, not legal advice, and you should have your own counsel review it if you are redistributing the library or bundling it into a product.

Editorial conclusion

Adopt leptonai if your workloads already live in an NVIDIA DGX Cloud Lepton workspace and you want the lep CLI, the Client, or the bundled agent skill to drive them. Do not adopt it as a self-hosted model server or as a way to avoid a cloud control plane; the README describes a client for that platform, not a runtime you own. Before committing, verify three things: whether the endpoint create path in your workspace generates a token by default, whether your SDK code uses create_with_response rather than the older create, and whether your workspace permits --allow-unauthenticated-access at all, since that flag is the only documented way to opt out of API-token authentication.

Official sources

1. leptonai/leptonai on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases