flyte-sdk: type-safe Python orchestration for ML pipelines and agents

What the Flyte 2 SDK actually orchestrates

The package on PyPI is named flyte, and pyproject.toml describes it as "The open source durable AI runtime for model training and batch/online inference." The SDK is the Python surface over that runtime. You write async functions, decorate them, and the runtime decides where they execute, how they retry, and whether a previous result can be reused.

The intended audience is narrow but deep: teams running ML training jobs, batch inference, or agent loops that need to survive a pod restart. The README frames it as "Reliably orchestrate ML pipelines, models, and agents at scale, in pure Python." If your workload is a single script that finishes in seconds, the SDK adds a control plane you do not need.

The type-safety claim rests on ordinary Python annotations. Task signatures use standard typing, and the SDK validates inputs against them. That is a smaller promise than it sounds, but it matters when a pipeline is assembled from dozens of tasks written by different people.

TaskEnvironment, async tasks and the caching mechanism

The central object is flyte.TaskEnvironment. It bundles a name and an image, and tasks declared inside it inherit both. The example in the README builds an image from a Debian base with a pinned Python version:

import asyncio
import flyte

env = flyte.TaskEnvironment(
    name="hello_world",
    image=flyte.Image.from_debian_base(python_version=(3, 12)),
)

@env.task(retries=3, cache="auto")
async def predict(x: int) -> int:
    return 2 * x + 5

The decorator arguments do the orchestration work. retries=3 tells the runtime how many times to re-run a failed task. cache="auto" lets the runtime decide whether a prior execution with the same inputs can be reused instead of recomputed. That is the durability story in one line: a task is a unit with an identity, not just a function call.

Composition uses asyncio.gather inside another task, so fan-out is plain Python concurrency. The README notes that a synchronous style also works via flyte.map, but states that async is recommended "for more control over concurrency and parallelism." The environment propagates to sub-tasks, which is how the Rust controller env var reaches child actions during a run.

Installing flyte-sdk and running a first task

The README gives a single install command. Python 3.10 or newer is required according to pyproject.toml.

pip install flyte

Write the example above into flyte_intro.py, then run it either as a plain Python file or through the CLI. The README shows both forms side by side.

python flyte_intro.py
flyte run flyte_intro.py main --data '[1,2,3]'

The CLI form passes the list as a JSON string to the main task. The README does not state what the CLI prints on success, so check the run output yourself rather than expecting a documented format.

For a richer local loop, the TUI is an optional extra. It is installed separately and enabled with a flag, and the --local flag keeps execution on your machine.

pip install flyte[tui]
flyte run --tui --local flyte_intro.py main --data '[1,2,3]'

If you want the full runtime locally, the README points at Flyte Devbox, a docker- and k3s-based environment. It starts with flyte start devbox, then requires a config file before you can run against it.

flyte start devbox
flyte create config \
    --endpoint localhost:30080 \
    --project flytesnacks \
    --domain development \
    --builder local \
    --insecure

After that, flyte run targets the devbox, and flyte get devbox reports run state, UI and registry endpoints, and the container image in use.

Serving a model from the same SDK

Flyte 2 does not stop at batch jobs. FastAPIAppEnvironment wraps a FastAPI app so it can be served through the same runtime. The README example defines the app, attaches it to the environment, and calls flyte.serve.

from fastapi import FastAPI
import flyte
from flyte.app.extras import FastAPIAppEnvironment

app = FastAPI()
env = FastAPIAppEnvironment(
    name="my-model",
    app=app,
    image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
        "fastapi", "uvicorn"
    ),
)

Note the image chaining: with_pip_packages adds fastapi and uvicorn to the base image, so the serving container carries its own dependencies. The CLI equivalent is flyte serve serving.py env.

This is the strongest argument for the SDK over a plain task runner. Training, batch inference and online inference share one task model and one image-building mechanism. The cost is that serving now depends on the same runtime and config as your pipelines, so a misconfigured endpoint breaks both.

The Rust controller is experimental and the README says so

The repository ships an alternative remote controller written in Rust, exposed to Python through maturin and pyo3, and distributed as a separate flyte_controller_base wheel so the main SDK keeps its build toolchain. It is optional: pip install flyte does not pull it in. The default Flyte task image bundles it, so tasks on that image work without the extra. You need flyte[rust-controller] only when running locally or bringing your own image.

pip install flyte[rust-controller]
_F_USE_RUST_CONTROLLER=1 python examples/basics/hello_v2.py

The env var is gated to 1, true or yes, and the driver propagates it to sub-task pods. The README is unusually candid about maturity, listing gaps: abort RPC on cancel, Code.ABORTED fast-fail, tunable retries and QPS, and graceful stop(). If your pipelines rely on prompt cancellation, that list is the thing to read before enabling the flag.

There is also a development constraint worth knowing: the wheel is not on PyPI until release, and the remote image builder installs all wheels in a layer at once, so it cannot resolve flyte_controller_base from a sibling layer. The README tells developers to switch to the local image builder in .flyte/config.yaml while working on it.

Flyte versus Airflow: different units of work

People searching for Flyte vs Airflow are usually asking whether they should replace a scheduler. The two do not model work the same way. Airflow orchestrates tasks as operators in a DAG, and the scheduler owns the graph. Flyte 2 starts from Python functions: a task is a decorated function, and the graph is expressed by calling tasks from other tasks, with asyncio.gather for fan-out.

That difference shows up in the type system. Flyte tasks carry typed inputs and outputs, and the environment carries the container image, so a task's dependencies travel with it. In Airflow, the equivalent wiring is spread across operator configuration and a separate environment setup.

Flyte 2 also folds model serving into the same SDK through FastAPIAppEnvironment, which Airflow does not attempt. The trade-off runs the other way too: Airflow has a much larger catalogue of prebuilt operators and connectors, while Flyte expects you to write the Python yourself. If your pipeline is mostly "call this SaaS API and wait," Airflow's operator library is the shorter path.

Licence, dependencies and what upgrades cost

The repository is Apache-2.0, which permits commercial use and modification with the usual notice and patent terms. That is a permissive licence, but it is not legal advice; read the LICENSE file for the actual text.

Upgrade cost is driven by pinned dependencies rather than by the licence. pyproject.toml pins flyteidl2==2.0.45 exactly, and caps connectrpc at >=0.9.0,<0.11 with a comment explaining that 0.11.0 turned RequestContext.request_headers into a property and broke the auth interceptor. Those pins mean an upgrade of the SDK can require coordinated bumps elsewhere.

The README also warns that important dependencies, notably flyteidl2, must be kept in lockstep across pyproject.toml, rs_controller/pyproject.toml and rs_controller/Cargo.toml. If you build from source or maintain a fork, that is manual work. The release cadence visible in the repository is fast: three releases between 2026-08-26 and 2026-08-27. The last push to main was on 2026-08-27.

Editorial conclusion

Adopt flyte-sdk if your team already runs Flyte or wants durable, retryable async Python tasks with a serving path in the same SDK. Skip it if you need only local parallelism, since asyncio and multiprocessing cover that without a control plane. Before committing, verify the connectrpc version pin in pyproject.toml, check whether the Rust controller gaps listed in the README (abort RPC on cancel, Code.ABORTED fast-fail) matter for your cancellation semantics, and confirm the default image builder setting for your environment.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes