databricks-sdk gives Python code a typed surface over the Lakehouse REST API

A generated client over the public REST API

The Databricks SDK for Python includes functionality to accelerate development with Python for the Databricks Lakehouse, and it covers all public Databricks REST API operations. That coverage is the point of the design. A REST API surface this wide is unpleasant to call by hand, because every operation means constructing a path, encoding query parameters, unpacking a JSON body, and deciding what to do with a paginated response or an error payload. A generated client turns those into named methods with typed arguments and typed results, and the repository shows the machinery that produces it, with a .codegen/ directory, a .codegen.json configuration file, a .package.json for the package layout and a .release_metadata.json. The documentation is hosted on readthedocs, the package is published to PyPI as databricks-sdk, and the project also points at the Java and Go SDKs and the Terraform provider, which tells you the APIs are the same and only the language differs.

Authentication that discovers itself

The authentication story is the part that saves the most code in practice. The documentation says that if you use Databricks configuration profiles or Databricks specific environment variables, the only code required to start working with a workspace is a client instantiation with no arguments, because the SDK then uses its default authentication flow. The flow is a search rather than a configuration, and the order is stated. The SDK tries Databricks native authentication first, then Azure native authentication, then Google Cloud Platform native authentication, and if none of those succeed it returns an authentication error and stops. The reason to do it this way is interoperability, and the documentation says so directly. If you run the Databricks Terraform Provider, the Databricks SDK for Go, the Databricks CLI, or applications targeting the SDKs in other languages, they will most likely all interoperate nicely together, which is only true if they share one credential resolution order. There are also sections on unified host support, on overriding the .databrickscfg file, and on additional configuration options, so a deployment with unusual networking is not left to the default path.

The boring parts of an API client, done for you

Three of the sections in the table of contents are the ones that determine whether a client is pleasant to live with. Paginated responses are handled by the SDK rather than by the caller, which is the difference between one call and a loop you have to get right. Retries are built in, and the documentation states that the internal HTTP client handles failures on different levels by performing intelligent retries, which is a claim about behaviour across the stack rather than a single request. Long running operations are the third, since starting a job on the Lakehouse returns something you have to poll, and the SDK exposes that as an operation you can wait on. The examples directory makes this concrete, with starting_job_and_waiting.py, flask_app_with_oauth.py and local_browser_oauth.py among the files, and the project points to other code that leans on it, naming Unity Catalog Automated Migration as heavily reliant on the Python SDK for working with Databricks APIs, and an IP access list analyzer that checks and prunes invalid entries from IP access lists. A client that is used by other people's production tooling has had its rough edges found already.

Installing it and the first client call

The install and the first call are short. The project description gives the install step and the client instantiation together:

1. Please install Databricks SDK for Python via `pip install databricks-sdk` and instantiate `WorkspaceClient`:

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
for c in w.clusters.list():
    print(c.cluster_name)

The WorkspaceClient object exposes one attribute per service area, and the documentation notes that the conventional name for the workspace level client is w, which is shorthand for workspace. There is a second snippet worth knowing about, because it is the interface to the auth section:

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
w. # press <TAB> for autocompletion

On version compatibility, the packaging metadata is the more current signal, with requires-python set to 3.10 or newer and classifiers for 3.10 through 3.13. An older line in the project description still carries about compatibility with Python 3.7 until June 2023 along with 3.8, 3.9, 3.10 and 3.11, so if you are on an older interpreter check the metadata rather than that sentence. There is also a runtime specific note. Databricks Runtime starting from version 13.1 includes a bundled version of the Python SDK, and the documentation recommends upgrading it in a notebook cell, followed by a restart of the Python interpreter:

%pip install --upgrade databricks-sdk

dbutils.library.restartPython()

Beta, but with a stated stability policy

The project calls itself a Beta and the documentation is careful about what that means. The SDK is supported for production use cases, but the project does expect future releases to have some interface changes, and there is a dedicated interface stability section that defines what those changes are. That combination is unusual and worth reading closely. A Beta label usually means do not use this in production, and here it means something narrower, namely that the interface may still move even though the code is supported for production. For a team with a test suite around their Databricks calls, that is a manageable risk. For a team whose Databricks access is a few lines in a script nobody tests, a minor version bump is an outage. The version history explains the pace. v0.152.0 on 2026-10-13, v0.153.0 on 2026-10-15 and v0.154.0 on 2026-10-19, with the last push to the repository on 2026-10-22, so releases are frequent and the version numbers are pre 1.0 while doing so. The repository is also a read-only mirror, published from Databricks internal repository with each release, and pull requests are reviewed there but merged internally, which tells you the release process is not something outside contributors can slow down.

Where a generated SDK is the wrong tool

There is a clear case for not choosing this. If your organisation has standardised on the Databricks CLI or the Terraform provider for automation, adding a third way to reach the same APIs widens your surface rather than narrowing it, and the documentation itself notes the interoperability with the Terraform Provider, the Go SDK and the CLI. That is a fact about credentials, but it is also a fact about the number of places a breaking change can land. The second case is a workspace you do not own. The SDK is designed for a Databricks Lakehouse, and if what you need is a generic REST client against a hundred services, a generated client for one vendor is not the answer even if it is a good one. The third is scale of surface. A client that covers all public REST API operations is large, and the package pulls in requests, google-auth, protobuf and urllib3, with a long list of excluded protobuf versions in the metadata that exists specifically to avoid vulnerable releases. That is careful work and it is also evidence that the dependency footprint of a full platform SDK is not trivial, and a smaller HTTP client for the two or three endpoints you actually call may be less to carry. A last practical point: the SDK is a client, and it still needs a workspace, credentials and network reachability, so none of this removes the operational work of running against a workspace you do not administer.

Editorial conclusion

Adopt databricks-sdk if your Python code talks to a Databricks workspace today over raw HTTP, because the generated client removes pagination handling, retry policy and long running operation polling from your code. Do not adopt it if your organisation standardises on the Databricks CLI or the Terraform provider instead, since the same APIs are reachable that way with a different operational model. Before you upgrade, read the interface stability notes, because the project is in Beta and expects interface changes in future releases, and pin the version rather than tracking the latest.

Frequently asked questions

Official sources

1. databricks/databricks-sdk-py on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases