# dataframely puts schema validation inside Polars instead of beside it

> dataframely is a BSD-3-Clause library from Quantco that validates Polars data frames against a declarative schema and returns a typed DataFrame, with a Rust core built through maturin. The schema doubles as a type hint, which is the part that changes how a pipeline reads.

**Quantco/dataframely** — A declarative, 🐻‍❄️-native data frame validation library.

- Repository: https://github.com/Quantco/dataframely
- Website: https://dataframely.readthedocs.io
- Stars: 620 · Forks: 21
- Language: Python
- License: BSD-3-Clause
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/quantco-dataframely

## A schema class is both the rule set and the type annotation

The central design decision is that one class does two jobs. It declares what the data must look like, and it is used as the type of the validated frame, which is how the documentation describes the goal of making pipelines more readable by adding schema information to data frame type hints. The declaration is a subclass of dy.Schema with one attribute per column, each carrying a dataframely type.

```python
import dataframely as dy
import polars as pl


class HouseSchema(dy.Schema):
    zip_code = dy.String(nullable=False, min_length=3)
    num_bedrooms = dy.UInt8(nullable=False)
    num_bathrooms = dy.UInt8(nullable=False)
    price = dy.Float64(nullable=False)
```

The types are Polars native, so dy.String and dy.UInt8 and dy.Float64 correspond to what a Polars user already knows, and the nullable flag and constraints are declared on the column rather than checked after the fact. The alternative approach to the same problem is to write a validation function that takes a frame and returns a boolean or raises, and the difference shows up in how a pipeline reads. Here the contract is a type, so a reader of the code sees which schema a frame is expected to satisfy, and a checker can see it too.

## Rules cover what column types cannot express

Column types handle nullability, length and range, but a good chunk of real data quality is about relationships between columns, and that is what the rule mechanism is for. A rule is a method on the schema class decorated with dy.rule(), returning a Polars expression, and the classmethod form is what the ruff configuration expects, since the pyproject lists dataframely.rule among the classmethod decorators.

```python
    @dy.rule()
    def reasonable_bathroom_to_bedroom_ratio(cls) -> pl.Expr:
        ratio = pl.col("num_bathrooms") / pl.col("num_bedrooms")
        return (ratio >= 1 / 3) & (ratio <= 3)

    @dy.rule()
    def minimum_zip_code_count(cls) -> pl.Expr:
        return (pl.len() >= 2).over("zip_code")
```

The first rule encodes a plausibility constraint on a ratio, and the second uses a window expression to require a minimum count per group, which is the kind of check that would otherwise need a groupby and a join. Returning a Polars expression rather than a Python predicate is what keeps the evaluation in the dataframe engine, so the checks are vectorised rather than row by row. The rule that a zip code must appear at least twice is a good illustration of the kind of business invariant a schema library can hold that a type system cannot.

## Validation returns a typed frame and can cast on the way

The call site is a single method on the schema class, and the parameter that matters is cast.

```python
validated_df: dy.DataFrame[HouseSchema] = HouseSchema.validate(df, cast=True)
```

The annotation on the left is the payoff of the whole design, since the variable is not an untyped frame but a DataFrame parameterised by HouseSchema. The cast=True argument means the columns are cast to the expected types during validation, so a column that arrived as something else is converted rather than rejected. Whether that is the behaviour you want is a decision, not a detail, because a cast will happily turn a wrong type into a plausible one. The example frame the documentation builds deliberately contains bad rows, a zip code of a single character, null bedroom counts and a bathroom count of zero, so a run against that data is a good first check of how strict the default behaviour is. The documentation points to the guide at dataframely.readthedocs.io for more advanced usage examples.

## The Rust core is pinned to a specific Polars revision

The build metadata explains where the speed comes from and also where the friction is. The Python package is built with maturin, and the native module is exposed as dataframely._native, which means there is a compiled extension in the install rather than pure Python. The Cargo manifest names the crate dataframely with a cdylib crate type and an edition of 2021, and the release profile sets codegen-units to 1 with link time optimisation enabled, which is the usual sign of a library optimised for runtime rather than build time. The dependencies are the interesting part. Polars and polars-arrow and polars-core are required at 0.50 or newer with the dtype-full feature, pyo3-polars at 0.23.1 or newer brings the derive and dtype-full features, and pyo3 is configured with the abi3-py311 and extension-module features. Then the patch section pins polars, polars-arrow, polars-core and pyo3-polars to a specific git revision of the Polars repository, with a comment explaining that the Polars crates are kept on the same revision as pyo3-polars for compatible Rust types, and a FIXME noting the intent to switch back to a pyo3-polars release once an upstream pull request is released. A library that patches its core dependency to a git commit is a library where upgrade planning is a real task.

## Packaging, tooling and the shape of the repository

The pyproject declares Python 3.11 or newer, classifiers for 3.11 through 3.14, and dependencies of numpy and polars 1.35 or newer, with optional extras for pydantic 2 and for sqlalchemy. That the extras exist at all tells you the intended integration points, since a pydantic extra suggests the schemas can be tied to Pydantic models and the sqlalchemy extra suggests data can be pushed into a database layer with the schema in hand. The repository is a hybrid project, with a Python package directory alongside src/, a Cargo.toml, a rust-toolchain.toml, a Cargo.lock, a pixi.toml and pixi.lock, a docker-compose.yml, a lefthook configuration and a .prettierrc, plus docs, tests and skills directories. The docker-compose file is a single service, an azure-sql-edge image with an sa user and a password and a port mapping of 1455 to 1433, which is the kind of thing you would spin up to test the SQL side. The README badges show both a CI workflow and a nightly workflow, a conda-forge package alongside the PyPI one, and codecov coverage reporting.

## When the extra machinery is not worth it

There are cases where this library is the wrong shape of answer. If your pipeline is built on pandas, nothing here applies, because the whole library is written against Polars data frames and its types are Polars types, and there is no pandas backend described. If your data is small and the checks are simple, a few assert statements in a Polars pipeline will do the same job without a schema class, a compiled extension and a version alignment between two Polars revisions. If your validation needs to involve a reference table, a fuzzy match or a statistical outlier rule, you will end up writing that check outside the schema anyway, and the schema then covers only part of your rules. The honest comparison is with writing validation by hand in Polars expressions, and the difference in approach is that dataframely makes the contract explicit and typed and gives you a place to hang the rules that a column type cannot express. If your team already has a house style for this, the gain is smaller. The licence is BSD-3-Clause, which is permissive, and the last push to the main branch was on 2026-10-09 with version 3.1.2 released on 2026-09-20, so the project is being worked on at a steady rate.

## Conclusion

dataframely suits Polars users who want schema rules written once and enforced as data, with the schema usable as a type annotation, and it suits pipelines that need to reject bad rows rather than quietly coerce them. It does not suit teams on pandas, and it does not suit projects that want validation to be a separate step outside the dataframe library. Before adopting it, install it with pip install dataframely, define a schema with dy.rule() for the invariants a column type cannot express, and check the supported Python and Polars versions against your environment, since the project requires Python 3.11 or newer and Polars 1.35 or newer.

## FAQ

### How do I install dataframely?

Install it with the package manager of your choice, for example pixi add dataframely or pip install dataframely. The package requires Python 3.11 or newer and depends on numpy and polars 1.35 or newer.

### What is a dataframely rule?

A rule is a method on your schema class decorated with dy.rule() that returns a Polars expression, so you can express constraints that a column type cannot, such as a ratio between two columns or a minimum count per group.

### Does dataframely cast columns during validation?

Yes, when you pass cast=True to the validate method, the columns are cast to the expected types as part of validation. The documentation example calls HouseSchema.validate(df, cast=True).

### Is dataframely written in pure Python?

No. The package is built with maturin and exposes a native module named dataframely._native, backed by a Rust crate that depends on polars, polars-arrow, polars-core and pyo3-polars.

### What licence is dataframely released under?

The project is licensed under BSD-3-Clause, with the licence text in the LICENSE file at the top of the repository.

## Sources

- [License: BSD-3-Clause](https://github.com/Quantco/dataframely/blob/main/LICENSE)
- [Project website](https://dataframely.readthedocs.io)
- [Quantco/dataframely on GitHub](https://github.com/Quantco/dataframely)
- [README](https://github.com/Quantco/dataframely/blob/main/README.md)
- [Releases](https://github.com/Quantco/dataframely/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/quantco-dataframely