Quinn fills in the small PySpark functions you keep rewriting

A helper library, not a framework

Quinn describes itself as PySpark helper methods to maximize developer productivity, and the summary is precise about the scope. It provides DataFrame validation functions, useful column functions and DataFrame transformations, and performant helper functions. What it does not provide is a new way to work with Spark. Every function here is a wrapper that returns or takes a PySpark Column, so the library sits beside your existing job code rather than around it. The install is a single PyPI package.

pip install quinn

and the import is a single line.

import quinn

The packaging metadata makes one point worth noting for an adoption decision. The main dependencies section lists only a Python constraint of >=3.7,<4.0, with no runtime library dependencies at all, and everything else sits in optional groups for development, testing, linting and docs. That means importing quinn does not drag in a second version of anything, which matters in the Spark ecosystem where version conflicts with pyspark itself are the usual cause of a broken environment. The library is a thin, dependency-free layer that happens to talk to Spark, and the Spark dependency only appears in the development and testing groups, where pyspark is pinned at >2.

Validations that raise instead of silently returning null

The validation group is the part with the clearest value proposition, because it converts a class of silent data bug into an exception. There are three functions. validate_presence_of_columns() raises an exception unless the source DataFrame contains the named columns, which catches a renamed or dropped upstream field before it becomes a wall of nulls downstream.

quinn.validate_presence_of_columns(source_df, ["name", "age", "fun"])

validate_schema() raises unless the DataFrame contains all the StructFields defined in the required schema, which is the stricter version and the one to use when a contract has a fixed shape rather than a fixed column list.

quinn.validate_schema(source_df, required_schema)

validate_absence_of_columns() is the inverse check, raising if the DataFrame contains the listed columns, which is useful when a downstream consumer must not receive a field that carries personal data. That set of three covers the assertions teams write most often at the top of a pipeline, and the design choice is deliberate. These functions raise rather than return a boolean or a count, so a validation failure stops the job. That is the right behaviour for a data contract, and the wrong behaviour for a report that should degrade gracefully, so the calls belong at pipeline boundaries rather than in the middle of a transformation.

Four string functions with different trimming semantics

The string helpers look repetitive until you read the examples, because each one handles whitespace differently and the differences matter. single_space() replaces all multiple spaces with a single space, so "this has some" becomes "this has some".

actual_df = source_df.withColumn(
    "words_single_spaced",
    quinn.single_space(col("words"))
)

remove_all_whitespace() strips every whitespace character, turning "this has some" into "thishassome". anti_trim() removes inner whitespace but keeps leading and trailing whitespace, so " this has some " becomes " thishassome ". remove_non_word_characters() drops every non-word character, so "si%$#@!#$!@#mpsons" becomes "simpsons". The reason this set exists is worth stating plainly, because it explains the whole library. These are the normalisations a text column needs before tokenisation, before a name lookup, or before comparing two strings that should be equal but differ in spacing. The last one in particular, remove_non_word_characters, is the kind of thing teams end up copying between codebases, and the fact that quinn gives it a name and a test is most of the value. The helper also provides multi_equals(), which returns true when two columns are both equal to a given value, and approx_equal(), which compares two DataFrames column by column against an integer threshold, a function that is close to indispensable when you are asserting that a transform preserved its input.

Date helpers, UUIDs and regex extraction

Beyond the string functions there are several helpers that are less obvious but save real work. week_start_date() takes a column and a week_start_day, with Sun as the default, and returns a column holding the start date of the week, so several dates in the same week collapse to one value and can be grouped on. week_end_date() is the mirror image with a default of sat, and the two together give you a week key without a date_trunc or a custom expression. uuid5() generates a UUIDv5 in string form from a passed column, with the DNS namespace by default and an optional namespace and extra salt to reduce hash collisions, which is what you want for a stable surrogate key that is reproducible across reruns. The determinism matters more than it sounds, because a random UUID column cannot be used to join a fact table back to a dimension table after a rerun. regexp_extract_all() takes a string and a regular expression and returns every match, so the pattern can pull all words containing a given letter out of a free text column in one call. array_choice() takes a Column and returns a column containing a random value from it, which is a small tool for sampling and for test data generation.

Transformations that reshape the frame itself

The transformations group operates on the DataFrame rather than on a column, and both functions address a specific irritation. snake_case_col_names() converts all column names to snake_case, which the README frames plainly as annoying to write SQL queries when columns are not snake cased.

quinn.snake_case_col_names(source_df)

sort_columns() sorts the DataFrame columns in alphabetical order, including nested columns when sort_nested is set to True.

quinn.sort_columns(df=source_df, sort_order="asc", sort_nested=True)

The stated reason is that wide DataFrames are easier to navigate when they are sorted alphabetically, which is a workflow argument rather than a performance one. There is also a helper for renaming, with_columns_renamed(), which renames all or multiple columns in a DataFrame by implementing a common renaming logic, and the README illustrates the motivation with two order DataFrames arriving from source A and source B with the same column names and different values in the store_id column. A shared rename function means the two frames line up without each ingestion path hardcoding its own mapping. These transformations are the kind of code that is trivial to write and tedious to get consistent, which is exactly the category a shared library is good at.

How the project is built and tested

The repository layout shows a package with real engineering around it. The quinn/ directory holds the source, tests/ the test suite, benchmarks/ a performance comparison area, docs/ the documentation sources with an mkdocs.yml at the root, and a Makefile with the common development commands. The Makefile is short and shows the workflow.

test:
	@poetry run pytest tests

The install targets use poetry install with different dependency group combinations, so a contributor can set up just the development and testing groups or everything including linting and docs. Linting is ruff, run as ruff check --fix against the quinn package, and the docs stack is MkDocs with mkdocstrings for API reference, the Material theme, jupyterlab and a macros plugin, which explains how the documentation pages include executable examples. The testing group is worth a closer look because it shows how a PySpark library is tested at all. It uses pytest with pytest-rerunfailures, since Spark jobs are flaky by nature, pytest-describe for readable test naming, and chispa pinned at 0.9.4, a library of PySpark DataFrame comparison helpers, which is how you assert on DataFrame contents without writing brittle row-by-row checks. The build uses poetry with a build backend of poetry.masonry.api, and the project carries a .licenserc.yaml and a pre-commit configuration.

The maintenance question and where quinn fits

The release history is the thing to weigh. The most recent version on PyPI is 0.10.3 from 2025-02-26, preceded by 0.10.2 in February 2024 and 0.10.1 days earlier, and the last push to the repository was on 2026-06-09. So the code is being touched, but the version line is still in the 0.x range, and there is a long gap between 2024 and 2026 with no release at all. For a package whose entire job is convenience functions over Column expressions, that is a mild concern rather than a disqualifying one, since the functions are small, the API surface is stable by nature, and the code is Apache 2.0 licensed so a fork is straightforward. The realistic risk is not that quinn breaks, it is that a Spark API change makes a specific helper quietly wrong. The alternative to adopting it is writing the same helpers yourself, which many teams already do, and the difference in effort between maintaining your own utils module and depending on this one is small either way. The real decision is about consistency across a team, and about whether the helper list matches your own. If your pipeline is mostly SQL with only a little Python, quinn will not earn its place. If your Python jobs are full of these small string, validation and date operations, it will, and the cost of finding out is one pip install and a test suite to compare against.

Editorial conclusion

Adopt quinn if your team already uses PySpark and keeps writing the same DataFrame assertions and string normalisation by hand, since the library is a small, well tested helper layer with no runtime dependencies beyond Python itself. Skip it if your pipeline is SQL only, or if your team would rather own those helpers, since the functions are thin wrappers over Column expressions and a fork costs little. Before you adopt it, read the string function list closely, because the trimming and whitespace helpers have subtly different semantics, and check the maintenance record, since the last push was on 2026-06-09 and the most recent release, v0.10.3, dates from 2025-02-26.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. mrpowers-io/quinn on GitHub
3. Project website
4. README
5. Releases