fg-data-quality: a pandas DataFrame quality engine with a one-call evaluate()
Data Quality assessment with one line of code
At a glance
- What is it?
- fg-data-quality bundles duplicate detection, collinearity checks, erroneous-value scans and missing-data profiling behind a single DataQuality object that returns a prioritised warning list. It is a fast first pass over a DataFrame, not a validation framework, and its licence situation needs checking before adoption.
- Who is it for?
- Adopt fg-data-quality if you already work in pandas and want a prioritised warning list before a modelling or feature-engineering step; skip it if you need column-level schema contracts enforced in CI, or if your data does not fit in memory.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 146 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap fg-data-quality fills: a single entry point over many checks
Most quality checks live in separate libraries. Missing-value profiling is one call, correlation analysis is another, duplicate detection is a third, and each returns a differently shaped object. fg-data-quality puts them behind one class. The README describes it as an engine that evaluates data "in a modular way wrapped into a single Data Quality engine", and the module list in the tutorials directory backs that up: bias and fairness, data expectations, data relations, drift, duplicates, labelling (split into categorical and numerical notebooks), missings, and erroneous data. The intended user is an engineer or analyst sitting in a pandas workflow who wants a broad sweep before feature engineering or model training, and who does not want to wire up eight separate checks by hand. The output is not a pass or fail. It is a warning summary grouped by priority, where priority 1 means "heavy impact expected" and priority 2 means "usage allowed, limited human intelligibility". That framing matters: the library assumes a human will read the warnings and decide, which makes it a triage tool rather than a gate.
What evaluate() actually returns, and why the priority tiers are the interesting part
The quickstart shows the whole flow in four lines: read a CSV into a DataFrame, construct DataQuality(df=df), call dq.evaluate(), and read the printed summary. The sample output in the README lists five warnings split as one priority 1 and four priority 2. The priority 1 item is a duplicate column: one column whose values are exactly the same as another column's. The priority 2 items are three numerical variables with a Variance Inflation Factor above 5.0, ten categorical variables with significant collinearity at a p-value below 0.05, 1,960 erroneous values matched against a predefined list, and three exact duplicate rows. Two design choices stand out. First, the collinearity checks are split by dtype: numerical columns go through VIF, categorical columns go through a significance test, and the categorical results are sorted in descending order of propensity so you can drop variables from the top. Second, the warning text carries the remediation hint inline. The VIF warning says high collinearity "will make model explainability harder and potentially give way to issues like overfitting", and then tells you to consider removing the highest VIF variables. That is more useful than a bare metric, though it also means the library is opinionated about what counts as a problem, and those opinions are baked into thresholds you did not choose.
Getting it running, and the import rename that breaks existing code
Installation is a single command: pip install fg-data-quality. The package is published on PyPI, and the README shows Python 3.6, 3.7 and 3.8 badges, so anything newer is untested territory as far as the documentation goes. The rename from the earlier ydata-quality distribution is covered by a migration section, and it is worth reading before you upgrade, because the import path changed. The old code is import ydata_quality, the new code is import data_quality, and both expose DataQuality from the same module path. The README gives a grep to find affected files: grep -r "ydata_quality" . --include="*.py". The migration steps are pip uninstall ydata-quality, then pip install fg-data-quality, then the find-and-replace. If you have pinned ydata-quality in a requirements file or a lockfile, that pin will not resolve to the new name, and no compatibility shim is mentioned. There is no configuration file, no CLI, and no environment variables in the supplied material. Everything is driven from Python: you construct the object with your DataFrame and call methods on it. The README also shows dq.get_warnings() as the way to pull the warnings as a list instead of reading the printed summary, which is the path you would take if you wanted to log them or attach them to a run record.
The DataFrame-in-memory constraint and the licence mismatch
The constructor takes a pandas DataFrame. That is the whole input surface described in the material. There is no mention of a connection string, a file path argument, a chunked reader, or a Spark or Dask backend. If your table does not fit comfortably in memory alongside whatever you are doing with it, this library is the wrong tool, and no amount of tuning changes that. The second constraint is licensing, and it is not a small one. The README's License section links to GNU General Public License v3.0, while the repository metadata supplied for this review says MIT. Those are very different terms, and the discrepancy is unresolved in the material available. GPL-3.0 carries copyleft obligations that MIT does not. I am not giving legal advice, and I cannot tell you which one is authoritative from here. What I can say is that the two sources disagree, that the README text is the more specific claim, and that anyone embedding this in a distributed product should read the LICENSE file in the repository directly rather than trusting either the badge or the metadata line.
Where it stops: no schema contract, no CI gate, no streaming
The data expectations module is the closest thing to a contract mechanism, and the README does not describe how expectations are declared or what happens when one fails. It lists a tutorial notebook and nothing else. That is a real gap for anyone hoping to use this as a build-time check. Compare the shape of the output: a warning summary with priorities is designed for a person to read once and act on. It has no exit code, no threshold configuration shown, and no documented way to say "this column must never contain nulls, fail the build if it does". If that is what you need, you want a declarative validation library instead, where expectations are written as code and a validation run returns a boolean you can branch on. The other boundary is state. Drift analysis implies comparing two datasets, and the README includes a drift tutorial, but the quickstart only ever constructs DataQuality from one DataFrame. How the reference and comparison frames are supplied is not shown in the material. Do not assume a two-frame API exists until you have read tutorials/drift.ipynb.
The honest comparison: fg-data-quality versus Great Expectations
Great Expectations and fg-data-quality are often reached for in the same situation and they solve different problems. Great Expectations is a validation framework. You declare Expectations as code, such as a column being non-null or a value falling inside a range, and a validation run produces a structured result you can branch on in a pipeline. The contract is written down before the data arrives, and the framework checks the data against it. fg-data-quality works the other way round. You hand it a DataFrame, it profiles that DataFrame, and it tells you what looks off using heuristics it chose: VIF above 5.0 for numerical collinearity, p-value below 0.05 for categorical, exact value matching for duplicates. Nothing is declared in advance, which means nothing is enforced. The trade is real in both directions. fg-data-quality needs one line to produce a useful first look at an unfamiliar dataset, and Great Expectations needs a suite of expectation definitions before it produces anything at all. But Great Expectations can fail a build, and fg-data-quality, based on the material here, cannot. If you are onboarding a new dataset and want to know what is in it, the profiling approach wins. If you are protecting a production pipeline from a schema change, the declaration approach wins.
Maintenance, versions and what the release history suggests
Three releases are listed: 0.1.1, 0.1.2 and 0.2.0, all dated 2026-04-23, with the 0.1.x pair landing within about twelve minutes of each other. That pattern looks like packaging fixes rather than feature work, and the gap to 0.2.0 is where the substantive change presumably sits. The project is not archived, and the last push matches the 0.2.0 release date. The version number itself is the useful signal: at 0.2.x, the public API is still moving, and the ydata-quality to fg-data-quality rename is evidence that it has already moved once. Pin the version in your requirements file and read the changelog before bumping. The README points contributors at issues labelled good first issue and at a Discord server for support and feature requests, which tells you where answers come from: a community channel, not a paid support contract. Budget for that. If your team cannot absorb a breaking rename in the import path, this is a dependency to isolate behind a thin wrapper of your own rather than call directly from twenty files.
Editorial conclusion
Adopt fg-data-quality if you already work in pandas and want a prioritised warning list before a modelling or feature-engineering step; skip it if you need column-level schema contracts enforced in CI, or if your data does not fit in memory. Before writing it into a pipeline, verify two things: the actual licence text in the repository, since the README states GNU General Public License v3.0 while the repository metadata says MIT, and the module behaviour on your own columns by running the tutorials/main.ipynb walkthrough against a sample of your data rather than trusting the census_10k example output.
Community notes