pySarah: a TensorFlow toolkit for handwriting recognition and synthesis

What pySarah is for, and who ends up using it

Handwritten text recognition is usually assembled from parts: a preprocessing stage that cleans scans, a segmentation stage that finds lines or words, a recognition model that turns those crops into characters, and a post-processing stage that repairs the output. pySarah bundles all of those stages into one repository and exposes them through a single argument parser. The README describes the project as providing a solution for HTR using TensorFlow, with a tutorial and tools for data processing, model training, testing and inference. It also states that the HTR model can be trained on various datasets and supports different levels of recognition, and that generative and language models make up the workflow for handwriting synthesis and spelling correction.

The intended user is someone who already has a dataset in a known handwriting corpus format and wants to move from raw images to trained models without writing the data pipeline themselves. That is a narrower audience than a general OCR library serves. The parameter list assumes familiarity with text-level choices such as line versus word, with image normalization by character width, and with augmentation as a configurable set of named transforms. If those terms are unfamiliar, the CLI will be a wall of flags rather than a starting point.

The five model slots and how they connect

The architecture is best understood as five named roles rather than one monolithic model. The Models section of the parameter list defines them: --synthesis, --recognition, --segmentation, --writer-identification and --spelling. Each takes a model name, and the examples given are flor for the first four and openai for spelling. So the pipeline can synthesize handwriting, segment pages, recognize text, identify writers and correct spelling, and each role is selected independently.

The glue is MLflow. Each role also has a corresponding run identifier flag: --synthesis-run-id, --recognition-run-id, --segmentation-run-id and --writer-identification-run-id. Those flags accept a run id or an index, which means a stage can be pointed at a previously logged model rather than retrained. The --experiment-name flag groups runs, and --finished-runs restricts selection to runs that completed. That last flag is the clearest signal of how the project expects you to work: you train stages separately, log them, and then compose a pipeline by referencing run identifiers. The README notes that MLflow Tracking enables logging and comparing experiments, tracking metrics and storing trained models for reproducibility, and that the dashboard can be opened with mlflow ui.

Datasets, fonts and the input surface

Twelve datasets are listed as already integrated: Bentham, BRESSAY, CVL-Database, CVL-Digits, EMNIST, IAM, MNIST, ORAND-CAR, Parzival, RIMES, Saint Gall and Washington. Each entry links to a download location and a paper. The range is wide in script and era, from digit corpora such as MNIST and CVL-Digits to historical manuscripts such as Bentham and Saint Gall. The --source flag selects one by name, with iam given as the example.

Fonts are a second input path. The README states that folders of .ttf files placed under fonts/ are loaded by the batch generator for training and evaluation, and points to a collection of open fonts. That matters for synthesis specifically: a font prototype gives the generator a target style without needing paired handwriting samples in that style. The --enable-graph flag enables prototype graph inputs, and there are graph-specific augmentations (--graph-rotate, --graph-scale, --graph-noise) alongside the image augmentations. The README does not explain what a prototype graph represents or how it is constructed, which is the largest documentation gap in the input story.

Getting it running: the commands the README gives

Installation is four steps. Clone the repository, change into it, create a virtual environment with python3 -m venv .venv, activate it (source .venv/bin/activate on Linux or Mac, .venv\Scripts\activate on Windows), then run pip install -r requirements.txt. The stated Python constraint is >=3.11, <3.13, so 3.13 and later are excluded and 3.10 and earlier are excluded too. That is a tight window and worth checking before you spend time on the install.

Training is invoked through the same parser that defines every other flag, using --training to perform the training pipeline. The dataset flags that shape the run include --source, --text-level, --image-shape (height, width, channels), --char-width, --training-ratio, --validation-ratio and --test-ratio. Preprocessing options include --illumination for illumination compensation, --binarization for a binarization method, --lazy-mode for lazy loading, --mask-by-text and --order-by-text. Training itself is controlled by --epochs, --batch-size, --learning-rate and --training-step-factor. The synthesis workflow adds --discriminator-steps, --generator-steps and --monitor-samples, which the README describes as repetition of discriminator steps, skipping of generator steps and the number of sample images saved by the training monitor. The README text supplied here cuts off mid-sentence at --learning-rate, so the remaining training and inference flags are not visible.

Augmentation is a first-class configuration surface

The Augmentor section is the longest in the parameter list, and it is the clearest statement of what the maintainer thinks matters. Each transform takes a probability plus its own parameters: --mixup takes probability, opacity and iterations; --erode and --dilate take probability, kernel size and iterations; --elastic takes probability, kernel size and alpha; --perspective, --shear, --rotate, --scale, --shift-y and --shift-x each take probability and alpha. Noise and blur are covered by --salt-and-pepper, --gaussian-noise and --gaussian-blur, the last taking probability and kernel size. A single --disable-augmentation flag turns the whole set off.

This is a sensible design for HTR, where scanner variation, skew and ink bleed are the dominant sources of error, and where the amount of labeled handwriting is usually small relative to the number of parameters being trained. The trade-off is that these flags are numerous and interacting. Nothing in the supplied material gives recommended values, a preset, or a mapping from dataset to augmentation recipe, so tuning is left to the user. The graph augmentations are grouped with the image ones but operate on a different input type, which makes the flat list harder to reason about than it needs to be.

Where pySarah is the wrong tool

The project is tied to its corpus list. Recognition models are trained per --source, and the preprocessing flags (image shape, character width, binarization, illumination) are tuned to the page geometry of those corpora. If your input is photographed forms, receipts, or mixed print and handwriting, none of the twelve datasets resembles that distribution and the pipeline gives you no layout analysis stage to fall back on. Segmentation exists as a model slot, but the README does not describe its output format or how it hands regions to recognition.

There are no releases. The repository metadata shows no releases retrieved, so there is no tagged version to pin and no changelog to consult. The only installation path documented is cloning master and installing requirements.txt, which means your build tracks the default branch. For a research workflow that is acceptable. For anything that needs a reproducible dependency set across machines, it is a liability.

The documentation supplied here is also incomplete. The README ends mid-sentence in the Training section, and there is no visible inference or testing command, no description of output formats, and no accuracy figures. The homepage points to a Medium article, which suggests the tutorial lives outside the repository. Anyone evaluating this should read that article before assuming the README is the whole story.

A comparison worth making: Tesseract and Kraken

The obvious alternative for text extraction is Tesseract, which ships as a packaged OCR engine with language models and a stable command-line interface. The difference in approach is fundamental. Tesseract applies a pretrained model to an image and returns text; it does not ask you to supply a corpus, choose a text level, or train a recognizer. pySarah does the opposite. It is a training framework first, and inference is the last stage of a workflow you assemble from logged MLflow runs. If you want text out of a scan today, Tesseract is the shorter path. If you want a recognizer adapted to a specific hand or a specific historical script, pySarah's dataset-and-training orientation is the point.

A closer comparison is Kraken, which also targets historical handwriting and also expects you to train or fine-tune on transcribed pages. Both treat the corpus as the unit of work. pySarah's distinguishing feature relative to that class of tool is the synthesis side: the --synthesis model and the fonts/ input path mean the same repository can generate handwriting as well as read it, and the --spelling model with openai as its example name adds a correction stage. That combination, synthesis plus recognition plus correction under one parser, is what the repository is actually selling.

Maintenance cost, licensing and what to check first

The license is MIT, which permits commercial use and modification provided the copyright notice and permission notice are retained. That is a permissive choice, but it says nothing about the datasets. Each corpus is linked to its own download location and paper, and those datasets carry their own terms, which the README does not summarize. If you plan to train on IAM, RIMES or Bentham and ship the resulting model, the dataset license is the constraint to check, not the repository license. This is not legal advice; read the source terms for whichever corpus you pick.

Upgrade cost is dominated by the Python window and the absence of releases. The stated range is >=3.11, <3.13, so a jump to 3.13 will require either a code change or a pinned interpreter, and there is no version tag to fall back to when master moves. Dependencies are unpinned as far as the supplied material shows, since the only install instruction is pip install -r requirements.txt. The MLflow run store adds a second thing to maintain: run identifiers are part of your pipeline configuration, so deleting or moving an experiment directory breaks the flags that reference it.

Before committing, verify three concrete things. Confirm your interpreter satisfies the >=3.11, <3.13 constraint. Confirm that the run you intend to reference with --recognition-run-id or --synthesis-run-id is actually finished, or pass --finished-runs so that selection cannot land on a partial run. And confirm that your target script or page geometry matches one of the twelve integrated sources, because the preprocessing flags are written against those corpora and there is no documented path for adapting them to a different layout.

Editorial conclusion

Adopt pySarah if you are working with line-level or word-level HTR on one of the twelve bundled datasets and you want synthesis, recognition and spelling correction sharing a single CLI and MLflow run store. Do not adopt it if you need a supported API surface, a released version to pin, or recognition on document layouts outside the integrated corpora. Before committing, verify two things on your own data: that the Python >=3.11, <3.13 constraint matches your environment, and that the recognition run you intend to reuse was logged as finished, since the --finished-runs flag exists precisely because unfinished runs are selectable otherwise.

Official sources

1. arthurflor23/handwritten-text-recognition on GitHub
2. Issues
3. License: MIT
4. Project website
5. README