SimpleHTR: A Minimal TensorFlow Baseline for Handwritten Word and Line Recognition
Handwritten Text Recognition (HTR) system implemented with TensorFlow.
At a glance
- What is it?
- SimpleHTR is a stripped-down TensorFlow implementation of a CNN-LSTM-CTC handwriting recognizer trained on IAM. It is a readable reference and a baseline, not a production OCR stack, and it only reads words or single lines.
- Who is it for?
- Adopt SimpleHTR if you need a small, readable TensorFlow reference for line-level handwriting recognition, or a baseline to compare a custom model against on IAM. Do not adopt it if your input is a scanned page, a form, or a photographed document: the README states the model takes images of single words or text lines, and full-page reading is handled by the separate HTRPipeline repository.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 82 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap SimpleHTR fills: a readable CNN-LSTM-CTC baseline
Most handwriting recognition code you find online is either a wrapper around a hosted API or a research repository with the training loop buried under configuration layers. SimpleHTR sits in between. The README describes it as a stripped-down version of the HTR system the author implemented for his thesis, keeping what it calls the bare minimum to recognize text with an acceptable accuracy. That framing is honest and it sets expectations correctly. The model is five CNN layers, two LSTM layers, and a CTC loss and decoding stage. Nothing else. The audience is therefore narrow: someone who wants to read a complete HTR pipeline end to end, or who needs a known baseline to measure a different architecture against. The README reports that three quarters of the words from the validation set are correctly recognized and that the character error rate is around 10 percent. Those numbers apply to IAM and to the pretrained models, not to arbitrary handwriting.
Input shape is the constraint that decides everything
The model accepts an image of a single word or an image of a text line containing multiple words, and returns the recognized string. That is the whole contract. Line mode was added in the 2021/2 update, and the README is explicit that the word model only handles single words per image while the line model can handle multiple words. There is no page segmentation, no line detection, no deskewing, no layout analysis in this repository. If you feed it a full page, nothing in the documentation suggests it will find the lines for you. The README points to the separate HTRPipeline repository for reading full pages, which tells you the author treats page-level work as a different problem with a different codebase. Anyone evaluating SimpleHTR for a document digitization project should read that as a boundary, not a missing feature.
Running inference with the pretrained snapshots
The README gives a two-step path. Download either the word model or the line model from the Dropbox links, put the contents of the zip file into the model directory of the repository, then change into src and run the inference script. For a word image the command is python main.py. For a text line the command is python main.py --img_file ../data/line.png. The README shows the expected console output: an Init line naming the snapshot, a Recognized line with the text, and a Probability value. The sample word run reports a probability of roughly 0.98 and the sample line run roughly 0.67, which is a useful reminder that line-level confidence is lower even on the author's own example. The --img_file argument is how you point inference at your own image, and --mode defaults to infer, so you do not need to pass it for a quick check.
Training on IAM: dataset preparation and the real cost
Training requires the IAM off-line HTR dataset, which you obtain by registering at the website linked in the README, downloading words/words.tgz and ascii/words.txt, then creating a dataset directory with img and gt subdirectories. words.txt goes in gt, and the contents of words.tgz (the a01, a02 directories and so on) go in img. Training is then python main.py --mode train --data_dir path/to/IAM. The README states the dataset is split 95 percent training and 5 percent validation, and that training stops after a fixed number of epochs without improvement. The two commands the author used for the published models are given verbatim: the word model with --fast, --batch_size 500 and --early_stopping 15, and the line model with --fast, --batch_size 250 and --early_stopping 10. The README says training on single words with --fast on a GTX 1050 Ti takes around 3 hours, and that line training takes a bit longer. That is the maintenance cost you are signing up for: a few hours of GPU time per retrain, plus the IAM registration step, which is manual and cannot be scripted.
The fast path: LMDB, SSD, and why it exists
The README is unusually direct about the bottleneck. Loading and decoding the PNG files from disk is the bottleneck even when using only a small GPU, so the project uses LMDB to speed up image loading. You run create_lmdb.py --data_dir path/to/iam from src, which creates an lmdb subfolder inside the IAM data directory, and then add --fast when training. The README also states the dataset should be located on an SSD drive. This is a concrete operational detail that is easy to miss when reading the command list: --fast is not a generic speed knob, it changes where images come from, and it only works after create_lmdb.py has produced the database. If you skip the LMDB step and pass --fast anyway, the documentation gives no fallback behaviour, so treat the two steps as a pair.
Word beam search and the dictionary trade-off
Three CTC decoders are selectable through --decoder: bestpath (the default), beamsearch, and wordbeamsearch. The word beam search decoder is not shipped with TensorFlow. You clone the separate CTCWordBeamSearch repository, run pip install . at its root, and then pass --decoder wordbeamsearch. The mechanism matters: words are constrained to those contained in a dictionary, while arbitrary non-word character strings such as numbers and punctuation marks can still be recognized. The README says the dictionary is built automatically during training and validation from all words in the IAM dataset, including words from the validation set, and saved to data/corpus.txt, with the word-character list in model/wordCharList.txt. Beam width is set to 50 to match vanilla beam search. The trade-off is plain: a dictionary built from IAM will not contain your domain vocabulary, and the README does not describe a supported way to substitute your own corpus, only where the file lives. For a fixed vocabulary such as a form field, that constraint is an advantage. For free text with names or technical terms, it is a liability.
Where SimpleHTR is the wrong tool, and what to compare it against
SimpleHTR is the wrong tool when your input is a page, a form, or a photograph of a document. It has no line segmentation, so a full-page scan has to be cut into lines by something else before the model ever sees it. It is also the wrong tool if you need a supported package with a stable API: the README points readers to HTRPipeline for full-page reading, which is the author's own signal that SimpleHTR is a model repository rather than a document pipeline. As an alternative, Tesseract occupies a genuinely different position. Tesseract is a complete OCR engine with its own layout analysis and line finding, and it ships as a system binary with language data, whereas SimpleHTR is a Python and TensorFlow training and inference script that expects pre-cropped word or line images. The difference is not accuracy on handwriting, which the README does not compare; it is that Tesseract solves the segmentation problem and SimpleHTR assumes you have already solved it. If your pipeline already produces clean line crops, SimpleHTR is a reasonable model to drop in. If it does not, you are choosing the harder half of the problem.
Licence, maintenance, and what to verify before you commit
The repository is MIT licensed, which permits commercial use and modification provided the copyright notice and permission notice are retained. That is a permissive arrangement, and it removes the licensing question that usually surrounds OCR engines. It does not remove the dependency questions: TensorFlow version compatibility is your problem, and the README notes only that the code is compatible with TF2 following the 2020 update. There are no releases listed for this repository, so there is no versioned artifact to pin against; you are tracking the master branch. The pretrained models are hosted on Dropbox links in the README, which is a fragile distribution channel for anything you intend to depend on long term. Before adopting, confirm that you can reach the IAM registration page, that the model zip you download matches the mode you plan to run, and that you have a plan for the dictionary file if you enable wordbeamsearch. The web demo linked in the README is the fastest way to see the model's behaviour on an image before you install anything.
Editorial conclusion
Adopt SimpleHTR if you need a small, readable TensorFlow reference for line-level handwriting recognition, or a baseline to compare a custom model against on IAM. Do not adopt it if your input is a scanned page, a form, or a photographed document: the README states the model takes images of single words or text lines, and full-page reading is handled by the separate HTRPipeline repository. Before committing, verify three things yourself: that you can obtain the IAM dataset through the registration page linked in the README, that the pretrained snapshot in the model directory matches the mode you intend to run (word model versus line model), and what character error rate the model reaches on your own scan quality rather than on IAM.
Community notes