ASRT Speech Recognition: a Keras and CTC Chinese ASR you train yourself

The gap ASRT fills: Mandarin ASR you can retrain

Most speech recognition products arrive as an API. You send audio, you get text, and you have no way to adapt the acoustic model to a call centre in Sichuan, a set of medical dictation recordings, or a noisy factory floor. ASRT is the opposite arrangement. The repository ships training scripts, a data loader, a datalist format and a configuration file, so the model is something you produce rather than something you rent. The README states the project is built on tensorFlow.keras with a deep convolutional neural network, LSTM layers, attention and CTC. The audience is narrow and specific: Mandarin speech, an NVIDIA GPU, and a team willing to run training jobs. The README puts the hardware floor at a 4-core x86_64 CPU, 16 GB of RAM, a GPU with 11 GB of graphics memory (it names the 1080 Ti as the starting point) and a 500 GB disk. Training is documented for Ubuntu 20.04 or CentOS 7 and later; Windows 10 and 11 are listed for inference only. If you only want to transcribe audio, the project's own download page offers a packaged server with trained models, and the Docker image runs CPU inference without any training at all.

Two models in series: DCNN plus CTC, then a maximum-entropy HMM

ASRT splits recognition into two stages with a different model for each. The speech model is a DCNN with CTC; the README states the input audio has a maximum length of 16 seconds and the output is a sequence of Hanyu Pinyin. That is the acoustic stage. The language model is described as a maximum-entropy hidden Markov model built on a probabilistic graphical model, and its input is the Pinyin sequence while its output is Chinese characters. So the data flow is audio, then Pinyin, then Hanzi, with a separate file, language_model3.py, handling the second hop. This split is worth understanding before you build anything on top of it, because it means an acoustic error and a decoding error look different. If the Pinyin comes out wrong, the acoustic model or the audio is the problem. If the Pinyin is right but the characters are wrong, the language model is the problem. The repository also carries torch_speech_model.py and train_speech_model_pytorch.py alongside the Keras files, which suggests a PyTorch path exists in the tree, but the README's quick start and its dependency list are written entirely around TensorFlow, so treat the PyTorch files as an unadvertised second track rather than the documented one.

Installing ASRT and running a first transcription

The README's quick start is written for Linux. Clone the repository, create a directory to hold the datasets, and unpack the archives into it. The README uses /data/speech_data as the example location and notes that a symbolic link works just as well.

git clone https://github.com/nl8590687/ASRT_SpeechRecognition.git
cd ASRT_SpeechRecognition
mkdir /data/speech_data
tar zxf <dataset archive> -C /data/speech_data/

The repository's asrt_config.json already lists six datasets by default: Thchs30, ST-CMDS, Primewords, aishell-1, aidatatang200 and MagicData. The README tells you to delete the entries you do not want, and warns that any other dataset has to be added by hand and reformatted into the standard layout ASRT expects. Next, fetch the Pinyin label files for the default datasets.

python download_default_datalist.py

Install the Python dependencies. The README lists TensorFlow, numpy, wave, matplotlib, scipy, requests, flask, waitress and the grpcio, grpcio-tools and protobuf trio, and gives requirements.txt as the shortcut. Note that the file pins tensorflow-gpu==2.8.4 while the Dockerfile installs tensorflow-cpu==2.5.3, so the two paths do not resolve to the same TensorFlow build.

pip install -r requirements.txt

Training, evaluation and single-file prediction are three separate scripts. The README notes that before evaluating you must make sure the model file path written in the code actually exists.

python3 train_speech_model.py
python3 evaluate_speech_model.py
python3 predict_speech_file.py

To serve recognition over the network, start the HTTP server, which the README pairs with client_http.py for a local check, or the gRPC server, paired with client_grpc.py. The Dockerfile exposes port 20001 for HTTP and the README's docker run example publishes both 20001 and 20002.

python3 asrserver_http.py
python3 client_http.py
python3 asrserver_grpc.py
python3 client_grpc.py

The four available model variants are named 24, 25, 251 and 251bn. Switching away from 251bn means editing the corresponding from speech_model.xxx import xxx line in the code. If you would rather not build the environment at all, the README gives a Docker route that runs CPU inference only, with no training.

docker pull ailemondocker/asrt_service:1.3.0
docker run --rm -it -p 20001:20001 -p 20002:20002 --name asrt-server -d ailemondocker/asrt_service:1.3.0

The accuracy ceiling and the 16-second input window

The README is unusually direct about quality: it states that the current best model reaches roughly 85 percent Pinyin accuracy on the test set. That is Pinyin accuracy, not character accuracy, and it is measured on a test set rather than in your deployment. Anything you build on top of ASRT inherits that ceiling, and the second-stage language model can only rearrange a Pinyin sequence that is already partly wrong. The 16-second maximum input length is the other constraint that shapes real integrations. A long recording has to be segmented before it reaches the model, and where you cut matters, because a cut through the middle of a phrase loses context on both sides. Neither the README nor the file list shows a VAD or segmentation utility, so that work sits with you. The language model is also domain-bound: a maximum-entropy HMM trained on general text will not know your product names, your internal jargon or your place names, and the README does not describe a procedure for adapting it. Finally, if your audio is not Mandarin, none of this applies. The Pinyin intermediate stage is the whole architecture, and there is no documented path to another language.

ASRT against cloud ASR APIs and Kaldi-style toolkits

The obvious alternative is a hosted Mandarin ASR API from a cloud provider. The difference is not accuracy, which you cannot compare from this material, but where the work happens. A hosted API needs no GPU, no 500 GB of disk and no dataset download, and it improves without you doing anything. ASRT needs all of those, and it does not improve on its own. What you get in return is control: the audio never leaves your machines, the model is a file you own, and you can fine-tune on your own transcripts. The second alternative is a traditional Kaldi-style recipe, which typically combines a TDNN or similar acoustic model with an n-gram decoder and a pronunciation lexicon. ASRT instead uses a neural acoustic model with CTC, which removes the need to align audio to frame-level labels before training, and pairs it with a maximum-entropy HMM for decoding. That is a lighter setup to reason about than a full Kaldi recipe, at the cost of the tuning surface Kaldi exposes. The project also publishes client SDKs in several languages, with repositories for a Windows client, Python 3, Go and Java, so integration on the calling side is not limited to Python even though the server is.

Maintenance, licensing and what an upgrade costs

The last push to the repository was on 2026-04-10, and the most recent tagged release is v1.3.0 from 2022-05-20, with v1.2.0 and v1.1.2 before it in the same year. So the tagged releases are old while the tree has seen commits since. Plan for the master branch as your source of truth, and expect to read the code rather than a changelog when something moves. The dependency pins are the practical upgrade cost. requirements.txt holds tensorflow-gpu==2.8.4, numpy==1.24.1, Flask==2.2.2, protobuf==3.19.6, scipy==1.10.0 and Wave==0.0.2, while the Dockerfile installs tensorflow-cpu==2.5.3, numpy==1.19.2, grpcio==1.34.0 and grpcio-tools==1.34.0. Those two sets have to be kept straight depending on which path you take, and the README's stated support window is TensorFlow 2.5 to 2.11 with Python 3.9 to 3.11. Moving past that window is untested territory. The licence is GPL-3.0, which matters more here than for a library you merely import. If you distribute a product that links against this code, the GPL's copyleft terms are likely to reach your distribution, and running it as a network service raises questions the licence text answers rather than the README. That is a question for your own legal review, not something this article can settle.

Editorial conclusion

Adopt ASRT if you need a Chinese ASR pipeline you can train on your own transcribed Mandarin audio and serve behind your own HTTP or gRPC endpoint, and if you can supply the GPU and the roughly 500 GB of disk the README asks for. Do not adopt it if you need streaming recognition, multilingual coverage or a model that is refreshed on a schedule, because the last push was on 2026-04-10 and the newest tagged release, v1.3.0, dates from 2022-05-20. Verify three things before you commit: that your TensorFlow version sits inside the 2.5 to 2.11 window, that the model file path in evaluate_speech_model.py points at a checkpoint you actually have, and that the six datasets preloaded in asrt_config.json are the ones you want.

Frequently asked questions

Official sources

1. License: GPL-3.0
2. nl8590687/ASRT_SpeechRecognition on GitHub
3. Project website
4. README
5. Releases