LTP 4: Six Chinese NLP Tasks Behind One Pipeline API

What LTP 4 Is For, and Who It Is Actually For

LTP stands for Language Technology Platform, and the README describes it as a set of tools for Chinese text: word segmentation, part-of-speech tagging, syntactic parsing and related work. The project comes from HIT-SCIR, the social computing and information retrieval lab at Harbin Institute of Technology, and the associated N-LTP paper (Che et al., EMNLP 2021 system demonstrations) frames the design against toolkits such as Stanza. The stated difference is that Stanza uses an independent model per task, while N-LTP uses a multi-task framework with a shared pre-trained model, so tasks can share knowledge. The paper also describes a knowledge distillation step in which a single-task model teaches the multi-task model.

The audience is narrow and concrete. If your input is Chinese and you need tokenisation plus labels attached to those tokens, this is the target case. If your input is English, or if you need a general-purpose multilingual pipeline, nothing here applies. The README is written primarily in Chinese, the pre-trained models are Chinese, and the six tasks are the six tasks of Chinese lexical, syntactic and semantic analysis. A team building a Chinese search index, a Chinese content moderation filter, or a Chinese annotation pre-pass for human reviewers is the intended user.

The 4.2.0 Split: Legacy Perceptron Versus PyTorch Models

The most consequential fact about LTP 4 is that 4.2.0 divided it into two parts. The first is described as a legacy model, rewritten in Rust from a perceptron-based algorithm, aimed at users who care about inference speed. The release notes claim accuracy comparable to LTP3 and a speed of 3.55 times LTP v3, rising to 17.17 times with multithreading enabled. Those numbers come from the project's own release notes; I have not run them. The important constraint is in the same sentence: the legacy path currently supports only three tasks, segmentation, POS tagging and named entity recognition.

The second part is the deep learning model, implemented in PyTorch, and it covers all six tasks: word segmentation, POS tagging, named entity recognition, semantic role labelling, dependency parsing and semantic dependency parsing. The 4.2.0 notes also state that decoding for segmentation, dependency parsing (Eisner) and semantic dependency parsing (Eisner) was moved to Rust for speed, and that models are hosted on Hugging Face Hub for automatic download. So the split is not clean: the Rust rewrite touches the deep learning path too, but only for decoding, and the fully Rust-supported task set remains three tasks. Anyone reading "rewritten in Rust" as "parsing is now fast in Rust" will be wrong.

The Pipeline API and What the Output Object Holds

Version 4.2.0 introduced a breaking change: inference moved to a Pipeline API. The README's example constructs an LTP object with a model identifier, optionally moves it to a GPU, then calls pipeline with a list of strings and a task list. The documented task names are cws, pos, ner, srl, dep, sdp and sdpg, where sdp is the semantic dependency tree and sdpg the graph. The release notes give a rationale for the pipeline shape rather than a bare function call: SDP and SDPG overlap heavily, and reusing that work speeds up inference. That is a design decision with a visible cost, because the breaking change means code written against the 4.1.x API will not run unchanged.

The return value is a structured object, not a list of strings. The README shows output.cws, output.pos and output.sdp, and notes that the same fields can be reached by index (output[0]) or by key (output['cws']). There is also a to_tuple() method that converts the result into tuple form. The README's own commented-out line is informative: attempting to run tasks=["cws", "ner"] and convert to a tuple is marked as an error, with the explanation that NER requires the POS tagging result. That is a real dependency edge in the task graph, and it means the task list is not freely composable. If you want NER, you pay for POS whether you display it or not.

Installation, Model Loading and Custom Dictionaries

The README gives two installation routes, both using the Tsinghua TUNA mirror. The first installs dependencies and packages in one sequence: pip install -i https://pypi.tuna.tsinghua.edu.cn/simple torch transformers, then pip install -i https://pypi.tuna.tsinghua.edu.cn/simple ltp ltp-core ltp-extension. The second sets the mirror globally with pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple and then installs the same packages without the -i flag. Note that three distributions are installed, not one: ltp, ltp-core and ltp-extension. The README does not explain the boundary between them, and a reader planning a vendored or offline install will need to work that out from the package contents.

Model selection happens at construction. LTP("LTP/small") loads the small model by default, and the README states that the default download path goes through Hugging Face, which may require a proxy in some networks. A local directory can be passed instead, provided it contains config.json and the other model files. The legacy path uses a different identifier, LTP("LTP/legacy"). Two methods, add_word and add_words, inject domain vocabulary with a frequency argument, shown as ltp.add_word("汤姆去", freq=2) and ltp.add_words(["外套", "外衣"], freq=2). This is the mechanism for handling names, product terms or jargon the base model splits wrongly, and it is the one project-specific knob most users will actually touch.

The Rust Interface Is Not a Mirror of the Python One

The repository ships a Rust crate, published as ltp on crates.io, with its own README. The example loads three separate model files, cws_model.bin, pos_model.bin and ner_model.bin, each through ModelSerde::load with Format::AVRO(Codec::Deflate). That is a materially different shape from the Python side. Python takes a model identifier and downloads it; Rust takes file handles and a serialisation format you must know in advance. The Rust example also pulls in itertools for multizip, which suggests the caller is expected to zip the per-task outputs together rather than receive a unified result object.

So the two interfaces are not interchangeable. If you are embedding Chinese segmentation in a Rust service, you get the three legacy tasks and you manage model files yourself. If you need SRL, DEP, SDP or SDPG, the Rust path as documented does not offer them, and you would be calling into the Python stack instead. This is the clearest example of a trade-off in the project: the fast path is deliberately narrower, and the README says so rather than hiding it.

Training, Hydra Configuration and the Upgrade Surface

The 4.2.0 notes state that training scripts and training examples were added so users can train customised models on private data. For the deep learning models, training configuration is handled with hydra, which the notes describe as making it easier to modify training parameters and extend LTP, for instance by using modules from other packages. Models can also be uploaded to Hugging Face Hub by users, and the notes say LTP can then run inference with them. That is a genuine extension point, but it also defines the maintenance cost: if you train your own model, you own the training configuration, the data pipeline and the compatibility of that model with future LTP releases. The project's own release cadence gives a sense of the risk. v4.2.0 is dated 2022-08-15, and the two preceding releases, 4.1.5.post1 and 4.1.5.post2, are from July 2021. The last push to the default branch is dated 2026-03-11, which is later than any listed release, so repository activity and release activity are not the same thing here. Plan for a slow release cadence and pin your versions.

On licensing, the supplied material does not state a licence for the repository, and the licence field is unknown. The README includes a BibTeX entry asking users to cite the N-LTP paper, which is an academic attribution request, not a licence grant. Before shipping LTP inside a product, check the repository for a LICENSE file and read it; I cannot tell you from this material what terms apply, and this is not legal advice.

Where LTP 4 Is the Wrong Choice

The first failure mode is a task mismatch. If your pipeline needs semantic role labelling or dependency parsing and your deployment target is Rust or another non-Python runtime, LTP 4 does not give you a supported route in the material provided. You would be running Python and calling into PyTorch, which changes your container size and your latency profile.

The second is the legacy model's accuracy. The README says the perceptron-based path is faster but slightly lower in accuracy, without quantifying the gap. If your application is sensitive to segmentation errors, the speed claim is not free, and the release notes' comparison is against LTP v3 rather than against the deep learning models in the same release.

The third is the dependency constraint inside the pipeline. The README marks cws plus ner without pos as an error. Pipelines that assume tasks are independent will need restructuring.

The fourth is the download path. The default model load goes through Hugging Face and the README warns it may need a proxy. In an air-gapped or restricted-egress environment, you must pre-stage the model directory and pass a local path, and you must confirm which files that directory needs beyond config.json.

As an alternative, consider Stanza, which the N-LTP paper names directly. The difference is architectural: Stanza uses an independent model per task, whereas N-LTP uses a shared pre-trained model across tasks with distillation from single-task teachers. Independent models are simpler to reason about and let you swap one task without touching the others. A shared model is what lets LTP expose six tasks through one object and one pipeline call, at the cost of the coupling described above. If you only need segmentation, that shared architecture buys you nothing and a smaller single-task model may serve you better.

Editorial conclusion

Adopt LTP 4 if you need Chinese word segmentation, POS tagging and NER at speed and can accept the legacy model's narrower task set, or if you need all six tasks and can run PyTorch. Do not adopt it if you need a confirmed open source licence before shipping, or if you expected the Rust rewrite to cover parsing. Verify first: check the repository for a LICENSE file, then run the pip install line and confirm that ltp.pipeline with tasks=["cws", "pos", "ner"] returns before you wire it into a service.

Official sources

1. HIT-SCIR/ltp on GitHub
2. Issues
3. Project website
4. README
5. Releases