shibing624/similarity: a Java text similarity toolkit that ships its own dictionaries

What shibing624/similarity computes, and for whom

This is a Java library, not a service. The README describes it as a toolkit whose stated goal is to spread similarity calculation methods from natural language processing, and its feature list is organised by text length: word, phrase, sentence, paragraph. Each length has a recommended method marked in the README, and the rest are alternatives you can call when the recommended one does not fit.

The audience is narrow but real. If you are in a JVM codebase and need to score how close two Chinese strings are, without adding a Python process or a model server, the library gives you a single entry class. The README's own example imports org.xm.Similarity and calls static methods on it, so there is no object graph to wire up and no configuration file to load. The sentiment feature is a smaller side offering: a HownetWordTendency class that returns a polarity value for a single word.

Where it is not aimed: English text. Every recommended method leans on Chinese resources, from the Tongyici Cilin synonym dictionary to Hownet sememes. The README does not present an English pipeline, and the character-based method it does expose compares characters, which behaves differently on space-separated languages.

The mechanism: dictionary lookup, character overlap and weighted cosine

The four text lengths map to four different mechanisms, and they are not variations of one algorithm.

At word level, the recommended method is cilinSimilarity, which the README describes as based on the Tongyici Cilin synonym dictionary. Similarity comes from where two words sit in that coded hierarchy, so the score is a lookup, not a computation over the words themselves. The README also lists a pinyin method, a concept method built on Hownet sememes, and a character-based method, all callable side by side.

At phrase level, phraseSimilarity works on shared characters and their positions, according to the README. That is a positional overlap measure, which means reordering the characters in a phrase changes the score even when the character set is identical.

At sentence level, morphoSimilarity combines matching surface text with the order in which that text appears. The README frames it as considering both the identical text between two sentences and the sequence of that text. The three edit distance variants are the fallback when you want pure string distance.

At paragraph level, CosineSimilarity tokenises both texts, weights terms by frequency and part of speech, and computes cosine similarity. This is the one method in the set that behaves like a conventional vector space model rather than a dictionary lookup. The README's own example returns 0.399143 for cosine against two Chinese news paragraphs, and 0.0875 for the edit distance method on the same pair. That gap is the useful signal: on longer text, the tokenised method and the raw string method are not interchangeable, and the README presents both numbers without recommending one.

Installing from JitPack with Maven, and a first call

The README routes installation through JitPack rather than Maven Central. You add the JitPack repository, then the dependency. The version it documents is 1.1.6.

<repositories>
  <repository>
    <id>jitpack.io</id>
    <url>https://jitpack.io</url>
  </repository>
</repositories>

<dependency>
  <groupId>com.github.shibing624</groupId>
  <artifactId>similarity</artifactId>
  <version>1.1.6</version>
</dependency>

Gradle is mentioned in the README, but the section only carries the JitPack badge and no snippet, so you will need to translate the coordinates above yourself. After the dependency resolves, the first useful call is a word pair. The README's word demo compares 教师 and 教授 across four methods and prints each score.

import org.xm.Similarity;

double result = Similarity.cilinSimilarity("教师", "教授");
System.out.println(result);

What you should see is a double between 0 and 1. The README shows the demo output as an image rather than text, so it does not state the expected value for this pair. Run all four methods on the same pair before choosing one, because they disagree by design. For a longer input, the paragraph demo instantiates a TextSimilarity implementation and calls getSimilarity:

TextSimilarity cosSimilarity = new CosineSimilarity();
double score1 = cosSimilarity.getSimilarity(text1, text2);
System.out.println("cos相似度分值：" + score1);

The README gives the working range for this method as paragraphs between 25 and 500 characters. Below or above that, the tokenisation and weighting assumptions behind the score are outside what the project documents.

The dictionary is the dependency, and the README does not say how to extend it

The README states that the internal modules keep low coupling, that models load lazily, and that dictionaries are published as plain text so users can train their own corpus. That last claim is the one to test before you commit, because the README does not show where the dictionary files live, what format they use, or how to point the library at a modified copy. The repository does have corpus/ and data/ directories at the top level, which is consistent with the claim, but the README gives no procedure.

The practical consequence: if your domain vocabulary is absent from the bundled Tongyici Cilin dictionary, cilinSimilarity has nothing to look up and the score is not meaningful. The same applies to the Hownet sememe methods. You are not tuning a model; you are querying a fixed resource. For general-purpose Chinese this is often fine, and it is why the library has no training step and no model download.

The second limitation is the release cadence. The newest release listed is 1.1.6 from 2022-04-12, with 1.1.4 before it in 2022-02-18. The repository's last push was on 2026-07-16, so work has continued on master, but the tagged artifact you would depend on is years behind that date. If your policy is to depend only on tagged releases, budget for the possibility that a fix on master is not in any version you can resolve.

Third, the sentiment path is word-level only. The README is explicit that the Hownet sememe tree analysis is at word granularity and points to a separate project, pytextclassifier, for text-level sentiment with neural or SVM models. Do not read the sentiment feature as a document classifier.

How this differs from a sentence-embedding library

The obvious alternative for a JVM team is an embedding-based approach: encode both strings with a sentence transformer and take cosine similarity of the vectors. The difference in mechanism is the whole story. An embedding model generalises, so it will score two paraphrases as similar even when they share no characters and no dictionary headword. This library's recommended word method cannot do that, because it resolves each word to a position in a fixed synonym hierarchy, and its phrase method counts shared characters at shared positions.

That cuts both ways. An embedding model needs weights, a runtime and a tokeniser, and its scores drift when you change the model version. The dictionary methods here are deterministic given the same corpus files, and the dependency is a jar. If your similarity task is closer to deduplication of near-identical strings, or to detecting whether two product titles refer to the same headword, the deterministic path is easier to reason about and to explain to whoever reviews the output.

A second alternative is to stay in Python with a cosine implementation over your own tokeniser, which is what most of the search interest around cosine similarity assumes. That gives you full control of tokenisation and weighting at the cost of a second runtime. The README's paragraph method is essentially that pipeline, pre-built and with part-of-speech weighting already applied, inside the JVM.

Licence and what upgrading actually costs

The repository is Apache-2.0, per the LICENSE file and the badge in the README. For most teams that means you can use it in a closed product, and the usual Apache obligations around notices and attribution apply. Whether the bundled corpus files carry the same terms is not something the README addresses, and the corpus/ and data/ directories are not described in the licence section. If you plan to redistribute the jar or the dictionaries, that is the question to put to your own legal review, not something this article can settle.

Upgrade cost is low in the mechanical sense. The public surface the README documents is static methods on org.xm.Similarity plus a handful of classes such as CosineSimilarity, EditDistanceSimilarity and HownetWordTendency. There is no configuration file and no migration step described. The cost sits on the other side: because the newest tagged release is 1.1.6 from 2022-04-12, any fix you want may only exist on master, and depending on a JitPack build of a branch commit is a different risk profile from depending on a release. The README does not document a changelog or a rollback path, so pin the version explicitly in your build file and treat a bump as a deliberate change to verify against your own sentence pairs.

Editorial conclusion

Adopt it if you are writing Java and need Chinese word, phrase, sentence or paragraph similarity without standing up a model server; the Maven coordinates and the static entry points on org.xm.Similarity are the whole integration. Skip it if your text is not Chinese, since the recommended word method reads the Tongyici Cilin dictionary, or if you need a maintained release train: the last push to master was on 2026-07-16 but the newest tagged release, 1.1.6, dates from 2022-04-12. Before committing, run Similarity.cilinSimilarity and Similarity.morphoSimilarity on your own sentence pairs and check that the bundled corpus covers your vocabulary, because the README does not document how to extend it.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. shibing624/similarity on GitHub