# elasticsearch-analysis-vietnamese: a word segmenter for Lucene

> This plugin brings Vietnamese word segmentation into Elasticsearch by embedding the CocCoc C++ tokenizer, exposing a vi_analyzer, a vi_tokenizer and a vi_stop filter. It suits teams indexing Vietnamese text who need syllable-aware terms rather than whitespace tokens, and it asks you to build and install a native shared library before the plugin will work.

**duydo/elasticsearch-analysis-vietnamese** —  Vietnamese Analysis Plugin for Elasticsearch

- Repository: https://github.com/duydo/elasticsearch-analysis-vietnamese
- Stars: 565 · Forks: 223
- Language: Java
- License: Apache-2.0
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/duydo-elasticsearch-analysis-vietnamese

## Why whitespace tokens fail on Vietnamese

The problem this plugin solves is word segmentation. Vietnamese is written with spaces between syllables rather than between words, so a sentence that reads as a handful of words arrives at the analyzer as a handful of long syllable groups. A query for one word will not match a document that used a different syllable boundary, and search quality suffers in a way that is invisible until users complain.

The plugin integrates Vietnamese language analysis into Elasticsearch and does so with a C++ tokenizer for Vietnamese, the library developed by the CocCoc team for their Search Engine and Ads systems. The tokenizer is a native component, not a Java implementation, and the README is explicit about the consequences: the plugin uses the dynamic library `libcoccoc_tokenizer_jni.so` directly, and that library has to be installed on the machine.

Three things are exposed. The `vi_analyzer` is a composed analyzer built from the `vi_tokenizer` tokenizer plus the standard `stop` and `lowercase` filters. The `vi_tokenizer` is the segmenter on its own, usable inside a custom analyzer. The `vi_stop` is a stop filter, so the stop word list can be applied to a tokenizer of your own choosing rather than only through `vi_analyzer`.

The default behaviour is visible in a single example. Analysing the sentence Cộng hòa Xã hội chủ nghĩa Việt Nam produces the terms cộng hòa, xã hội, chủ nghĩa and việt nam, four multi-syllable words rather than the seven whitespace-delimited groups.

## The vi_analyzer parameters, including the dictionary path assumption

The composed analyzer takes a small set of parameters, and one of them is the one that will bite you on a fresh install.

`dict_path` is the path to the tokenizer dictionary on the system, and it defaults to `/usr/local/share/tokenizer/dicts`. That default is not arbitrary: it is where the CocCoc `make install` step puts the dictionary files, and the README says the plugin uses that path for `dict_path` by default. Install the library somewhere else, or install it on a host with a different layout, and the analyzer will be looking for a directory that is not there.

`keep_punctuation` decides whether punctuation marks survive as tokens, and it defaults to `false`. `split_url` controls domain handling, and it defaults to `false`. With it enabled, a domain such as `duydo.me` is split into duy, do and me; with it disabled, the same domain becomes duydo and me.

Stop words come from one of two parameters. `stopwords` accepts a pre-defined list such as `_vi_` or an array of words, and it defaults to the `stopwords.txt` file shipped in the plugin's resources under `src/main/resources/org/apache/lucene/analysis/vi/stopwords.txt`. `stopwords_path` points at a file on disk instead, which is the option for a list maintained outside the plugin.

The combination of a filesystem dependent default and a file based override is the design point to remember. This analyzer is configured partly through the host's filesystem, so the same index definition behaves differently on two machines with different directory layouts.

## Building the CocCoc tokenizer and the plugin from source

The build is three steps, and the first one is native code, not Java. You build the C++ tokenizer library with the JNI binding enabled, then you build the plugin against it, then you install the plugin into Elasticsearch.

Step one clones the tokenizer, configures it with CMake, and installs it. The `make install` step puts three commands under `/usr/local/bin`, namely `tokenizer`, `dict_compiler` and `vn_lang_tool`, puts `libcoccoc_tokenizer_jni.so` under `/usr/local/lib`, and puts the dictionary files under `/usr/local/share/tokenizer/dicts`. The shared library is then linked into `/usr/lib` so the plugin can find it at runtime.

```bash
git clone https://github.com/duydo/coccoc-tokenizer.git
cd coccoc-tokenizer && mkdir build && cd build
cmake -DBUILD_JAVA=1 ..
make install
# Link the coccoc shared lib to /usr/lib
sudo ln -sf /usr/local/lib/libcoccoc_tokenizer_jni.* /usr/lib/
```

You can verify the tokenizer on its own before going near Elasticsearch. The binary takes a sentence and prints the segments, tab separated.

```bash
/usr/local/bin/tokenizer "Cộng hòa Xã hội chủ nghĩa Việt Nam"
# cộng hòa	xã hội	chủ nghĩa	việt nam
```

Step two is the Maven build of the plugin itself. The `pom.xml` carries the Elasticsearch version to build against, and the README notes it is the same as the plugin version, so you change that one value to target a different Elasticsearch release. The build produces a zip under `target/releases`.

```bash
git clone https://github.com/duydo/elasticsearch-analysis-vietnamese.git
```

```bash
cd elasticsearch-analysis-vietnamese
mvn package
```

Step three installs the built artefact into an Elasticsearch node with the plugin manager.

```bash
bin/elasticsearch-plugin install file://target/releases/elasticsearch-analysis-vietnamese-9.3.0.zip
```

## Running the whole thing in Docker instead

There is a Docker route, and it removes the host level native library problem because the shared object is built inside the image. The `Dockerfile` is a multi-stage build with three stages: one that compiles coccoc-tokenizer, one that builds the plugin with Maven, and a final runtime stage. The tokenizer stage clones the tokenizer with a shallow clone, configures it with CMake using the Elasticsearch bundled JDK, and runs `make install`. The plugin stage downloads a pinned Maven version, copies the POM first so dependency downloads are cached separately from source changes, and then packages the plugin with the Elasticsearch version passed in as a build argument.

The Compose file is a single Elasticsearch service built from that Dockerfile, exposed on port 9200, with the data directory mounted from the host, four gigabytes of heap set through `ES_JAVA_OPTS`, a single node discovery type, and security enabled with an `elastic` user whose password comes from the environment. The environment file has two variables, the Elasticsearch version and the password.

The README's Compose instructions start by copying the sample environment file and editing the version and password, then creating the data directory and giving it open permissions, and finally building and starting the stack.

```bash
# Copy, edit ES version and password for user elastic in file .env. Default password: changeme
cp .env.sample .env

# Give everything permission to data/ folder
mkdir data/
sudo chmod a+rw data/

docker compose build
docker compose up
```

The verification step is a single curl against the analyze endpoint, and the output shows the token, its offsets, its type and its position, which is the level of detail Elasticsearch returns for any analyzer.

```bash
curl -k http://elastic:changeme@localhost:9200/_analyze -H 'Content-Type: application/json' -d '
{
  "analyzer": "vi_analyzer",
  "text": "Cộng hòa Xã hội chủ nghĩa Việt Nam"
}'
```

## Building a custom analyzer over vi_tokenizer for accent-insensitive search

The `vi_analyzer` is a fixed composition, and for many applications it is not the one you want. The plugin gives you the tokenizer separately precisely so you can build your own analyzer around it, and the README's second example shows why.

The default `vi_analyzer` produces accented terms only. Adding the standard `ascii_folding` filter with `preserve_original` set to `true` produces both forms, so a search typed without diacritics still matches accented content while the original spelling is retained in the index.

```json
PUT my-vi-index-00002
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_vi_analyzer": {
          "tokenizer": "vi_tokenizer",
          "filter": [
            "lowercase",
            "ascii_folding"
          ]
        }
      },
      "filter": {
        "ascii_folding": {
          "type": "asciifolding",
          "preserve_original": true
        }
      }
    }
  }
}
```

The result of analysing the same sentence through that analyzer is a list of eight terms, four accented and four without diacritics, for the same four words.

```json
["cong hoa", "cộng hòa", "xa hoi", "xã hội", "chu nghia", "chủ nghĩa", "viet nam", "việt nam"]
```

The cost is index size, and it is a real one. Every term is now stored twice, once in each spelling, so the index grows and the term dictionary carries more entries. A team deciding whether to fold accents should weigh that against the fraction of real user queries that omit diacritics, which for a mobile audience typing quickly is often not small.

The same technique applies to the stop word list. `vi_stop` exists so that the Vietnamese stop words can be applied to a tokenizer of your own, and the `stopwords` parameter of `vi_analyzer` accepts either a pre-defined list name or an explicit array, so a custom list can be supplied inline in the index settings.

## Version compatibility is a per-branch decision, not a free upgrade

Elasticsearch plugins are not forward compatible, and this project treats that as a per-branch decision rather than shipping one artefact for all versions. The README has a table mapping plugin versions to Elasticsearch versions: master and develop track 9.3.0, the 9.3.0 plugin covers Elasticsearch 9.0.0 through 9.3.0, the 8.7.0 plugin covers 8.7.0, the 8.4.0 plugin covers 8.4.0 through 8.7.1, and the 8.0.0 plugin covers 8.0.0 through 8.0.x. Below that the mapping is one to one for each release line, down to versions in the 2.x range.

There is also a fork in the project's own history. From v7.12.11 the plugin uses the CocCoc C++ tokenizer instead of the VnTokenizer by Le Hong Phuong, and the author states plainly that the VnTokenizer version is no longer maintained, with development continuing on the `vntokenizer` branch if that is what you need.

The release history in the repository matches the table. The newest release is v9.3.0 on 2026-03-15, before it v8.7.0 on 2023-05-08 and v7.17.1 on 2022-05-26, so the gap between the 8.x and 9.x lines is long and the 9.x line is young. The repository is not archived and the last push was on 2026-10-11, so the 9.3.0 branch is where the work is happening.

For an operator, the practical consequence is that an Elasticsearch upgrade is not just a cluster change, it is a plugin rebuild and reinstall on every node, and the version pair has to be chosen deliberately.

## What this plugin cannot do, and what you use instead

The limitations are structural rather than bugs. The first is the native dependency. Because the plugin loads `libcoccoc_tokenizer_jni.so` directly, every node that runs the analyzer needs that library and the dictionary files at the expected path. On a managed Elasticsearch service you cannot install a shared library, so this plugin is not an option there, full stop. On Elastic Cloud the analysis has to be done differently, and the README does not describe a path for that.

The second limitation is that the plugin does one job. It segments Vietnamese. It does not stem, it does not remove diacritics for you, it does not handle Chinese or Japanese text in the same index, and it offers no synonym set, no dictionary update mechanism and no way to add your own words. The stop word list is a file or an array, and the tokenizer's vocabulary is whatever the CocCoc dictionary contains.

That last point is the one to check before committing. If your organisation has domain vocabulary, a product name or a transliterated term, there is no documented way to teach this tokenizer about it.

The alternative is to segment in the application, or to use a different analysis approach entirely. A pipeline that tokenizes Vietnamese in the service that produces the documents, and sends the already segmented terms into Elasticsearch with a whitespace or keyword analyzer, avoids the native library entirely and gives you full control over the vocabulary. The cost is that the segmentation happens at index time only, so a search for a partial word cannot be handled by the analyzer, and the dictionary and the segmentation logic live outside the cluster. For a Vietnamese corpus where the same segmentation must be applied consistently at index and search time, an in-cluster plugin is the cleaner answer, and for everything else the application side is simpler to operate.

## Conclusion

Adopt elasticsearch-analysis-vietnamese if you index Vietnamese text and need syllable aware terms, since splitting on whitespace leaves most Vietnamese words joined to their neighbours and the plugin fixes that at the analysis stage rather than in the application. Do not adopt it if you cannot add a native library to the Elasticsearch nodes, because the CocCoc shared object has to be present on every node that runs the analyzer. Before installing, check the compatibility table in the README for a plugin version that matches your Elasticsearch version, confirm that `dict_path` points at a readable dictionary directory, and run `bin/elasticsearch-plugin install file://target/releases/elasticsearch-analysis-vietnamese-9.3.0.zip` on a staging node first, since a plugin that fails to load takes the node down with it.

## FAQ

### What does the Vietnamese analysis plugin for Elasticsearch provide?

It provides a `vi_analyzer` analyzer composed from the `vi_tokenizer` tokenizer with the `stop` and `lowercase` filters, plus the `vi_tokenizer` and a `vi_stop` stop filter so you can build your own analyzer. Segmentation is done by the CocCoc C++ tokenizer for Vietnamese.

### How do I install the Vietnamese analysis plugin on an Elasticsearch node?

Build the plugin with `mvn package` and then install the produced archive with `bin/elasticsearch-plugin install file://target/releases/elasticsearch-analysis-vietnamese-9.3.0.zip`. The native CocCoc shared library must be installed on the node as well.

### Which Elasticsearch versions does the plugin support?

The compatibility table in the README maps plugin versions to Elasticsearch versions: the 9.3.0 plugin covers Elasticsearch 9.0.0 through 9.3.0, 8.7.0 covers 8.7.0, 8.4.0 covers 8.4.0 through 8.7.1, and older plugin releases map to their own lines. Master and develop track 9.3.0.

### How do I search Vietnamese text without diacritics?

Build a custom analyzer on the `vi_tokenizer` with the `lowercase` and `ascii_folding` filters, and set `preserve_original` to `true` on the asciifolding filter. The indexed terms then contain both the accented and the unaccented spelling, at the cost of a larger index.

### Can I run the plugin with Docker?

Yes. The repository ships a multi-stage Dockerfile that builds the CocCoc tokenizer and the plugin, and a docker-compose.yaml that runs a single node Elasticsearch with security enabled. You copy `.env.sample` to `.env`, create the `data/` directory, then run `docker compose build` and `docker compose up`.

## Sources

- [duydo/elasticsearch-analysis-vietnamese on GitHub](https://github.com/duydo/elasticsearch-analysis-vietnamese)
- [Issues](https://github.com/duydo/elasticsearch-analysis-vietnamese/issues)
- [License: Apache-2.0](https://github.com/duydo/elasticsearch-analysis-vietnamese/blob/master/LICENSE)
- [README](https://github.com/duydo/elasticsearch-analysis-vietnamese/blob/master/README.md)
- [Releases](https://github.com/duydo/elasticsearch-analysis-vietnamese/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/duydo-elasticsearch-analysis-vietnamese