kagome: a Japanese morphological analyzer that ships its dictionaries

Pure Go means the dictionary travels with the binary

Kagome is an open source Japanese morphological analyzer written in pure Go, and the pure part is the whole point of the design.

It tokenizes Japanese text into words and analyzes parts of speech, and the dictionaries are embedded in the binary for easy deployment. The key features list calls this out directly: self-contained binaries with embedded dictionaries, specifically MeCab-IPADIC and UniDic, and it names the improvements over version one.

The practical effect is a deployment story that is hard to beat. There is no shared library to install, no system dictionary to point at, no compiler toolchain needed on the target machine, and no mismatch between the binary and the dictionary format. A container image is one file. A lambda is one file. A Go program that embeds the library is one binary.

The dictionaries are separate Go modules, and the module list shows the split: the base kagome-dict module, an ipa module for the IPADIC dictionary and a uni module for UniDic. That split is why the library example imports the ipa dictionary package explicitly rather than having a default, because the dictionary is a deliberate choice rather than a hidden default.

The project also maintains a Korean tokenizer, judging by the topic list, and the repository includes a filter package, a tokenizer package, a command directory, an examples directory and test data.

Four commands, and the flags that matter

The command line tool is a single binary with a handful of sub-commands, and the default one does the obvious thing.

usage: kagome <command>
The commands are:
   [tokenize] - command line tokenize (*default)
   server - run tokenize server
   lattice - lattice viewer
   sentence - tiny sentence splitter
   version - show version

Tokenize is the default, so piping text into the binary with no arguments produces the analysis. The lattice command is the interesting one for anyone who has used a Japanese analyzer before, because a lattice view shows the candidate segmentations the dictionary proposed and which one was chosen, which is how you understand why a word was split the way it was.

The tokenize flags are where the behaviour is controlled. The system dictionary is selected with the sysdict flag, which takes ipa or uni, and the default is ipa. A user dictionary can be supplied with the userdict flag, and a separate dictionary file with the dict flag. The mode flag takes normal, search or extended, with normal as the default. The split flag turns on the tiny sentence splitter, the simple flag displays abbreviated dictionary contents, the json flag switches the output format, and the file flag reads from a file instead of standard input.

The json output is the one to build against if you are writing an integration, because it gives you structured fields rather than a tab separated line: an id, a start and end offset, the surface form, a token class, a part of speech list, a base form, a reading, a pronunciation and the full feature list. The reading and the pronunciation fields are what you want for anything that feeds text to speech or furigana generation, and the readme shows exactly that use of the JSON output with the jq command line tool.

As a library, the dictionary is an argument

Integrating the analyzer into a Go program takes a module fetch and a constructor call.

go get github.com/ikawaha/kagome/v2

The tokenizer is created from a dictionary, and there is an option to omit the beginning and end of sentence markers:

t, err := tokenizer.New(ipa.Dict(), tokenizer.OmitBosEos())

Two operations matter, and they are different things. The Wakati method does simple word splitting, which is the segmentation without the morphological analysis, and the Tokenize method does the full analysis and returns tokens you can read.

Each token carries a surface form and a Features method that returns the feature fields, which are the part of speech, the sub-classification, the inflection type and form, and the reading and pronunciation. The readme joins those features with commas for display, which is how the tab separated output on the command line is produced.

The design decision worth noting is that the dictionary arrives as a value from a separate module. That means the same binary can carry IPADIC, UniDic or a custom dictionary, and it means the dictionary version is pinned in your go.mod rather than being whatever happens to be on the machine.

The module requires Go 1.24.0, and the example code in the repository is organised as a directory of runnable examples that are also exposed in the package documentation, so you can read a working program for each use case rather than assembling one from a snippet.

Segmentation modes and the choice of dictionary

Japanese segmentation is ambiguous by nature, and the tools that handle it differ in how they resolve ambiguity. Kagome exposes that as a mode rather than hiding it.

The three modes are normal, search and extended, and the mode flag selects between them with normal as the default. The readme does not spell out the algorithmic difference between them, so the honest statement is that they produce different segmentations and that the choice is yours to make. That is not a gap in the documentation so much as a fact about the domain, and the sensible approach is to run the same sample text through all three and see which matches how your corpus is written.

The dictionary choice is the second axis, and it is the more consequential of the two. The system dictionary flag takes ipa or uni, defaulting to ipa, so the two supported dictionaries are MeCab-IPADIC and UniDic. The two are not interchangeable. IPADIC is the long-standing general purpose dictionary, and UniDic is a newer academic dictionary with a different set of unit types, and the features they produce, including the part of speech labels and the readings, do not line up one to one. A system that trains on one set of features will need translating if you switch to the other.

The simple flag and the abbreviated dictionary contents output point at a third practical concern. When you are debugging a tokenization result, being able to see what the dictionary actually contains for a surface form is worth more than a page of documentation, and the lattice view plus the simple output are the two tools for that.

Four ways to deploy it, including outside Go

The version 2 feature list is really a list of deployment shapes, and it is broader than most tokenizer projects offer.

The first two are the ones a Go developer expects. The library, imported into your program, and the command line tool, installed as a binary.

The third is a RESTful API server mode for production use. The server command runs a tokenize server, and the readme links a web app that talks to it. That is a meaningful option when several services need analysis, or when the analysis has to happen in a language other than Go, because the JSON output is the contract and the HTTP interface is stable regardless of what is behind it.

The fourth is WebAssembly support for browser environments, which puts a Japanese tokenizer inside a page with no server round trip. For a client-side application that highlights words, this is the difference between shipping a dictionary to the browser and shipping a request to a backend.

The fifth is the C library API for FFI integration, and the readme is explicit about the point: Kagome is written in pure Go but can be compiled as a C shared library and used from other languages via the foreign function interface, with Python and PHP named as examples. This is the option that lets a Python team use it without rewriting in Go or running a service, and it is unusual for a Go project to offer.

For a C library build, the practical questions are the boring ones: how the shared object is built, how the dictionary is embedded in it, and how the calling convention exposes tokens and feature strings. The readme refers to the FFI section of the documentation for the details and the examples.

A container image with nothing in it

The Dockerfile is three lines long, and it is a good illustration of what the pure Go rewrite bought.

FROM scratch
COPY kagome /
ENTRYPOINT ["/kagome"]

There is no base image, no shell and no package manager. The binary is copied in and made the entry point, which produces an image with nothing in it but the program. No libc, no shell, no configuration files.

That image is useful for two reasons. The size argument is the obvious one, and for a tool that people often want to run as a sidecar or in a pipeline, a few megabytes changes whether it is a reasonable dependency. The security argument is the one that lasts longer, because a scratch image has no shell for an attacker to run and no package inventory to keep patched, and for a service that parses untrusted text from the internet, that is a real reduction in exposure.

The project publishes images on Docker Hub, and the readme also points at a Docker Compose section, which suggests the server mode is the shape meant for a container deployment.

The manual install path exists too, and it has one requirement worth noting: you download the archive for your operating system and architecture from the releases page, and the extracted binary must be placed in an accessible directory with execution permission. No installer, no package database entry, no post-install step.

Release discipline on a v2 branch

The maintenance story is straightforward and worth a short paragraph.

The default branch is v2, which is the convention of keeping the current major version on its own branch so that the older line can be maintained separately. The project publishes tagged releases, with v2.10.2 on 2025-04-01, v2.10.3 on 2026-01-03 and v2.11.0 on 2026-03-03, and the last push was on 2026-10-15 with the repository not archived. The licence is MIT.

The engineering setup around the code is the usual for a Go library that cares about quality. There is a Go build workflow badge, a release workflow badge, a coverage status badge pointing at Coveralls for the v2 branch, a Docker image on Docker Hub, a golangci lint configuration, a DeepSource configuration, a goreleaser configuration for the release binaries, and a Dockerfile in the root.

The install options follow from that setup. The recommended route is the Go toolchain, installing the latest version of the module and putting a binary in your Go bin directory. Homebrew is supported for macOS and Linux on both AMD64 and Arm64 through a tap. The manual route is the release archive, and Docker is the fourth.

Two things to watch when you pin a version. The go.mod requires Go 1.24.0, so a Go 1.23 toolchain cannot build the current line, and the module path carries the major version suffix, so imports have to use the v2 path rather than the original one. Both are ordinary Go module mechanics, and both will surface as a confusing error if you are following an older example from the web.

Editorial conclusion

kagome is the right choice if you need Japanese tokenization and part of speech tagging without a cgo dependency, a runtime data directory or a Python service in the middle. It is a poor fit if you need the accuracy of a large language model, if you rely on a segmentation mode it does not implement, or if your platform has no supported install path. Before you adopt it, compare the normal, search and extended segmentation modes on your own text rather than assuming the default is right, check that the dictionary you want is the one embedded, and decide early whether you will call the library, the server or the WebAssembly build.

Frequently asked questions

Official sources

1. ikawaha/kagome on GitHub
2. Issues
3. License: MIT
4. README
5. Releases