Elasticsearch Learning to Rank: storing query templates as features and ranking with trained models

What the plugin actually adds to Elasticsearch

Elasticsearch ranks by query score. That score is a sum of term statistics, and every attempt to shape it means editing boosts, functions or query structure by hand. This plugin replaces that hand-tuning with a learned model, but it does not replace the queries themselves. The README lists four things the plugin does: store features as Elasticsearch query templates, log feature scores to build a training set, store linear, xgboost or ranklib ranking models, and rank results with a stored model. Each of those is an Elasticsearch-side operation, which is the point. Feature definitions and model files live in the cluster, not in an application service that has to stay in sync with the index.

The audience is narrow and specific. You need an Elasticsearch cluster you control, because this is a plugin and plugins are installed per node. You need graded judgements, because a model trained on clicks or on nothing at all will not beat the default scorer in any way you can defend. The README points at Wikimedia Foundation and Snagajob as places where it powers search, and the acknowledgements credit Wikimedia, Snagajob, Bonsai and Yelp engineering with significant contributions. That is a fair signal of the kind of team it was built for: someone with a search relevance problem large enough to justify offline training.

Features as stored query templates, and where the model sits

The mechanism is a two-stage ranking pipeline. A normal query retrieves a candidate set. Then a rescore phase runs a stored model over those candidates, and the model's score replaces or adjusts the order. The model does not see documents directly. It sees feature values, and each feature is a named Elasticsearch query template that produces a number for a given document.

That indirection is the design choice worth understanding. A feature is not a field or a vector. It is a query, stored in the cluster, with placeholders for the terms of the incoming request. At query time the plugin renders each template with the current request parameters and runs it, collecting one score per feature per document. The model is a separate stored object that maps feature names to weights or tree conditions. So the same feature set can be reused across models, and a model can be swapped without touching the retrieval query.

The cost is that feature extraction is query execution. If you define twelve features, you are running twelve query templates over your candidate set in addition to the retrieval query. The README does not publish latency figures, and none should be assumed. What the README does say is that the plugin logs feature scores to create a training set. That logging path runs the same templates, which means the training data is generated by the same machinery that will serve it. It is consistent by construction, and it is also the reason feature templates have to be cheap enough to run in both modes.

Installing the plugin from a release zip

The README is explicit that you should pick the prebuilt version matching your Elasticsearch version from the releases page, and only build from source if no matching build exists. Installation is the standard Elasticsearch plugin command pointed at the release asset URL. The README gives this example, and says to replace it with the appropriate prebuilt version zip:

./bin/elasticsearch-plugin install https://github.com/o19s/elasticsearch-learning-to-rank/releases/download/v1.5.4-es7.11.2/ltr-plugin-v1.5.4-es7.11.2.zip

Elasticsearch will ask you to confirm security exceptions because the plugin is installed from a URL. The README notes that you can pass `-b` to `elasticsearch-plugin` to accept them automatically. If Elasticsearch is already running, restart it. Nothing in the README suggests a rolling install that avoids a restart.

If you need a version that has no prebuilt artifact, the README gives the build path. It is a Gradle build, and the resulting zip is installed from the local filesystem rather than a URL:

./gradlew clean check
./bin/elasticsearch-plugin install file:///path/to/elasticsearch-learning-to-rank/build/distributions/ltr-<LTR-VER>-es<ES-VER>.zip

The placeholder names in that second path are the README's, not invented ones: the build output is named with the LTR version and the Elasticsearch version it targets. The README also states that the project aims to officially support `*.*.1` releases of Elasticsearch and invites pull requests for dot-zero compatibility or unsupported versions. Read that as a real constraint on your upgrade calendar, not as a formality.

For a first real use, the README no longer ships the demo in this repository. It points to a separate project, Hello LTR, which contains both Elasticsearch and Solr examples, and says to follow the Elasticsearch directions there and start with the TMDB notebook at notebooks/elasticsearch/tmdb/hello-ltr.ipynb. That is the honest starting point: build the feature set and the model against a known dataset before pointing any of it at production traffic.

Where this plugin is the wrong tool

The plugin is Elasticsearch-only, and that is a hard boundary rather than a missing feature. If your retrieval layer is Solr, OpenSearch, or a vector database, none of the stored feature templates or model objects transfer. The README's own acknowledgements point to Bloomberg's separate Learning to Rank work for Solr, which is a different implementation for a different engine.

The second limitation is version coupling. A plugin runs inside the Elasticsearch process, so a plugin artifact is built against a specific Elasticsearch version. The release list shows this clearly: v1.5.13-es9.3.5, v1.5.13-es9.3.2 and v1.5.13-es9.2.4 are separate artifacts of the same plugin version, each pinned to a different Elasticsearch version. Upgrading Elasticsearch means finding or building a matching plugin artifact first. Teams that upgrade Elasticsearch on a schedule they do not control will feel this.

The third limitation is that learning to rank needs labels. The plugin gives you the logging machinery to produce feature scores, but it does not produce judgements. If you have no graded relevance data, the plugin adds a rescoring stage and a model file without giving you anything to train the model on. In that situation a simpler approach, such as tuning the existing query, is the better use of the same effort.

Finally, the README itself points to KNOWN_ISSUES.md for current issues and possible workarounds. That file exists because the plugin is not exempt from problems, in the README's own words. Anyone evaluating it should read that file before committing, since the README does not reproduce its contents.

How it differs from tuning boosts or using a separate reranker

The obvious alternative is doing nothing new: adjust boosts and function scores until the ordering looks right. The difference is where the knowledge lives. With boosts, relevance knowledge is encoded in the query, and every change is a query change that has to be reviewed and deployed. With this plugin, the knowledge lives in a model object and the query stays fixed. That makes the ranking behaviour testable offline against a judgement set, and it makes experiments reversible by swapping a model rather than rewriting a query.

The other alternative is a reranking service outside Elasticsearch: retrieve candidates, send them to an application that scores them, then reorder. That gives you any model framework you like and no plugin version constraints. It also adds a network hop, a second service to operate, and a synchronization problem, because the features the reranker computes must match the features used at training time. The plugin's answer to that last problem is that features are stored query templates executed by the same engine in both logging and serving, so the feature definitions cannot drift between the two. That is the trade the project makes: less flexibility in where scoring happens, more consistency in what gets scored.

Maintenance, licensing and the upgrade tax

The repository is not archived, and the last push was on 2026-06-16. The most recent release, v1.5.13-es9.3.5, carries the same date, with v1.5.13-es9.3.2 four days earlier and v1.5.13-es9.2.4 on 2026-02-19. The pattern in those three releases is the real maintenance story: the plugin version stays at 1.5.13 while the Elasticsearch version in the artifact name moves. Maintaining this plugin is largely a matter of rebuilding it against new Elasticsearch releases, which is also why the README asks users to submit pull requests for versions the project does not support.

That shapes the upgrade cost. Budget for a plugin artifact per Elasticsearch version you run, and check the releases page before planning an Elasticsearch upgrade rather than after. If you run a version with no prebuilt artifact, the Gradle build in the README is the fallback, and you own the result.

The licence is Apache-2.0, and the repository carries both LICENSE.txt and NOTICE.txt alongside a licenses/ directory. Apache-2.0 is a permissive licence that generally allows commercial use and modification, but the NOTICE file and the bundled dependency licences under licenses/ are part of what you are distributing when you ship a built plugin. This is not legal advice; if your organisation has a policy on third-party licence review, the files to hand to that process are LICENSE.txt, NOTICE.txt and the licenses/ directory.

Editorial conclusion

Adopt it if you already run Elasticsearch, have graded relevance judgements, and want model scores to decide the order of the top results without leaving the cluster. Do not adopt it if you need ranking inside a system that is not Elasticsearch, or if you cannot commit to matching plugin versions to Elasticsearch versions on every upgrade. Before installing, check the releases page for a prebuilt zip that matches your exact Elasticsearch version, read KNOWN_ISSUES.md, and confirm the feature templates you plan to log are cheap enough to run twice, once for logging and once for rescoring.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. o19s/elasticsearch-learning-to-rank on GitHub
3. Project website
4. README
5. Releases