Eland: pandas-style DataFrames and model uploads for Elasticsearch
Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
At a glance
- What is it?
- Eland is Elastic's Python client that puts a pandas-compatible DataFrame in front of an Elasticsearch index and adds a CLI for importing trained scikit-learn, XGBoost, LightGBM and NLP models. It is useful when the data is too large to pull into memory, and awkward the moment you need behaviour pandas does not expose through Elasticsearch.
- Who is it for?
- Adopt Eland if your working set lives in an Elasticsearch 9 cluster and you want notebook-style exploration without downloading the index, or if you need to push a trained scikit-learn, XGBoost or LightGBM model into Elasticsearch for inference. Do not adopt it as a general pandas replacement: the DataFrame delegates to Elasticsearch, so any operation outside the supported API surface fails rather than falling back to local execution.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 92 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap Eland fills between pandas and an Elasticsearch index
The README states the premise directly: in general, the data resides in Elasticsearch and not in memory, which allows Eland to access large datasets stored in Elasticsearch. That is the whole pitch. If you have a 27-column flights index and it fits in RAM, pandas is simpler and you should use pandas. Eland targets the case where the index does not fit, or where pulling it down is wasteful because the cluster can filter and aggregate faster than your laptop can.
The second audience is narrower and less obvious from the tagline. Eland ships tools to upload trained models from scikit-learn, XGBoost and LightGBM into Elasticsearch, plus a separate path for NLP models that pulls a model from Hugging Face Hub and installs it into the cluster. That makes Eland a deployment tool, not just an exploration tool. The two halves share a repository and a release cadence but almost nothing else in terms of workflow.
How eland.DataFrame defers work to the cluster
The README describes eland.DataFrame as wrapping an Elasticsearch index in a pandas-like API and deferring all processing and filtering of data to Elasticsearch instead of your local machine. The mechanism is visible in the connection example: ed.DataFrame takes either a host string or an elasticsearch.Elasticsearch instance, plus an es_index_pattern naming the index.
The memory reporting is the clearest signal of the architecture. The sample df.info() output shows memory usage: 80.0 bytes alongside Elasticsearch storage usage: 5.043 MB. Eighty bytes for a 13,059-row frame means the client is holding metadata, not rows. Dtypes are inferred from the index mapping, and the frame prints rows only when you ask for them (df.head() returns five).
Filtering follows the same path. The README example df[(df.Carrier=="Kibana Airlines") & (df.AvgTicketPrice > 900.0) & (df.Cancelled == True)].head() is expressed as pandas boolean operations, but the README's framing is that these become Elasticsearch queries. That is the trade-off in one line: you write pandas, the cluster executes Elasticsearch, and the two must agree on what each method means.
Installing Eland and the version-matching rule that catches people
Installation is two commands from the README. python -m pip install eland for the core package, or python -m pip install 'eland[pytorch]' if you intend to upload NLP models, since PyTorch is an optional extra. Conda users get conda install -c conda-forge eland. Debian-based systems may need build-essential, pkg-config, cmake, python3-dev, libzip-dev and libjpeg-dev for transitive dependencies; the README notes other distributions will need different package names.
The compatibility section is where the real constraint lives. Eland supports Python 3.10 through 3.13 and pandas 1.5 and 2. For Elasticsearch 9+ clusters, the major version must match Eland's major version. For the NLP with PyTorch feature specifically, the minor version must match too, and the README points Eland 8.x users at Elasticsearch 8.x. A mismatch here produces a failure at import time rather than a silent wrong answer, which is the better failure mode but still costs an afternoon if you did not read the matrix.
There is also a Docker image at docker.elastic.co/eland/eland, runnable interactively with docker run -it --rm --network host docker.elastic.co/eland/eland. The README shows it being used to run eland_import_hub_model without a local install, which is the pragmatic route if you only need the model upload path.
Uploading models: eland_import_hub_model and the sklearn path
The model upload tooling splits by model family. For transformer models, the README gives a concrete invocation inside Docker: eland_import_hub_model with --url http://host.docker.internal:9200/, --hub-model-id elastic/distilbert-base-cased-finetuned-conll03-english, and --task-type ner. The hub-model-id is a Hugging Face identifier, so the flow is: pick a model from the Hub, name the task type, point at your cluster, and Eland handles the transfer.
For scikit-learn, XGBoost and LightGBM, the README only says Eland provides tools to upload trained models into Elasticsearch. The exact CLI flags for those libraries are not in the supplied README text, so I cannot state them here. What is clear is the direction of travel: you train locally in the library you already use, then push the fitted model into the cluster so inference happens next to the data rather than in a separate serving process.
The version-matching rule applies hardest here. If you are uploading an NLP model, matching only the major version is not enough; the README requires the minor version to match as well. That means an Elasticsearch 9.2 cluster wants Eland 9.2, not 9.0. The release list shows v9.2.0, v9.0.1 and v8.18.2, so the pairing exists, but you have to check it deliberately rather than installing the latest and assuming.
Where the pandas-compatible claim breaks down
The README says eland.DataFrame has the same API as pandas.DataFrame except all data is in Elasticsearch. That "except" is doing a lot of work, and it is the single most important thing to understand before adopting Eland.
A pandas DataFrame executes locally. Every method either works or raises. An Eland DataFrame translates to Elasticsearch queries, which means the supported method set is bounded by what can be expressed as an Elasticsearch query and what the Eland authors have implemented. The README links to an Eland DataFrame API documentation page rather than claiming full parity, which is the honest framing. If a method you rely on is absent, you do not get a slower local fallback; you get an error or an unexpected result.
This matters most for anything involving row-wise Python functions, custom transforms, or reshaping operations that have no natural Elasticsearch equivalent. The practical test is to take your existing pandas analysis, list every method it calls, and check each one against the API reference before you commit to the port. The README also does not describe write-back semantics for DataFrame modifications, so treat Eland as read-oriented for analysis unless you confirm otherwise in the docs.
Eland against elasticsearch-py and plain pandas
The obvious comparison is elasticsearch-py, which Eland uses underneath. The README states Eland uses the Elasticsearch low level client to connect, and you can pass an elasticsearch.Elasticsearch instance straight into Eland APIs. So the choice is not Eland versus the client; it is whether you want a DataFrame abstraction on top of it.
With elasticsearch-py alone you write query DSL by hand: a bool query with term and range clauses, then you parse hits yourself. You get exact control over what the cluster does and no translation layer to debug. With Eland you write df[(df.Carrier=="Kibana Airlines") & (df.AvgTicketPrice > 900.0)] and the library builds the query. For exploratory work in a notebook, that is a real reduction in friction. For a production ETL job where you need to reason about the exact query sent to the cluster, the abstraction is a liability because you have to know what it generated anyway.
The second comparison is pandas itself, and the honest answer is that pandas wins whenever the data fits in memory. Eland's df.info() example reports 5.043 MB of Elasticsearch storage for 13,059 rows. At that size, loading into pandas costs nothing and you get the complete API. Eland becomes worth its constraints at the scale where that number is gigabytes, not megabytes.
Maintenance, licensing and what the release history suggests
Eland is licensed Apache-2.0, which permits commercial use and modification with the usual attribution and notice requirements. That is a permissive licence and imposes no copyleft obligation on your own code. I am not a lawyer and this is not legal advice; if you are redistributing Eland or a modified version, read LICENSE.txt in the repository rather than this summary.
The repository is not archived and the last push is dated 2026-06-16. Releases v9.2.0, v9.0.1 and v8.18.2 all landed in 2025, with v9.2.0 in October and the other two in April. That pattern is consistent with a project that tracks Elasticsearch release trains rather than one that ships on an independent schedule. The practical consequence for you: upgrading Elasticsearch is what forces an Eland upgrade, and the version matrix in the README is not optional reading. Budget for Eland and cluster upgrades as a single coordinated change, not two independent ones.
The maintenance cost that is easy to miss is the dependency surface. Eland supports pandas 1.5 and 2, Python 3.10 through 3.13, and optionally PyTorch. Each of those is a version range you have to keep inside, and the PyTorch extra pulls a large dependency tree if you use the NLP path. If your environment pins an older pandas, check that pin against the supported list before you add Eland to it.
Editorial conclusion
Adopt Eland if your working set lives in an Elasticsearch 9 cluster and you want notebook-style exploration without downloading the index, or if you need to push a trained scikit-learn, XGBoost or LightGBM model into Elasticsearch for inference. Do not adopt it as a general pandas replacement: the DataFrame delegates to Elasticsearch, so any operation outside the supported API surface fails rather than falling back to local execution. Before committing, verify three things against your own cluster: that your pandas version (1.5 or 2) is on the supported list, that your Eland major version matches your cluster major version, and that the specific methods in your analysis pipeline appear in the Eland DataFrame API reference.
Community notes