Addax ETL: Docker Setup, Plugins, and Limits

Addax for Database and File Integration

Addax addresses batch data movement between systems that do not share a straightforward import path. The README describes it as an ETL tool supporting more than 30 SQL and NoSQL data sources. Named examples include MySQL, PostgreSQL, Oracle, SQLServer, ClickHouse, Elasticsearch, MongoDB, Redis, Kafka, S3, Hive, Excel, and several analytical databases. Its likely audience is infrastructure and data engineers who need repeatable transfers across heterogeneous stores without building a separate application for every pair of systems.

The project presents itself as the actively maintained successor to Alibaba DataX, which the README says has been frozen since 2023. Addax says its maintenance releases bundle dependency updates, CVE updates, and bug fixes. Plugin behavior is deliberately kept stable, while new data-source plugins are added when community demand justifies them. This is a conservative development model. It favors preserving established jobs and periodically incorporating maintenance changes into numbered releases rather than changing plugin interfaces frequently.

Addax is not presented as a visual workflow designer or a general transformation platform. Its documented control surface is a JSON job description, while source and destination behavior comes from plugins. It is best suited to batch transfer where the required systems appear in `support_data_sources.md` and the operator can follow the plugin-specific configuration. The README does not explain whether the job engine provides record-level transformation, schema inference, or scheduling. Those capabilities should not be inferred from the word ETL.

The project may fit organizations that already understand Alibaba DataX job conventions. However, Addax's identity as a successor does not prove complete compatibility. Engineers should compare connector names, parameters, column handling, and failure behavior before reusing an existing DataX configuration unchanged.

Addax JSON Jobs and Plugin Boundaries

The clearest documented execution mechanism is configuration-driven. An operator supplies a JSON job description, Addax runs it, and plugins provide access to the selected data source or target. The repository layout supports this modular interpretation: `core/` contains central project code, `plugin/` contains connector modules, `server/` contains server-side code, and `e2e/` contains end-to-end assets. The README does not expose class names or the internal scheduling sequence, so a more detailed description of the executor internals would be speculation.

Plugin breadth should be checked per integration rather than inferred from the total. The README names databases, analytical systems, messaging products, object stores, and file formats, but directs readers to `support_data_sources.md` for the full list. It also identifies `difference.md` as a plugin comparison document. Both files should be checked for the particular source and destination pair. A connector existing in the catalog does not itself document write modes, type conversion rules, bulk-operation support, or compatibility with a specific database version.

The top-level `pom.xml` and the organization of `core/` and `plugin/` identify a Maven-based Java project. The `Dockerfile` uses `maven:3.8.3-openjdk-17` as its build base. It defines a `release` target containing an assembled distribution and leaves a default `from-source` stage that compiles inside the container. The file comments say CI builds the release image from the distribution it just assembled, so the release image does not perform that compilation during the image build.

That separation has practical value for deployment consistency. A release container can correspond more closely to the archive produced by CI, while a source build can package code directly from a clone. The repository also includes `build-module.sh` and `shrink_package.sh`, but their accepted arguments and intended build contexts are not documented in the README. Engineers should read those scripts before constructing a private image pipeline rather than guessing their flags.

Running a First Addax Job Through Docker

The README provides Docker commands for downloading Addax and invoking a job. Docker must already be installed, and the selected image must contain a readable job file at `/opt/addax/job/job.json`. These commands do not create that configuration file, so the real first task is preparing a valid job for the chosen plugins.

The first command downloads the image tagged `latest` from Quay.io:

docker pull quay.io/wgzhao/addax:latest

After a successful pull, Docker should report that the image is available locally. Because `latest` is a mutable tag, the command does not identify a fixed Addax release. A production invocation should use a concrete image version after the project documents its available tags and compatibility policy.

The README then invokes the Addax shell entry point with a job file:

docker run -ti --rm --name addax \
  quay.io/wgzhao/addax:latest \
  /opt/addax/bin/addax.sh /opt/addax/job/job.json

This command allocates an interactive terminal, removes the container after the process exits, and passes `/opt/addax/job/job.json` to `/opt/addax/bin/addax.sh`. It does not mount a host directory into the container. As written, the example depends on the job file already existing at that path inside the image.

The README does not include the complete JSON schema or a runnable `job.json` in the displayed setup section. Adding reader, writer, parameter, or column names would create configuration that the project has not documented. Obtain the first job from Addax's online documentation or its connector documentation, then confirm that the referenced job path is available inside the chosen image.

The README also lists `install.sh` and compilation from source as alternatives, but the displayed installation-script section ends before giving the complete script command. The Docker sequence is the only complete runnable tutorial provided there.

Addax Recovery, Scheduling, and Configuration Gaps

The main production risk is connector-specific configuration mismatch. Addax depends on plugins, yet the README does not define plugin naming, credential fields, schema selection, column mapping, type conversion, or write modes. A JSON file can be well formed and still fail because a plugin is absent, the target rejects converted values, or a required parameter is missing. The connector documentation and complete `support_data_sources.md` file are therefore part of the effective configuration contract.

Operational recovery is a larger gap. The README does not document transaction boundaries, retries after partial failure, duplicate handling, exactly-once delivery, checkpointing, or rollback. It also does not say whether a failed load can resume from a safe point without repeating previously written records. The Docker example returns the Addax process's exit result through Docker, but that alone does not establish atomic loading or safe retry behavior. A team with strict delivery requirements must confirm those semantics for the selected reader and writer plugins.

Scheduling is also absent from the documented setup. Addax appears to accept an executable job file, but the README does not state whether it includes a scheduler or must be launched by cron, an orchestration service, or another external system. Resource controls are not described either. There are no documented settings for parallelism, batch size, connection limits, rate limiting, or memory management. Those omissions limit how confidently the Docker command can be treated as a production deployment design.

Secret handling is also undocumented. The README does not specify environment variables, mounted credential files, key-management integration, or whether credentials may appear directly in JSON. Operators should avoid placing secrets into an image layer or committing them to version control, but the README does not define an approved Addax-specific secret mechanism. This boundary must be resolved against the online documentation before a production job is built.

Addax and Alibaba DataX Have Different Maintenance Paths

Alibaba DataX is the most relevant alternative because Addax explicitly identifies itself as its successor. The practical difference described by the README is maintenance direction. Addax says Alibaba DataX has been frozen since 2023, while Addax publishes releases containing fixes and dependency or CVE updates. For organizations deciding whether to retain an old DataX deployment or evaluate Addax, that distinction matters.

Compatibility should not be assumed merely because Addax follows DataX. The README calls the configuration "simple JSON-based" but does not state that every old job can run without edits. The `difference.md` file is the repository's designated plugin comparison, so it is the first place to check reader and writer behavior. Engineers should compare exact connector names and configuration keys before migration.

A hand-written integration program is a different alternative. It can contain source-specific validation, type conversion, and recovery behavior directly in application code. That gives the team tighter control over unusual requirements, but every new source and destination combination may require additional implementation work. Addax instead centralizes that work in reusable plugins and describes the job through JSON, which reduces repeated connector code when the required integrations already exist.

Addax is therefore the more suitable option for repeatable batch movement across several supported systems. A hand-written program is easier to justify for one specialized integration whose error handling, transformation, or delivery guarantees cannot be expressed by the documented plugin interface. Neither choice should be made solely from the count of named systems.

Addax Release Cadence and Apache-2.0 Obligations

The repository is not archived, and the last push was on 2026-10-16. Recent releases include 6.2.0 on 2026-10-13, 6.1.0 on 2026-09-18, and 6.0.13 on 2026-08-31. These records support the project's maintenance-oriented positioning, but they do not prove that every connector behaves identically across releases.

Addax says maintenance releases occur roughly every month and may contain dependency updates, CVE updates, and bug fixes without a major new feature. That cadence can simplify upgrade planning when connector behavior remains stable. It still requires review of release notes before changing a production image, particularly when a maintenance release alters dependencies used by a database driver. The repository includes `release-notes/` and `RELEASING.md`, which are the relevant files for checking release-specific changes and the maintainer release process.

The project uses the `Apache-2.0` license. The repository contains a `LICENSE` file, and organizations normally need to retain applicable license and notice information when redistributing the software. This is a factual statement about the project's declared license, not legal advice. Dependencies and connectors may introduce separate notices or terms, and the README does not inventory those conditions. An adoption review should therefore inspect `LICENSE`, `pom.xml`, and the notices associated with the chosen dependencies and container image.

The upgrade cost depends on whether a job relies on changed dependencies, connector parameters, or bundled scripts. The README's policy of fixing bugs and updating dependencies reduces churn, but it does not document automatic job migration or a compatibility guarantee between every version.

Choosing Addax for a Java Batch Integration Stack

Addax is a reasonable candidate for Java-oriented data teams that need repeatable batch movement among the systems listed in `support_data_sources.md`. Its JSON job model and plugin architecture reduce the need to implement a separate transfer program for every supported connection. The documented Docker path also gives operators a concrete starting point without requiring a source build.

The decision should be narrower than the claim of support for more than 30 data sources. The repository does not document universal recovery, scheduling, rollback, or credential behavior. It should not be selected without confirming that the exact reader and writer pair supports the required operation and data types. The project's monthly maintenance releases and last push on 2026-10-16 show recent maintenance, while its Apache-2.0 declaration provides a defined license basis for engineering review.

The first adoption gate is therefore connector-specific: inspect `support_data_sources.md` and `difference.md`, select a fixed release, and validate a representative job. If that job requires guarantees Addax does not document, use a purpose-built application or an orchestration platform with explicit recovery semantics. If it only needs supported-plugin batch transfer, Addax is a more direct fit than writing another database integration program.

Editorial conclusion

Adopt Addax when repeatable batch movement between supported databases, analytical stores, messaging systems, or files is the primary requirement and JSON-based plugin jobs suit the team. Do not adopt it solely for the connector count when strict checkpointing, rollback, scheduling, or recovery semantics are required but undocumented. Verify the exact connector pair in `support_data_sources.md`, inspect `difference.md`, and pin a release before moving the documented Docker test job into production.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. wgzhao/Addax on GitHub