# ChunJun runs batch and streaming ETL jobs on Apache Flink

> ChunJun, formerly FlinkX, is a Flink-based data integration framework that abstracts databases into reader, writer and lookup plugins, configured by JSON template or Flink SQL. Its README is candid that the Flink version must match the branch exactly or jobs fail at runtime.

**DTStack/chunjun** — A data integration framework

- Repository: https://github.com/DTStack/chunjun
- Website: https://dtstack.github.io/chunjun/
- Stars: 4,104 · Forks: 1,684
- Language: Java
- License: Apache-2.0
- Published: 2026-10-16 · Updated: 2026-10-16 · Language: en
- Canonical page: https://hysenlabs.com/projects/dtstack-chunjun

## Reader, writer and lookup: the abstraction that defines the project

ChunJun describes itself as a distributed integration framework based on Apache Flink, and the abstraction it applies to a database is the whole design.

Databases are turned into three kinds of plugin: reader/source plugins, writer/sink plugins and lookup plugins. A task then reads from one, writes to another, and optionally looks up against a third. The claimed consequence is extensibility: a newly added data source plugin can integrate with the existing ones instantly, and the plugin developer does not need to care about the code logic of other plugins.

That last clause is the interesting claim. In a framework where every connector shared one big switch statement, adding Oracle would mean touching Oracle-adjacent code. Here a plugin is a boundary, so the work is confined to the new boundary.

The README claims support for synchronisation and calculation across more than 20 data sources, naming MySQL, Oracle, SQLServer, Hive and Kudu among them. Note the word calculation as well as synchronisation: this is not only a copy tool, it also transforms while moving.

The repository layout backs up the plugin model. `chunjun-connectors/` sits alongside `chunjun-core/`, `chunjun-ddl/`, `chunjun-formats/`, `chunjun-clients/`, `chunjun-server/`, `chunjun-assembly/`, `chunjun-examples/`, `chunjun-dirty/`, `chunjun-metrics/`, `chunjun-restore/`, `chunjun-docker/` and `chunjun-catalog/`. There is also a `.gitmodules` entry, so some connectors live in their own repositories and are pulled in as submodules.

## Version alignment is the failure mode you will hit first

The README puts the branch-to-Flink correspondence in a table, and warns about it directly: if the versions are not aligned, problems such as `Serialization Exceptions` and `NoSuchMethod Exception` occur in tasks.

| Branches | Flink version |
| master | 1.16.1 |
| 1.12_release | 1.12.7 |
| 1.10_release | 1.10.1 |
| 1.8_release | 1.8.3 |

This is worth dwelling on because of the shape of the failure. The warning does not describe a job that refuses to start. It describes a job that starts and then fails while deserialising, which is the harder kind of problem to diagnose because the submission looks successful.

It also constrains the maintenance decision. Releases visible for the project include v1.16.1 on 2025-12-02, then v1.12.8 in February 2023 and v1.12.7 in November 2022. The repository is not archived and the last push was on 2026-10-09, so work continues, but the version story runs along Flink lines rather than ChunJun's own.

A related detail: the project was initially called FlinkX and renamed to ChunJun on February 22, 2022. Recipes, documentation and blog posts from before that date refer to a name that no longer matches the artefact, which matters when you are searching for prior art.

## Building from source and the javac-shaded wall

The build is a Maven multi-module build with a wrapper, so the source tree builds without a system Maven.

```shell
git clone https://github.com/DTStack/chunjun.git
./mvnw clean package
```

An alternative path is the build script:

```shell
sh build/build.sh
```

The README then documents a specific and annoying failure in the Common problem section. Compiling the `ChunJun-core` module can throw `Failed to read artifact descriptor for com.google.errorprone:javac-shaded`. The underlying cause is the Spotless plugin at `com.diffplug.spotless:spotless-maven-plugin:2.4.2`, whose `spotless-check` goal pulls `com.google.googlejavaformat:google-java-format:jar:1.7`, which in turn depends on a javac-shaded artifact that the configured mirror refuses to serve.

The documented workaround is to fetch the jar from Maven Central directly and install it locally:

```shell
mvn install:install-file -DgroupId=com.google.errorprone -DartifactId=javac-shaded -Dversion=9+181-r4173-1 -Dpackaging=jar -Dfile=./jars/javac-shaded-9+181-r4173-1.jar
```

Two things follow. The first is that formatting is enforced at build time, so a build that fails in the style check is telling you something about code style, not about ChunJun. The second is that the error mentions `aliyunmaven`, which suggests the default or common configuration uses an Aliyun mirror. If your environment cannot reach that host, the same step fails at a different point.

A Linux user who built on Windows gets one more warning: run `sed -i "s/\r//g" bin/*.sh` to strip carriage returns from the shell scripts.

## Running a job: local, standalone, yarn-session

Submission modes are documented one at a time, and each has a different environment requirement.

Local mode needs neither a Flink environment nor Hadoop. It starts a JVM process in the local environment, and you run it from the distribution directory:

```shell
sh bin/chunjun-local.sh  -job $SCRIPT_PATH
```

`$SCRIPT_PATH` is the path to the task script. After that the task runs locally, which is the fastest way to check a configuration file before submitting it anywhere.

Standalone mode needs a Flink standalone cluster and no Hadoop. The jars are copied into the Flink lib directory:

```shell
cp -r chunjun-dist $FLINK_HOME/lib
sh $FLINK_HOME/bin/start-cluster.sh
```

The copy must be done on every machine of the cluster, and the README explains the consequence plainly: otherwise some jobs fail with `ClassNotFoundException`. That is a specific enough failure that it is worth knowing in advance.

The submission is then:

```shell
sh bin/chunjun-standalone.sh -job chunjun-examples/json/stream/stream.json
```

The Flink web UI defaults to port 8081 and is configurable in `flink-conf.yaml`, and the README says task status can be observed there.

Yarn session mode needs both the Flink jars and a Hadoop environment, and the yarn session has to be started before the job is submitted. Yarn-per-job is listed among the supported submission methods. Docker one-click deployment and Kubernetes deployment are also claimed in the feature list, with a `chunjun-docker/` module in the repository.

One naming detail that catches people: if you built with Maven the distribution directory is `chunjun-dist`, while a downloaded tar.gz unpacks to something like `chunjun-assembly-${revision}-chunjun-dist`.

## What the feature list claims, and where to be sceptical

The feature list in the README is long, so it is worth separating the structural claims from the ones that need verification.

Structural, and plausible from the Flink foundation: incremental synchronisation alongside full synchronisation, interval training, both offline and real-time scenarios, DDL synchronisation including `CREATE TABLE` and `ALTER COLUMN`, and breakpoint resuming with disaster recovery through the Flink checkpoint mechanism.

Needing verification: the claim of stable production deployment in thousands of companies, and the monitoring statement. There is a `chunjun-metrics/` module and the README mentions indicator monitoring plus dirty data storage with a `chunjun-dirty/` module, but it does not describe what is collected, where it goes or what a dashboard looks like. Treat that as a capability to test rather than a feature to rely on.

Configuration is either a JSON template or a SQL script, and the SQL is compatible with Flink SQL syntax. That compatibility is doing real work in the design: a team already writing Flink SQL can express the transformation rather than learning a bespoke expression language, which is the main alternative reason to choose this kind of framework over writing a Flink job directly.

The project also carries a Chinese-language README at README_CH.md and documentation under `website/`, with the official site at dtstack.github.io/chunjun. Licence is Apache-2.0, which permits use inside proprietary products and modification with attribution.

## Conclusion

ChunJun fits a Java and Flink shop that already runs Flink clusters and needs batch or streaming synchronisation between MySQL, Oracle, SQLServer, Hive, Kudu and similar sources without writing Flink jobs by hand. It does not fit a team whose source of truth is a single modern lakehouse table format, where a native connector would do, and it does not fit an environment without a working Maven mirror, given the build failure the README documents. Verify first by aligning your Flink version with the branch table before building, because master tracks Flink 1.16.1 and a mismatch shows up as SerializationException or NoSuchMethodException inside the job rather than as a startup error.

## FAQ

### Which Flink version does the ChunJun master branch support?

The README table maps master to Flink 1.16.1, with 1.12_release at 1.12.7, 1.10_release at 1.10.1 and 1.8_release at 1.8.3. It warns that a mismatch produces SerializationException and NoSuchMethodException problems in tasks.

### How do I build ChunJun from source?

Clone https://github.com/DTStack/chunjun.git and run ./mvnw clean package in the project directory, or run sh build/build.sh. If the ChunJun-core module fails on javac-shaded during the Spotless check, install that jar locally with mvn install:install-file as documented in the README.

### How do I run a ChunJun job locally?

Go to the chunjun-dist directory and run sh bin/chunjun-local.sh -job $SCRIPT_PATH, where SCRIPT_PATH is the path to the task script. Local mode needs neither a Flink environment nor Hadoop.

### What was ChunJun called before?

It was initially known as FlinkX and was renamed to ChunJun on February 22, 2022. Documentation and examples written before that date use the older name.

### What license is ChunJun released under?

Apache-2.0, with the LICENSE file at the repository root. That permits use inside proprietary products and modification, provided the notice is kept.

## Sources

- [DTStack/chunjun on GitHub](https://github.com/DTStack/chunjun)
- [License: Apache-2.0](https://github.com/DTStack/chunjun/blob/master/LICENSE)
- [Project website](https://dtstack.github.io/chunjun/)
- [README](https://github.com/DTStack/chunjun/blob/master/README.md)
- [Releases](https://github.com/DTStack/chunjun/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/dtstack-chunjun