# zdh_web is the visual control plane for the ZDH data collection and ETL suite

> zdh_web is the web front end of the ZDH platform, a Chinese big data suite whose configuration side lets teams define data collection jobs, ETL pipelines, scheduling dependencies, permissions and approval flows, while the execution side lives in separate Spark and FlinkX repositories. It suits a Chinese speaking data engineering team, and it is a control plane rather than an engine.

**zhaoyachao/zdh_web** — 大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批流,私域营销等模块

- Repository: https://github.com/zhaoyachao/zdh_web
- Stars: 536 · Forks: 188
- Language: HTML
- License: Apache-2.0
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/zhaoyachao-zdh-web

## The control plane, not the engine

The repository description is a data collection and extraction platform, and zdh_web is described as the visual management platform of the ZDH series of services, with modules for data collection, scheduling, permissions, approval flows and private domain marketing. The most important thing to understand is the split. The README states plainly that ZDH has two parts, a front end configuration side and a back end data ETL processing side, and that this repository contains only the front end configuration part. The execution side is in separate repositories, with zdh_server and zdh_flinkx given as GitHub and Gitee links, zdh_mock for a Netty based HTTP mock service, zdh_rqueue for a priority queue intended to control ETL task priority, zdh_auth for unified big data permission management across Hadoop, Hive, HBase and Presto which is still in development, and zdh_magic_mirror for customer management, customer profiling and intelligent marketing with the common, label, plugin, ship and variable modules. So zdh_web is the place where jobs are defined, not where they run. The primary language of the repository is listed as HTML because the interface is a Bootstrap front end served by a Spring Boot backend.

## The stack and the modules the web side offers

The technology list in the README is short: Bootstrap on the front end, and Spring Boot with Shiro, Redis and MyBatis on the back end, with the data ETL engine built on Spark where Hadoop and Hive are optionally deployed. The project description says the main function of ZDH is to pull data from sources such as HDFS, Hive, JDBC and HTTP JSON interfaces and write it to other sources such as HDFS, Hive and JDBC. The features list expands on the control side: support for standard SQL functions, interface-based configuration, quick copying of an existing task, support for external scheduling tools which requires adding specific interfaces, elastic scaling from a single machine to a cluster, customer level permissions, a built in scheduler with timed tasks, time series tasks and run counts, scheduling dependencies, SQL data warehouse processing, quality checks with reports, support for SHELL commands, SHELL scripts, JDBC queries and HDFS queries as scheduled tasks, local file upload and download, multi source ETL, task monitoring, and dynamic Drools rules for cleanup. The use case list adds data encryption, data migration, metadata and metric management, intelligent marketing, financial risk control, and OA style capabilities such as permissions and approval flows.

## Version compatibility is the hard part

The README devotes more space to versions than to features, and that is a fair signal of where the friction is. 4.7.18 is the last 4.x release, and it switched to Maven for dependency management and refactored the Quartz source, so it cannot share anything with earlier 4.x versions. The 5.x line rewrites all of the code, and the README notes that no new features were developed between 2022-02-06 and 2022-06-01 while the documentation was improved. From 5.1.0 onward there is no prebuilt package download, and users have to build from GitHub themselves, while earlier versions are downloadable and are not maintained. The README also says that the zdh_web and zdh_server sides stay in sync and that major versions are compatible, so a 1.0 zdh_web works with a 1.x zdh_server. For secondary development the README advises choosing the dev branch, because the dev branch is merged into master only after tests pass, so master may not be the newest code but is guaranteed to work. The repository is Apache-2.0 licensed, is not archived, has no GitHub releases, and the last push was on 2026-06-07.

## Preparing the environment and the configuration

The base environment is three things: MySQL 8 and Redis installed in advance for running a prebuilt package, and JDK 1.8, with the README noting that versions above JDK 8 are not supported, while from version 5.5.0 onward at least JDK 1.8 is required and the Spring Boot version is 2.3.4. The configuration list is where most of the work sits. The server port defaults to 8081 and is set with the parameter server.port. The datasource connection has to point at an external database, with MySQL 8 supported by default. Redis is configured through the parameters beginning with spring.redis. Each service instance needs its own myid value in the application configuration files, set to 1, 2, 3 and so on across multiple services. And the README flags one step as the most important and non optional: editing the RUN_MODE parameter in the bin/start.sh and bin/zdh_web.sh scripts, which selects the configuration file, with the default RUN_MODE=prod pointing at the application-prod configuration. The FAQ adds that an external database must be introduced, and that the data table structure is defined by the SQL files under release/db.

## Compiling and starting the service

The source build is a single script, and the README says to run it from the project root.

```bash
    sh build.sh
```

The run instructions differ by version, which is worth internalising before you debug a start failure. Before 4.7.13 you find zdh_web.jar in the target directory and run it with the file encoding flag set. Between 4.7.13 and 5.1.1 you run it from the xxx-RELEASE directory with an extra loader path pointing at libs/ and conf/ so that the external configuration and libraries are picked up. From 5.1.1 onward the recommended route is the start scripts, with sh bin/start.sh or sh bin/zdh_web.sh start to launch and sh bin/stop.sh or sh bin/zdh_web.sh stop to shut down. For developers running the source inside IntelliJ IDEA, the README says an environment variable has to be set in advance, ZDH_RUN_MODE with the optional values tmp, prod or dev. The repository carries build.sh and build.bat for the two platforms, an mvnw wrapper alongside the Maven wrapper files, pom.xml for the build definition, and the screw.sh and smart-doc.sh scripts.

## The FAQ answers that will save you an afternoon

Four entries in the README's FAQ are the ones a new user hits immediately. The first is a SQL structure error, and the cause is that the Quartz related tables must be upper case. The second is log level, and the answer is to change the level in the logback configuration. The third is the serial versus parallel scheduling mode, and the difference is concrete: serial mode checks whether the previous task run has finished, while parallel mode does not check the previous task state and generates the time automatically. The fourth is the data table structure, which follows the release/db/zdh_xxx.sql files. The last entry is a genuine gap: Kerberos authenticated Hadoop, Hive and HBase services are not supported for reading, and the README says Kerberos authentication is expected in a 5.x version. That single line is the clearest statement of a limitation in the whole document, and it is the one to check against your own cluster before you plan an install.

## Where zdh_web is the wrong choice

Four boundaries are worth naming. The first is that this repository alone will not move any data, and a team that installs only zdh_web ends up with a configuration interface for jobs that cannot run, so the execution repositories have to be deployed alongside it. The second is the version history, and it is not a small thing: a 4.x to 5.x move is a code rewrite rather than an upgrade, the Quartz refactor in 4.7.18 makes it incompatible with every earlier 4.x, and from 5.1.0 you build the artefact yourself with no packaged binary to fall back on. The third is the Kerberos gap and the partially finished work. The unified permission module for Hadoop, Hive, HBase and Presto is listed as under development, the intelligent marketing module is described as not fully complete with only some functions working, and the queue project for ETL priority control is still a plan. The fourth is the documentation. The README says at the top that the project has no other documentation and asks users and developers to read it in full, and the detailed function descriptions are promised for later in the form of a blog series, so the README is where the operational detail has to come from, and it is written in Chinese with only a brief English description document alongside. The alternative to weigh is not another Chinese big data suite but a different approach: configuring pipelines in a general purpose orchestration tool with a code-first definition and a thin web UI, which gives you version control over the pipeline definitions themselves and avoids the requirement to run a Java platform with MySQL, Redis and Quartz. For a team that already runs the ZDH stack, or that wants a drag and drop configuration surface over Spark and FlinkX without writing each pipeline by hand, zdh_web has a clear role, and the module split makes that role explicit.

## Conclusion

Adopt zdh_web if your team runs a Chinese speaking big data group that wants collection, transformation, scheduling and permissions defined through a web interface rather than hand written pipelines, because the platform bundles the configuration side with a scheduler that resolves task dependencies for you. Do not adopt it if you need the execution engine in the same repository, since the Spark and FlinkX processing side lives in zdh_server and zdh_flinkx, and do not adopt it if you need Kerberos authenticated access to Hadoop, Hive or HBase, which the README lists as unsupported. Verify two things before you start: which ZDH version you are on, because 4.7.18 is the last 4.x release, the 5.x line rewrites the code, and versions from 5.1.0 onward ship no prebuilt package, and which branch you build from, since the README says the dev branch is merged to master only after tests pass so master may not be the newest code but is the usable one. The practical first step is to install MySQL 8 and Redis, set the myid value and the RUN_MODE parameter in the application configuration and the start scripts, and compile the source with sh build.sh before you touch any job definition.

## FAQ

### What is zdh_web?

It is the visual web management platform of the ZDH big data suite, covering data collection, ETL configuration, scheduling, permissions, approval flows and intelligent marketing modules. It is the configuration side only, with the data processing engine living in separate repositories such as zdh_server and zdh_flinkx.

### What does zdh_web need installed before it runs?

MySQL 8 and Redis must be installed in advance for a prebuilt package, and JDK 1.8 is required, with versions above JDK 8 not supported according to the README. From version 5.5.0 onward at least JDK 1.8 is needed and the Spring Boot version is 2.3.4.

### How do I build and run zdh_web?

Compile the source with sh build.sh from the project root. Before 4.7.13 you run zdh_web.jar from the target directory, between 4.7.13 and 5.1.1 you run it from the xxx-RELEASE directory with a loader path for libs/ and conf/, and from 5.1.1 onward you use sh bin/start.sh or sh bin/zdh_web.sh start to launch.

### Which configuration values must be changed before zdh_web will start?

The server port via server.port, which defaults to 8081, the datasource connection to an external database, the Redis settings under the spring.redis parameters, the myid value in each application configuration file, with 1, 2, 3 across services, and the RUN_MODE parameter in bin/start.sh and bin/zdh_web.sh, which the README calls the most important and non optional step.

### Does zdh_web support Kerberos authenticated Hadoop, Hive or HBase?

Not in the versions documented. The README lists reading from Kerberos authenticated Hadoop, Hive and HBase services as not supported and says Kerberos authentication is expected in a 5.x version.

## Sources

- [Issues](https://github.com/zhaoyachao/zdh_web/issues)
- [License: Apache-2.0](https://github.com/zhaoyachao/zdh_web/blob/master/LICENSE)
- [README](https://github.com/zhaoyachao/zdh_web/blob/master/README.md)
- [zhaoyachao/zdh_web on GitHub](https://github.com/zhaoyachao/zdh_web)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/zhaoyachao-zdh-web