EFAK-AI: a Kafka monitoring platform with an LLM assistant built in

What EFAK-AI covers that a plain Kafka UI does not

Most Kafka web UIs stop at read-only inspection: list brokers, list topics, show consumer-group lag. EFAK-AI aims wider. The README describes four feature groups: an AI assistant with multi-model support and function calling, cluster monitoring across brokers, topics, partitions and consumer groups, distributed task scheduling sharded through Redis, and alerting over email, DingTalk and WeCom. The intended user is the operations engineer who already has a Kafka cluster and wants one place to look at health, lag and alerts. The AI assistant is the differentiator the project leads with, and it is also the part with the least verifiable behaviour in the documentation.

How the modules fit together

The repository is a Maven multi-module build with five modules: efak-ai for the LLM integration, efak-core for Kafka connection and monitoring logic, efak-dto for data transfer objects, efak-tool for utilities, and efak-web for controllers, services and the Thymeleaf front end. The Dockerfile builds all of them with maven:3.9-eclipse-temurin-17, then copies a single artifact, efak-web/target/KafkaEagle.jar, into an openjdk:17-jdk-slim runtime image. That single-JAR shape matters: the AI module is compiled into the same process as the monitoring code, so an LLM call path and a consumer-lag query share one JVM heap. The documentation states the stack is Spring Boot 3.4.5, MyBatis 3.0.4, MySQL 8.0 or later and Redis 6.0 or later, with Kafka 4.0.0 listed as the message queue. Redis is not optional: the README describes it as the backing store for distributed locks and task scheduling, so a deployment without Redis loses the scheduling features.

Installing EFAK-AI with Docker and running a first query

The README gives a quick-start script and two deployment paths. The Docker path is the shorter one. Clone the repository and bring the stack up, then follow the log output:

git clone https://github.com/smartloli/EFAK-AI.git
cd EFAK-AI
docker-compose up -d
docker-compose logs -f efak-ai

The application then answers at http://localhost:8080 with the default account admin / admin123. Note that docker-compose.yml uses network_mode: host and points at an external MySQL and Redis on 192.168.31.72, so the compose file assumes those services already exist and that you are on Linux, where host networking is fully supported. If you prefer the package route, the README builds and unpacks a tar.gz instead:

./build-package.sh
tar -zxvf efak-ai-5.1.0.tar.gz
cd efak-ai-5.1.0

That route needs a database before the first start, so create it and load the schema:

mysql -u root -p
CREATE DATABASE efak_ai CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
USE efak_ai;
SOURCE /opt/efak-ai-5.1.0/sql/ke.sql;

Then point config/application.yml at your MySQL and Redis, and start the service:

spring:
  datasource:
    url: jdbc:mysql://localhost:3306/efak_ai?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
    username: root
    password: your_password
  data:
    redis:
      host: localhost
      port: 6379

./bin/start.sh
tail -f logs/efak-ai.log

The README says the running process shows up as KafkaEagle, which is what you should see in ps aux | grep KafkaEagle. For a first real query, the documented monitoring endpoints are plain GETs, for example /api/cluster/list for the cluster inventory and /api/consumer/groups?clusterId=xxx for consumer-group state. Health is exposed at /actuator/health and returns db and redis component status, which is the quickest way to tell whether the datasource and Redis configuration took effect.

The AI assistant and its SSE endpoint

The assistant is exposed as a streaming endpoint rather than a request-response call. The README documents a GET to /api/chat/stream with modelId, message, clusterId and enableCharts parameters, returning Server-Sent Events with typed frames: thinking, content, chart and end. The chart frame carries chartData as a JSON string, which is how the assistant renders time-series results without a separate charting request. Function calling is the mechanism that lets the model pull live cluster data instead of answering from its own knowledge, and the README lists OpenAI, Claude and DeepSeek as supported providers. Because the endpoint is a GET with the prompt in the query string, long questions and cluster identifiers travel in the URL, which is worth remembering when you put a reverse proxy in front of it.

Where EFAK-AI is the wrong tool

Two constraints stand out. First, the dependency footprint: MySQL 8.0 or later plus Redis 6.0 or later plus JDK 17 is a real install, and the README does not document a SQLite or embedded mode. If you want a single container that holds its own state, this is not that project. Second, the AI layer is coupled to an external provider. The README lists OpenAI, Claude and DeepSeek as integrations but does not describe an offline or self-hosted model option, so an air-gapped cluster cannot use the assistant. There is also an operational trap in the shipped compose file: the datasource URL and Redis host are hardcoded to a private address, and the password appears as admin123 with a comment telling you to change it. Deploying that file unchanged points the application at someone else's network range.

Alternatives and the difference in approach

Kafdrop, which appears in the related searches for this project, takes the opposite design stance: a read-only viewer that you point at a broker list, with no application database and no scheduler. That difference decides the choice. If your question is only what is in this topic and how far behind is this consumer group, Kafdrop answers it without MySQL or Redis. EFAK-AI answers a broader question, adding alert routing, historical trend storage and the assistant, and charges you the operational cost of two backing services for it. KnowStreaming, also present in the search data, is the closer comparison in scope, since it is likewise a management platform rather than a viewer, but the documentation here does not describe its internals, so treat that as a name to evaluate separately rather than a documented equivalence.

Maintenance, licence and upgrade cost

The repository is not archived and the last push was on 2026-09-12. No releases were retrieved, so version history is not visible from the repository metadata; the README and Dockerfile both label the build 5.1.0. Upgrades are not free: the application owns a MySQL schema, and the README shows the initial load through sql/ke.sql without documenting a migration path between versions, so a schema change in a future release would need manual handling. On licensing, the README carries an Apache 2.0 badge and the repository is described as open source, but the licence field in the project metadata is empty and no LICENSE file is listed among the top-level entries. Confirm the actual licence text before you build a distribution on top of it. Nothing here is legal advice.

Editorial conclusion

Adopt EFAK-AI if you already run MySQL and Redis, want one web UI for broker, topic and consumer-group state, and are willing to treat the LLM assistant as an experiment rather than a dependency. Skip it if you need a single static binary with no database, or if the AI feature is the only reason you are looking. Before deploying, verify three things: that the Apache 2.0 badge in the README matches the LICENSE file in the repository, that the hardcoded 192.168.31.72 datasource URL in docker-compose.yml has been replaced, and that the default admin/admin123 credentials are changed before the port is exposed.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. smartloli/EFAK on GitHub