# EFAK-AI: Kafka Monitoring with an Embedded AI Operations Assistant

> EFAK-AI is an open-source Kafka cluster monitoring and management platform that integrates large language model assistants for conversational cluster diagnosis, built on Spring Boot 3 and JDK 17 with Docker deployment, MySQL, and Redis.

**smartloli/EFAK** — A AI-Driven, Distributed and high-performance monitoring system, for comprehensive monitoring and management of kafka cluster.

- Repository: https://github.com/smartloli/EFAK
- Website: https://www.kafka-eagle.org/
- Stars: 3,183 · Forks: 788
- Language: Java
- License: not declared
- Published: 2026-09-16 · Updated: 2026-09-16 · Language: en
- Canonical page: https://hysenlabs.com/projects/smartloli-efak

## What EFAK-AI Monitors and Who Uses It

EFAK-AI targets Kafka operations teams who need ongoing visibility into broker node status, partition health, throughput metrics, consumer group lag, and storage capacity across one or more Kafka clusters simultaneously. The web interface displays real-time and historical data, and the README describes multi-cluster support as a core feature.

The project is a renamed and extended version of the earlier Kafka Eagle project. Its current name, EFAK-AI (Eagle For Apache Kafka - AI), reflects the addition of large language model integration. Earlier Kafka Eagle releases focused purely on metrics display; EFAK-AI adds an AI assistant layer that can be queried conversationally about the cluster state, with the AI able to call backend functions to retrieve live data rather than operating only on pre-loaded context.

The README identifies the primary target runtime as Spring Boot 3.4.5 on JDK 17, with MySQL 8.0 or higher as the main data store and Redis 6.0 or higher for distributed task coordination. This dependency profile means EFAK-AI is oriented toward teams already running a JVM-based operations stack, not toward single-operator setups where a statically compiled monitoring tool would be simpler to maintain.

## The AI Assistant Architecture and Function Calling

The AI assistant in EFAK-AI connects to external LLM providers, with the README listing OpenAI, Claude, and DeepSeek as supported integrations. The assistant uses server-sent events for streaming responses, so answers appear incrementally in the web UI rather than after a full generation delay.

The design uses Function Calling, which means the AI model can invoke server-side functions to retrieve real-time cluster data as part of answering a question. According to the README, this allows the assistant to query live Kafka metrics and generate charts from time-series data within the same conversational response. The streaming endpoint is /api/chat/stream, which accepts the model identifier, the user's message, a cluster identifier, and an enableCharts flag. The stream returns a sequence of SSE events with type fields indicating thinking, content, chart, and end states. This means the frontend can render partial answers and chart data progressively as the model responds. The chat history is persisted across sessions, and the assistant supports Markdown rendering with code highlighting and Mermaid diagram output.

Configuring the AI assistant requires an external API key from one of the supported providers. The README does not document how to set that key through the UI; it refers to configuration documentation in DEPLOY.md and FEATURE_PREVIEW.md.

## Docker Deployment and First Login

The simplest deployment path uses Docker Compose. Clone the repository and start all services with:

```bash
git clone https://github.com/smartloli/EFAK-AI.git
cd EFAK-AI
docker-compose up -d
```

The docker-compose.yml file builds the application image from the local Dockerfile (a multi-stage Maven build producing a KafkaEagle.jar artifact) and expects external MySQL and Redis instances. The compose file uses host network mode on Linux so that the container can reach localhost services directly; the README notes that macOS and Windows users should switch to bridge mode instead.

The application starts on port 8080. The default login is admin with the password admin123. To follow the application log during startup:

```bash
docker-compose logs -f efak-ai
```

For the traditional tar.gz path, the build script produces an efak-ai-5.1.0.tar.gz archive with a bin/ directory containing start.sh, stop.sh, restart.sh, and status.sh. The manual installation requires creating the efak_ai database and running the SQL initialization scripts before the first start:

```bash
./bin/start.sh
tail -f logs/efak-ai.log
```

The core configuration lives in config/application.yml. The SPRING_DATASOURCE_URL, SPRING_DATASOURCE_USERNAME, SPRING_DATASOURCE_PASSWORD, SPRING_DATA_REDIS_HOST, and SPRING_DATA_REDIS_PORT variables can also be set as environment variables when using Docker, which makes it straightforward to pass secrets through an orchestrator rather than baking them into a config file.

## Distributed Task Scheduling and Alert Channels

EFAK-AI uses Redis for distributed task coordination across multiple application nodes. The README describes a sharding system where monitoring tasks are partitioned across available instances, with automatic failover if a node goes offline and load-balancing that redistributes tasks dynamically. When running on a single node, the application detects the single-node environment and skips the sharding logic.

The alert system supports email, DingTalk, and WeCom (Enterprise WeChat) as notification channels. Thresholds can be configured through the web UI, and the README mentions dynamic threshold adjustment based on historical data, which reduces alert noise from short-lived spikes. Alert aggregation prevents repeated notifications for the same condition over a configurable time window.

The JVM memory settings are documented in the README for three common server sizes. For a server with 4 GB of RAM, the recommended setting is:

```bash
JAVA_OPTS="-Xms512m -Xmx2g -XX:+UseG1GC"
```

For 8 GB or more, the README suggests -Xmx4g with the G1GC garbage collector and -XX:MaxGCPauseMillis=200 to cap GC pause times. These settings apply whether the application is started through the shell script or through Docker's environment variable JAVA_OPTS.

## Where EFAK-AI Is a Heavier Choice Than Kafdrop

Kafdrop is an open-source Kafka web UI that runs as a single JVM process with no external database or cache dependencies. It displays topic lists, partition details, and message contents, making it easy to inspect a Kafka cluster quickly without standing up additional infrastructure.

EFAK-AI requires MySQL 8.0, Redis 6.0, and an optional Nginx reverse proxy in addition to the application server itself. The multi-stage Docker build and the SQL initialization scripts add deployment complexity that Kafdrop does not have. For a team that needs nothing more than cluster topology inspection and message browsing, Kafdrop is simpler to run and maintain.

The case for EFAK-AI is the AI assistant, the alert management system, and the historical trend storage that MySQL provides. If you need to retain performance data over weeks or months, set threshold-based alerts across multiple clusters, and ask conversational questions about cluster health backed by live metric lookups, EFAK-AI covers those requirements and Kafdrop does not. The trade-off is a deployment that requires three services instead of one.

## Tech Stack, License, and Upgrade Considerations

EFAK-AI is built on Spring Boot 3.4.5, which requires JDK 17 as the minimum Java version. The ORM layer is MyBatis 3.0.4, and the frontend uses Thymeleaf templates with TailwindCSS. The build system is Maven 3.6 or newer, and the Docker image uses a two-stage build starting from maven:3.9-eclipse-temurin-17 and producing a runtime image based on openjdk:17-jdk-slim.

The README states the project is licensed under Apache License 2.0. The repository metadata shows the license as unknown, so readers should verify the LICENSE file in the repository before redistributing.

The last push to the repository was on September 12, 2026. The project targets Apache Kafka 4.0.0 according to the tech stack table in the README. If your cluster runs an earlier Kafka version, verify client compatibility before deploying, since the Apache Kafka client API has evolved across major versions and some management operations differ between Kafka 2.x, 3.x, and 4.x.

## Conclusion

EFAK-AI is a reasonable choice for Kafka operators who want a single web interface that covers cluster health metrics, consumer lag, alert routing, and conversational AI diagnostics, and who are willing to run the full MySQL and Redis dependency stack. It is not suitable as a lightweight inspection tool: the dependency list is substantial and the AI assistant requires configuring an external LLM API key for any of the intelligence features to work. Before deploying, verify that your Kafka version is compatible (the README targets Apache Kafka 4.0.0 in the tech stack table) and that you have a MySQL 8.0 or newer database instance available.

## FAQ

### What LLM providers does EFAK-AI support for its AI assistant?

The README lists OpenAI, Claude, and DeepSeek as supported integrations for the AI assistant. The system uses Function Calling so the AI model can query live Kafka metrics during a conversation rather than relying only on pre-loaded context.

### Does EFAK-AI require Kafka itself to be running before deployment?

EFAK-AI monitors existing Kafka clusters; it does not bundle or start a Kafka broker. You need a running Kafka cluster to add as a monitored target after logging in. The Docker Compose file in the repository starts only the EFAK-AI application, MySQL, and Redis.

### Can EFAK-AI manage multiple Kafka clusters from a single dashboard?

Yes, the README describes multi-cluster support as a core feature. You can add multiple clusters through the web interface, and the monitoring data for each cluster is stored separately in the MySQL database.

## Sources

- [Issues](https://github.com/smartloli/EFAK/issues)
- [Project website](https://www.kafka-eagle.org/)
- [README](https://github.com/smartloli/EFAK/blob/main/README.md)
- [smartloli/EFAK on GitHub](https://github.com/smartloli/EFAK)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/smartloli-efak