LLM Engineer's Handbook repository: the code behind the book

What the LLM Engineer's Handbook repository is, and what it is not

This is the official companion repository for the book LLM Engineer's Handbook by Paul Iusztin and Maxime Labonne, published by Packt. The README frames the goal plainly: to build an end-to-end LLM-based system, covering data collection and generation, an LLM training pipeline, a simple RAG system, AWS deployment, monitoring, and a testing and evaluation framework. The book itself is the product. The repository is the code that goes with it.

That distinction matters more than it usually does. There is no PyPI package named llm-engineering to pip install. The pyproject.toml declares the project name llm-engineering at version 0.1.0, but the README never describes publishing it. Nothing in the repository layout suggests a library interface meant for third parties. The llm_engineering package is application code: it wires crawlers, embedding models, a vector store, a NoSQL warehouse, a FastAPI inference service and SageMaker deployment together.

So the audience is narrow and specific. You are a Python engineer or ML engineer who already knows what embeddings and retrieval are, and you want a full worked example of how those pieces fit into a production-shaped system. If you are looking for a drop-in RAG library, this is the wrong shape of artifact, and you will spend your first hour fighting imports rather than reading retrieval code.

The DDD layering and the one-way import rule

The most transferable idea in the repository is not the RAG code. It is the package structure. The README states that llm_engineering follows Domain-Driven Design principles and splits into four subpackages: domain for core business entities, application for business logic, crawlers and the RAG implementation, model for LLM training and inference, and infrastructure for external service integrations such as AWS, Qdrant, MongoDB and FastAPI.

The README then states the import direction explicitly: infrastructure, then model, then application, then domain. Read that as a dependency rule. Domain code sits at the bottom and knows nothing about MongoDB or FastAPI. Infrastructure sits on top and depends inward. This is what lets you swap Qdrant for another vector store without touching the retrieval logic, at least in principle.

In practice the rule is a convention enforced by review, not by tooling. Nothing in pyproject.toml or the top-level files enforces layering. There is a ruff.toml in the repository root, and ruff can be configured with import rules, but the repository does not show such configuration. So if you copy this structure into your own project, understand that you are copying a discipline, not a guardrail. The payoff is real when the discipline holds; the cost is that a single careless import from domain into infrastructure quietly undoes it.

The pipelines and steps directories are separate from the package on purpose. Pipelines hold ZenML pipeline definitions, steps hold the reusable ZenML steps that pipelines compose, and configs holds the YAML files that control pipeline and step execution. That split means the same step can be reused across pipelines with different configuration, which is the point of the ZenML layer.

Installing the LLM Engineer's Handbook code and running the first pipeline

The README's installation section begins with cloning and then insists on Python 3.11. The dependency table lists pyenv 2.3.36 or newer as optional, Python 3.11 as required, Poetry between 1.8.3 and 2.0, Docker 27.1.1 or newer, AWS CLI 2.15.42 or newer, and Git 2.44.0 or newer. Note the Poetry upper bound. Poetry 2.x is outside the stated range, and pyproject.toml pins the package metadata to the older toolchain.

Start with the clone and the version check:

git clone https://github.com/PacktPublishing/LLM-Engineers-Handbook.git
cd LLM-Engineers-Handbook
python --version  # Should show Python 3.11.x

The README comments that the version output should read Python 3.11.x. If it does not, the README recommends pyenv for a project-specific interpreter, and the repository carries a .python-version file at the root, which is what pyenv reads.

Next, bring up the local backing services. The docker-compose.yml defines exactly two: MongoDB on port 27017 with the root username and password both set to llm_engineering, and Qdrant exposing ports 6333 and 6334. Both mount named volumes and restart always.

docker compose up -d

After that, the .env.example file tells you what the code expects. It marks OPENAI_API_KEY, HUGGINGFACE_ACCESS_TOKEN and COMET_API_KEY as required even for local work, with OPENAI_MODEL_ID defaulting to gpt-4o-mini. The MongoDB connection string in the example points at 127.0.0.1:27017 with the same llm_engineering credentials the compose file sets. Qdrant has a USE_QDRANT_CLOUD flag that defaults to false, so local Qdrant is the default path.

Finally, the README points at tools/run.py as the entry point for ZenML pipelines, and pyproject.toml defines Poe the Poet tasks that wrap it. One of them is a worked example of the shape:

poetry run python -m tools.run --run-etl --no-cache --etl-config-filename digital_data_etl_maxime_labonne.yaml

That task is named run-digital-data-etl-maxime. The flags are --run-etl, --no-cache and --etl-config-filename, and the config file lives in configs/. What you should see is the ETL pipeline executing under ZenML, with artifacts landing in the orchestrator. If you have not configured a ZenML stack, expect to do that first; the README defers that setup to Chapters 10 and 11 of the book.

Where the repository assumes you already have accounts

The cloud services table lists eight external dependencies: HuggingFace for the model registry, Comet ML for experiment tracking, Opik for prompt monitoring, ZenML for orchestration and artifacts, AWS for compute and storage, MongoDB for the NoSQL database, Qdrant for the vector database, and GitHub Actions for CI/CD. The README says you do not have to do anything about these yet, because the installation and deployment sections will guide you.

That guidance is in the book, not the repository. The README states that Chapter 2 walks through each tool and that Chapters 10 and 11 give step-by-step setup instructions. This is the single biggest friction point for someone arriving from GitHub alone. You can clone the repository, install dependencies and start the two containers without reading a page of the book. You cannot run a training pipeline or deploy the inference service without accounts on the services above and credentials wired into .env.

The .env.example makes the split visible. It labels the first block as required even when working locally (OpenAI, HuggingFace, Comet ML) and the second block as required when deploying, with default values otherwise (MongoDB host, Qdrant Cloud URL and API key, and the AWS variables: AWS_ARN_ROLE, AWS_REGION defaulting to eu-central-1, AWS_ACCESS_KEY, AWS_SECRET_KEY). Note that AWS_REGION defaults to eu-central-1 rather than a US region, which is a small signal that the authors wrote this for a European audience and did not generalize the default.

There is also a paid tier hidden in plain sight. Opik is a Comet ML product, and the README links to Comet ML's product page for both the experiment tracker and prompt monitoring rows. The repository does not document a free tier, a self-hosted Opik option, or what happens when you exceed a quota. If cost matters to you, that is a question to answer before you start, not after.

The Dockerfile, the Chrome dependency and the AWS path

The Dockerfile is worth reading before you build anything. It starts from python:3.11-slim-bullseye, sets POETRY_VERSION to 1.8.3, and installs Google Chrome from Google's apt repository. The system dependency list then adds build-essential, gcc, python3-dev, libglib2.0-dev and libnss3-dev.

Chrome is there because the ETL stack is browser-based. pyproject.toml lists selenium, webdriver-manager, beautifulsoup4, html2text, jmespath and chromedriver-autoinstaller under the comment Digital data ETL. This is a scraping pipeline, not an API client. That has consequences. Chrome images are large, headless scraping is fragile against sites that change their markup or add bot detection, and the dependency list includes fake-useragent in the RAG group, which tells you the authors anticipated being blocked. None of this is a flaw in the design; it is the honest cost of collecting digital data rather than buying it.

The AWS dependency group is separate and opt-in. pyproject.toml puts sagemaker, s3fs, aws-profile-manager, kubernetes and sagemaker-huggingface-inference-toolkit under tool.poetry.group.aws.dependencies, and the Dockerfile installs with --without dev. So a container build does not pull the AWS SDK unless you change the install flags. The README's feature list calls the deployment production-ready, and the tooling supports that claim: SageMaker for endpoints, S3 for storage, Kubernetes in the dependency tree. What the README does not document is teardown. There is no described command to remove a deployed endpoint or empty a bucket, and that omission is the one that costs money if you walk away.

The trained model is published, the pipeline that made it is not reproducible for free

The README links to mlabonne/TwinLlama-3.1-8B-DPO on Hugging Face as the final trained model, and says you can download and use it. That is a genuine convenience. If your interest is the model rather than the pipeline, you can skip the training sections entirely and pull the weights.

If your interest is the pipeline, the training path is heavier than the RAG path. torch is pinned at 2.2.2, datasets at 3.0.1, and the AWS group adds SageMaker. The README's dependency table lists AWS CLI 2.15.42 or newer as a local requirement, which is a hint about how central cloud compute is to the intended workflow. Training an 8B model is not something you do on a laptop, and the repository does not pretend otherwise.

The RAG side is more approachable. tools/rag.py is described as demonstrating usage of the RAG retrieval module, and the RAG dependencies are ordinary: langchain, langchain-openai, langchain-community, sentence-transformers, qdrant-client, jinja2 and tiktoken. If you want to evaluate this repository cheaply, read tools/rag.py and the application-layer retrieval code before you touch the training pipeline. That is where most readers will get the most transferable ideas per hour invested.

How this differs from a RAG framework like LlamaIndex

The obvious comparison is a RAG framework such as LlamaIndex. The difference is not quality; it is what each artifact is for.

A framework gives you abstractions and expects you to supply the application. You install it, import it, and compose its building blocks into your own system. It is maintained for that purpose, versioned for that purpose, and its documentation is written for strangers.

This repository inverts that. It is a complete application with opinions baked in: MongoDB as the warehouse, Qdrant as the vector store, ZenML as the orchestrator, Opik for prompt monitoring, FastAPI for serving, SageMaker for deployment. You do not compose these; you inherit them. The value is in seeing the whole system assembled and in the DDD layering that keeps the pieces separable. The cost is that adopting it means adopting its entire stack. If you already run Pinecone and Weights and Biases, you are rewriting the infrastructure layer before the code does anything useful for you.

LangChain sits in an interesting middle position here. It is a dependency of this repository, listed alongside langchain-openai and langchain-community, so the project uses a framework rather than replacing one. The repository's contribution is the architecture around LangChain, not the retrieval primitives themselves. That is a more honest framing than calling it an alternative to anything.

Editorial conclusion

Adopt this repository if you are working through the LLM Engineer's Handbook and want the runnable code, or if you want a worked example of DDD layering in an ML codebase. Do not adopt it as a dependency: there is no published package, and every pipeline expects HuggingFace, Comet ML, Opik, ZenML, MongoDB, Qdrant and AWS accounts. Before you start, verify that your Python is 3.11, that Poetry is between 1.8.3 and 2.0, and that you have read Chapter 10 and Chapter 11 of the book, because the README does not document rollback or teardown for the AWS deployment it describes.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. PacktPublishing/LLM-Engineers-Handbook on GitHub
4. Project website
5. README