oreilly-ai-agents: a course repository for comparing agent frameworks, not a framework itself

What the repository is, and the problem it addresses

Agent frameworks arrived faster than documentation. A team evaluating LangGraph, CrewAI, AutoGen, SmolAgents and OpenAI's Agents SDK faces five different mental models, five different tool-calling conventions, and no shared example to compare them on. This repository is an attempt to close that gap by putting a working notebook next to each framework inside one Python 3.11 environment. It accompanies an O'Reilly live online training course and a video series, both linked from the README, so the notebooks are the written half of taught material rather than a standalone library. The audience is a developer or architect who already writes Python and wants to see how a ReAct loop, a supervisor pattern or an MCP connection looks in practice before committing a codebase to one vendor's abstractions. Nothing here is a runtime you install and depend on. It is a set of examples you read, run and adapt.

How the material is organised: notebooks as the unit of delivery

Everything is a Jupyter notebook, which is the primary language the repository is classified under. The README groups them by framework and by theme. Under third-party frameworks there are introductions to HuggingFace's SmolAgents, Google's Agent Development Kit with Google Search and custom tools, CrewAI, and Microsoft's Autogen. OpenAI gets two: Swarm and the newer Agents SDK. LangGraph gets the largest cluster, covering a RAG workflow, an evaluation of that workflow, simple ReAct agents, ReAct agents against local llama models through Ollama, a ReAct agent combined with MCP that also tests tool positional bias, a middleware notebook described as chat compression and guardrails, and reasoning-capable LLMs such as o1 and o3 used as ReAct agents. A separate evaluation group holds rubric-based output scoring, a longer alignment analysis, and tool-selection accuracy measurements extended for DeepSeek distilled models, DeepSeek R1 and Llama 4. Multi-agent material sits in its own folder with parallel fan-out/fan-in node execution, a dynamic network using a generic handoff tool, and a supervisor that delegates to sub-agents. There is also a track on building your own framework, pointing at a separate repository called squad-goals, with a multimodal variant that brings in Dalle-3 for image generation. The data flow in each case is the standard one for the framework being demonstrated: a prompt enters an agent, the agent selects tools, tools return observations, and the loop repeats until a stop condition. What the repository adds is the side-by-side placement, not a new execution engine.

Getting the notebooks running: the documented path

The README gives one supported path. You need Python 3.11, and it states that requirement explicitly. Verify it with python3.11 --version, then create the environment with python3.11 -m venv .venv. Activation differs by platform: source .venv/bin/activate on macOS and Linux, .venv\Scripts\activate on Windows. Confirm the interpreter with python --version, install dependencies with pip install -r requirements.txt, and start the notebooks from the activated environment with python3 -m jupyter notebook. If Python 3.11 is missing, the README offers brew install python@3.11 on macOS, sudo apt update followed by sudo apt install python3.11 python3.11-venv on Ubuntu and Debian, and a Windows installer from python.org with the Add Python 3.11 to PATH option checked. That is the whole documented setup. There is no environment.yml, no container image, no make target, and no per-notebook dependency file. Because requirements.txt is a single flat list covering every framework in the repository, you should expect it to be heavier than any one notebook needs, and you should expect version drift to be your problem rather than the repository's.

The evaluation notebooks are the part worth studying closely

Most of the repository teaches framework mechanics. The evaluation group does something less common: it treats the judge and the tool selector as objects of measurement. One notebook explores a rubric prompt for scoring generative output and notes positional biases when an LLM chooses between agent responses. A longer notebook extends that into alignment analysis. A third measures tool-selection accuracy across models and quantifies positional bias in auto-regressive LLMs, with variants for DeepSeek distilled models, DeepSeek R1 and Llama 4. The LangGraph MCP notebook folds the same concern into an agent that selects among tools, so bias testing is not confined to a standalone exercise. If you take one thing from this repository, take the framing that an agent's choice between two equivalent tools is a measurable property, not an implementation detail. The notebooks supply the method; they do not supply a benchmark suite you can run in CI, and no numeric results are published in the material available here.

Where the repository stops being the right tool

The first limitation is that it is courseware. Notebooks are written to be read top to bottom in a session, with narrative cells between code cells, which makes them poor inputs for automated testing or packaging. There is no setup.py, no pyproject.toml, no published release, and no versioned artefact to pin. The second is the environment constraint. Python 3.11 is required at the time of writing, and a single requirements.txt serves every notebook, so an incompatibility in one framework's dependencies can block the rest. The third is licence. The repository metadata shows no licence, and the README does not state one. Without a licence file, the default position is that the author retains rights, so copying notebook code into a commercial product is not something the material grants you. Treat the code as reference reading until you have clarified terms with the author. The fourth is that the frameworks themselves move. Notebooks against Swarm, the Agents SDK, Autogen and the various LangGraph APIs will age at the pace of those projects, and nothing in the repository pins them to a known-good combination beyond requirements.txt as it stood at the last push.

What to compare it against, and how the approaches differ

The closest alternative is a framework's own documentation and quickstart, for example the LangGraph tutorials or the CrewAI getting-started guide. Those are maintained by the framework authors, track the current API, and carry the project's own licence. The difference in approach is that a framework's own material is written to make that framework look coherent, and it will not show you the same task implemented in a competitor. This repository's reason to exist is the opposite: it puts CrewAI, Autogen, SmolAgents, OpenAI's SDKs and LangGraph in one place under one Python version so the comparison is at least nominally fair. A second alternative is a dedicated evaluation library for LLM outputs, which will give you statistical machinery, dataset handling and reporting that these notebooks do not. The trade-off is real in both directions. Framework docs are more current and better supported; this repository is more comparative and more honest about cross-framework friction, but only for as long as someone keeps updating it.

Maintenance, upgrades and the licence question

The repository is not archived and the last push is recent relative to the course it supports, so it is being kept alive alongside the live training. That is the maintenance model to plan around: updates follow the course schedule, not a release cadence. There are no releases to upgrade between, which means there is no changelog telling you what broke when a framework changed. Your upgrade work is re-running the notebooks and fixing whatever fails, notebook by notebook. The dependency surface is the entire agent ecosystem in one requirements.txt, so that work is not trivial. On licensing, the material supplied here shows no licence identifier, and I cannot infer one from the topics or the README. That matters more than usual for a repository whose whole purpose is to be copied into your own projects. If you intend to reuse notebook code beyond personal study, the specific next step is to open an issue asking the author to add a licence file, and to hold off on redistribution until one exists.

Editorial conclusion

Adopt this repository if you are learning agent frameworks side by side and want runnable notebooks for LangGraph, CrewAI, AutoGen, SmolAgents, OpenAI Swarm and MCP, including the evaluation notebooks on rubric judging and tool-selection positional bias. Do not adopt it as a production dependency or as a library to import: there is no package, no release, and no licence file visible in the supplied material, so redistribution rights are unresolved. Before using any notebook, verify the Python 3.11 requirement against your own interpreter, check that requirements.txt resolves for your platform, and confirm the current API of whichever framework the notebook targets, since these SDKs change between versions and the repository ships no lockfile.

Official sources

1. Issues
2. Project website
3. README
4. sinanuozdemir/oreilly-ai-agents on GitHub