everything-curl: documenting the C source, not just the API

SUMMARY.md is the table of contents, and the build greps it

The most informative thing about this repository is that its structure is not hand-maintained, and the top of the Makefile is where that becomes visible.

OUT = bookindex.md

MDS := $(shell grep -o '[a-z0-9/-]\+\.md' SUMMARY.md | grep -v bookindex.md) README.md

IMDS := $(shell grep -o '[a-z0-9/-]\+\.md' SUMMARY.md | grep -vE '(bookindex.md|how-to-read.md)') README.md

SUMMARY.md is the table of contents file that mdBook requires. The build does not have its own list of chapters; it greps SUMMARY.md for markdown filenames and uses the result as the input to every downstream target. Two variants are computed, one excluding bookindex.md and one excluding both bookindex.md and how-to-read.md, because those two files are generated or are front matter that should not be indexed or concatenated.

That is a small design decision with a large payoff. Adding a chapter means adding it to SUMMARY.md and nothing else. Rename a file and the index, the single-file concatenation, the link check, the prose check and the word check all follow, because they all derive their file list from the same grep. There is no manifest to keep in sync and no way for the index to drift from the table of contents.

It also explains why the repository's primary language is recorded as Perl. The chapter list is extracted with grep and shell, and the three scripts that do the real work are mkindex.pl, which generates bookindex.md, uni.pl, which produces the single concatenated uni.md, and badwords.pl, which checks the prose against a blacklist. A book about a C project is maintained by a Perl toolchain, and the choice of Perl for text munging over a hundred markdown files is as reasonable as it sounds.

The grep pattern itself is worth noting. It matches lowercase letters, digits, slash and hyphen followed by .md, which means it finds chapter paths and ignores anything with an uppercase letter or a character outside that set. That is a deliberate constraint on how chapters may be named, enforced implicitly by the build rather than by a documented convention.

mdBook for the web since 2024, pandoc for everything else

There are two rendering pipelines in this repository and they have coexisted since mdBook took over the website.

The README states that the book website is hosted by Fastly, at everything.curl.dev, and that the content is rendered by mdBook since March 18th, 2024. That date is specific enough to be a deliberate record of a migration, and the consequence is that the website is now built by a Rust tool that expects a SUMMARY.md, which is exactly the file the Makefile now greps.

The other formats still go through pandoc, and the targets are longer:

everything-curl.pdf: uni.md pdf.txt
	pandoc --lua-filter=warn_bad_links.lua  -o everything-curl.pdf pdf.txt uni.md --toc

everything-curl.epub: uni.md epub.txt
	pandoc --lua-filter=warn_bad_links.lua -o everything-curl.epub --epub-cover-image=cover.jpg epub.txt uni.md

The PDF and the ePub are pandoc runs over uni.md, a single file that uni.pl builds by concatenating every chapter in SUMMARY.md order, with a prepended pdf.txt or epub.txt for format-specific front matter and a table of contents flag. The ePub adds cover.jpg as the cover image. The Lua filter is applied to both.

There is a third target, a standalone HTML bundle, and it is the oldest one:

everything-curl.html: uni.md $(MDS)
	pandoc --lua-filter=warn_bad_links.lua -o everything-curl.html uni.md
	rm -rf everything-curl
	mkdir -p everything-curl
	cp -p --parents `grep -oe 'img src="[0-9a-z/.-]*' everything-curl.html | cut -c10-` everything-curl/
	sed '/Generated Content/r everything-curl.html' template.html > everything-curl/index.html
	zip -r everything-curl.zip everything-curl

That one pandocs to HTML, then goes back into the HTML to find every image src with a grep, copies those files preserving directory structure with cp --parents, injects the whole HTML into template.html at a line matching Generated Content with sed, and zips the result. It is a shell script wearing a Makefile costume, and it is doing what single-file HTML distribution did before mdBook existed.

Which is exactly the point. The web version went modern in March 2024; the downloadable artefacts stayed on pandoc, because pandoc is what produces a PDF with a table of contents and an ePub with a cover image, and mdBook does neither. The legacy template.html and .htaccess in the repository root are the remains of the older hosting arrangement, and they are still in use by the HTML target.

The cost of the split is that there are two things to keep working and one canonical source. uni.md is the single source for the downloadable formats and the individual .md files are the source for the web, so a bug in the concatenation or in the chapter ordering shows up in the PDF and not on the website. The Makefile's dependency graph handles that correctly, since uni.md depends on all the chapters, but the reviewer has to know which surface they are looking at.

Five checks keep a document that is never finished

The book describes itself as never finished, and the repository is the answer to how a document in that state stays correct.

There are five distinct check targets, and each one catches a different class of error.

mdcheck:
	@./checkmd $(MDS)

check:
	@proselint $(MDS)

wordcheck:
	@./badwords.pl $(MDS) < badwords.txt

fixup:
	for i in $(MDS); do ./lang.sh $$i; done

mdcheck runs a compiled checkmd binary over every chapter, which is a structural check. check runs proselint, which is a Python prose linter, and is the only target in the file that shells out to something outside the repository. wordcheck runs the Perl badwords.pl script with the repository's own badwords.txt on standard input, which is a blacklist check against a list the project maintains. fixup runs lang.sh over every chapter, which is a mechanical rewriting pass.

Alongside those, two more mechanisms sit outside the Makefile's obvious targets. There is a pyspelling.yaml, which configures pyspelling, a Python wrapper around a Hunspell-style spell checker, and there is wordlist.txt, which is the project-specific vocabulary that pyspelling needs because libcurl has hundreds of function names and option spellings that no general dictionary contains. And there is warn_bad_links.lua, a pandoc Lua filter applied to every pandoc run, which is how broken links get reported during a build rather than discovered by a reader.

So a full quality gate on this book is: structure via checkmd, prose style via proselint, spelling via pyspelling plus a curated wordlist, vocabulary via badwords.txt, links via the pandoc filter, and Unicode and formatting normalisation via lang.sh. That is a serious toolchain for a prose project, and it is the reason a document that describes itself as never finished can still be trusted on any given page.

The wordcount and statistics scripts, lines.sh, stats.sh, showall.sh and tbd.sh, sit alongside. tbd.sh and firsttbd.sh are for tracking to-be-done markers, which tells you the book has an explicit convention for unfinished passages rather than pretending to be complete. And the epub.txt and pdf.txt files are per-format front matter, so the same body text can carry a different introduction depending on whether it is a web page or a downloadable book.

The one thing the toolchain does not do is version anything, which is the subject of the next section but one.

A reference arranged by API area, protocol and C source

The README gives one instruction about how to read it, and it is the correct one: do not read this book from front to back. Read the chapters or content you are curious about and flip back and forth as you see fit.

The directory listing shows what the arrangement is. There are chapters per functional area: cmdline/ for the command line tool, usingcurl/ for using it, libcurl/ for the library, libcurl-http/ for its HTTP interface, urlapi/ for the URL API, headerapi/ for the header-level API, bindings/ for language bindings, protocols/ for protocol coverage, transfers/ for transfers, http/ for HTTP, ftp/ for FTP, ws/ presumably for WebSocket, install/ for installation, internals/ for the implementation, source/ for the source code, project/ for the project itself, build/ for building it, and share/ for shared material.

Two of those directories are the reason this project exists in the form it does. An API reference for libcurl is available: the man pages ship with curl, and the option catalogue is in the tool itself with --help. What is not available anywhere else is internals/ and source/, an explanation of how the implementation works, written by the person who wrote it.

That is a genuinely different kind of documentation and it is the scarce one. A function signature tells you what a call accepts. A chapter on the transfer state machine tells you why a handle has to be reused, why a callback returning the wrong thing silently truncates a download, and why an easy handle inside a multi handle behaves differently. The topics list confirms the scope with libcurl-multi named explicitly, which is the interface where the behaviour is hardest to infer from signatures.

The example files are a good indicator of the register. Alongside get.md and getinmem.md there is login.md, http-ul-nonblock.md, and a pair of WebSocket chapters, ws-callback.md and websocket.md. Non-blocking HTTP and WebSocket callbacks are exactly the cases where the man page gives you a signature and an example, and neither explains the failure modes.

The book's own framing explains why it must be ongoing. The book is never finished, and the reason given is that the curl project continues to move so there are always things to update in the book as well. A document that tracks an implementation rather than an interface cannot have a last edition, because the thing it describes has no last version. That is a structural property of the subject, not a failure of the author.

One detail is worth flagging for anyone using this as a reference. It is a book about the project written by its founder, with the aspiration stated in the README of becoming a co-authored work. So the authority is unambiguous and the coverage is by one person's priorities, supplemented by the contributors listed at the end.

Three distribution channels and no version numbers

The repository has no GitHub releases, and the book is distributed three ways, which for a document of this size is an unusually thin release process.

The channels are: the website at everything.curl.dev, hosted by Fastly and rendered by mdBook since March 18th, 2024; a PDF at daniel.haxx.se/everything-curl/everything-curl.pdf; and an ePub at daniel.haxx.se/everything-curl/everything-curl.epub. All book content is hosted on GitHub in this repository.

Two things are worth noticing. The website and the downloadable files are served from different hosts on different infrastructure, with the web version on a CDN and the PDF and ePub on the author's personal domain. That is a deliberate separation, and it means the downloadable artefacts are not versioned by the CDN and not covered by anything like a cache policy you control.

The second is that neither the PDF nor the ePub carries a visible version. The Makefile names them everything-curl.pdf and everything-curl.epub with no version component, and the Makefile's clean target removes uni.md and everything-curl.pdf without producing a numbered artefact. So a reader who downloads the PDF in March and another who downloads it in September have files with the same name and different content.

For a reference book that you might cite in a design document, in a code review, or in a book of your own, that is a real problem. There is no way to say which revision you read, no way to check whether a page you remember is still there, and no way to submit a correction against a specific state.

The repository state is the only identifier that works, and the last push was on 2026-08-14. If you need a stable reference, record that commit hash alongside whatever you quote.

The contrast with the project it documents is instructive. curl the software is released obsessively, with dated tarballs, a changelog, a security process and a version in every response header. Everything curl the book has none of that apparatus. The most likely reason is that the book is a moving document that would need a release discipline its author has not wanted to take on, and the second most likely reason is that a book served from a web page does not feel like something that needs releasing.

Both reasons are understandable. Neither is a reason to leave the PDF without a version, and it would be a small change.

CC BY 4.0 on the document, and a permissionless correction process

Two things about how this book is licensed and edited are worth separating out, because both are unusual and both are deliberate.

The licence first. The README states that this document is licensed under the Creative Commons Attribution 4.0 license, and the repository has a LICENSE file. That is a deliberate difference from the software it documents. curl itself is permissively licensed, and many projects document themselves under a permissive or copyleft software licence to keep one set of terms. Everything curl is CC BY 4.0, which means you may quote it, translate it, adapt it and redistribute it commercially, provided you attribute Daniel Stenberg and the project.

That is the right choice for a reference work and it has a practical consequence. If you are writing a book, an internal design document or a course, you can copy long passages out of Everything curl and translate them, with attribution, without asking. A project that documented itself under a software licence would have made that awkward. The separation between document terms and software terms is the correct boundary and it is drawn explicitly.

The correction process second. The README asks contributors to send a refreshed version of the affected paragraph: if you find mistakes, omissions, errors or blatant lies in this document, please send us a refreshed version of the affected paragraph and we amend and update. Errors and pull requests on the book's GitHub page are described as preferable, but the paragraph-replacement model is stated first.

That is a specific editorial model and it is well matched to the problem. A pull request against an 8000-line markdown file requires the contributor to find the right section, match the surrounding style and be confident they have not broken the chapter ordering. Sending a replacement paragraph for one section requires none of that, and the maintainer can apply it. The threshold in the request is deliberately low, and the phrase blatant lies is doing real work: it signals that this is a document the author considers fallible and actively soliciting correction on, rather than an official reference that discourages it.

The contributor list shows the model working. There are seventy-six named people, from curl regulars such as Viktor Szakats, Dan Fandrich, Steve Holme, Senthil Kumaran, Jeroen Ooms, strupo, Ms2ger and Jay Satiro to names that appear to have come in with a single corrected section. A book with seventy-six contributors and no releases is a document maintained by accretion, and the two facts are the same fact.

The Build.md and GUIDELINES.md files at the repository root are where the mechanical side of that lives, and the fact that they exist means the editorial process has a written procedure rather than living in the maintainer's head.

Perl, Python, Rust and pandoc in one build

Reproducing this book's outputs means installing four toolchains from three language ecosystems, and that is worth knowing before you try.

The Perl part is the core. mkindex.pl generates the index, uni.pl concatenates the chapters into the single file that the PDF and ePub are built from, badwords.pl checks the prose against badwords.txt, and urlify.pl is in the root alongside them. The Makefile invokes these with @perl, so you need Perl on PATH.

The Python part is the quality gate. The check target runs proselint with no wrapper, so proselint has to be on your PATH, and pyspelling.yaml implies pyspelling. Neither is vendored, neither is pinned in the repository, and there is no requirements file. Their versions are whatever your machine has.

The Rust part is mdBook, but only for the website, and only since March 18th, 2024. The Makefile has no mdbook target, so the web build happens in the website's own pipeline, not here. That is a useful division: the repository builds the downloadable artefacts, and something else builds the site.

The pandoc part is the converter for HTML, PDF and ePub, and it has a dependency the project wrote itself, warn_bad_links.lua, which is applied via --lua-filter on all three pandoc invocations. Producing the PDF also needs whatever LaTeX engine pandoc reaches for, which the repository does not mention.

So the honest picture is: clone, get Perl, get proselint and pyspelling, get pandoc with a LaTeX toolchain, read BUILD.md, and run make. The mdBook toolchain is not needed unless you are working on the website.

None of this is unusual for a long-lived project. Documentation toolchains accrete, and the Makefile shows the accretion plainly: a Perl era, a Python linting era, a Lua filter added later, an mdBook era for the web, and a zip step in the HTML target that belongs to an era before either. The parts are small and the Makefile is readable.

The thing that would improve it most is not a new tool but a container. A single Dockerfile with perl, proselint, pyspelling, pandoc and a LaTeX engine would make the build reproducible for a contributor on any platform, and for a project that explicitly invites people to send corrected paragraphs, the difference between a contributor who can build and one who cannot is the difference between a pull request and an email. There is a docker/ directory in most projects of this kind. This one does not have one, and the build directory is chapters, not tooling.

Editorial conclusion

Use everything-curl if you are working with libcurl and have found that the man pages tell you a function exists without telling you why it behaves the way it does, because the source/ and internals/ chapters are the only public explanation of the implementation that ships with the project. Do not expect a versioned edition, because the repository has no GitHub releases and the PDF and ePub are hosted on the author's personal domain, so if you need to cite a specific state of the text you have to record a commit. Do not try to build it casually: the pipeline is mdBook for the web plus pandoc, Perl, proselint and pyspelling for everything else. Verify four things. Read the chapter you need rather than the book, since the README explicitly tells you not to read it front to back and the structure is a reference index rather than a curriculum. Note the licence, which is CC BY 4.0 on the document and therefore lets you quote and translate it freely, which is unusual for a project's own documentation. Run the checks before you send a correction, since wordcheck, mdcheck and check are the project's definition of correct prose. And contribute the way the project asks, by sending a refreshed version of the affected paragraph rather than only opening an issue. The deciding fact is that this is the one piece of curl documentation written by the person who wrote curl, and its value is exactly the value of that: the reasoning that never made it into a man page.

Frequently asked questions

Official sources

1. curl/everything-curl on GitHub
2. Issues
3. License: CC-BY-4.0
4. Project website
5. README