HPy is a finished experiment in abstracting the Python C API

The C API problem, stated plainly

The core argument is short. The old C API is specific to the current implementation of CPython, and it exposes a lot of internal details, which makes it hard to implement for other Python implementations such as PyPy, GraalPy, Jython and IronPython, hard to experiment with new things inside CPython itself, such as using a garbage collector instead of refcounting or removing the GIL, and hard to guarantee binary stability for. HPy is a specification of a new API and ABI for extending Python that is Python implementation agnostic, designed to hide and abstract internal details so that it can stay binary compatible even if the underlying Python internals change significantly, and so it does not hinder internal progress in CPython or elsewhere. The audience was extension authors, and the second audience, arguably the more important one now, is the CPython core developer community, because a reference design that the interpreter itself can borrow from is worth more than another extension.

Handles, and why the API looks the way it does

The name explains the central idea. The H in HPy stands for handle, and the key idea of the new API is to use fully opaque handles to represent and pass around Python objects, rather than exposing the interpreter's object structures. The project says the H stands for handle as one of the key ideas of the new API. That single decision produces most of the rest of the design. If an extension never sees a struct it cannot depend on its layout, so the implementation is free to change it, which is what buys binary stability and implementation independence at the same time. The features the documentation highlights follow from the same logic. There is a debug mode for a better developer experience, support for incremental porting from the CPython API to HPy so a large extension can be converted a piece at a time, and a CPython ABI for raw performance when you are on CPython and want the last few percent. The repository contains the API and ABI specification and the implementation for the CPython interpreter, while GraalPy and PyPy provide their own builtin HPy implementations.

The status is the story: it is over, and CPython took the ideas

The project status section is the first thing in the README and it is unambiguous. HPy is no longer actively maintained, and the project considers itself a successful demonstration of an abstract, JIT-friendly and alternative-Python-friendly C API for Python extensions, and of a possible path for evolving CPython without exposing implementation details. Then it lists what happened next. Several of the ideas explored by HPy have since become part of CPython's C API evolution, whether directly inspired by HPy or developed in parallel: opaque interpreter and type structures, getter and setter APIs instead of direct field access, converting macros to functions, and avoiding borrowed references in new APIs. The project points at PEP 620, PEP 670 and PEP 674 for some of that work, and notes that HPy's perspective was presented at the CPython core developer sprint and C API Summit in Brno in 2023. The last release was 0.9.0 on 2023-09-22, with 0.9.0rc1 and 0.9.0rc2 earlier in 2023, and the last push was on 2026-09-05.

Where the work continues

The continuation is named and it is not HPy itself. Ideas that require a substantially new API are being explored by the PyNI PEP work, and the README says that work is progressing gradually as community time and support allow, with contributions welcome from people seriously interested in moving it forward, and with experience in CPython development and C extensions described as helpful. The project offers to help onboard new contributors alongside the existing maintainers and authors. The community channels are a Discord server, the hpy-dev mailing list on python.org, the project website, and a documentation site on readthedocs. For a reader, the useful consequence is that a CPython core developer today looking for prior art on an API shape has a working implementation and a documented set of trade-offs, which is the kind of artefact that a PEP discussion can point at rather than argue from scratch.

Building the debug and universal builds

The Makefile is small and shows the shape of the project. The default target builds hpy.universal, which runs python3 setup.py build_clib -f build_ext -if, and there is a debug target that sets HPY_DEBUG_BUILD=1 before building all, a dist-info target, an autogen target that runs python3 -m hpy.tools.autogen ., and a cppcheck target that runs the static analyser over the C sources with a long list of include paths covering the devel headers for cpython, universal, runtime, debug and trace. The cppcheck invocation is worth a second look because of the comment explaining why it writes errors to a file and cats them later, since the Azure Pipelines configuration does not show stderr, which is a small piece of real world CI knowledge. There is also an infer target that builds a compilation database and runs the Infer analyser, with a blacklist for one file and a comment referring to a specific commit for why it is excluded, and a valgrind configuration pointing at suppression files for Python and HPy.

setup.py refuses to install on the wrong interpreter

One piece of the implementation is a small but instructive piece of defensive design. The setup.py comment says the package is supposed to be installed only on CPython and GraalPy, and the code tries to bail out with a meaningful error message in other cases. The check is on sys.implementation.name, and if the implementation is not cpython or graalpy, it builds a message that includes the implementation name and then inspects the builtin module names for _hpy_universal. If that module is present, the interpreter ships its own HPy implementation, so the code imports it, asks for its version with get_version if that method exists, and tells the user their interpreter comes with its own version of hpy along with the exact version string, recommending that they put hpy with that version in their requirements.txt so pip does not try to reinstall it. If the module is absent, the message says the implementation does not seem to support HPy and points at the vendor. The design goal is the same as the whole project, taken to the packaging layer, where a wrong install is refused with an explanation rather than a crash.

Why the C API was a problem for everybody else too

The section asking why you should care about this stuff makes the case in four ways, and one of them is about other implementations rather than about CPython. The existing C API is becoming a problem for CPython and for the evolution of the language itself, and this project makes it possible to make experiments that might be officially adopted later. For PyPy it would give obvious speed benefits, because data scientists would get the benefit of fast C libraries and fast Python code at the same time, which is hard to achieve now. The current implementation is too tied to CPython and proved to be a problem for almost all the other alternative implementations, and an API designed to work well on two implementations makes the job much easier for future ones, on the argument that going from 2 to N is much easier than going from 1 to 2. And it is arguably easier to learn than the massive CPython C API. The repository layout matches those goals, with the API and headers under hpy/, a proof-of-concept directory, a microbench directory, a c_test directory, gdb-py.test for debugging, and the test/ suite.

Editorial conclusion

HPy is worth reading rather than adopting, because it is a worked example of how a C extension API can be made implementation agnostic, and because the project is candid about having succeeded by being absorbed into CPython. It is the wrong dependency for a new extension today, since the last release was 0.9.0 in September 2023 and the tree is not actively maintained. Before you study it, read the project status section, because everything else in the repository is written as if the work continues, and if you want to build on the ideas rather than the code, follow the PyNI PEP work that the project points to as the continuation.

Frequently asked questions

Official sources

1. hpyproject/hpy on GitHub
2. License: MIT
3. Project website
4. README
5. Releases