gpython, a Python 3.4 interpreter in Go built for embedding rather than for speed

An experiment that turned into an embeddable runtime

The origin story is stated plainly and it explains the shape of everything else. gpython began as an experiment to find out how hard porting Python to Go would be. The answer turned out to be that the C modules are the significant barrier to making it a complete replacement for CPython. That finding is the reason the project is not a general purpose Python distribution, and the project is explicit that its real audience is people who want to embed a popular and well known language into a Go application, for whom it could be a better choice than less capable or less known alternatives. The list of what is included names the pieces: a lexer, parser and compiler, a runtime with high level convenience functions, multi-context interpreter instancing, easy embedding into a Go application, and an interactive mode, with a hosted REPL available to try online. The primary language is Go, and the module path is github.com/go-python/gpython. If you have Go installed, the install is one command.

go install github.com/go-python/gpython

A prebuilt binary is also published on the releases page.

Five modules, and the ones that were never portable

The standard library coverage is the single most important fact for anyone evaluating this. gpython does not include many Python modules, because many of the core modules are written in C rather than in Python, and a C extension cannot simply be translated to Go. The converted modules are builtins, marshal, math, time and sys. That is the whole list.

builtins
marshal
math
time
sys

A script that imports anything else, including anything from the standard library that is pure Python, will not run as written, which is a much tighter boundary than the phrase Python interpreter suggests. The project handles this honestly by pairing the module list with a status section: it parses all the code in the Python 3.4 distribution, and it runs Python 3 for the modules that are currently supported. The version target is 3.4, which is a clue about the vintage of the effort as well as the semantics you get.

Multi-context execution, and why it is the interesting part

The feature that distinguishes gpython from a toy interpreter is concurrent multi-interpreter execution, called multi-context in the project documentation. Each interpreter instance is independent, and the examples directory carries both an embedding example and a multi-context example that demonstrates it. The stated motivation is server use, where complete interpreter independence is paramount, and the point is stronger than it first appears. A shared interpreter has shared global state, so one script that rebinds a name or mutates a module affects every other script in the same process. Independent contexts remove that class of bug, which is what makes embedding plausible in a multi-tenant service rather than only in a build script. The embedding example demonstrates how to embed and invoke gpython from any Go application, and it is the first thing to read if you are evaluating the project for that purpose.

Performance was explicitly not the goal

The performance figures in the project documentation are given with an unusual amount of hedging, and the hedge is the message. Speed has not been a goal of the conversions, and the interpreter runs the pystone benchmark at about 20 percent of the speed of CPython. There is one place where it is faster, and the explanation is instructive: a pi computation test runs quicker under gpython, because the Go long integer primitives are likely faster than the Python ones. So the picture is an interpreter that is roughly five times slower on a general bytecode benchmark and faster on one specific numeric workload, which is roughly what you would expect from a port that prioritised correctness and embeddability over throughput. If your use case is throughput-bound, this is the wrong tool and the number is there so you do not have to guess. The pi test and the pystone script both live in the examples directory, so the benchmarks are available to look at rather than only described.

The repository layout, and the shape of the code

The tree maps closely onto the phases of an interpreter. There is a `parser/` directory, an `ast/` for the syntax tree, a `compile/` stage, a `vm/` for the virtual machine, a `py/` directory holding the runtime and the convenience functions, a `repl/` for the interactive mode, a `stdlib/` and an `importlib/` for module loading, a `symtable/` for symbol tables, and `bin/` for the command line entry point. There is also a `py3test.py` driver and a `pytest/` directory, a `testdata/` tree, and a `ci/` directory. The `main.go` and `version.go` at the root are the program entry point and the version constant. A `notes.txt` file is mentioned in the contributing guidance as holding bigger ideas, and the recommended starting point for a new contributor is adding more convenience functions to `py/util.go`, which is a reasonable way to learn the codebase. A `RELEASE.md` and a `CONTRIBUTE.md` sit alongside the usual `LICENSE` and the CI configuration.

A dependency set of three, and a Go version floor

The module file is short and tells you what the project actually needs. It requires Go 1.18 or newer. The direct dependencies are google go-cmp for comparisons in tests, gopherjs and gopherwasm, and peterh/liner. The indirect requirements are the unicode width handling that liner pulls in and the x/sys package. There is a `bin/` directory in the tree and a `.goreleaser.yml` file, which is the configuration for a release tool, so the binary distribution is produced by goreleaser rather than by a hand-written script.

The licence story has three layers

Licensing here needs care, and the project is candid about it. The project states that gpython is licensed under the MIT licence, and that it contains code which was ported fairly directly from the CPython source code under the PSF licence, with a link to the CPython licence file. The licence identifier in the repository metadata is BSD-3-Clause instead, which does not match the statement in the documentation. Anyone who copies code out of this repository into a product should read the LICENSE file directly rather than trusting any of the labels, because the code is of mixed provenance and the MIT statement cannot by itself cover the CPython-derived parts. The practical effect for most users is small, since a dependency is normally consumed rather than vendored, but it matters for anyone who plans to lift a module or a file. The same mixed-provenance question is the reason a port like this is not a drop-in replacement for CPython: the parts that are hard to port are the parts that carry the licence-sensitive code.

Editorial conclusion

Adopt gpython if you need to run Python inside a Go program with several fully independent interpreter instances, for example in a server where one tenant's script must not affect another's, and skip it if you need the standard library, speed, or a general Python runtime, because only five modules are converted and the project says so itself. Verify three things before you commit: that the module your script imports is one of builtins, marshal, math, time or sys, since the C-written core modules were never ported; that your Go version is 1.18 or newer, which the module file requires; and which licence applies to the parts you copy, because the project states MIT while also carrying code ported directly from CPython under the PSF licence, and the licence identifier in the repository metadata is a different one again.

Frequently asked questions

Official sources

1. go-python/gpython on GitHub
2. Issues
3. License: BSD-3-Clause
4. README
5. Releases