Jupyter Enterprise Gateway: running notebooks against kernels on a cluster

The gap between a notebook front end and a cluster scheduler

A Jupyter Notebook client normally talks to a kernel running on the same machine. That works for a single user on a laptop. It stops working the moment a notebook has to reach a Spark cluster, a Kubernetes namespace or a Docker Swarm service, because the kernel now needs to be placed somewhere by a scheduler, configured for that user, and reached over the network.

Jupyter Enterprise Gateway is the web server that sits in that gap. It gives Jupyter Notebook headless access to kernels across a distributed cluster, including Apache Spark managed by YARN, IBM Spectrum Conductor, Kubernetes and Docker Swarm.

Three kernel types ship with out of the box support: Python through IPython, R through IRkernel, and Scala through the Apache Toree kernel.

The project also states its own boundary explicitly. Enterprise Gateway does not manage multiple Jupyter Notebook deployments, and for that the documentation points to JupyterHub. So the intended architecture has three pieces, with different projects responsible for each: a notebook front end for users, JupyterHub if you need to manage many notebook deployments, and Enterprise Gateway for the kernel side.

For an analyst who runs a notebook against a Spark cluster from a managed notebook service, the piece that is missing without this project is the ability to say launch the kernel over there, on that node, with those settings.

Four kernel launch modes, and which one you need

The technical overview describes Enterprise Gateway as providing feature parity with Kernel Gateway's jupyter-websocket mode, plus a specific set of additions. The additions start with where a kernel can live. A kernel can be launched local to the gateway server, which is the behaviour of Kernel Gateway, on specific nodes of the cluster using a round-robin algorithm, or on nodes identified by an associated resource manager.

That third option is the interesting one, because a round-robin placement is easy to implement and wrong for real workloads. A Spark job needs its executor on a node with the data, and a resource manager knows which node that is. Enterprise Gateway's support for Kubernetes, YARN, Spectrum Conductor and Docker Swarm out of the box is what makes the round-robin mode a fallback rather than the main path.

The other additions listed are secure communication from the client through the gateway to the kernels, multi-tenant capabilities, persistent kernel sessions, and the ability to associate profiles of configuration settings to a kernel for a given user.

The profiles mechanism is the part most likely to matter to a platform team. Configuration that varies by user, such as environment variables, resource requests or an image, gets attached server side rather than being handed to the client, which means the client cannot bypass it by starting its own kernel.

Installing from PyPI and seeing the configuration surface

The README keeps installation short and defers the detail to the Users Guide in the project documentation. The quick start uses pip:

# install from pypi
pip install --upgrade jupyter_enterprise_gateway

# show all config options
jupyter enterprisegateway --help-all

# run it with default options
jupyter enterprisegateway

The second command matters more than it looks. A gateway that routes to a resource manager needs more configuration than a local kernel server, and --help-all prints the full set of options rather than the short help. The README then points at the configuration options in the Operators Guide for what each one does.

The package name on PyPI uses an underscore separator, jupyter_enterprise_gateway, while the command it installs is invoked with a space, jupyter enterprisegateway. The project name in pyproject.toml is written the same way, and the description there reads a web server for spawning and communicating with remote Jupyter kernels.

The build backend is hatchling, and the distribution targets Python 3.10 and 3.11 with requires-python set to at least 3.10. The classifier list also marks the licence as OSI Approved BSD License, while the repository carries a LICENSE.md file that GitHub cannot classify on its own, so check the file text before assuming the terms match the classifier.

The dependency pins show where the project still lags

The dependency list in pyproject.toml is more informative about the project's state than the feature list is, because the comments next to several pins explain what is waiting on what.

jupyter_client is pinned to a range that stops below version 7, with a comment saying the cap should be removed once the project supports kernel provisioners. jupyter_server is pinned below 2.0 for the same reason. pyzmq is capped below 25.0, and the comment records that pyzmq 25 removes deprecated code that jupyter_client 6 uses.

In other words, the project is waiting on the new kernel provisioning mechanism before it can move to current versions of the Jupyter stack. If you are running jupyter_client 7 or jupyter_server 2 in the same environment, these caps will need attention, and the project's own dependency resolution will fight you.

The rest of the dependency list shows the scope of the integrations. kubernetes at 18.20.0 or later and docker at 3.5.0 or later cover the two container orchestrators, yarn-api-client at 1.0 or later covers Spark on YARN, paramiko and pexpect handle remote process control over SSH, tornado serves the websocket side, and pycryptodomex provides the cryptography. jinja2 and traitlets cover templating and configuration, and watchdog and requests round out the runtime pieces.

Two integrations are worth calling out for anyone planning a deployment. paramiko and pexpect mean the gateway executes commands on remote nodes, so it needs credentials and network reachability to those nodes. And the kubernetes and docker clients mean the gateway needs permission to create pods or containers in the cluster, which is a privilege worth scoping tightly rather than granting broadly.

Building, testing and packaging from the Makefile

The Makefile at the top level of the repository is the operational map of the project. Its declared targets list is long and reads as an inventory of what the project produces: a wheel and an sdist, documentation, development and development over HTTP environments, test targets, release and check_dists targets, image cleaning and pushing, and a helm-chart target.

The kernel image targets are named after the kernel and cluster combination they build. There are separate targets for a Python kernel, a Spark Python kernel, an R kernel, a Spark R kernel, a Scala kernel, a TensorFlow Python kernel and a TensorFlow GPU Python kernel, each with a clean-kernel counterpart and a push-kernel counterpart. That naming tells you the project treats containerised kernels as first class deliverables rather than an afterthought, which is what you would expect from a gateway whose whole job is placing kernels on specific nodes.

Version handling is explicit. VERSION defaults to 3.4.0.dev0 and SPARK_VERSION defaults to 3.2.1, and the build tags are dev when the version string contains dev, otherwise the version itself. The helm chart version is read straight out of etc/kubernetes/helm/enterprise-gateway/Chart.yaml with a sed expression, and the helm tool version is pinned to v3.18.3 with a comment noting that it was the latest release as of June 2025.

The image pushing targets also cover an enterprise-gateway demo image and a demo base image, so there is a demo path in addition to the production one. The Makefile header notes it is distributed under the terms of the Modified BSD License.

Version history and the gap between the last two tags

The published releases are v3.3.0 on 2026-06-01, v3.2.3 on 2024-02-20 and v3.2.2 on 2023-03-18.

That sequence shows a project that spent a long stretch on 3.2.x and then made a jump to 3.3.0, a long gap between February 2024 and June 2026. The last push to the repository was on 2026-07-14, so work has continued since the 3.3.0 tag.

The development version in pyproject.toml is 3.4.0.dev0, which matches the Makefile default, so the next release is being prepared on a 3.4 line. If you are evaluating this project now, the choice is between the tagged 3.3.0 and whatever the main branch currently holds, and the gap between the tag and the branch tip means the difference between them is not small.

The homepage is the readthedocs documentation at jupyter-enterprise-gateway.readthedocs.io, and the documentation is organised into separate tracks for users, operators and contributors, with the Operators Guide holding the configuration reference, the System Architecture page describing the remote kernel, process proxy and launcher frameworks, and a Project Roadmap page that the README links for the profiles feature.

Nothing in the documentation suggests the project is archived, and the release cadence of 3.3.0 in 2026 indicates active work on a feature that only becomes useful with the newer kernel provisioning path.

Editorial conclusion

Adopt Jupyter Enterprise Gateway when notebook clients need kernels placed on specific cluster nodes and you need per-user configuration applied on the server side. Do not adopt it to run multiple notebook deployments, since the documentation points that job at JupyterHub, and do not adopt it if your workload only needs a local kernel. Before deploying, read the Operators Guide configuration options and confirm your resource manager integration, since the documented dependency caps on jupyter_client and jupyter_server show the project still carries constraints tied to the older kernel provisioning path.

Frequently asked questions

Official sources

1. Issues
2. jupyter-server/enterprise_gateway on GitHub
3. Project website
4. README
5. Releases