Cog: Generating Docker Images for ML Models from cog.yaml and a Python Runner Class
Containers for machine learning
At a glance
- What is it?
- Cog turns a short cog.yaml plus a typed Python class into a Docker image with an OpenAPI-validated HTTP prediction endpoint. It removes CUDA and Dockerfile work for single-model inference services, and it stops being the right tool the moment your deployment needs anything more than that one endpoint.
- Who is it for?
- Adopt Cog if you are shipping a single model as an HTTP inference service and want the Dockerfile, the CUDA and cuDNN pairing, and the request schema handled for you. Do not adopt it if your service needs several models behind one process, custom routing, or a runtime other than Python.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Go, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem Cog addresses: Dockerfiles and CUDA version pairing
Cog targets a specific bottleneck. The README states that it is really hard for researchers to ship machine learning models to production, and it names the moving parts: Dockerfiles, pre- and post-processing, Flask servers, CUDA versions. The stated failure mode is organisational as much as technical. More often than not, the README says, the researcher has to sit down with an engineer to get the model deployed. Cog exists to remove that meeting.
The audience is therefore narrow and identifiable: someone who has a working model in Python and needs it reachable over HTTP on a machine with a GPU. Not a training pipeline, not a feature store, not a multi-tenant serving platform. One model, one container, one endpoint. The project's own framing is that you can deploy the packaged model to your own infrastructure or to Replicate, and the homepage points at cog.run. The README also notes that the authors had seen similar systems built internally at Spotify and Uber, which is the clearest statement of what Cog is: an open source version of an internal model-packaging tool.
If your problem is scheduling, autoscaling, or routing between many models, Cog is not aimed at it. It produces an image. What happens to that image afterwards is out of scope.
What cog.yaml declares and what the generated image contains
The build side is a YAML file. The README's example sets build.gpu to true, lists system_packages as libgl1 and libglib2.0-0, pins python_version to 3.13, and points python_requirements at requirements.txt. A separate top-level run key names the Python entry point, in the example run.py:Runner.
That is the whole environment definition. There is no Dockerfile in the repository's user-facing workflow. Cog reads the config and emits a Docker image, and the README is explicit about which decisions it makes on your behalf: Nvidia base images, caching of dependencies, installation of the specific Python version, and environment variable defaults. The compatibility claim is the more interesting one. Cog, according to the README, knows which CUDA, cuDNN, PyTorch, TensorFlow and Python combinations are compatible and sets them up correctly. That means the version matrix lives inside Cog rather than in your head, and it also means the matrix is a thing Cog has to keep current as upstream libraries move.
The caching detail matters for iteration. Dependency installation is ordered so that a change to run.py does not invalidate the layer that installed requirements.txt. That is standard Docker layer discipline, but it is the kind of thing that is easy to get wrong by hand, and Cog applies it without being asked.
BaseRunner, Input types and the generated /predictions endpoint
The runtime contract is a Python class. In the README example, run.py defines a class named Runner that inherits from BaseRunner. It has a setup method, documented as loading the model into memory so that running multiple inferences is efficient, and a run method whose signature uses Cog's Input and Path types. The run method takes an image path and returns a path.
The type annotations are not decoration. The README states that defining inputs and outputs with standard Python lets Cog generate an OpenAPI schema and validate inputs and outputs, and that those same types are used to dynamically generate a RESTful HTTP API served by a Rust/Axum server. So the flow is: annotations in run.py become a schema, the schema becomes request validation, and the server enforces it before your Python code is called. The setup method is separated from run precisely so that expensive model loading happens once per container rather than once per request.
This is a clean design, but it constrains you. The shape of your API is the shape of one method signature. There is no routing layer to configure, no middleware to register, and no second endpoint to add. The README's curl example posts to /predictions with a JSON body of the form {"input": {"image": "https://.../input.jpg"}}, and that is the interface you get. If your product needs health semantics beyond what the server provides, or an endpoint that returns model metadata, Cog does not offer a documented hook for it in this material.
Installing Cog and running a model locally
Installation is a single binary. On macOS the README gives brew install replicate/tap/cog as the easiest route, plus an install script at https://cog.run/install.sh and a manual path that curls the release asset for the current platform into /usr/local/bin/cog, runs chmod +x, and clears the quarantine attribute with sudo xattr -d com.apple.quarantine. Linux uses the same script or the same manual download without the quarantine step. Windows is not natively supported; the README directs Windows 11 users to WSL 2 and then the Linux instructions. There is also a Dockerfile snippet for installing Cog inside an image, using INSTALL_DIR and a SUDO variable passed to the install script.
Prerequisites are Docker, and if you install Docker Engine rather than Docker Desktop, Buildx as well. That is a real dependency, not a formality: Cog is a front end over a container build.
The commands are short. cog run -i image=@input.jpg builds the image, runs the model with the named input, and writes output.jpg. cog build -t my-classification-model produces a deployable image, which you then start with docker run -d -p 5000:5000 --gpus all my-classification-model. cog serve -p 8080 combines build and run behind a server, and the README shows the matching curl against localhost:8080/predictions. Note the two different ports in the README's examples, 5000 for the docker run path and 8080 for cog serve. Both are configurable, but they are not the same default, and mixing them up is an easy first mistake.
Where Cog is the wrong tool
The README contains a development-environment section that is commented out, with a note explaining that the instructions were intentionally left out so as not to confuse the ship a model to production message. That note is the clearest signal of scope. Commands like cog exec python train.py and the Jupyter notebook path exist, but they are deliberately not part of the advertised product. If your workflow is exploratory, Cog is fighting you.
The harder limitation is architectural. Cog generates one HTTP surface from one Python class. Teams that need several models loaded in one process, per-request model selection, custom authentication, or non-Python inference code will find themselves either wrapping Cog's output or abandoning it. The Apache-2.0 licence means you can modify the tool, and the Go codebase is there to be changed, but that is a different project from using Cog.
There is also a version-coupling risk that the README does not address. Cog's value proposition rests on knowing which CUDA, cuDNN, PyTorch, TensorFlow and Python combinations are compatible. That knowledge has to be maintained against upstream releases. The release history shows a cadence of roughly two months between v0.21.0 in June 2026 and v0.22.0 in August 2026, with a release candidate before the June release. That is active maintenance, but the material does not say how quickly a new CUDA or PyTorch release is absorbed, and it does not describe a support window for older combinations. If your weights require a pinned, older framework build, verify that Cog still emits it before you plan around it.
How Cog differs from writing your own Dockerfile and FastAPI service
The obvious alternative is the manual stack: a Dockerfile based on an Nvidia image, a requirements file, and a FastAPI or Flask app exposing a predict route. The difference is not effort in the abstract, it is where the knowledge sits. With the manual stack, you choose the base image tag, you resolve the CUDA and cuDNN pairing, you decide the layer order, and you write the request validation. Every one of those decisions is visible in your repository and auditable by anyone who reads it. Nothing is generated.
With Cog, those decisions move into the tool. You get the compatibility matrix and the caching order for free, and you give up the ability to inspect them without building an image and looking at the result. You also give up the freedom to structure the HTTP layer. A hand-written FastAPI service can have as many routes as you want; a Cog model has one.
The trade is defensible when the model is the product and the API is incidental. It is a poor trade when the API is the product and the model is one component. The README's own framing, that you can deploy to your own infrastructure or to Replicate, hints at the intended shape: Cog is the packaging step for a service whose only job is inference.
Maintenance, licensing and what to check before adopting
Cog is Apache-2.0 and is not archived. The last push recorded is 2026-09-10, and the most recent release is v0.22.0 from 2026-08-14, following v0.21.0 in June 2026 and a v0.21.0-rc.3 before it. The project ships release candidates before minor versions, which suggests a testing step rather than continuous deployment to users. This is not legal advice, but the practical implication of Apache-2.0 is that you can vendor, modify and redistribute the Go binary and the generated build logic, provided you preserve the licence and notices; if you fork the compatibility matrix, you inherit the job of keeping it current.
Upgrade cost is the thing to weigh. Because cog.yaml pins python_version and names a requirements file, and because Cog selects the CUDA and framework combination, a Cog upgrade can change the base image underneath a model that was working. The README does not describe a lockfile or a way to freeze the resolved matrix. That is the gap to probe first: build the same model with two Cog versions and diff the resulting image configuration.
The concrete first step is small. Write a cog.yaml with build.gpu set, a Runner class with one Input and one output, run cog build -t my-classification-model, and inspect the Dockerfile Cog produced. If the base image and the framework versions in that output match what your weights expect, Cog is doing real work for you. If they do not, you have found the boundary before it found you in production.
Editorial conclusion
Adopt Cog if you are shipping a single model as an HTTP inference service and want the Dockerfile, the CUDA and cuDNN pairing, and the request schema handled for you. Do not adopt it if your service needs several models behind one process, custom routing, or a runtime other than Python. Before committing, build one image with cog build -t my-classification-model and inspect the generated Dockerfile, then confirm the CUDA and PyTorch combination Cog selected matches the one your weights were trained against.
Community notes