Self-hosted service
replicate/cog avatar
replicate/cog

Cog turns a Python model into a Docker image with a generated inference API

Containers for machine learning

9,497 stars703 forksGoApache-2.0

At a glance

What is it?
Cog is the Go CLI from Replicate that packages a machine learning model into a standard container from a cog.yaml file and a BaseRunner class. It covers the build, the schema, and an HTTP inference server.
Who is it for?
Cog earns its place when a model has to leave a notebook and reach a machine that runs Docker, because the gap it closes, a checked in environment plus a generated schema, is the part that normally needs an engineer. Skip it when your team already maintains its own Dockerfiles and serving code, and when you need custom routes, authentication, or asynchronous jobs, since none of that fits the type hint interface.
Can I use it commercially?
Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 8 days ago.
What is it written in?
Mainly Go, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 13, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • replicate/cog9,497 ★ · +21
Star history
Repository2026-09-132026-11-13
replicate/cog9.5k9.5k

21 Stars in the last 61 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

cog.yaml replaces the Dockerfile with five keys

Cog does not ask you to write a Dockerfile. The environment lives in a YAML file made of a `build` block and a single `run` target:

yaml
build:
  gpu: true
  system_packages:
    - "libgl1"
    - "libglib2.0-0"
  python_version: "3.13"
  python_requirements: requirements.txt
run: "run.py:Runner"

`gpu: true` is what selects an Nvidia base image, `system_packages` lists the system libraries the model links against, `python_version` pins the interpreter, and `python_requirements` points at a pip requirements file. The `run` value is a dotted path of file, colon, class, so one config names one entrypoint, and the README does not say what happens when a directory holds a second model. Cog's argument is that it knows which CUDA, cuDNN, PyTorch, TensorFlow and Python combinations are compatible and sets them up for you.

Type annotations become the HTTP contract

The Python file named in `run` does the schema work. A class derives from `BaseRunner`, a `setup` method loads weights into memory, and every argument of `run` becomes a field Cog can describe:

python
from cog import BaseRunner, Input, Path
import torch

class Runner(BaseRunner):
    def setup(self):
        """Load the model into memory to make running multiple inferences efficient"""
        self.model = torch.load("./weights.pth")

    # The arguments and types the model takes as input
    def run(self,
          image: Path = Input(description="Grayscale input image")
    ) -> Path:
        """Run the model"""
        processed_image = preprocess(image)
        output = self.model(processed_image)
        return postprocess(output)

Cog reads those annotations to emit an OpenAPI schema, checks incoming values against it, and generates a RESTful HTTP server in Rust with Axum. The type annotations therefore define the generated API's inputs and outputs.

Four install routes, and only one avoids sudo

macOS has the shortest path:

console
brew install replicate/tap/cog

Linux and macOS share a script, which also has a fish spelling and a wget spelling for anyone who does not want process substitution:

sh
# bash, zsh, and other shells
sh <(curl -fsSL https://cog.run/install.sh)

# fish shell
sh (curl -fsSL https://cog.run/install.sh | psub)

# download with wget and run in a separate command
wget -qO- https://cog.run/install.sh
sh ./install.sh

A manual route exists too, which downloads a release binary straight into a system path and then makes you clear Gatekeeper yourself when macOS refuses the unsigned file:

console
sudo xattr -d com.apple.quarantine /usr/local/bin/cog

That last command matters more than it looks. The manual path needs root on a shared machine, and on a locked down laptop a curl into `/usr/local/bin` is often refused by policy, which leaves an internal mirror or a managed package as the only option. Teams on a fully managed fleet should read the Linux script route and the Homebrew tap before assuming the binary will land cleanly.

Windows only arrives through WSL 2, and Docker Engine needs Buildx

Two prerequisites carry real cost. Cog does not support Windows natively; the supported path is Windows 11 with WSL 2, and once WSL 2 is set up you follow the Linux instructions. Your container then runs inside the WSL 2 virtual machine rather than on the Windows host, and the README does not explain how GPU access behaves in that arrangement, it points at `docs/wsl2/wsl2.md`. The second prerequisite is Docker, and the two Docker distributions are not equivalent here. Docker Desktop arrives with what Cog expects. Install Docker Engine instead and you must also install Buildx, which the prerequisites section states in a single line. That is the only place the requirement appears, so a contributor who installs Docker Engine from a package repository and skips Buildx is debugging a build failure whose cause sits one paragraph away from the install instructions. The supported set is macOS, Linux, and Windows 11.

cog run, cog build, and cog serve lead to different artifacts

Three commands sit at the top level and they differ in what they leave you holding. `cog run` is the loop for a single prediction: it builds when it has to, starts the container, and writes the returned path to disk.

console
$ cog run -i [email protected]
--> Building Docker image...
--> Running...
--> Output written to output.jpg

`cog build` produces the artifact you deploy, and everything after that point is plain Docker:

console
$ cog build -t my-classification-model
--> Building Docker image...
--> Built my-classification-model:latest

$ docker run -d -p 5000:5000 --gpus all my-classification-model

$ curl http://localhost:5000/predictions -X POST \
    -H 'Content-Type: application/json' \
    -d '{"input": {"image": "https://.../input.jpg"}}'

`cog serve -p 8080` merges the two and answers the same request on the port you name. The `--gpus all` flag also means the example assumes an Nvidia device and has to come off to exercise the same image on CPU.

The developer commands sit commented out inside the README

There is a shell inside the container, and the README admits it exists. The whole block is wrapped in an HTML comment carrying a note from bfirsh: development environment instructions are intentionally left out for now, so as not to confuse the ship a model to production message. Inside that comment sit the two commands you would expect.

console
$ cog exec python train.py
...
console
$ cog exec -p 8888 jupyter notebook --allow-root --ip=0.0.0.0

Because the markup is a comment, neither line renders on the project page and no visible section points at them. A link to `docs/notebooks.md` is buried in the same hidden block. Anyone who wants to retrain inside the packaged environment, or open a notebook against the built image, has to read the raw README source, and the cost lands hardest on exactly the people the tool is written for, a researcher debugging a model rather than deploying it. The examples directory does carry separate `hello-train` and `notebook` directories, so the capability is there even where the walkthrough is not.

A Go 1.26 binary wrapped around a Python package capped at 3.13

The command line tool is Go. `go.mod` declares module `github.com/replicate/cog` with a `go 1.26` directive, and the dependency list shows what it drives: `docker/docker` and `moby/buildkit` for the container work, `google/go-containerregistry` for image handling, `getkin/kin-openapi` and `xeipuuv/gojsonschema` for the schema, `spf13/cobra` for the commands, and `testcontainers-go` for tests. The Python side is a separate distribution. `pyproject.toml` sets `requires-python = ">=3.10"` and classifies only 3.10, 3.11, 3.12, and 3.13, takes its version from `setuptools_scm`, and depends on `coglet`, `pyyaml`, `structlog`, `requests`, and `typing_extensions`. Two practical edges for a contributor. A model on a newer interpreter is not among the classified versions, and the tracing extra pins opentelemetry api, sdk, and both OTLP exporters to `1.44.0` exactly, so those pins have to move as a set. Second, the Makefile is now a shim that prints a deprecation notice and forwards to mise, so `make cog` still works while `mise run build:cog` is the form the project asks for.

Editorial conclusion

Cog earns its place when a model has to leave a notebook and reach a machine that runs Docker, because the gap it closes, a checked in environment plus a generated schema, is the part that normally needs an engineer. Skip it when your team already maintains its own Dockerfiles and serving code, and when you need custom routes, authentication, or asynchronous jobs, since none of that fits the type hint interface. Before committing, check the interpreter your model needs against the >=3.10 floor and the classifiers, which list versions up to 3.13, and read the opentelemetry 1.44.0 pins in the tracing extra if you plan to export traces.

Frequently asked questions

What is Cog in Python?

It is the Python half of a two-language project. The `cog` distribution in `pyproject.toml` declares `requires-python >=3.10` and provides `BaseRunner`, `Input`, and `Path`, the three imports used to declare a model's inputs, outputs, and memory setup.

Is replicate free to use?

The README does not answer this. It names Replicate as one deployment target next to your own infrastructure, links [email protected], and states no pricing for the hosted service or for Cog itself.

What are some good alternatives to Replicate?

Cog answers half of that question. The README names Spotify and Uber's Michelangelo as companies that built similar Docker based systems, and calls Cog the open source version of that work. For the hosted side, the only alternative it offers is your own infrastructure.

How do I get an API key for Replicate?

The README sections on building and running a model do not cover it; that flow uses `cog.yaml`, the `BaseRunner` class and local Docker commands, with Replicate named as one deployment target.

Official sources

  1. License: Apache-2.0
  2. Project website
  3. README
  4. Releases
  5. replicate/cog on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/replicate-cog.svg)](https://hysenlabs.com/projects/replicate-cog)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/replicate-cog.svg)](https://hysenlabs.com/projects/replicate-cog)

Was this useful?