Ego4D: the ego4d Python module, downloader CLI and feature extraction API

What the ego4d repository actually ships, and what it does not

Ego4D is described in its README as the world's largest egocentric (first person) video ML dataset and benchmark suite, with over 3700 hours of annotated first-person video. Ego-Exo4D is a separate, larger collection in the same repository: time-synchronized first-person Aria and third-person GoPro footage, with V2 containing 1286.30 video hours across 5035 takes according to the announcement at the top of the README.

The repository is not the dataset. It is the tooling. The `ego4d` Python module contains two downloader CLIs, exposed as the console commands `ego4d` and `egoexo`; a small API that abstracts over TorchAudio and PyAV for reading video; a feature-extraction API with wrappers for models such as Omnivore and SlowFast; notebooks used as tutorials; and research code including clep, the Contrastive Language Ego-centric video Pre-training directory. There is also a visualization engine under `viz/`.

That split matters when you are deciding whether to adopt it. If your problem is "I have a licence, I have a manifest of clip UIDs, and I need the videos on disk plus frame tensors in a dataloader", this repository is aimed at you. If your problem is "I want to look at egocentric video today", the README points you at the hosted visualizer instead, and that requires a licence.

Two CLIs, one module: how the download and read path is wired

The entry points in setup.py tell you most of the architecture. `ego4d` maps to `ego4d.cli.cli:main`, `egoexo` maps to `ego4d.egoexo.download.cli:main`, and there are two further internal entry points, `egoexo_internal` and `ego4d_validation`, which the README does not describe. The two public datasets therefore have separate download code paths under `ego4d/cli/` and `ego4d/egoexo/download/`, each with its own README.

Runtime dependencies declared in setup.py are deliberately thin: boto3, tqdm, regex, dataclasses_json and iopath. The presence of boto3 is a strong hint about the mechanism, since the downloaders talk to object storage rather than serving files from a web front end. The heavier packages, torch, torchvision, torchaudio, pytorchvideo, timm and pycocotools, sit in requirements.txt instead, because they are only needed for feature extraction and research code, not for downloading.

For reading, the module provides what the README calls a simple API abstracting common video reading libraries, with TorchAudio and PyAV backends in `ego4d/research/readers.py`. That abstraction is the part worth caring about if you write your own dataloader: it means your training code can be written against one reader interface and switch backends without changing the rest of the pipeline. The feature-extraction API is documented separately in `ego4d/features/README.md`, which describes using it as an API rather than only as a script.

Installing the ego4d Python module and running a first import

The README gives two installation routes. The PyPI route is the short one, and it requires Python 3.10 or newer. Install it inside an activated conda or pyenv environment.

pip install ego4d --upgrade

After that, `ego4d` should be on your PATH as a console command, because the package declares that entry point in setup.py.

The second route is for working from a clone, which you need if you want the notebooks, the research code or the visualization engine. The README's example environment uses Python 3.11, and installation is a plain editable-style install from the repository root.

conda create -n ego4d python=3.11 -y
conda activate ego4d
pip install .

To confirm the module resolves to the copy you just installed, the README suggests printing it. The output shows the path of the `ego4d` module on your filesystem, which is how you tell a clone install apart from a stale PyPI one.

python3 -c 'import ego4d; print(ego4d)'

Downloading is a separate step, and the README does not inline the flags. It points at `ego4d/cli/README.md` for Ego4D and `ego4d/egoexo/download/README.md` for Ego-Exo4D. Read those files before running either command, because the downloaders are the part most likely to need an argument you have not thought about.

Licence gate, Python floor and the stale requirements.txt pins

The first real limitation is not technical. The repository is MIT licensed, but the README is explicit that exploring Ego4D or Ego-Exo4D through the visualizer requires a licence, and it sends you to the Start Here and Getting Started pages to access the data. So the code is permissive while the data is gated. Nothing in the repository removes that gate, and the two licences apply to different things. Treat them separately.

The second limitation is version drift. The PyPI package is versioned 1.7.3 in setup.py, while the most recent release listed for the repository is 1.5.2 from 2023-12-15. Those are different numbering tracks, and the README does not explain the relationship. Meanwhile requirements.txt pins torch==1.11.0, torchvision==0.12.0, torchaudio==0.11.0, numpy==1.22.3, av==9.0.2 and pandas==1.4.1. Those are exact pins, not floors. If you install the module with pip and then separately install a current torch for your own training code, you are running a combination the repository never pinned. The README does not document a supported upgrade path for those pins.

The third limitation is scope. Baseline code for the Ego4D benchmarks lives in separate GitHub repositories under the EGO4D organization, and the README states that baseline code for Ego-Exo4D is coming soon. If your goal is to reproduce a benchmark number, this repository is not where that code lives. The README also does not document rollback or pinning behaviour for the downloaders, so plan for that gap yourself.

How Ego4D compares with a general video dataset loader

The closest alternative is not another egocentric dataset. It is treating Ego4D like any other video corpus and pulling it through a generic loader such as a Hugging Face datasets pipeline. The difference in approach is where the work sits. A generic loader assumes the data is reachable and that a URL or dataset identifier is enough. Ego4D's own tooling assumes the opposite: you hold a licence, you have an approved manifest of clip identifiers, and the downloader's job is to fetch exactly that set from object storage and lay it out on disk in the structure the rest of the module expects.

That distinction shows up in the dependencies. A generic loader pulls in a dataframe and streaming stack. The `ego4d` module pulls in boto3 and iopath, which is what you would expect from something that resolves paths and fetches from storage rather than from a public hub. It also shows up in the reader layer: the module ships its own abstraction over TorchAudio and PyAV instead of delegating to a library's decoder, so feature extraction and research code can share one interface.

If your corpus is public and small, the generic loader is less machinery. If your corpus is licence-gated, multi-view and synchronized, the downloader and reader in this repository are doing work you would otherwise write yourself.

Maintenance, upgrade cost and what the licence split means for you

The repository is not archived, and the last push was on 2026-07-25. That is recent enough that the README's announcements, including the Ego-Exo4D V2 and Ego4D V2.1 Goal-Step additions, reflect current state rather than a frozen snapshot. The release history is a different story: the most recent release entry is 1.5.2 from 2023-12-15, so versioned releases are not the channel through which changes arrive. If you track this project, track the default branch, not the release list.

Upgrade cost is dominated by the pinned stack in requirements.txt rather than by the module's own code. The module's declared install_requires is five small packages, so `pip install ego4d --upgrade` is cheap. Moving the research and feature-extraction code forward means moving torch, torchvision and torchaudio together, and the repository does not document which combinations have been checked. Budget for that as a migration, not a version bump.

On licensing: the repository is MIT, which is permissive and imposes few obligations on the code you write against it. The dataset is a separate matter, and the README's note that the visualizer requires a licence makes clear that access is governed elsewhere. This is not legal advice, and the access terms are not reproduced in the repository, so read the Start Here page for Ego4D and the Getting Started page for Ego-Exo4D before you build anything you intend to publish.

Editorial conclusion

Adopt this repository if you are already cleared for the Ego4D or Ego-Exo4D licence and you want the downloader CLI plus the reader and feature-extraction APIs in one installable module; the pip route needs Python 3.10 or newer. Do not adopt it if what you actually want is the raw video, because the repository does not contain the dataset and the visualizer at visualize.ego4d-data.org requires a licence. Before you commit, verify which of the two licences covers your intended use, confirm whether you need the PyPI package or a clone of the repository, and check that your own torch and torchvision versions are compatible with the pins in requirements.txt rather than assuming those pins are current.

Frequently asked questions

Official sources

1. facebookresearch/Ego4d on GitHub
2. License: MIT
3. Project website
4. README
5. Releases