Habitat-Lab: Meta's End-to-End Embodied AI Research Framework

What Habitat-Lab Provides and Who It Is For

Embodied AI research requires simulating an agent with sensors and actuators in a physical environment, defining a task with success criteria, and measuring whether training improves performance on that task. Habitat-Lab packages this workflow into a set of Python libraries built around a gym-compatible environment interface.

The four core capabilities documented in the README are:

1. Flexible task definitions: users can define single-agent and multi-agent tasks, including navigation, object rearrangement, instruction following, question answering, and human following, and can define novel tasks in the framework.

2. Diverse embodied agents: the library configures commercial robots and humanoid agents, specifying their sensors (cameras, depth sensors) and capabilities.

3. Training and evaluation: built-in algorithms for single and multi-agent training via imitation learning, reinforcement learning, or scripted SensePlanAct pipelines. Benchmarking tools measure performance with standard metrics.

4. Human-in-the-loop interaction: a framework for humans to interact with the simulator, collecting embodied demonstration data or interacting with trained agents.

The primary audience is robotics and AI researchers at universities and labs who publish work on embodied AI tasks. The README links to three academic papers: Habitat 1.0 (2019), Habitat 2.0 (2021), and Habitat 3.0 (2023, describing co-habitat for humans and robots). The citation block in the README is the standard form for academic attribution.

Habitat-Lab does not include the 3D scene assets needed for training. Those are downloaded separately using Habitat-Sim's data download utility.

Architecture: Habitat-Lab on Top of Habitat-Sim

Habitat-Lab is the high-level library. Habitat-Sim is the low-level physics and rendering simulator. The two are separate repositories: `facebookresearch/habitat-lab` (this project) and `facebookresearch/habitat-sim`.

Habitat-Lab uses Habitat-Sim as its core simulator, as the README states. Habitat-Sim handles rendering, collision detection, and physics. Habitat-Lab defines the task logic, the reward functions, the agent configurations, and the training pipelines.

The Python package is organized into two installable components: `habitat-lab` (the core, installed with `pip install -e habitat-lab`) and `habitat-baselines` (the training algorithms and baselines, installed separately with `pip install -e habitat-baselines`). The split allows users who only need the task environment without the RL training code to install the smaller core package.

Configuration is handled through YAML files using a hierarchical config system. The `CONFIG_KEYS.md` reference documents what each key controls. The `habitat-lab/habitat/config/benchmark/` directory contains YAML configs for the benchmark tasks. Override keys at the command line or in Python using the `overrides` argument to `habitat.get_config()`.

The Dockerfile in the repository uses `nvidia/cudagl:10.1-devel-ubuntu16.04` as its base image. This is an older CUDA version and Ubuntu release, indicating that the Docker setup has not been updated recently.

Installing Habitat-Lab with Conda

The README requires Python 3.9 or higher and cmake 3.14 or higher. The recommended path uses conda:

conda create -n habitat python=3.9 cmake=3.14.0
conda activate habitat

Install Habitat-Sim with bullet physics:

conda install habitat-sim withbullet -c conda-forge -c aihabitat

Clone the stable branch of Habitat-Lab and install the core package:

git clone --branch stable https://github.com/facebookresearch/habitat-lab.git
cd habitat-lab
pip install -e habitat-lab

Install the baselines package (includes RL training dependencies):

pip install -e habitat-baselines

After installation, download test assets using Habitat-Sim's download utility:

python -m habitat_sim.utils.datasets_download --uids habitat_test_scenes --data-path data/

And download navigation episodes for the test scenes:

python -m habitat_sim.utils.datasets_download --uids habitat_test_pointnav_dataset --data-path data/

Run the example pick task script to verify everything works:

python examples/example.py

This uses the configuration at `habitat-lab/habitat/config/benchmark/rearrange/skills/pick.yaml` to set up a virtual robot performing a pick task with random actions.

Task Types, Robots, and the Gym Interface

The gym interface is the primary way to interact with Habitat-Lab tasks. The README shows two patterns. The first uses `gym.make` with a registered task ID:

import gym
import habitat.gym

env = gym.make("HabitatRenderPick-v0")
observations = env.reset()

The second uses a YAML config for more control:

config = habitat.get_config(
  "benchmark/rearrange/skills/pick.yaml",
  overrides=["habitat.environment.max_episode_steps=20"]
)
env = habitat.gym.make_gym_from_config(config)

The `examples/` directory contains scripts for multiple tasks: navigation (`shortest_path_follower_example.py`), vision-language navigation (`vln_benchmark.py`, `vln_reference_path_follower_example.py`), and interactive play (`interactive_play.py`). The `register_new_sensors_and_measures.py` example shows how to extend Habitat-Lab from outside the source code, without modifying the installed package.

Habitat 3.0 (described in the 2023 paper) added humanoid agents and human-robot collaboration tasks under the `habitat-hitl/` directory. HITL stands for Human In The Loop. This component supports keyboard and mouse control of a Fetch robot in the ReplicaCAD scene dataset.

Habitat-Lab vs. Habitat-Sim: What Each Project Does

The distinction between Habitat-Lab and Habitat-Sim is a common source of confusion. Habitat-Sim is the C++ and Python simulator that handles rendering, navigation mesh computation, physics integration (via Bullet), and low-level sensor simulation. It is a separate repository (`facebookresearch/habitat-sim`) with its own installation and release cycle.

Habitat-Lab depends on Habitat-Sim but adds the task abstraction layer: episode datasets, reward functions, agent policies, training loop utilities, and evaluation metrics. If you want to render a 3D scene and move an agent around, Habitat-Sim alone may be sufficient. If you want to train a policy to complete a specific task (navigation, pick-and-place, human following), you need Habitat-Lab.

The conda install in the README's step 2 installs `habitat-sim` as a package from the `aihabitat` channel. For features added after the most recent release of that package, the README says you may need `aihabitat-nightly`.

For researchers who want a simulator without the full training framework, Isaac Lab (from NVIDIA) and PyBullet are alternative simulation environments. Isaac Lab focuses on physical manipulation with GPU-accelerated physics and is not limited to indoor navigation tasks. Habitat-Lab's advantage is its large collection of pre-built indoor scene assets and task definitions specifically designed for navigation and manipulation research.

Maintenance Status and License

The README includes a prominent warning at the top: "Beyond v0.3.4 this project is no longer receiving official active development or maintenance by Meta internal teams. Please feel free to continue forking and developing the software independently."

The last official release, v0.3.4, was published on 2026-05-07. The last push to the repository was also on 2026-05-07, corresponding to that release. Meta's statement means that bug reports and pull requests may not receive official responses.

The three academic papers (Habitat 1.0, 2.0, 3.0) remain the authoritative documentation for the research platform's design decisions. The project website at `aihabitat.org` hosts documentation and a demo linked from the README.

The license is MIT. ROS integration is documented separately in the repository as ROS-X-Habitat, mentioned in the README's table of contents. The `DATASETS.md` file catalogs which scene datasets and episode datasets are available and how to download them.

Editorial conclusion

Habitat-Lab suits robotics researchers and embodied AI engineers who need a full pipeline from task definition through training and evaluation, and who are building on the Habitat-Sim physics simulator. The library's breadth (single and multi-agent tasks, multiple robot types, RL and imitation learning baselines) makes it a practical starting point for academic research. However, Meta's internal teams have stated that official active development stopped after v0.3.4. New adopters should read v0.3.4 before deciding whether the current state meets their needs, check the issue tracker for community forks, and confirm that the task types and robot configurations they require are already implemented. The last official release was v0.3.4 on 2026-05-07.

Frequently asked questions

Official sources

1. facebookresearch/habitat-lab on GitHub
2. License: MIT
3. Project website
4. README
5. Releases