# ARKitScenes is an indoor RGB-D dataset with a laser scanner in the room > ARKitScenes is the dataset and code that accompanied Apple's NeurIPS paper on 3D indoor scene understanding. It holds 5,047 captures across 1,661 unique indoor scenes, captured with the Apple LiDAR scanner, and the part that makes it expensive is the ground truth: high resolution depth from a stationary laser scanner placed in the same rooms, plus manually labelled oriented 3D bounding boxes for a large furniture taxonomy. **apple-aiml-research/ARKitScenes** — This repo accompanies the research paper, ARKitScenes - A Diverse Real-World Dataset for 3D Indoor Scene Understanding Using Mobile RGB-D Data and contains the data, scripts to visualize and process assets, and training code described in our paper. - Repository: https://github.com/apple-aiml-research/ARKitScenes - Stars: 974 · Forks: 80 - Language: Python - License: NOASSERTION - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/apple-aiml-research-arkitscenes ## What the dataset is, in the authors' own terms ARKitScenes accompanies a research paper presented at the Thirty-fifth Conference on Neural Information Processing Systems, in the Datasets and Benchmarks Track, and the repository is the paper's companion rather than a library. The claim the documentation makes is twofold. It is the first RGB-D dataset captured with a now widely available depth sensor, the Apple LiDAR scanner, and it is the largest indoor scene understanding data ever collected. The rest of the dataset's shape follows from how the first claim was achieved. Because a consumer device was used to capture the data, the scenes are ordinary rooms rather than staged ones, which is the point of the dataset, and the paper's stated ambition is to push the boundaries of existing state of the art methods and to introduce new challenges that better represent real world scenarios. The author list is long and includes people from both sides of the collaboration, and the documentation asks that the paper be cited when the data or the source code is used. ## Five thousand captures, and a high resolution subset The headline figures are in the key features list, and they are the numbers to quote. ARKitScenes is the largest indoor 3D dataset, consisting of 5,047 captures of 1,661 unique scenes. The distinction between a capture and a scene matters, since a capture is one scan of one room, so a single room can contribute several captures from different positions and scan patterns. The documentation also includes a comparison table of RGB-D datasets and their ground truth assets, and mentions that high resolution and low resolution variants are available for a subset of 2,257 captures of 841 unique scenes. So a little under half of the captures have a high resolution counterpart, which is the subset where the expensive ground truth was collected, and the comparison figure exists precisely because the ground truth assets differ in kind from one dataset to the next, which is what makes a table of them useful rather than decorative. ## The ground truth is the expensive half Two ground truth assets are named and both are the reason a dataset like this takes a team. The first is high resolution depth maps captured using a stationary laser scanner, which is a different instrument from the handheld device and a different class of measurement, and the difference in accuracy between a consumer depth sensor and a laser scanner is what allows the depth to be used as a training target rather than as a feature. The second is manually labelled 3D oriented bounding boxes for a large taxonomy of furniture, which is annotation work rather than measurement work, and oriented boxes are what a 3D detection model is actually trained against. On top of that the dataset provides the camera pose and the surface reconstruction for each scene, and the documentation summarises the ground truth as registered RGB-D frames plus oriented bounding boxes of room defining objects. Registered is the word doing work in that sentence, since a registration error between the RGB frame and the depth frame would corrupt every downstream label. ## Two devices in the room, and the scan patterns The data collection section explains the setup, and the explanation is the most useful part of the repository for anyone reproducing it. The figure shows an iPad Pro on its scanning setup, a mesh overlay used to assist the data collection, and an example of one of the scan patterns captured with the iPad Pro, where the red markers show the chosen locations of the stationary laser scanner in that room. So the protocol is that a person carries the iPad Pro around a room to record the RGB-D video, and a laser scanner is placed at marked positions to be run alongside, and the markers exist because the scanner has to be repositioned and the operator has to know where. The mesh overlay is the practical aid, since it lets the operator see what has already been covered and what has not. This is a workflow detail that a dataset card usually omits and that a person about to collect their own data would find immediately useful. ## Two downstream tasks ship with the data The repository does not only host data. It provides helper scripts for two downstream tasks, 3D object detection and RGB-D guided upsampling, and each has its own directory with its own readme, `threedod/` and `depth_upsampling/`. The detection task is the one the dataset's oriented bounding boxes exist for, so the code in that directory is the reference implementation of how the labels are meant to be used, and the upsampling task is the one the laser scanner depth exists for, since guided upsampling needs a high resolution target to learn from. The training code described in the paper is in these directories. The practical value for a new user is that the two tasks double as worked examples, since they show how the data is laid out, how the annotations are formatted, and what a working training setup looks like against this dataset, which is usually the slowest part of adopting a research dataset. ## Getting the data, and what is in the tree The data is not in the repository, and the download is a separate step. The README points at a `DATA.md` document for the download instructions, and the tree has a `download_data.py` script, so the practical workflow is to follow the document and use the script rather than to pull files from a repository. The rest of the top level is small and tells you what else to expect. There is a `raw/` directory for the raw captures, the two task directories, `DATA.md`, the licence file, an `ACKNOWLEDGEMENTS.txt` and a `.gitignore`. The acknowledgements file is a good sign, because a dataset built on a lot of people's rooms and a lot of people's scanning hours is the kind of thing where who helped matters. The documentation also says that other questions can go in a GitHub issue or to a group address, and that questions about the licence specifically should go to a separate licence address, which is the arrangement you want to see in a dataset with terms as specific as these. ## The licence is the first thing to read The licence section is short and points at the `LICENSE` file for the detailed terms on using the dataset, and the repository records that licence as one GitHub cannot classify, which in practice means it is not one of the standard permissive licences. The documentation asks you to refer to that file for details and gives a dedicated address, `ARKitScenes-license@group.apple.com`, for any additional inquiries about the licence, with a separate address for other questions. This review does not interpret the terms, and a dataset released by a company rather than a foundation is exactly the case where the terms are worth reading carefully, especially for a commercial use or a model trained on it. Two practical points to carry into that reading. The ground truth was produced by Apple's own hardware and staff, so the dataset is a corporate asset rather than a community one, and the citation request in the README asks you to cite the paper, which is a scholarly obligation rather than a licence condition but is the expectation the project states. ## Conclusion ARKitScenes suits a researcher who needs indoor RGB-D data with real depth ground truth rather than a simulator's, and who needs oriented 3D boxes for a large furniture taxonomy, since that combination is what the dataset was built to provide. It is a poor fit if you need the data to be easy to get, since the download is a separate documented process and the volume is what it is, and it is a poor fit for commercial use without reading the licence, because the terms are in a file of their own and Apple publishes a dedicated address for licence questions. Before you build on it, check three things: what the licence permits, since the repository records a custom licence and asks you to read it, how much of the dataset you actually need, since the high resolution subset covers a fraction of the captures, and the version of the data you download, since the scripts in the tree are the ones that fetch it and the last push to the repository was on 2026-09-11. ## FAQ ### What is in the ARKitScenes dataset? It is an indoor RGB-D dataset with 5,047 captures of 1,661 unique scenes, captured with the Apple LiDAR scanner, including the camera pose and surface reconstruction for each scene, high resolution depth maps from a stationary laser scanner, and manually labelled 3D oriented bounding boxes for a large taxonomy of furniture. ### How do I download the ARKitScenes data? Follow the data documentation in the repository, which is the DATA.md file, and use the download_data.py script in the tree. The raw captures are not committed to the repository itself. ### What code comes with ARKitScenes? Helper scripts and training code for two downstream tasks, 3D object detection in the threedod directory and RGB-D guided depth upsampling in the depth_upsampling directory, each with its own readme describing how the data and the labels are meant to be used. ### How was the ARKitScenes depth ground truth collected? With a stationary laser scanner placed at marked positions in the room while the iPad Pro was used to capture the RGB-D data, and the documentation shows the scan patterns and the red markers indicating the chosen scanner locations for each room. ### Can I use ARKitScenes commercially? The repository records a custom licence, and the documentation asks you to refer to the LICENSE file for the detailed terms and to contact the dedicated ARKitScenes licence address for any additional questions. The paper's NeurIPS citation is also requested when the data or the code is used. ## Sources - [apple-aiml-research/ARKitScenes on GitHub](https://github.com/apple-aiml-research/ARKitScenes) - [Issues](https://github.com/apple-aiml-research/ARKitScenes/issues) - [README](https://github.com/apple-aiml-research/ARKitScenes/blob/main/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/apple-aiml-research-arkitscenes