cityscapesScripts: file naming, label conversion and evaluators for the Cityscapes street-scene dataset

One path template decides where every Cityscapes file lives

Every file in the dataset follows the same path: {root}/{type}{video}/{split}/{city}/{city}_{seq:0>6}_{frame:0>6}_{type}{ext}. The scripts all assume this layout, so a dataset that has been renamed or flattened will not work with them.

root is the top folder. Many scripts look for an environment variable named CITYSCAPES_DATASET pointing at it and use that as the default, which saves repeating the path on every command. type is the modality, such as gtFine for fine ground truth or leftImg8bit for left 8-bit images. city is one of the 50 cities where the footage was recorded. seq and frame are six-digit numbers padded with zeros.

The ext part carries the file extension and sometimes a suffix, for example _polygons.json for ground truth files. One detail that surprises people: cities differ in how they were recorded. Some have very few but very long sequences, while others have many short sequences in which only the 19th frame is annotated. Code that assumes dense annotation along a sequence will break on the second kind.

Empty split folders are expected, not a broken download

The five splits are train, val, test, train_extra and demoVideo, and not every data type exists for every split. Finding an empty folder is normal and the documentation says not to be surprised by it.

train holds 2975 images with fine and coarse annotations. val holds 500 images with both annotation kinds and is meant for validating hyper-parameters, though it can also be used for training. test is used on the evaluation server, and its annotations are not public. Ego-vehicle and rectification border annotations are included for convenience. train_extra holds 19998 images with coarse annotations only. demoVideo contains video sequences for qualitative evaluation and has no annotations at all.

If a loader walks every split and every type and fails on a missing directory, the fault is in the loader. Decide per split which types you actually need. A training script that reads gtFine from train_extra will find nothing, because that split only has coarse labels.

Fine and coarse labels come in different sizes and serve different jobs

gtFine is the fine annotation set: 2975 training, 500 validation and 1525 testing images. It is used for validation, testing and optionally training. The labels are stored as json files with individual polygons, and as png images where pixel values encode labels. helpers/labels.py and the scripts in preparation explain the encoding.

gtCoarse covers all training and validation images plus the 19998 extra training images in train_extra. It can be combined with gtFine or used alone for weakly supervised training.

Other annotation types are narrower. gtBbox3d holds 3D bounding boxes of vehicles and points to the Cityscapes 3D paper from CVPRW 2020 for details. gtBboxCityPersons holds pedestrian boxes for training and validation images, with the values (x, y, w, h) where (x, y) is the top-left corner. gtBboxValidated holds validated 2D boxes for eight instance classes: person, rider, car, truck, bus, train, motorcycle and bicycle. Its boxes use a different coordinate convention, (x1, y1, x2, y2), so reusing a box parser between those two types gives wrong widths and heights.

Sixteen-bit PNGs and disparity maps need a library that can read them

Two data types are stored as 16-bit PNG files, and the documentation warns that this is non-standard and not supported by all libraries. leftImg16bit holds left images in 16-bit HDR format with more information in very dark or bright parts of a scene. rightImg16bit is the same for the right stereo view. The 8-bit variants, leftImg8bit and rightImg8bit, are the standard annotated images.

disparity holds precomputed depth maps in the same 16-bit format. To get a disparity value, take each pixel p greater than zero and compute d = ( float(p) - 1. ) / 256. A pixel value of 0 is an invalid measurement. If your image reader silently converts to 8 bits, every disparity value you compute will be wrong without raising an error, so check the dtype after loading.

leftImg8bit_blurred has faces and license plates blurred, and the instructions are specific: compute results on the original images but use the blurred ones for visualization. The blurring was done by Mapillary. timestamp files give the recording time in nanoseconds, and the first frame of each sequence has a timestamp of 0. camera and vehicle hold calibration, odometry, GPS and outside temperature, documented in docs/csCalibration.pdf.

The pip install and the separate [gui] extra for Qt5 tools

The package is on PyPI under the name cityscapesscripts. The base install is one command:

python -m pip install cityscapesscripts

The viewer and the label tool use Qt5 and are only installed with the extra:

python -m pip install cityscapesscripts[gui]

The split exists because the evaluation and conversion tools do not need a GUI toolkit, so a headless server can install the base package without Qt5. In setup.py the tools are registered as console scripts. csDownload, csEvalPixelLevelSemanticLabeling, csEvalInstanceLevelSemanticLabeling, csEvalPanopticSemanticLabeling, csEvalObjectDetection3d, csCreateTrainIdLabelImgs, csCreateTrainIdInstanceImgs, csCreatePanopticImgs and csPlot3dDetectionResults all work from the command line. csViewer and csLabelTool are the graphical entries.

After installation the package is importable as the Python module cityscapesscripts, so you can call the evaluation code from your own scripts as well as from the shell.

Evaluation commands that score validation and also match the test server

Four evaluators ship as commands. csEvalPixelLevelSemanticLabeling scores pixel-level semantic labeling. csEvalInstanceLevelSemanticLabeling scores instance-level labeling. csEvalPanopticSemanticLabeling scores panoptic segmentation. csEvalObjectDetection3d evaluates 3D object detection, and csPlot3dDetectionResults plots those detection results. Each of the first three runs on the validation set, and the documentation says the same tool is also used to evaluate results on the test set.

That matters if you report a number from your own validation run and later submit to the server: the scoring code is shared, not reimplemented. The preparation commands convert annotations into the form training code usually wants. csCreateTrainIdLabelImgs and csCreateTrainIdInstanceImgs produce train-ID label images, and csCreatePanopticImgs produces panoptic images.

The README text available here is cut off in the middle of the csEvalObjectDetection3d entry, so options for that command and for the converters should be read from each command's help output or from the source under cityscapesscripts/evaluation and cityscapesscripts/preparation.

A Cython build for the evaluators, and a download step that needs an account

setup.py has an optional build for faster evaluation. The comment at the top says to install cython and numpy with python -m pip install cython numpy, then build with CYTHONIZE_EVAL= python setup.py build_ext --inplace. The setup script only compiles when the CYTHONIZE_EVAL variable exists in the environment, and it then sets both CC and CXX to g++ and builds addToConfusionMatrix.pyx from cityscapesscripts/evaluation.

On macOS the setup comment warns that you may have to export the numpy headers through CFLAGS. The build therefore needs a g++ compiler on the machine. The comment calls the result slightly faster, and no figure is given.

csDownload fetches the dataset packages from the command line. The dataset details and download are hosted at www.cityscapes-dataset.com. The repository itself is MIT licensed and its last push was on 4 October 2026. It has no GitHub releases, so the version comes from the VERSION file in the package and from PyPI.

Editorial conclusion

Install it if you train or score segmentation or detection models on Cityscapes, because the evaluators are the same ones used for the test server and the folder convention is what other Cityscapes code expects. Skip it if you want the images themselves, since they are downloaded from the dataset site with an account. Before building a pipeline, check which split each annotation type exists for, because several types are missing from test and train_extra and the folders may simply be empty.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. mcordts/cityscapesScripts on GitHub
4. README