# iHarmony4: the benchmark that made image harmonization measurable > A dataset and reference implementation for adjusting a composited foreground to match its background, covering four sub-datasets with paired masks and real images, generated with four color transfer methods chosen to span the parametric and non-parametric, correlated and decorrelated axes. **bcmi/Image-Harmonization-Dataset-iHarmony4** — [CVPR 2020] The first large-scale public benchmark dataset for image harmonization. The code used in our paper "DoveNet: Deep Image Harmonization via Domain Verification", CVPR2020. Useful for image harmonization, image composition, etc. - Repository: https://github.com/bcmi/Image-Harmonization-Dataset-iHarmony4 - Stars: 812 · Forks: 96 - Language: MATLAB - License: MIT - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/bcmi-image-harmonization-dataset-iharmony4 ## Why harmonization needed a benchmark at all Image harmonization is defined in the project documentation as adjusting the foreground appearances of a composite image so that they are consistent with the background region. A real composite is generated by taking a foreground region from one image and combining it with the background of another image. The problem the dataset addresses is a chicken and egg situation. It is easy to create real composite images, but producing the harmonized output for them is too time consuming and skill demanding, which means there was no high quality publicly available dataset for image harmonization to be trained and measured on. That is the gap iHarmony4 fills, described as the first large scale public benchmark dataset for image harmonization. The repository description also names the paper it accompanies, DoveNet: Deep Image Harmonization via Domain Verification, from CVPR 2020, and the code used in that paper is included. The audience is therefore research oriented. If you are building a production compositing pipeline, the practical question of how to make a pasted object look native is a different problem, solved at inference time by a model rather than by a dataset. This repository is for the people who train and evaluate those models. There is also an online demo for image composition, described as built upon the libcom toolbox, which links out to the separate bcmi/libcom repository. The demo is for trying composition interactively, and the dataset is what a model developer trains against. ## Four sub-datasets, three orders of magnitude apart in size The dataset is not one collection but four, each built on a different source, and the sizes differ enormously. HCOCO is generated from Microsoft COCO using the object segmentation masks provided by COCO as the foreground region. Within that region, the appearance of the COCO image is edited using various color transfer methods. It is described as containing 42k synthesized composite images, with 38545 in the training set and 4283 in the test set. HAdobe5k is generated from MIT-Adobe FiveK. That source provides six editions of the same image, and the foreground region was manually segmented and exchanged between two versions, giving 19437 training and 2160 test images. HFlickr comes from 4833 images collected from Flickr. After manually segmenting the foreground region, the same method used for HCOCO generates the composites, giving 7449 training and 828 test images. Hday2night is built on the day2night dataset. The foreground is manually segmented, cropped and overlaid on another image captured at a different time of day, giving 311 training and 133 test images. In total the four make up iHarmony4 with 65742 training images and 7404 test images. Each sub-dataset contains synthesized composite images, foreground masks of the composite images, and the corresponding real images, which is the pairing that makes the task learnable. The size spread is the important design detail. Hday2night is a niche illumination case with a few hundred pairs, while HCOCO is the general case at tens of thousands. A model evaluated only on Hday2night is being tested on something quite different from one evaluated on HCOCO. ## Four colour transfer methods chosen to span a taxonomy The synthetic composites are produced by color transfer, and the selection of methods is explained rather than asserted. Color transfer methods fall into four groups depending on whether they are parametric or non-parametric and whether they work in a correlated or a decorrelated color space. The project picked one representative method from each group, so the synthetic data covers the space of approaches rather than one family's quirks. The first is global color transfer, described as a parametric method in a decorrelated color space, implementing the paper Color transfer between images. The second is global color transfer in RGB color space, a parametric method in a correlated color space. The third is cumulative histogram mapping, a non-parametric method in a decorrelated color space. The fourth is IDT regrain color transfer, a non-parametric method in a correlated color space, following the paper on automated colour grading using colour distribution transfer. Three of the four implementations were specialised for the foreground based on the code released by Wei Xu with the survey paper, which is what made foreground restricted versions possible at all. The fourth, IDT regrain, was downloaded from the author's own GitHub repository. The colour transfer code lives in its own `color_transfer_methods/` directory in the repository, so it is usable separately from the dataset. If your own compositing pipeline needs foreground colour matching, that directory is the part worth reading. There is also a related dataset, ccHarmony, built with a color checker, which the documentation says can more faithfully reflect illumination variation. That is the dataset to look at if the synthetic composites here look too clean to represent your case. ## Downloading the data, which is archives rather than a package There is no install step for the dataset itself, because the deliverable is image files rather than code. The full iHarmony4 release is provided on Baidu Cloud and on Dropbox, and each sub-dataset is also available separately with its own training and testing split on both services. The documentation gives direct links for the full release and for each of HCOCO, HAdobe5k, HFlickr and Hday2night. Two distributions is a practical choice rather than redundancy. Baidu Cloud is the faster option from mainland China and Dropbox is the one that works from most other networks, and the project links both rather than asking you to guess. The base images also need to be understood before you download anything. HCOCO is derived from Microsoft COCO, HAdobe5k from MIT-Adobe FiveK, HFlickr from Flickr, and Hday2night from the day2night dataset, so if you want to regenerate or extend any of these yourself, the upstream sources are named. For the DoveNet code that accompanies the dataset, the installation steps are separate and assume you already have the data: ```bash git clone https://github.com/bcmi/Image_Harmonization_Datasets.git cd Image_Harmonization_Datasets ``` The prerequisites are Linux, Python 3, and either a CPU or an NVIDIA GPU with CUDA and cuDNN. PyTorch 1.2 and other dependencies including torchvision, visdom and dominate are needed, and the documentation notes that Conda users can create an environment with `conda env create -f environment.yaml`. One naming detail to be aware of when you clone: the repository you are reading is iHarmony4, while the clone command points at the Image_Harmonization_Datasets repository, which is the umbrella location. ## DoveNet and the code of other papers sharing the repository The repository is a collection of several research efforts rather than one codebase, and the top level layout shows it. `DoveNet/` holds the PyTorch implementation of DoveNet and the trained model, which is the paper the dataset was released with. `color_transfer_methods/` holds the four transfer implementations. `data/` holds the dataset scripts and `examples/` holds example images and a results animation. Alongside those are directories named for other approaches: `DIH/`, `Lalonde and Efros/`, `U-Net+attention/`, `Xue et al/` and `Zhu et al/`. These are the prior and contemporaneous works in image harmonization, and having their code in the same tree lets you run the comparisons that a paper needs rather than reimplementing each baseline. There is also an `evaluation.py` at the root, which is the metric script, and the primary language recorded for the repository is MATLAB, which reflects that the earlier methods in the tree are MATLAB implementations while DoveNet is PyTorch. The training workflow uses visdom for visualisation. To view training results and loss plots you run: ```bash python -m visdom.server ``` and then open the URL at http://localhost:8097. Two auxiliary datasets are worth knowing about because they extend the same idea in different directions. SycoNet, from the same group, can augment the small scale dataset by synthesizing high quality composite images for real images, which is the natural answer to a sub-dataset with only a few hundred pairs. And the ccHarmony dataset adds a color checker so that illumination variation is reflected more faithfully. ## Where the benchmark falls short, and what to do about it Three limitations are worth naming. The first is the synthetic composite problem itself. Every sub-dataset except HFlickr and Hday2night is built by color transfer rather than by a human compositing workflow, and the documentation is explicit that creating real composites is easy while creating high quality harmonized versions of them is not. So the ground truth foreground appearance comes from a color transfer method, which means the target is a particular algorithmic idea of what consistent appearance looks like rather than a photographer's judgement of it. That is inherent to a benchmark of this size, not a flaw in the execution, but it does mean a model trained here learns the behaviour of the four transfer methods. The second is the size imbalance. Hday2night at 311 training images is a different regime from HCOCO at 38545, and a paper reporting on iHarmony4 as a whole can hide that. If your use case is illumination transfer, the aggregate number tells you little. The third is the version age of the code. The prerequisites name PyTorch 1.2 and Visdom, and the repository pins to that era of tooling. Getting DoveNet to train today is likely to involve dependency work, and the project has no releases to pull a known good environment from. The alternative to adopting this dataset is to build your own composite pairs from a source corpus with segmentation masks, which you can do with the colour transfer code included here. That gives you control over the domain, at the cost of no common benchmark to compare against published numbers. If you need comparability with the published literature, use iHarmony4. If you need the domain to match your own images, generate your own. ## Conclusion Adopt iHarmony4 if you are training or evaluating an image harmonization model and need paired composite, mask and real image data rather than assembling your own from COCO. Do not adopt it expecting a ready to run library, since the repository is a dataset release with a research implementation and MATLAB and Python code from several papers, and note that the data is distributed as archives on Baidu Cloud and Dropbox rather than through a package manager. Verify first which sub-dataset matches your domain, since Hday2night has only 311 training and 133 test images against HCOCO's 38545, and read the colour transfer section before you assume the synthetic composites reflect real compositing artefacts. ## FAQ ### What is the iHarmony4 dataset? It is described as the first large-scale public benchmark dataset for image harmonization, released with the CVPR 2020 paper DoveNet. It contains four sub-datasets, HCOCO, HAdobe5k, HFlickr and Hday2night, each with synthesized composite images, foreground masks and corresponding real images. ### How many images are in iHarmony4? The table gives 65742 training images and 7404 test images in total, made up of HCOCO with 38545 and 4283, HAdobe5k with 19437 and 2160, HFlickr with 7449 and 828, and Hday2night with 311 and 133. ### How were the iHarmony4 composite images generated? By color transfer, with one representative method from each of four groups based on parametric or non-parametric and correlated or decorrelated color space: global color transfer, global color transfer in RGB, cumulative histogram mapping, and IDT regrain color transfer. Foregrounds were manually segmented for HAdobe5k, HFlickr and Hday2night, and taken from COCO masks for HCOCO. ### Where can I download the iHarmony4 dataset? The full release and each sub-dataset with its training and testing split are provided on both Baidu Cloud and Dropbox, with direct links given for iHarmony4, HCOCO, HAdobe5k, HFlickr and Hday2night. ### Does the iHarmony4 repository include the DoveNet model code? Yes. The DoveNet directory holds a PyTorch implementation and the trained model, and the repository also contains code from earlier works such as DIH, Lalonde and Efros, U-Net plus attention, Xue et al. and Zhu et al, plus evaluation.py. ## Sources - [bcmi/Image-Harmonization-Dataset-iHarmony4 on GitHub](https://github.com/bcmi/Image-Harmonization-Dataset-iHarmony4) - [Issues](https://github.com/bcmi/Image-Harmonization-Dataset-iHarmony4/issues) - [License: MIT](https://github.com/bcmi/Image-Harmonization-Dataset-iHarmony4/blob/master/LICENSE) - [README](https://github.com/bcmi/Image-Harmonization-Dataset-iHarmony4/blob/master/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/bcmi-image-harmonization-dataset-iharmony4