ruoqianguo/FPN_Pytorch: four patches that moved VOC mAP to 80.5

The layer four stride decides where anchors land

The first change is the one that matters, and the README explains it as the fundamental reason for low accuracy in the base code. Anchor positions and the number of anchors per layer are computed from the stride. The designed feature strides in the configuration are given as a list of four, eight, sixteen, thirty-two and sixty-four. When the stride of ResNet layer four is changed from one to two, that list should also change to four, eight, sixteen, sixteen and thirty-two, but in the original code it is left at its default. The consequence is stated plainly: the highest two pyramid levels end up generating about three quarters of their anchors outside the image, which is a large amount of the training signal spent on positions that cannot contain anything.

A config switch is what makes VOC07+12 trainable

The second change solves a data problem rather than a modelling one. In the original code, images are cropped so that batches work and memory stays reasonable. When training on the combined 2007 and 2012 sets, the author found that some images no longer contained the target object after that crop, so a large amount of the training data was quietly mislabelled by the pipeline itself. The fix is a parameter called ASPECT_CROPPING added to the config file, set to false, which stops the cropping from happening at all. It is a one-line configuration change with a large effect on what the model actually sees, and it is the specific reason this fork can train on VOC07 plus 12 where the base repository could not.

The log base change is listed and then disclaimed

The third change is a single identifier: in a function that pools a region of interest over pyramid feature maps, a natural logarithm is changed to a base two logarithm, on the grounds that the original paper uses log2. The README then says in the same sentence that it does not seem to affect the training results. It is worth reading that as what it is, a correctness alignment with the paper rather than an accuracy improvement, and it is listed among the four changes without distinguishing it from the two that do move the numbers. If you are evaluating this fork on its benchmark, the stride change and the cropping switch are where the difference comes from, and this one is a detail you could apply to the original code yourself.

The prerequisites are from 2018

The environment list is short and dated. Python 2.7 or 3.6, PyTorch 0.2.0 or higher, CUDA 8.0 or higher, and tensorboardX for logging. The last of those is a third-party writer for TensorBoard events, which is what the training flag turns on. Note the second item in the fork's own list of changes: support for both Python 2 and Python 3, which in 2018 was a real distinction for a detection codebase. Everything else about the project is consistent with that era. There is no container definition, no dependency lock file, and no CI configuration named in the tree, which holds a library directory, a configuration directory, a demo script, a path setup module and the two network scripts.

The clone command names a different account

The first instruction is a clone, and the URL in it does not match the repository you are reading.

git clone https://github.com/guoruoqian/FPN_Pytorch.git

The account in that address is spelled differently from the repository's own owner, which is the sort of thing that costs an afternoon if you assume the URL was generated from the page you clicked. The second command then creates the data directory.

cd FPN_Pytorch && mkdir data

After that, dataset contents are not committed. The instructions point you at two other repositories: for VOC2007 you are sent to the py-faster-rcnn instructions, and for the combined 07 and 12 sets you are sent to py-R-FCN, with the author saying plainly that the second is more helpful. Either way the expectation is that you create symbolic links inside the data folder rather than copying files into it, and the pretrained model and compilation steps are delegated the same way, to the instructions of another FPN implementation.

Two benchmark tables, and one cell was never filled in

The results are given for two setups, both at scale 600 with ROI Align, both with a ResNet 101 on a single GTX 1080 Ti, a learning rate of 1e-3, a decay of 10 and twelve epochs.

| Setup | Batch | Memory per GPU | Speed per epoch | mAP | | --- | --- | --- | --- | --- | | VOC 2007, 07trainval to 07test | 2 | 6137MB | 0.22 hr | 75.7 | | VOC 07+12, 07+12trainval to 07test | 1 | 9011MB | backslash in the source | 80.5 |

The headline figure is 80.5 on the combined set, which is what the repository description quotes. Two observations for anyone reusing the numbers. The batch size drops from two to one for the larger set, which is consistent with the memory figure rising by nearly three gigabytes, and the speed column for the combined run contains a stray backslash instead of a measurement, so the cost of that configuration is undocumented.

Training and testing are two long command lines

Usage is expressed as complete commands rather than a config tutorial, and each one is worth reading as the actual interface.

CUDA_VISIBLE_DEVICES=3 python3 trainval_net.py exp_name --dataset pascal_voc --net res101 --bs 2 --nw 4 --lr 1e-3 --epochs 12 --save_dir weights --cuda --use_tfboard True

The training entry point is a script called trainval_net.py, and the flags carry the dataset name, the network, the batch size, the number of workers, the learning rate, the epoch count, an output directory, the CUDA switch and the TensorBoard logging switch. Testing uses a second script with checkpoint flags.

CUDA_VISIBLE_DEVICES=3 python3 test_net.py exp_name --dataset pascal_voc --net res101 --checksession 1 --checkepoch 7 --checkpoint 5010 --cuda --load_dir weights

The combined set is trained with the same script under a different dataset name, `pascal_voc_0712`, and a different experiment name, which is what makes the ASPECT_CROPPING switch relevant. A demo script also sits at the top level for a single image.

Editorial conclusion

Take this fork if you want the original faster-rcnn style FPN working on VOC07+12 without editing the config yourself, since the aspect cropping switch is the reason it can be trained on combined data at all. Do not take it as a modern baseline: the prerequisites name Python 2.7 or 3.6, PyTorch 0.2.0 or higher and CUDA 8.0 or higher, and the last push to master is dated 2018-08-07. Two details to verify before you start, because both would otherwise cost you time. The clone command in the README names a different account than the repository itself, and the benchmark table leaves one cell unmeasured.

Frequently asked questions

Official sources

1. Official README
2. Project repository
3. README
4. Releases