Improve the accuracy of Detection & Segmentation models by using SOTA recipes and primitives

[datumbox]

🚀 The feature

Similar to #3995 but focus on Object Detection and Segmentation.

Kick-off a Batteries Included phase 2 project that will focus on improving object detection and segmentation. After adding the necessary primitives, create a new recipe that improves the accuracy of existing models and retrain them to offer better weights to the community.

Results

Best currently available models achieved:

• RetinaNet ResNet50 FPN: Add RetinaNet improved weights #5756
  • Old retinanet_resnet50_fpn: 36.4 mAP
  • New retinanet_resnet50_fpn_v2: 41.5 mAP (+5.1)
• MaskRCNN ResNet50 FPN: Add MaskRCNN improved weights #5773
  • Old maskrcnn_resnet50_fpn: 37.9 box mAP / 34.6 mask mAP
  • New maskrcnn_resnet50_fpn_v2: 47.4 box mAP / 41.8 mask mAP (+9.5/+7.2)
• FasterRCNN ResNet50 FPN: Add FasterRCNN improved weights #5763
  • Old fasterrcnn_resnet50_fpn: 37.0 mAP
  • New fasterrcnn_resnet50_fpn_v2: 46.7 mAP (+9.7)

The above results were achieved by building on top of work done by @rbgirshick, @pdollar, @vaibhava0, @fmassa and @xiaohu2015.

[RangiLyu]

We recently did some experiments on the pre-trained backbone and found that using TIMM's ResNet training method as pretrain
can boost Faster R-CNN from 37.4 to 40.8 mAP (+3.4 mAP).

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.408
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.625
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.446
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.255
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.449
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.532
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.367
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.580
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.682

More details can be found in open-mmlab/mmdetection#7001

And I learned that torchvision also updated a new resnet pre-training method recently in #5201 and it is a SOTA ResNet50. Do you have some experiments on faster rcnn using this pretrained model? Wondering how many improvements can achieve.

[datumbox]

@RangiLyu I don't have yet these numbers but we plan to do such experiments soon after we add some new primitives for detection. I'm currently scoping which techniques should be added (see here for some early work). The metrics that appear on this issue were moved from #3995 and was written prior doing any work on ResNet50. BTW I wouldn't be surprised if at the end we end up training the detection models from scratch using longer cycles, as this has been the trend for strong recipes the last few years.

[xiaohu2015]

We recently did some experiments on the pre-trained backbone and found that using TIMM's ResNet training method as pretrain can boost Faster R-CNN from 37.4 to 40.8 mAP (+3.4 mAP).

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.408
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.625
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.446
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.255
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.449
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.532
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.367
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.580
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.682

More details can be found in open-mmlab/mmdetection#7001

And I learned that torchvision also updated a new resnet pre-training method recently in #5201 and it is a SOTA ResNet50. Do you have some experiments on faster rcnn using this pretrained model? Wondering how many improvements can achieve.

Hi, I run the expriment of RetinaNet with new ResNet50 on detectron2, with the new weights, we can get 41.9 mAP (+about 3.6 compared 38.3 ) (GN + GIoU + multi-scale training trick)

code: https://github.com/xiaohu2015/nndet2