Example for rotated faster rcnn

[cjt222]

Thanks for great work, i have a question, how can i set config file to detect oriention target in faster rcnn or other network?

[ppwwyyxx]

What do you mean by "oriention target"?

If you mean objects that are often not horizontal/vertically aligned, like texts, detectron2 supports rotated bounding boxes. But an example is not available now.

[cjt222]

Thanks for your reply!! I still have some question:
1、what kind of network can support rotated bounding boxes？
2、what kind of datasets have you tried when detecting with rotated bounding boxes and how to calculate mAP while eval and could you give me a baseline?
3、Could you share papers in rotated bounding boxes in detectron2?

[ppwwyyxx]

There will be more details when a complete training example is released.

[Aswantm]

State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet and Fast R-CNN have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully-convolutional network that simultaneously predicts object bounds and objectness scores at each position. RPNs are trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. With a simple alternating optimization, RPN and Fast R-CNN can be trained to share convolutional features. For the very deep VGG-16 model, our detection system has a frame rate of 5fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007 (73.2% mAP) and 2012 (70.4% mAP) using 300 proposals per image. The code will be released.

[issa-tingzon]

Hi, would like to follow up on this. Would you have a minimal working example on training and evaluating models using rotated bounding boxes? Thanks!

[freesunshine]

We almost cry out for it! A minimal working example is needed!Thanks!

[WeihongM]

@ppwwyyxx Any updates?

[edwardchaos]

I'm attempting to do this as well, so far I've found these configs:

cfg.MODEL.PROPOSAL_GENERATOR.NAME = "RRPN"
cfg.MODEL.ANCHOR_GENERATOR.NAME = "RotatedAnchorGenerator"

Currently, the angles in anchors are at 90 degree intervals so i'll need to add more angles.

I'm using the faster_rcnn_R_101_FPN_3x pretrained network.

The issue now is using detectron2's dataloader with rotated bounding box annotations.
In the documentation it says that box must be a member of detectron2.structures.BoxMode which only supports horizontal bounding box annotations.

To get it to work now I'm thinking i'll have to use another way of loading data.
@ppwwyyxx do you think it will be available in days, weeks, or months?

[ppwwyyxx]

Yes a custom dataloader is needed to provide rotated boxes as input. We don't have a concrete timeline.

[edwardchaos]

Update:

I found another config that needs to be overridden:
MODEL.RPN.BBOX_REG_WEIGHTS: (1.0, 1.0, 1.0, 1.0, 1.0)

The default configs file doesn't have a section for RRPN, assuming it's intentional and we're supposed to override RPN configs

[freesunshine]

@edwardchaos Appreciate you for working on it! Can you keep on updating your progress here? That will be very helpful!

[edwardchaos]

Yup, now I'm taking a step back and getting horizontal bounding boxes to work first, found another config that needs to be changed for rotated boxes:

MODEL.ROI_BOX_HEAD.POOLER_TYPE = "ROIAlignRotated"