An internal-external optimized convolutional neural network for arbitrary orientated object detection from optical remote sensing images

Figures & data

Figure 1. The architecture of the proposed network contains the backbone network, the coarse detection head and the Refining detection head, where the Backbone network extracts pyramid features of input images with feature pyramid network (FPN) (Lin et al. 2017a), the coarse detection head obtains a set of refined anchors and the refining detection head aims to achieve accurate object detection result. The regression loss used in the proposed network is IOU balanced loss

Figure 2. The schematic diagram of the internal optimization mechanism

Figure 3. The schematic diagram of arbitrary orientated bounding box regression using smooth L1 loss. The $x$ axis and the $y$ axis point to the positive direction of the screen coordinates. The term $\theta p$ denotes the angle from the positive direction of $x$ axis to the direction of the long side of the predicted box where ${\theta }_P\in [0,\pi )$

Table 1. Dataset description

Data set	Images	Categories	Size	Crop size	Training images	Instances
DOTA	2806	15	800 × 800 ~ 4000 × 4000	1024 × 1024	21,046	188,282
HRSC2016	1061	1	300 × 300 ~ 1500 × 900	-	617	2976

Table 2. Detection result on DOTA.(%)

Method	RetinaNet-R	IENet	FR-O	R-DFPN	R2CNN	RRPN	ICN	ROI-Trans	Ours
Plane	88.92	57.14	79.09	80.92	80.94	88.52	81.40	88.64	88.67
BD	67.67	80.20	69.12	65.82	65.67	71.20	74.30	78.52	79.51
Bridge	33.55	64.54	17.17	33.77	35.34	31.66	47.70	43.44	52.44
GTF	56.83	39.82	63.49	58.94	67.44	59.30	70.30	75.92	70.01
SV	66.11	32.07	34.20	55.77	59.92	51.85	64.90	68.81	75.01
LV	73.28	49.71	37.16	50.94	50.91	56.19	67.80	73.68	74.00
Ship	75.24	65.01	36.20	54.78	55.81	57.25	70.00	83.59	84.03
TC	90.87	52.58	89.19	90.33	90.67	90.81	90.80	90.74	90.85
BC	73.95	81.45	69.60	66.34	66.92	72.84	79.10	77.27	80.49
ST	75.07	44.66	58.96	68.66	72.39	67.38	78.20	81.46	85.35
SBF	43.77	78.51	49.40	48.73	55.06	56.69	53.60	58.39	55.64
RA	56.72	46.54	52.52	51.76	52.23	52.84	62.90	53.54	66.08
Harbor	51.05	56.73	46.69	55.10	55.14	53.08	67.00	62.83	66.17
SP	55.86	64.40	44.80	51.32	53.35	51.94	64.20	58.93	66.25
HC	21.46	64.24	46.30	35.88	48.22	53.58	50.20	47.067	51.02
mAP	62.02	57.14	52.93	57.94	60.67	61.01	68.20	69.56	72.37

Table 3. Detection result on HRSC2016 (%)

Method	Fast-RCNN +SRBBS	BL2 (Liu et al. 2017)		R2CNN	IENet	RRD (Liao et al. 2018)	ROI-Trans	Ours
mAP	55.70		69.60	73.07	75.01	84.30	86.20	87.26

Figure 4. Selected examples of detection results from the DOTA test set

Figure 5. Selected examples of detection results on the HRSC2016 dataset

Table 4. Speed-accuracy trade-off comparison with selected methods on DOTA dataset

Method	Image size	mAP	Test speed(s)
FR-O	1024 × 1024	52.93	0.141
ROI-Trans	1024 × 1024	69.56	0.170
Ours	1024 × 1024	72.37	0.137

Figure 6. Comparison of detection results on the HRSC2016 dataset with IOU balanced loss (even rows) and with smooth L1 loss (odd rows)

Table 5. Ablation study on HRSC2016 dataset

Method	Deformable conv layer	IOU balanced loss	mAP
Baseline + Smooth L1 loss	×	×	78.00
Baseline + Deformable conv layer+ Smooth L1 loss	√	×	79.10
Baseline + IOU balanced loss	×	√	86.11
Baseline + Deformable conv layer + IOU balanced loss	√	√	87.26

Lin, T. Y., P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. 2017a. “Feature Pyramid Networks for Object Detection.” In Proceedings of the IEEE conference on computer vision and pattern recognition. Honolulu, Hawaii, America. July 21–26.

 Google Scholar

Liu, Z., J. Hu, L. Weng, and Y. Yang. 2017. “Rotated Region Based CNN for Ship Detection.” In 2017 IEEE International Conference on Image Processing (ICIP). Beijing, China. September 17–20.

 Google Scholar

Liao, M., Z. Zhu, B. Shi, G. S. Xia, and X. Bai. 2018. “Rotation-sensitive Regression for Oriented Scene Text Detection.” In Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City, Utah, America. June 18–23.

 Google Scholar

Data availability statement

The DOTA dataset that support the findings of this study is openly available in https://captain-whu.github.io/DOTA/dataset.html, and the HRSC2016 dataset that support the findings of this study is available from the corresponding author, upon reasonable request.