Bilateral-Head Region-Based Convolutional Neural Networks: A Unified Approach for Incremental Few-Shot Object Detection

Abstract

Practical object detection systems are highly desired to be open-ended for learning on frequently evolved datasets. Moreover, learning with little supervision further adds flexibility for real-world applications such as autonomous driving and robotics, where large-scale datasets could be prohibitive or expensive to obtain. However, continual adaption with small training examples often results in catastrophic forgetting and dramatic overfitting. To address such issues, a compositional learning system is proposed to enable effective incremental object detection from non-stationary and few-shot data streams. First of all, a novel Bilateral-Head Framework is proposed to decouple the representation learning of base (pre-trained) and novel (few-shot) classes into separate embedding spaces, which takes care of novel concept integration and base knowledge retention simultaneously. Moreover, to enhance learning stability, a robust parameter updating rule, i.e., Recall and Progress mechanism, is carried out to constrain the optimization trajectory of sequential model adaption. Beyond that, to enforce inter-task class discrimination with little memory burden, we present a between-class regularization method that expands the decision space of few-shot classes for constructing unbiased feature representation. Finally, we deeply investigate the incomplete annotation issue considering the realistic scenario of iFSOD and propose a semi-supervised object labeling mechanism to accurately recover the missing annotations for previously encountered classes, which further enhances the robustness of the target detector to counteract catastrophic forgetting. Extensive experiments conducted on both Pascal VOC and MS-COCO datasets demonstrate the effectiveness of our method.