Non-Rectangular RoI Extraction and Machine Learning Based Multiple Object Recognition Used for Time-Series Areal Images Obtained Using MAV

Abstract

This paper presents a novel method to recognize multiple objects from aerial scene images obtained using a micro air vehicle (MAV). As a robot vision extended platform, MAVs measure objects in time-series images obtained from various angles and altitudes with advanced outstanding active vision characteristics. The proposed method consists of four major steps: region of interest (RoI) extraction using binarized normed gradients (BING), feature point detection and description using accelerated-KAZE (AKAZE), generation of codebook as histogram features quantized using self-organizing map (SOMs), and semantic recognition of multiple objects using category maps created using counter propagation networks (CPNs). We obtained five datasets of time-series aerial images at an atrium while flying a MAV manually. The original video images were downsampled from 30 fps to 1 fps in consideration of calculation cost and appearance changes between frames. We annotated ground truth (GT) labels to all images used for teaching signals for learning and validation signals for testing. Experimentally obtained results with leave-one-out cross-validation (LOOCV) revealed the mean recognition accuracy for all datasets as 71.96%. For each dataset, the maximum and minimum recognition accuracies were, respectively, 77.26% in Dataset 3 and 65.25% in Dataset 2.