Toward Scene Understanding with Depth and Object-Aware Clustering in Contested Environment

摘要

Scene understanding in a contested battlefield is one of the very difficult tasks for detecting and identifying threats. In a complex battlefield, multiple autonomous robots for multi-domain operations are likely to track the activities of the same threat/objects leading to inefficient and redundant tasks. To address this problem, we propose a novel and effective object clustering framework that takes into account the position and depth of objects scattered in the scene. This framework enables the robot to focus solely on the objects of interest. Our system model incorporates YOLOv5 object detection model (ODM) and a pre-trained depth estimation model (DEM), whose outputs are fed into the KMeans Clustering (KM C) model. To train YOLOv5, we create our own battlefield dataset entitled “Common Objects in Battlefield (COBA)” which is split into a training dataset and a validation dataset using our customized object-aware stratified 7-fold cross-validation technique. We present the creation of minimum optimal clusters assuming the minimum number of available battlefield robots. These clusters exhibit accurate relative distances among objects, further validating the effectiveness of our approach for depth and object-aware clustering for scene understanding. Overall, our proposed approach facilitates efficient work division among robots, enhances decision-making capabilities, and helps to improve situational awareness in battlefield scenarios.