Intelligent Robot for Worker Safety Surveillance: Deep Learning Perception and Visual Navigation

Abstract

The fatal injury rate for the construction industry is higher than the average for all industries. Recently, researchers have shown an increased interest in occupational safety in the construction industry. However, all the current methods using conventional machine learning with stationary cameras suffer from some severe limitations, perceptual aliasing (e.g., different places/objects can appear identical), occlusion (e.g., place/object appearance changes between visits), seasonal / illumination changes, significant viewpoint changes, etc. This paper proposes a perception module using end-to-end deep-learning and visual SLAM (Simultaneous Localization and Mapping) for an effective and efficient object recognition and navigation using a differential-drive mobile robot. Various deep-learning frameworks and visual navigation strategies with evaluation metrics are implemented and validated for the selection of the best model. The deep-learning model's predictions are evaluated via the metrics (model speed, accuracy, complexity, precision, recall, P-R curve, F1 score). The YOLOv3 shows the best trade-off among all algorithms, 57.9% mean average precision (mAP), in real-world settings, and can process 45 frames per second (FPS) on NVIDIA Jetson TX2 which makes it suitable for real-time detection, as well as a right candidate for deploying the neural network on a mobile robot. The evaluation metrics used for the comparison of laser SLAM are Root Mean Square Error (RMSE). The Google Cartographer SLAM shows the lowest RMSE and acceptable processing time. The experimental results demonstrate that the perception module can meet the requirements of head protection criteria in Occupational Safety and Health Administration (OSHA) standards for construction. To be more precise, this module can effectively detect construction worker's non-hardhat-use in different construction site conditions and can facilitate improved safety inspection and supervision.