Localization of mobile robots through optical flow and sensor fusion in mining environments

Abstract

Currently, autonomous mining vehicles are using GNSS for localization. Due to atmospheric phenomena, the GNSS signal becomes unstable, making autonomous equipment stop their movements, thus decreasing the mine's productivity. This paper presents a method to estimate the 2D localization of ground vehicles through the optical flow from images of a camera pointed at the ground, IMU, and wheel encoder, focusing on mining environments. Using a ground-facing camera is more robust to particulates in the air, like fog and dust, than techniques using horizon-facing sensors. We analyze five implementations for localization: (1) using wheel encoders, (2) a visual-only method, (3) using the IMU orientation and linear displacement by visual information, (4) obtained by merging wheel encoder and IMU data using Extended Kalman Filter (EKF), and (5) an EKF using visual, encoder, and IMU data. We perform tests in mining-like environments in simulation and field experiments. Simulations are implemented in CoppeliaSim software and make use of realistic textures. In the field experiments, we use a mobile robot equipped with a camera, IMU, and GNSS receiver with RTK correction, which we consider the robot's actual position (ground truth). Results show that the proposed methods are promising but need to become more accurate for use in heavy mining vehicles.