Deep reinforcement learning based approach for the search and engagement phase of the robotic screw unfastening process

Abstract

Abstract Fasteners such as screws or nuts play an essential role in product design due to their non-permanent joint behavior. In this context, the search and engagement phase during the automated disassembly of fasteners is crucial since position errors due to the inaccuracy of a fully automated robot cell must be compensated. Additional inaccuracy appears in product-specific mounting situation fasteners, where computer vision cannot capture the situation. This might leads to a time-consuming spiral search. In this regard, a deep reinforcement learning (DRL) approach is proposed to solve the search and engagement task. A deep Q-network (DQN) agent has been trained by using external force–torque sensor values and the intrinsic motor currents of the robot itself. The training has been conducted virtually and continued on the robot to improve the success rate. To find the optimal DQN architecture, we have combined a multilayer perceptron with convolutional neural networks with long short-term memory as feature extractors. Our proposed DRL-based approach is 2 s faster to engage than a spiral search for an initial error of 3 mm.