Visual Target Recognition and Tracking for Autonomous Manipulation Tasks

Abstract

Increased levels of robotic system autonomy will enable scientific exploration in previously unreachable destinations. Dexterous manipulator systems can be coupled with machine vision systems to increase perceptive capabilities and provide greater opportunities for scientific return through environmental interaction. Reliable autonomous vision systems are often specific to a certain task and quickly become complex as new operational tasks are added. This paper describes the development of a flexible stereovision system coupled with the University of Maryland’s Ranger manipulator system to identify, track, and sample arbitrary targets within the manipulator’s workspace. The vision system is designed to be deployed on an autonomous underwater vehicle (AUV) that will conduct deep-sea sampling missions autonomously, where lighting and visibility constraints are formidable. Laboratory testing demonstrates the vision system’s ability to repeatedly and reliably select desired targets and provide target position data to the manipulator during autonomous sampling tasks. Visual servoing results demonstrate closed-loop tracking of static and moving targets. I. Introduction Reliable and capable autonomous manipulation systems are in great demand for exploration in harsh, inaccessible environments. Development of such systems will allow for greater scientific return on missions where ground support, communications, and operator workload are prohibitive in terms of cost and factors such as time delay or communication bandwidth constraints. Enhanced robotic perception of the environment is a key enabler to reduced human interaction. Tasks utilizing robotic manipulators are notorious for the strain placed on human operators, both mentally and physically. Lack of sufficient camera views during teleoperation, hand strain from long-term use of hand controllers, and mental stress associated with difficult teleoperation tasks are all challenges that can be mitigated through effective automation. The University of Maryland Space Systems Lab (SSL) has developed multiple dexterous robotic manipulator systems. Their long history of working with teleoperated systems provides a strong foundation for ongoing research in autonomous robotics. The Ranger telerobotic manipulator system has been utilized extensively in 1-G and neutral buoyancy environments to simulate on-orbit servicing of spacecraft and satellites, as shown in Figure 1. By visually characterizing the environment with accurate 3-D position data, this work augments the Ranger system to autonomously manipulate targets within its workspace. This extension will significantly reduce the workload placed on the operator while simultaneously reducing discrepancies caused by time delays, operator error, and fatigue.