Modifiable Intuitive Robot Controller: Computer Vision-Based Controller for Various Robotic Designs

Abstract

This paper describes the Modifiable Intuitive Robot Controller (MIRC) framework, a vision-based remote controller for robotic systems. The MIRC system integrates a modern controller interface known as the Leap Motion, which uses multiple vision sensors to gather three-dimensional anatomical arm and hand data points in the form of numerical coordinates. The MIRC system publishes this data in the form of ROS pose messages and transforms, and converts these numerical coordinates into goals or motion commands for the robots. Complex robot designs, such as robotic arms, are controlled using MoveIt's motion planning algorithms and robot configuration package. By using MoveIt's library, the framework can accommodate an ever increasing number of complex robotic arm designs. Simpler rover based designs are controlled through a custom function that interprets the operator's hand position into speed and rotation for motion. The integrated system is demonstrated in both simulation and hardware using the Kinova MICO arm and TurtleBot 2 mobile robot.