Augmented reality visualisation for mobile robot developers

Abstract

Developer interactions with robots during the testing and
\ndebugging phases of robot development are more complex than and distinct from general software
\napplication development. One of the primary differences is the need to understand the robot's view
\nof the environment and the inconsistencies between this and the actual environment. Augmented
\nreality (AR) provides an ideal way to achieve this, allowing robot program data to be displayed in
\ncontext with the real world. This allows for easy comparison by the developer, highlighting the
\ncause for any bugs in the robot behaviour. An AR debugging space is created in this work that
\nallows the developer to have this enhanced understanding of the robot's world-view, thus improving
\ndeveloper efficiency.
\n
\nOver the past decade robots have begun to move out of industrial assembly lines and
\ninto environments that must be shared with human users. Many of the tasks that we wish robots to
\nperform in these environments require close interaction and collaboration with human
\nusers. The move away from the constrained environment of a production line means that the tasks
\nrequired of robots are more varied, their operating environment is far more complex and
\nunpredictable, and safety can no longer be achieved through isolation of the robot. 
\n
\nThe result of these influences has been to change robot programming from a simple task of
\ninstructing the robot to perform a sequence of steps to an open ended challenge of specifying
\ndynamic interactions that
\nrobot developers are still coming to terms with. Robot development is
\nmore than just design and code entry and a broader approach to improving robot development is
\nneeded. One of the founding principles of this thesis is that robot development should be
\napproached as a human-robot interaction issue, this particularly applies to the testing and
\ndebugging phases of the development.
\n
\nThe nature of the robot platform, the tasks the robot is required to perform and the
\nenvironments that robots work within are significantly different from those of the desktop
\napplication. Hence robot developers need a tailored tool chain that focuses on this unique
\ncombination of issues. Current robot programming research is dominated by robot APIs and frameworks,
\nleaving support tools to be developed in an ad hoc manner by developers as features are
\nrequired. This leads to disjointed tools that have minimal feature sets; tools that generally have
\npoor portability when applied to other robot developments. This work examines the needs of the
\ndeveloper in terms of a general purpose robot visualisation tool.
\n
\nOne of the fundamental requirements of a general purpose robot visualisation tool is that a set of
\nstock visualisations must be available for the developer. A prerequisite to providing these is to
\nhave a set of standard interfaces to provide the visualisations for. The open source
\nrobot framework Player/Stage was used throughout this work to provide standardised access to robot
\nhardware. As part of this research the author has contributed heavily to the Player/Stage project,
\nparticularly as one of the key developers of the 2.0 release of Player. This new release simplifies
\nPlayer development and increases the ease of maintenance of Player drivers and the
\nefficiency of the server core.
\n
\nTo evaluate the benefits of AR visualisation an intelligent debugging space was
\ndeveloped, which runs as a permanent installation in a robotic development lab providing an enhanced
\nview of the robot's behaviour to the developer. The space is capable of automatically detecting the
\npresence of robots and displaying visualisations of the standard interfaces of the robot, such as
\nits sensors and effectors. The deb