A methodology for testing unmanned vehicle behavior and autonomy

摘要

This paper discusses approaches developed at the Idaho National Laboratory (INL) for quantifying and analyzing the performance of human-robot teams across different domains. These methods reflect experience and insights gained from previous INL experiments that have focused on landmine detection and marking; mapping and localization for robot positioning, mobile manipulation for explosive ordinance disposal (EOD), radiation characterization, and urban search and rescue operations. An overarching goal of this work has been to enhance our understanding of how the robot, the control and display interface, the task context, and the human contribute to or hinder mission success. Our approach to performance measurement was developed in concert with the iterative design cycle of our intelligent robotic control system, the robot intelligence kernel (RIK). In extending and refining the RIK for various applications, three factors key to holistic human-robot performance assessment were identified: comprehensive planning; the inclusion of end users in the design and performance evaluation phases of the study; and combining automated data collection with subjective measures. The paper discusses lessons learned in developing and applying performance metrics and provides a brief overview of measures that we are currently using to support the assessment of autonomy. In particular, the paper emphasizes the application of these metrics to behaviors for complex and potentially dangerous missions.