A Quantitative Autonomy Quantification Framework for Fully Autonomous Robotic Systems

Abstract

Although autonomous functioning facilitates deployment of robotic systems in domains that admit limited human oversight on our planet and beyond, finding correspondence between task requirements and autonomous capability is still an open challenge. Consequently, a number of methods for quantifying autonomy have been proposed over the last three decades, but to our knowledge all these have no discernment of sub-mode features of variation of autonomy and some are based on metrics that violet the Goodhart's law. This paper focuses on the full autonomous mode and proposes a quantitative autonomy assessment framework based on task requirements. The framework starts by establishing robot task characteristics from which three autonomy metrics, namely requisite capability set, reliability and responsiveness are derived. These characteristics were founded on the realization that robots ultimately replace human skilled workers, from which a relationship between human job and robot task characteristics was established. Additionally, mathematical functions mapping metrics to autonomy as a two-part measure, namely of level and degree of autonomy are also presented. The distinction between level and degree of autonomy stemmed from the acknowledgment that autonomy is not just a question of existence, but also one of performance of requisite capability. The framework has been demonstrated on two case studies, namely autonomous vehicle at an on-road dynamic driving task and the DARPA subterranean challenge rules analysis. The framework provides not only a tool for quantifying autonomy, but also a regulatory interface and common language for autonomous systems developers and users. Its greatest feature is the ability to monitor system integrity when implemented online.