Evaluating situation awareness of autonomous systems

Abstract

Autonomous systems proved to be very successful in specialized problem domains. But their perception, reasoning, planning and behavior capabilities are generally designed to fit special purposes. For instance, a robotic agent perceives its environment in a way that was defined in advance by a human designer. The agent does not exhibit a certain perception behavior because it actually thinks it would be reasonable to do so. But with an increasing level of autonomy as well as a larger temporal and spatial scope of agent operation higher-level situation analysis and assessment become essential. This paper examines criteria for evaluating situation-awareness of autonomous systems and proposes methods to satisfy them. An example application scenario is presented that provides initial results for evaluating situation-aware systems.