Cognitive Evaluation of Human-Robot Systems: A Method for Analyzing Cognitive Change in Human-Robot Systems

Abstract

To help answer questions about the behavior of participants in human-robot systems, we propose the cognitive evaluation of human-robot systems (CEHRS) method based on our work with the personal exploration rover (PER). The CEHRS method consists of six steps: (1) identify all system participants, (2) collect data from all participant groups, including the system's creators, (3) analyze participant data in light of system-wide goals, (4) answer targeted questions about each participant group to determine the flow of knowledge, information, and influence throughout the system, (5) look for inconsistencies in the knowledge and beliefs of different participant groups, and (6) make recommendations for improvement. We offer this comprehensive, human-centered evaluation method as a starting point for future work in understanding cognitive change in human-robot interactions