Toward a Holistic Framework for Robotic Assessment: A Survey on Performance, Software, and Environmental Adaptability

Abstract

The evaluation of robotic systems has traditionally centered on task performance metrics such as accuracy, efficiency, and execution time. However, as robotics continues to expand across diverse domains, a broader and more structured assessment framework is necessary. This survey examines metrics from 113 research papers published between 2020 and 2024 to identify trends in robotic evaluation. The findings reveal a wide variety of metrics applied based on different research objectives, making standardization difficult. Despite this diversity, these metrics can generally be classified into task performance and non-task performance categories. Many studies prioritize task performance, often placing less emphasis on software contributions, which are frequently underrepresented in evaluations despite their integral role in control and decision-making. Additionally, robotic adaptability in harsh environments is often inconsistently assessed, limiting comparability across different studies. To address these challenges, this paper advocates for clearer frameworks that systematically integrate both hardware and software contributions. Furthermore, a standardized approach to evaluating robotic adaptability across various environmental conditions is recommended. Establishing structured methodologies will enhance comparability and lead to more precise assessments of robotic performance.