Performance Evaluation of Sensors on Mobile Vehicles Using a Large Data Repository and Ground Truth

Abstract

Progress in algorithm development and transfer of results to practical applications such as military robotics requires standard qualitative and quantitative measurements for performance evaluation and validation. Although the evaluation and validation of algorithms have been discussed for well over a decade, the research community still faces a lack of well-defined and standardized methodology. In this research, we describe three methods for creating ground truth databases and benchmarks using multiple sensors, for use in mobile robotics. The databases and benchmarks provide researchers with high quality data from suites of sensors operating in complex environments representing real-world problems. At NIST, we have equipped a High Mobility Multi-purpose Wheeled Vehicle (HMMWV) with a suite of sensors including a Riegl ladar,