A Comparison of Multi-Object Sub-Pattern Linear Assignment Metrics

Abstract

Error estimation in multi-target tracking and robotic mapping is critical for algorithmic evaluation. In multi-target tracking and robotic mapping, both target or feature detection, as well as localization errors often exist. For this reason, recently, the multi-object Optimal Sub-Pattern Assignment (OSPA) and Cardinalized Optimal Linear Assignment (COLA) metrics were introduced, based on the Wasserstein construct. These metrics have the ability to compare set-based multi-target or multi-feature estimates and penalize both detection as well as state (spatial) errors. After the OSPA and COLA metrics were introduced, a variant referred to as the Generalized Optimal Sub-Pattern Assignment (GOSPA) metric was introduced, based on the claim that it is a generalization of the OSPA metric. This article compares the OSPA, COLA and GOSPA metrics in terms of their abilities to penalize certain multi-target/multi-feature scenarios. The article will determine when these metrics can disagree with one another when evaluating multi-target/multi-feature set-based errors and will evaluate their sensitivities to their characteristic “cut-off” parameters. As a result, their intuitive penalizations of various multi-target/multi-feature configurations will be demonstrated to provide a more complete and nuanced understanding of their applicability and performance in different contexts.