Simultaneous AMCW ToF Camera and FMCW Radar Simulation

Abstract

As two of the main sensors utilized in robotics, environment representation and navigation, ToF cameras and radars are often combined in sensor fusion frameworks. Realistic and complete simulation plays a crucial role in fast prototyping, performance assessment, and model analysis of systems relying on these sensing technologies. Typical simulation environments, however, don't model key characteristics, such as the radar sensor's time domain raw data, antenna arrays' structure, or parasitic effects of depth estimation of ToF cameras. This article presents a simultaneous ToF camera and radar simulation based on the Unity 3D engine. The two sensors are modeled from a single custom RGB camera object and are inherently time synchronized. The simulation provides ToF depth and intensity estimates, together with time domain radar data and realistic detrimental effects, allowing testing of standard post-processing algorithms. Several parameters can be configured, such as field of view, modulation characteristics, and the number and position of antennas. Moreover, simulation objects can have different material properties for the two sensors making them more or less visible for each sensor. The approach focuses on user-friendly, realistic sensor simulation while keeping computational complexity within acceptable limits to ensure applicability. The capabilities of the proposed simulator are validated by comparing real and simulated sensors in different scenarios.