Neighbourhood Context Embeddings in Deep Inverse Reinforcement Learning for Predicting Pedestrian Motion Over Long Time Horizons

Abstract

Predicting crowd behaviour in the distant future has increased in prominence among the computer vision community as it provides intelligence and flexibility for autonomous systems, enabling the early detection of abnormal events and better and more natural interactions between humans and autonomous systems such as driverless vehicles and field robots. Despite the fact that Deep Inverse Reinforcement Learning (D-IRL) based modelling paradigms offer flexibility and robustness when anticipating human behaviour across long time horizons, compared to their supervised learning counterparts, no existing state-of-the-art D-IRL methods consider path planning in situations where there are multiple moving pedestrians in the environment. To address this, we present a novel recurrent neural network based method for embedding pedestrian dynamics in a D-IRL setting, where there are multiple moving agents. We propose to capture the motion of the pedestrian of interest as well as the motion of other pedestrians in the neighbourhood through Long-Short-Term Memory networks. The neighbourhood dynamics are encoded into a feature map, preserving the spatial integrity of the observed trajectories. Utilising the maximum-entropy based non-linear inverse reinforcement learning framework, we map these features to a reward map. We perform extensive evaluations on the publicly available Stanford Drone and SAIVT Multi-Spectral Trajectory datasets where the proposed method exhibits robustness towards lengthier predictions into the distant future, demonstrating the importance of capturing the dynamic evolution of the environment using the proposed embedding scheme.