Glass-to-Glass Delay Reduction: Encoding Rate Reduction vs. Video Frame Extrapolation

Abstract

Applications such as teleoperated driving, remote robot control, and telepresence rely on video services to ensure real-time interaction with a satisfying quality of experience. Reducing the Glass-to-Glass (G2G) delay, i.e., the time delay between the acquisition of a video frame and its display on a remote terminal is critical for these applications. Deep learning-based video frame extrapolation before video encoding has been recently considered as an interesting solution to reduce G2G delay, however, the latency introduced by extrapolation has not been taken into account. In this paper, considering the main sources of latency, including extrapolation delay, we examine the benefits and limitations of frame extrapolation at encoder in reducing the G2G delay in a point-to-point video transmission system. To this end, we compare the latency-quality trade-off for two latency compensation methods: encoding rate reduction and video frame extrapolation. Our aim is to determine the G2G delay reduction that may be achieved at the price of a given quality reduction. Our experiments show that extrapolation methods can provide a null perceived G2G delay with an acceptable loss in quality, particularly for applications with video contents with limited temporal information. Such delay reduction is unreachable via encoding rate reduction.