Forecasting in the presence of scale-free noise

Abstract

The extraction of signals from noise is a common problem in all areas of science and engineering. A particularly useful version is that of forecasting: determining a causal filter that estimates a future value of a hidden process from past observations. Current techniques for deriving the filter require that the noise be well described by rational power spectra. However, scale-free noises, whose spectra scale as a non-integer power of frequency, are ubiquitous in practice. We establish a method, together with performance guarantees, that solves the forecasting problem in the presence of scale-free noise. Via the duality between estimation and control, our technique can be used to design control for distributed systems. These results will have wide-ranging applications in neuroscience, finance, fluid dynamics, and quantum measurements.