Marine Snow Simulation and Elimination in Video

Abstract

Marine snow is formed by the aggregation of small particles of organic debris and inorganic sediments that settle through the water column and give the appearance of snow falling through the sky. Although critical to aquatic ecosystems, marine snow can potentially compromise underwater computer vision tasks. This works begins with a survey of techniques for overcoming such issues. Since many recent techniques rely on machine learning, which in turn relies on large amounts of training data, we also survey current approaches at generating such data. After discussing opportunities in existing approaches, we present novel solutions for simulation, isolation, and removal of marine snow in both images and video. Unlike other approaches, we focus on simulating marine snow in industrial scenarios with moving cameras and photorealistic discoloration, lens distortion, and motion blur typical in imagery collected by underwater robots. To remove the marine snow, we use only synthetic data, including a new dataset that we share online, to train a convolutional autoencoder extended to consider temporal information in consecutive video frames. Experiments are then conducted on two real datasets – one collected by an autonomous underwater vehicle in a shallow lake and the other by a remotely-operated vehicle in the deep sea.