Potsdam data set of eye movement on natural scenes (DAEMONS)

Abstract

In recent years, model-based simulation of eye-movement behavior has become a more and more powerful scientific tool to understanding visual cognition (Kümmerer and Bethge, 2023). Predicting eye movements requires understanding low-level vision properties, high-level cognitive aspects and taking image content into account. For this reason, the task of eye movement prediction has received a lot of attention from diverse fields such as vision science, cognitive science, and computer vision, as well as applied fields such as foveated rendering, compression, robotics, design. As eye-movement models increase in complexity, the role of high-quality, openly available data sets becomes increasingly crucial in advancing the field. Corpora, i.e. general-purpose data sets, serve as the foundation upon which researchers can train, test and validate their models. A powerful method that is particularly popular in the field of computer vision is rigorous benchmarking on commonly accepted data sets. The fair and statistically rigorous comparison of models is crucial in order to ascertain the behavioral relevance of hypothesized effects and mechanisms, to understand the advantages and shortcomings of different approaches, and to rank models with respect to their success in explaining behavior.The different research traditions contributing to the field of modeling eye movement each prioritize different ideas, methods, specific interests and goals. Nonetheless the cross-pollination between the fields may be an important asset to further our understanding of human vision. One important difference between the various modeling approaches concerns the data requirements. In order to achieve the best possible standard, different models require different properties from a data set. As an example, we can compare two computational models of scan path generation: the Deep Neural Network model DeepGaze III (Kümmerer et al., 2019), and the dynamical, biologically-inspired model SceneWalk (Engbert et al., 2015;Schwetlick et al., 2020Schwetlick et al., , 2022)). DeepGaze III requires large amounts of data with many different image examples in order to fit many thousands of model parameters. SceneWalk, by contrast, has only a handful of parameters and needs comparatively much less data to fit. It does, however, rely on long sequences (i.e., long presentation durations) in order to fit sequence effects accurately.The MIT/Tuebingen Saliency Benchmark (Kümmerer et al., 2018) is the commonly agreed-upon benchmark for free viewing saliency prediction on static images since 2013, when it was established as the MIT Saliency Benchmark (Judd et al., 2012). This benchmark scores the performance of fixation density prediction models (often denoted as "saliency models", even though the term saliency originally only referred to low level features) on two different data sets with a public training part and a hold-out test part, to avoid overfitting to the test set.Recently, there has been an increased interest in modeling not just the spatial distribution of fixations but also entire fixation sequences (scan paths), introducing sequence effects and temporal effects. While in principle the same data set could be used for both static and dynamic fixation prediction, in practice the requirements can differ. Some existing data sets use very short presentation times, resulting in relatively short scan paths with little information on sequential effects. In other public data sets, sequence and temporal data was not published initially and was lost. The issue is compounded by the fact that, even when data is published, preprocessing details are not consistently reported. This lack of transparency raises concerns about data quality, hindering retrospective addressing of the issue.In this study we address the above shortcomings of data sets and collect and openly publish a scene viewing corpus data set which is useful for many different modeling approaches, including machine learnin