On scientific foundation models: Rigorous definitions, key applications, and a comprehensive survey

Abstract

Scientific Foundation Models (SciFMs) represent a transformative paradigm for addressing complex scientific and engineering problems by leveraging large-scale pretraining and deep learning architectures. Unlike traditional numerical solvers, which require problem-specific discretization and extensive parameter tuning, SciFMs aim to learn generalizable representations of physical laws, thereby enabling broad applicability with minimal retraining. In the absence of a rigorous definition, this work categorizes their capabilities into four key dimensions-domain adaptation, domain generalization, problem adaptation, and problem generalization; thereby providing rigorous definitions for SciFMs, which we refer to as the sufficient conditions. To operationalize these conditions in practice, we further introduce the necessary conditions. We also propose a taxonomy of SciFMs based on model architecture, learning objectives, and training strategies. Although SciFMs demonstrate remarkable potential in computational science and engineering, several challenges persist. Ensuring physical consistency, interpretability, and robustness under extrapolation to extreme regimes remains a critical hurdle. Furthermore, the significant computational demands and the lack of standardized benchmarks present substantial barriers to widespread adoption. This work surveys existing SciFMs across diverse domains-including chemistry, materials science, biology, climate and weather, Earth observation, geophysics, chaotic dynamics, robotics and control, nuclear science, etc., while categorizing them under the proposed definitions. We further outline open research directions, emphasizing the integration of domain knowledge with data- and physics-driven approaches and the development of efficient architectures to enhance generalization. Addressing these challenges will be critical for SciFMs to accelerate scientific discovery and deliver real-world impact. The GitHub repository for this work is available at https://github.com/ParamIntelligence/Awesome-Scientific-Foundation-Models.