Multitask Learning 1997–2024: Part I Fundamentals

Abstract

In this three-part survey, we review the literature on multitask learning (MTL) from its inception in the 1990s to the present in 2024. Unlike single-task learning (STL), MTL is a learning paradigm that simultaneously learns multiple related tasks by leveraging both task-specific and shared information. MTL offers a range of benefits, including streamlined model architectures, improved performance, and enhanced generalizability across domains. Over the past 2 decades, it has gained widespread recognition as a flexible and powerful approach across diverse fields such as computer vision, natural language processing, recommender systems, disease prognosis and diagnosis, and robotics. This Part I lays the foundation by presenting a unified formalization of MTL, setting the stage for a deeper exploration of MTL methodologies in Parts II and III. Part II focuses on the technical aspects of MTL, detailing regularization and optimization methods that are essential for managing the complexities and trade-offs involved in learning multiple tasks. Part III bridges theoretical concepts of MTL and practical application, showcasing the application of MTL in real-world scenarios through the development of deep learning models. Together, these three parts aim to provide a comprehensive and accessible overview of the key developments, methodologies, and applications of MTL during 1997–2024. This project is publicly available at https://github.com/junfish/Awesome-Multitask-Learning <https://github.com/junfish/Awesome-Multitask-Learning> .“Multi-Task Learning 1997–2024” is a three-part article. Part II, “Regularization and Optimization” can be read here <https://doi.org/10.1162/99608f92.03dd064a> . Part III, “Applications” can be read here <https://doi.org/10.1162/99608f92.ca91645f> .