Loading your learn pack…
Read the summary, then start training!
Foundation models are trained on broad, large-scale data so they can be adapted to many downstream tasks, often across multiple modalities (text, images, audio, robotics).
A foundation model (FM), also called a large x model (LxM), is a machine learning or deep learning model trained on very large, broad datasets so it can be adapted to many different downstream tasks. Generative AI systems such as large language models (LLMs) are common examples, but foundation models also exist across modalities including images, music, and robotics. Because building them requires massive compute, sophisticated data pipelines, and advanced hardware (e.g., GPUs), training is extremely expensive, while adapting an existing foundation model to a specific task is typically much cheaper via fine-tuning or direct use. The term “foundation model” was coined in August 2021 by Stanford’s CRFM to describe models trained on broad data (often using self-supervision at scale) that can be adapted widely. The choice of “foundation” emphasizes their role as a reusable base rather than a set of fundamental principles. The concept emerged from advances in deep learning—especially self-supervised learning, transfer learning, and architectures like Transformers—along with greater parallel computing capability and the availability of large web-scale datasets. Public attention surged with major 2022 releases such as Stable Diffusion and ChatGPT (initially powered by GPT-3.5), and further momentum came from 2023 releases like LLaMA, Llama 2, and Mistral.
Foundation models are trained on broad, large-scale data so they can be adapted to many downstream tasks, often across multiple modalities (text, images, audio, robotics).
Training foundation models is resource-intensive and costly, while adaptation to specific tasks is comparatively cheaper using pre-trained capabilities and fine-tuning.
The term “foundation model” was coined in August 2021 by Stanford’s CRFM, emphasizing adaptability from broad self-supervised training rather than narrow “language model” framing.
Technological progress (Transformers, self-supervised learning, transfer learning, and scalable GPU training) in the late 2010s enabled the rise of foundation models, with widespread public discourse accelerated by 2022–2023 releases like ChatGPT, Stable Diffusion, and LLaMA.
A machine learning model trained on broad, large-scale data that can be adapted to a wide range of downstream tasks.
An alternative label for foundation models where “x” can represent different data types such as text, images, or sound.
A training approach that learns useful representations from unlabeled data by creating supervisory signals from the data itself.
A neural network architecture that has become the de facto choice for building foundation models across many modalities.
Stanford’s center that coined the term “foundation model” in August 2021.
“Can you explain what "Foundation models are trained on broad, large-scale data so they can be adapted to many downstream tasks, often across multiple modalities (text, images, audio, robotics)." means in simple terms?”