Foundation Models Learning Path

Definition and origin of foundation models

A foundation model (FM), also called a large x model (LxM), is a machine learning or deep learning model trained on very large, broad datasets so it can be adapted to many different downstream tasks. Generative AI systems such as large language models (LLMs) are common examples, but foundation models also exist across modalities including images, music, and robotics. Because building them requires massive compute, sophisticated data pipelines, and advanced hardware (e.g., GPUs), training is extremely expensive, while adapting an existing foundation model to a specific task is typically much cheaper via fine-tuning or direct use. The term “foundation model” was coined in August 2021 by Stanford’s CRFM to describe models trained on broad data (often using self-supervision at scale) that can be adapted widely. The choice of “foundation” emphasizes their role as a reusable base rather than a set of fundamental principles. The concept emerged from advances in deep learning—especially self-supervised learning, transfer learning, and architectures like Transformers—along with greater parallel computing capability and the availability of large web-scale datasets. Public attention surged with major 2022 releases such as Stable Diffusion and ChatGPT (initially powered by GPT-3.5), and further momentum came from 2023 releases like LLaMA, Llama 2, and Mistral.

0/2 modes complete

Foundation models versus related terminology

0/2 modes complete