Learning Path: Large Language Models

LLM definition and core capabilities

A large language model (LLM) is an AI model—typically a neural network—trained on large amounts of text to perform natural language processing tasks, especially language generation. LLMs can generate, summarize, translate, and analyze text across many contexts, and they are a foundational technology behind modern chatbots. Their outputs can be affected by biased or inaccurate training data, so reliability and safety are important considerations. LLMs’ core capabilities come from how they are trained and architected. They are commonly based on transformer architectures with attention mechanisms that let the model relate tokens across a context window, enabling effective next-token prediction (autoregressive) or missing-token prediction (masked models). Training usually involves pretraining (often next-word prediction) followed by post-training such as fine-tuning and instruction-following methods (e.g., RLHF), and models may also use extensions like mixture-of-experts to reduce inference cost. LLMs are evaluated using measures such as perplexity and task benchmarks that test reasoning, factual accuracy, alignment, and safety.

0/2 modes complete

Transformer architecture overview

0/2 modes complete