Reinforcement Learning Foundations and Algorithms

Reinforcement learning objective and reward maximization

In reinforcement learning, the objective is to learn a policy that maximizes an accumulated reward signal over time. The agent interacts with an environment modeled as a Markov decision process: at each step it observes a state, selects an action according to a policy, receives an immediate reward from the transition, and the environment moves to a new state. Because immediate rewards may be negative, the agent must reason about long-term consequences, creating a long-term vs. short-term reward trade-off. This reward maximization is formalized using value functions. The state-value function Vπ(s) estimates the expected discounted return starting from state s and following policy π, where future rewards are weighted by a discount factor γ. More generally, the optimality criterion uses the maximum expected discounted return across policies, leading to optimal value functions V*(s) and action-value functions Q*(s,a). Knowing the optimal action-value function is sufficient to act optimally by choosing, in each state, the action with the highest Q-value. Value-function methods and direct policy search are two broad approaches to optimize these objectives.

0/2 modes complete

Exploration versus exploitation dilemma

0/2 modes complete