Loading your learn pack…
Read the summary, then start training!
RL is framed as learning a policy that maximizes expected cumulative (often discounted) reward from interaction with an environment.
Reinforcement learning (RL) problem framing models an agent interacting with a dynamic environment over discrete time steps. At each step, the agent observes a state, selects an action, and receives a reward based on the transition to a next state. The objective is to learn a policy (a rule for choosing actions given states) that maximizes expected cumulative reward, typically expressed as discounted return. A common formalization is the Markov decision process (MDP), which assumes the next state depends only on the current state and action (full observability). If the agent cannot directly observe the true state or observations are noisy, the problem is framed as a partially observable MDP (POMDP).
RL is framed as learning a policy that maximizes expected cumulative (often discounted) reward from interaction with an environment.
The standard formal model is the MDP; partial observability leads to POMDP framing.
A central practical issue is balancing exploration (trying actions to learn) and exploitation (using current knowledge), known as the exploration–exploitation dilemma.
Optimality is defined via value functions (state-value and action-value), which guide how to choose actions to maximize long-term return.
A policy is a mapping from states to actions (or action probabilities) that the agent follows to decide what to do next.
An MDP is a formal model of RL where transitions and rewards depend on the current state and action, satisfying the Markov property.
The discounted return is the sum of future rewards weighted by a discount factor, emphasizing near-term rewards more than distant ones.
Vπ(s) is the expected discounted return starting from state s and following policy π thereafter.
Qπ(s,a) is the expected discounted return starting from state s, taking action a, and then following policy π.
The trade-off between trying new actions to gather information (exploration) and choosing the best-known actions to gain reward (exploitation).
A setting where the agent can directly observe the true environment state, allowing MDP framing.
A setting where the agent observes only incomplete or noisy information about the environment, requiring POMDP framing.
“Can you explain what "RL is framed as learning a policy that maximizes expected cumulative (often discounted) reward from interaction with an environment." means in simple terms?”