Off-Policy Learning

Learning only from current policy’s data.

Balancing learning new behaviors vs exploiting known rewards.

Continuous cycle of observation, reasoning, action, and feedback.

Models trained to decide when to call tools.

Expected return of taking action in a state.

Fundamental recursive relationship defining optimal value functions.

Optimizing policies directly via gradient ascent on expected reward.

Combines value estimation (critic) with policy learning (actor).