Outer Alignment

Ensuring learned behavior matches intended objective.

Model behaves well during training but not deployment.

Learned subsystem that optimizes its own objective.

Maintaining alignment under new conditions.

Model exploits poorly specified objectives.

Maximizing reward without fulfilling real goal.

Tendency for agents to pursue resources regardless of final goal.

Model optimizes objectives misaligned with human values.