Scaling Laws

Formal framework for sequential decision-making under uncertainty.

All possible configurations an agent may encounter.

Set of all actions available to the agent.

Strategy mapping states to actions.

Attention mechanisms that reduce quadratic complexity.

Routes inputs to subsets of parameters for scalable capacity.

Chooses which experts process each token.

Capabilities that appear only beyond certain model sizes.