Emergent Abilities

Empirical laws linking model size, data, compute to performance.

Formal framework for sequential decision-making under uncertainty.

All possible configurations an agent may encounter.

Set of all actions available to the agent.

Techniques to handle longer documents without quadratic cost.

Attention mechanisms that reduce quadratic complexity.

Routes inputs to subsets of parameters for scalable capacity.

Chooses which experts process each token.