Benchmark

Intermediate

A dataset + metric suite for comparing models; can be gamed or misaligned with real-world goals.

Full Definition

A dataset + metric suite for comparing models; can be gamed or misaligned with real-world goals.

Keywords

standardized evaluation

Domains

Evaluation & Benchmarking

Related Terms

Brier Score related to

A proper scoring rule measuring squared error of predicted probabilities for binary outcomes.

 Experiment Tracking related to

Logging hyperparameters, code versions, data snapshots, and results to reproduce and compare experiments.

 Observability related to

A broader capability to infer internal system state from telemetry, crucial for AI services and agents.

 Perplexity related to

Exponential of average negative log-likelihood; lower means better predictive fit, not necessarily better utility.

Concept Map

See how Benchmark connects to other concepts.

Open Knowledge Graph