Inference Cost

Intermediate

Cost to run models in production.

Full Definition

Cost to run models in production.

Keywords

tokens/sec

Domains

AI Economics & Strategy

Related Terms

Training Cost related to

Cost of model training.

 Model Moat related to

Competitive advantage from proprietary models/data.

 Open-Weight Model related to

Models whose weights are publicly available.

 Closed Model related to

Models accessible only via service APIs.

 Canary Tokens related to

Detecting unauthorized model outputs or data leaks.

 Compute Scaling related to

Increasing model capacity via compute.

 Data Scaling related to

Increasing performance via more data.

 Chinchilla Scaling related to

Scaling law optimizing compute vs data.

Concept Map

See how Inference Cost connects to other concepts.

Open Knowledge Graph