Compute

Techniques that fine-tune small additional components rather than all weights to reduce compute and storage.

PEFT method injecting trainable low-rank matrices into layers, enabling efficient fine-tuning.

Reducing numeric precision of weights/activations to speed inference and reduce memory with acceptable accuracy loss.

Removing weights or neurons to shrink models and improve efficiency; can be structured or unstructured.

Central system to store model versions, metadata, approvals, and deployment state.

Ability to replicate results given same code/data; harder in distributed training and nondeterministic ops.

Tracking where data came from and how it was transformed; key for debugging and compliance.

Time from request to response; critical for real-time inference and UX.