Information Gain

Intermediate

Reduction in uncertainty achieved by observing a variable; used in decision trees and active learning.

Full Definition

Reduction in uncertainty achieved by observing a variable; used in decision trees and active learning.

Keywords

entropy reduction

Domains

AI Economics & Strategy

Related Terms

Entropy related to

A measure of randomness or uncertainty in a probability distribution.

 Cross-Entropy related to

Measures divergence between true and predicted probability distributions.

 KL Divergence related to

Measures how one probability distribution diverges from another.

 Mutual Information related to

Quantifies shared information between random variables.

 VC Dimension related to

A measure of a model class’s expressive capacity based on its ability to shatter datasets.

 Rademacher Complexity related to

Measures a model’s ability to fit random noise; used to bound generalization error.

 Bias Term related to

Systematic error introduced by simplifying assumptions in a learning algorithm.

 Variance Term related to

Error due to sensitivity to fluctuations in the training dataset.

Concept Map

See how Information Gain connects to other concepts.

Open Knowledge Graph