= Entropy = '''Entropy''' quantifies uncertainty of a random variable. <> ---- == Description == Entropy is a measure of the information needed to encode an event drawn from a random distribution. === Cross-entropy === Cross-entropy is a comparative measure between two distributions. Generally it is framed as the information needed to encode an event with reference to a distribution ''q'' which differs from the true distribution ''p''. Cross-entropy can easily be implemented as a loss function for estimation. This provides a measure for how well a model classifies observations, as compared to true labels. In the field of [[Statistics/DecisionTrees#Classification_Trees|classification trees]], this is referred to as either '''cross-entropy loss''' or '''log loss'''. ---- CategoryRicottone