= Loss Function = A '''loss function''' is a [[Analysis/Functions|mapping]] that assigns a real valued 'loss' or 'cost' to a system of equations. <> ---- == Description == Given a prediction ''ŷ'', the observable case-level difference between that and the true value is called a '''residual''': ''y - ŷ''. A function can be designed to take an entire system of equations, i.e. a set of observations and a model, and produce a score that represents 'loss' or 'cost' that should be minimized. Such a function is known either as a loss function or a cost function. A loss function is conventionally notated as 𝓁 but for simplicity ''L'' will be used on this page. === Examples === The most obvious forms of loss functions are '''absolute loss''' (i.e., ''L(p,q) = |p-q|'') and '''squared loss''' (i.e., ''L(p,q) = (p-q)^2^ = (p-q)^T^(p-q)''; sometimes called squared error loss or SEL). When applied to probabilities (i.e., predictions range from 0 to 1), squared loss is also called a Brier score. Squared loss divided by the number of observations is '''mean squared error'''. '''Log loss''' is appropriate for classifications: ''L(p,q) = -p log(q) - (1-p) log(1-q)''. Note that it is equivalent to [[Statistics/Entropy#Cross-entropy|cross entropy]]. ---- CategoryRicottone