|
Size: 1201
Comment: Fix link
|
← Revision 3 as of 2026-08-13 15:58:06 ⇥
Size: 1339
Comment: Notes
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 17: | Line 17: |
| The most obvious forms of loss functions are absolute loss (i.e., ''L(p,q) = |p-q|'') and squared loss (i.e., ''L(p,q) = (p-q)^2^''; sometimes called squared error loss or SEL). When applied to probabilities (i.e., predictions range from 0 to 1), squared loss is also called a Brier score. | |
| Line 19: | Line 18: |
| Log loss is appropriate for classifications: ''L(p,q) = -p log(q) - (1-p) log(1-q)''. Note that it is equivalent to [[Statistics/Entropy#Cross-entropy|cross entropy]]. | === Examples === The most obvious forms of loss functions are '''absolute loss''' (i.e., ''L(p,q) = |p-q|'') and '''squared loss''' (i.e., ''L(p,q) = (p-q)^2^ = (p-q)^T^(p-q)''; sometimes called squared error loss or SEL). When applied to probabilities (i.e., predictions range from 0 to 1), squared loss is also called a Brier score. Squared loss divided by the number of observations is '''mean squared error'''. '''Log loss''' is appropriate for classifications: ''L(p,q) = -p log(q) - (1-p) log(1-q)''. Note that it is equivalent to [[Statistics/Entropy#Cross-entropy|cross entropy]]. |
Loss Function
A loss function is a mapping that assigns a real valued 'loss' or 'cost' to a system of equations.
Contents
Description
Given a prediction ŷ, the observable case-level difference between that and the true value is called a residual: y - ŷ. A function can be designed to take an entire system of equations, i.e. a set of observations and a model, and produce a score that represents 'loss' or 'cost' that should be minimized. Such a function is known either as a loss function or a cost function.
A loss function is conventionally notated as 𝓁 but for simplicity L will be used on this page.
Examples
The most obvious forms of loss functions are absolute loss (i.e., L(p,q) = |p-q|) and squared loss (i.e., L(p,q) = (p-q)2 = (p-q)T(p-q); sometimes called squared error loss or SEL). When applied to probabilities (i.e., predictions range from 0 to 1), squared loss is also called a Brier score. Squared loss divided by the number of observations is mean squared error.
Log loss is appropriate for classifications: L(p,q) = -p log(q) - (1-p) log(1-q). Note that it is equivalent to cross entropy.