Maximum Likelihood

Maximum likelihood is a method for fitting a distribution to observations.

Description

The most common probability distribution functions are parameterized so that shape, scale, etc., can vary to better fit a particular set of observations. A methodology is then needed to determine the values for those parameters with the 'best' fit.

For example, the probability density function for an exponential distribution with a rate of λ=λ* is:

The likelihood for a value of λ given an observation x_i is therefore given by:

Given many observations i in X, the probabilities are multiplied.

The method of maximum likelihood estimation relies on logarithmic differentiation and more generally on logarithm properties. The log of this product is taken, turning the expression into a summation that usually can be simplified, and that is still useful for optimization.

Let n be the number of observations in X so that:

The value of λ that maximizes likelihood is a critical point, i.e. identified by deriving the expression and setting to zero.

So the maximum likelihood estimator (MLE) for λ is λ_MLE = 1/X̅.

CategoryRicottone