Gradient

A gradient is a vector of partial derivatives. It describes the direction of steepest ascent for a differentiable function.

Notation

The gradient of function f is notated as ∇f. In terms of partial derivatives, the gradient of f(x₁, x₂, ... x_n) is:

At a given point p, as long as the function f is differentiable at p, the gradient vector is:

Note the assumption; it is not negligible. For example, (xy)/(x² + y²) is partially derivable but is itself not totally derivable at point p = [0 0]. Furthermore, it is not derivable if rotated; the basis must be orthonormal.

Also note that, technically, this is the transpose of the gradient; vectors are by default columns.

Relation to Jacobian Matrices

A Jacobian matrix is a square matrix of partial derivatives. Consider a vector-valued function, such as a parameterized function, but for simplicity let F(X) = <f₁(X), f₂(X)> where X is a vector as X = <x₁, x₂, ... x_n>. A gradient can be calculated for each of the component function.

Usage

By setting a gradient to 0, critical points (local minima, local maxima, and inflections) can be calculated.

More generally, gradient descent can be used to estimate minima.

CategoryRicottone