Projection

A projection is an approximation within a column space. A projection matrix describes the linear transformation of the approximation.

See also vector projection.

Vectors

Given vectors a and b, b can be projected into the column space of a (i.e., C(a)). A projection matrix describes the linear transformation from vector a to projection vector p. Such a projection matrix P satisfies p = Pa.

The column space of the projection matrix (i.e., C(P)) is equivalent to C(b). It follows that P is also of rank 1.

Properties

Projection matrices are symmetric (i.e., P^T = P) and idempotent (i.e., P² = P).

Matrices

Given vector b and matrix A, b can be projected into the column space of A (i.e., C(A)).

Given a linear system as Ax = b, if b is in C(A), there are solutions for x. If b is not in C(A) however, there is no possible solution. The best approximation is expressed as Ax̂ = p where projection p estimates b with an error term e. This should sound familiar.

The error term can be characterized by e = b - p or e = b - Ax̂. e is orthogonal to R(A), the row space of A; equivalently it is orthogonal to C(A^T). Orthogonality in this context means that e is in the null space, so A^T(b - Ax̂) = 0.

The system of normal equations is A^TAx̂ = A^Tb. This simplifies to x̂ = (A^TA)^-1A^Tb. Altogether, the projection is characterized by p = A(A^TA)^-1A^Tb.

The projection matrix P satisfies p = Pb. It is calculated as P = A(A^TA)^-1A^T.

b can also be projected onto e, which geometrically means projecting into the null space of A^T. Algebraically, if one projection matrix has been computed as P, then the projection matrix for going the other way is (I - P)b.

Properties

As above, the projection matrix P is symmetric and idempotent.

If A is square, the above equations simplify rapidly.

If b is in C(A), then P = I. Conversely, if b is orthogonal to C(A), then Pb = 0 and b = e.

CategoryRicottone