Pseudoinverse

A pseudoinverse is the reverse of a transformation.

Description

While only square matrices can be inverted, all matrices have a pseudoinverse. If a matrix describes a transformation from a row space into a column space (plus vectors in the null space being transformed into the zero vector), the its pseudoinverse describes the reverse mapping of columns space into row space (plus vectors in the left null space being transformed into the zero vector).

The pseudoinverse of A is notated A⁺.

It can be proven that every matrix has a unique pseudoinverse.

Pseudoinverses have applications in many of the same ways as true inverses. For example, given Ax = b, if there is a solution to b = AA⁺b (analogous to determining if b is in the row space of A), then x = A⁺b. Furthermore, if A⁺A = I (a mutation of I - A⁺A = 0, which is analogous to identifying the null space), then there is a unique solution given by x = A⁺b.

Properties

Pseudoinverses have the following multiplicative properties.

• AA⁺A = A

• A⁺AA⁺ = A⁺

• (AA⁺)^T = AA⁺

• (A⁺A)^T = A⁺A

Solution

Independent rows

In the case that all rows of A are independent, note that AA^T is symmetric and is always invertible.

Suppose then that for such a matrix A⁺ = A^T(AA^T)^-1. It can then be proven to satisfy all properties.

AA⁺A = AA^T(AA^T)^-1A

AA⁺A = (AA^T)(AA^T)^-1A

AA⁺A = A

therefore satisfying property 1.

A⁺AA⁺ = A^T(AA^T)^-1AA^T(AA^T)^-1

A⁺AA⁺ = A^T(AA^T)^-1(AA^T)(AA^T)^-1

A⁺AA⁺ = A^T(AA^T)^-1

which is A⁺, therefore satisfying property 2.

Note now that because AA^T, so is (AA^T)^-1. And furthermore because those are both symmetric, so it their product.

(AA⁺)^T = (AA^T(AA^T)^-1)^T

(AA⁺)^T = AA^T(AA^T)^-1

which is AA⁺, therefore satisfying property 3.

(A⁺A)^T = (A^T(AA^T)^-1A)^T

(A⁺A)^T = A(AA^T)^-1A^T

which is A⁺A, therefore satisfying property 4.

Independent columns

For similar reasons (i.e., when all columns of A are independent A^TA is symmetric), the pseudoinverse in this case is (A^TA)^-1A^T.

Limit Approach

The pseudoinverse of any matrix can be calculated as lim_{α -> 0} (A^TA + α²I)^-1A^T.

Singular Value Decomposition

Consider a m x n matrix A. Recall that the SVD of A is UΣV^T, where Σ is a m x n diagonal matrix with values of σ on the diagonal. Given this SVD, A⁺ = VΣ'U^T where Σ' is a n x m diagonal matrix with values of 1/σ on the diagonal.

CategoryRicottone