Singular Value Decomposition

The singular value decomposition (SVD) is a decomposition that follows from the definition of singular values.

Description

Any linear transformation can be rewritten as a rotation (i.e., from the original basis onto a convenient basis), a scaling and stretching, and another rotation (i.e., from the convenient basis onto the destination basis). Therefore A = UΣV^T.

Solution

The definition of singular values (AV = UΣ) can be restated as A = UΣV^-1. If the certain vectors v_i are chosen to be orthogonal bases of the row space, then it is also true that A = UΣV^T.

Note then that A^TA = (UΣV^T)^TUΣV^T = VΣ^TU^TUΣV^T = VΣ^TΣV^T. Because Σ is a diagonal matrix with values of σ on the diagonal, Σ^TΣ evaluates to a diagonal matrix with values of σ² on the diagonal.

Effectively, this is a diagonalization of A^TA. V is the eigenbasis. Λ is equal to Σ^TΣ, therefore λ = σ².

With V and Σ known, U can be solved for. Alternatively, repeat the above method on AA^T. U will be that eigenbasis. The eigenvalues will be the same.

CategoryRicottone