Singular Values

Singular values are

Introduction

Any linear transformation can be rewritten as a rotation (i.e., from the original basis onto a convenient basis), a scaling and stretching, and another rotation (i.e., from the convenient basis onto the destination basis). Therefore A = UΣV^T.

Alternatively, it is known that (for square matrices) there are certain vectors which the transformation only scales: Ax = λx.

For any matrix A of size m x n, there are certain orthogonal basis vectors u_i in the column space that correspond to certain orthogonal basis vectors v_i in the row space. They correspond such that Av = σu. And by composing these vectors and values together: AV = UΣ.

Description

Singular values are the square roots of eigenvalues. Specifically, a matrix A has singular values equal to the square roots of the eigenvalues of A^TA: σ = √λ.

Matrices of all sizes have singular values. For a matrix of size m x n, there are at most min{m,n} singular values. Furthermore, the number of non-zero singular values is equal to the rank of A.

These singular values are the numbers on the diagonal of Σ. Naturally Σ matches A as a m x n matrix.

CategoryRicottone