Singular Values

Singular values are a generalization of eigenvalues.

Introduction

It is known that (for square matrices) there are certain vectors which the transformation only scales: Ax = λx.

Similarly, for any matrix A of size m x n, there are certain vectors u_i in the column space that correspond to certain vectors v_i in the row space. They correspond such that Av = σu. And by composing these vectors and values together: AV = UΣ.

The size of Σ will always match A; that is, it will also be size m x n. V spans the column space of A, so it will be size n x n. U spans the row space, so it will be m x m.

Description

Singular values are square roots of eigenvalues. Specifically, a matrix A has singular values equal to the square roots of the eigenvalues of A^TA: σ = √λ.

Matrices of all sizes have singular values. The number of non-zero singular values is equal to the rank of A.

Singular values are also more stable as compared to eigenvalues. Consider the matrix:

┌        ┐
│ 0 1 0 0│
│ 0 0 2 0│
│ 0 0 0 3│
│ 0 0 0 0│
└        ┘

All eigenvalues are 0 and there is only one eigenvector: [1 0 0 0]. By comparison, the singular values are 3, 2, 1, and 0. If the 0 in row 4, column 1 were changed to an extremely small degree, then 4 unique eigenpairs would suddenly emerge. The first three singular values would be unchanged however, and the fourth would become non-zero.

CategoryRicottone