|
Size: 1629
Comment: More content
|
Size: 1655
Comment: Stability
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| '''Singular values''' are | '''Singular values''' are a generalization of [[LinearAlgebra/EigenvaluesAndEigenvectors|eigenvalues]]. |
| Line 13: | Line 13: |
| Any linear transformation can be rewritten as a rotation (i.e., from the original basis onto a convenient basis), a scaling and stretching, and another rotation (i.e., from the convenient basis onto the destination basis). Therefore '''''A''' = '''UΣV'''^T^''. | It is known that (for square matrices) there are [[LinearAlgebra/EigenvaluesAndEigenvectors|certain vectors]] which the transformation only scales: '''''A'''x = λx''. |
| Line 15: | Line 15: |
| Alternatively, it is known that (for square matrices) there are [[LinearAlgebra/EigenvaluesAndEigenvectors|certain vectors]] which the transformation only scales: '''''A'''x = λx''. For any matrix '''''A''''' of size ''m x n'', there are certain [[LinearAlgebra/Orthogonality|orthogonal]] [[LinearAlgebra/Basis|basis vectors]] ''u,,i,,'' in the column space that correspond to certain orthogonal basis vectors ''v,,i,,'' in the row space. They correspond such that '''''A'''v = σu''. And by composing these vectors and values together: '''''AV''' = '''UΣ'''''. |
Similarly, for any matrix '''''A''''' of size ''m x n'', there are certain vectors ''u,,i,,'' in the column space that correspond to certain vectors ''v,,i,,'' in the row space. They correspond such that '''''A'''v = σu''. And by composing these vectors and values together: '''''AV''' = '''UΣ'''''. |
| Line 25: | Line 23: |
| Singular values are the square roots of [[LinearAlgebra/EigenvaluesAndEigenvectors|eigenvalues]]. Specifically, a matrix '''''A''''' has singular values equal to the square roots of the eigenvalues of '''''A'''^T^'''A''''': ''σ = √λ''. | Singular values are square roots of [[LinearAlgebra/EigenvaluesAndEigenvectors|eigenvalues]]. Specifically, a matrix '''''A''''' has singular values equal to the square roots of the eigenvalues of '''''A'''^T^'''A''''': ''σ = √λ''. |
| Line 27: | Line 25: |
| Matrices of all sizes have singular values. For a matrix of size ''m x n'', there are at most ''min{m,n}'' singular values. Furthermore, the number of non-zero singular values is equal to the [[LinearAlgebra/Rank|rank]] of '''''A'''''. | Matrices of all sizes have singular values. The number of non-zero singular values is equal to the [[LinearAlgebra/Rank|rank]] of '''''A'''''. |
| Line 29: | Line 27: |
| These singular values are the numbers on the diagonal of '''''Σ'''''. Naturally '''''Σ''''' matches '''''A''''' as a ''m x n'' matrix. | Singular values are also more stable as compared to eigenvalues. Consider the matrix: {{{ ┌ ┐ │ 0 1 0 0│ │ 0 0 2 0│ │ 0 0 0 3│ │ 0 0 0 0│ └ ┘ }}} All eigenvalues are 0 and there is only one eigenvector: ''[1 0 0 0]''. By comparison, the singular values are 3, 2, 1, and 0. If the 0 in row 4, column 1 were changed to an extremely small degree, then 4 unique eigenpairs would suddenly emerge. The first three singular values would be unchanged however, and the fourth would become non-zero. |
Singular Values
Singular values are a generalization of eigenvalues.
Contents
Introduction
It is known that (for square matrices) there are certain vectors which the transformation only scales: Ax = λx.
Similarly, for any matrix A of size m x n, there are certain vectors ui in the column space that correspond to certain vectors vi in the row space. They correspond such that Av = σu. And by composing these vectors and values together: AV = UΣ.
Description
Singular values are square roots of eigenvalues. Specifically, a matrix A has singular values equal to the square roots of the eigenvalues of ATA: σ = √λ.
Matrices of all sizes have singular values. The number of non-zero singular values is equal to the rank of A.
Singular values are also more stable as compared to eigenvalues. Consider the matrix:
┌ ┐ │ 0 1 0 0│ │ 0 0 2 0│ │ 0 0 0 3│ │ 0 0 0 0│ └ ┘
All eigenvalues are 0 and there is only one eigenvector: [1 0 0 0]. By comparison, the singular values are 3, 2, 1, and 0. If the 0 in row 4, column 1 were changed to an extremely small degree, then 4 unique eigenpairs would suddenly emerge. The first three singular values would be unchanged however, and the fourth would become non-zero.