|
Size: 2664
Comment: Moved Euclidean distance elsewhere
|
← Revision 8 as of 2025-03-28 03:10:35 ⇥
Size: 2755
Comment: Rephrased everything
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 15: | Line 15: |
| The concept of '''orthogonality''' is a generalization of '''perpendicularity''' in 2-dimensional space. | The concept of '''orthogonality''' is a generalization of '''perpendicularity''' from 2-dimensional space. |
| Line 17: | Line 17: |
| Two vectors are proven to be orthogonal if they obey the [[LinearAlgebra/Distance#n_Dimensions|Pythagorean theorem]]. In the case of two vectors, this test simplifies to ''x^T^y = 0''. | Put simply, vectors ''a'' and ''b'' are proven to be orthogonal if their [[LinearAlgebra/VectorMultiplication#Dot_Product|dot product]] is 0. |
| Line 19: | Line 19: |
| ''x^T^x + y^T^y = (x+y)^T^(x+y)'' expands to ''x^T^x + y^T^y = x^T^x + y^T^y + x^T^y + y^T^x'', simplifies to ''0 = x^T^y + y^T^x'', is trivially proven to be equivalent to ''0 = 2x^T^y'', and finally simplifies to ''0 = x^T^y''. | More precisely: assuming orthogonality, vectors ''a'' and ''b'' will satisfy the Pythagorean theorem. The hypotenuse is [[LinearAlgebra/Distance|Euclidean distance]]: ''(a+b)^T^(a+b)''. Simplifying from there: ''a^T^a + b^T^b = (a+b)^T^(a+b)'' ''a^T^a + b^T^b = a^T^a + b^T^b + a^T^b + b^T^a'' ''0 = a^T^b + b^T^a'' ''0 = 2(a^T^b)'' ''0 = a^T^b'' |
Orthogonality
Orthogonality is an important property for relating two vectors, or two subspaces, or a vector and a plane. The math notation is ⊥, as in x ⊥ y.
Orthonormality is an expanded concept, requiring that the components be unit vectors.
Vectors
The concept of orthogonality is a generalization of perpendicularity from 2-dimensional space.
Put simply, vectors a and b are proven to be orthogonal if their dot product is 0.
More precisely: assuming orthogonality, vectors a and b will satisfy the Pythagorean theorem. The hypotenuse is Euclidean distance: (a+b)T(a+b). Simplifying from there:
aTa + bTb = (a+b)T(a+b)
aTa + bTb = aTa + bTb + aTb + bTa
0 = aTb + bTa
0 = 2(aTb)
0 = aTb
Subspaces
For a subspace S to be orthogonal to a subspace T, every vector in S must be orthogonal to every vector in T.
Vectors and Planes
The null space of a matrix A contains the vectors that are not in the row space. These vectors cancel out; they are not a linear combination of the rows; if the row space is a plane, then these vectors are not on that plane.
The null space of A (a.k.a. N(A)) is orthogonal to the row space of A (a.k.a. R(A)). The null space of AT (a.k.a. N(AT)) is orthogonal to the column space of A (a.k.a. C(A)). Commonly this means that the row and column spaces are planes while the null spaces of A and AT are vectors, but that isn't always true.
Matrices
If a matrix is composed of orthonormal columns, then it is a matrix with orthonormal columns. These are usually denoted as Q. This has an important property: QTQ = I.
The projection of A if A is a matrix with orthonormal columns simplifies from P = A(ATA-1)AT into P = QQT. Correspondingly, the system of normal equations simplifies from ATAx̂ = ATb into x̂ = QTb.
If a matrix with orthonormal columns is also square, only then can it be called an orthogonal matrix. This has an additional important property: QT = Q-1.
For example, if Q is square, then the projection matrix further simplifies to P = I.