|
Size: 3391
Comment: Matrices
|
← Revision 16 as of 2026-02-04 00:30:26 ⇥
Size: 3051
Comment: Cleanup
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| '''Orthogonality''' is an important property for relating two vectors, or two subspaces, or a vector and a plane. | '''Orthogonality''' is a generalization of perpendicularity. '''Orthonormality''' is a related concept, requiring that the components be [[Calculus/UnitVector|unit vectors]]. |
| Line 5: | Line 5: |
| '''Orthonormality''' is an expanded concept, requiring that the components be unit vectors. | See also [[Calculus/Orthogonality|vector orthogonality]]. |
| Line 13: | Line 13: |
| == Vectors == | == Description == |
| Line 15: | Line 15: |
| The concept of '''orthogonality''' is a generalization of '''perpendicularity''' in 2-dimensional space. | '''Orthogonality''' is an extension of perpendicularity to higher dimensions of Euclidean space, and also to arbitrary [[LinearAlgebra/InnerProduct|inner product spaces]]. To notate that ''x'' is orthogonal to ''y'', use ''x ⊥ y''. |
| Line 17: | Line 17: |
| The Pythagorean theorem specifies that the sides of a right triangle are characterized by ''x^2^ + y^2^ = z^2^''. | '''Orthonormality''' is a further constraint: the orthogonal vectors are also [[Calculus/UnitVector|unit vectors]]. |
| Line 19: | Line 19: |
| For a vector ''x'', the total length can be thought of as the sum of each components' absolute value. If ''x'' is ''[1 2 3]'', the length is 6. The squared length can be thought of as the sum of each components' square. For the same ''x'', this is 14. This can be generalized as ''x^T^x''. For a similar reason, the total length of the hypotenuse can be thought of as the sum of the other two vectors: ''x+y''. Continuing with the example for ''x'', if ''y'' were ''[2 -1 0]'', then ''z'' would be ''[3 1 3]''. Note that the squared length can be written as ''(x+y)^T^(x+y)''. If the vectors ''x'' and ''y'' are perpendicular then the Pythagorean theorem should hold: ''x^T^x + y^T^y = (x+y)^T^(x+y)''. This expands to ''x^T^x + y^T^y = x^T^x + y^T^y + x^T^y + y^T^x''. By cancelling out common terms, this simplifies to ''0 = x^T^y + y^T^x''. It must be understood that the last two terms are the same value. Therefore, this further simplifies to ''0 = 2x^T^y'' and finally to ''0 = x^T^y''. The test for orthogonality of two vectors is ''x^T^y = 0''. ---- |
For a two vectors (which are ''not'' the zero vector) to be orthogonal, their inner product should be equal to 0. For a set of vectors to be orthogonal, every possible pair from the set should be orthogonal. |
| Line 33: | Line 23: |
| == Subspaces == | === Matrices === |
| Line 35: | Line 25: |
| For a subspace S to be orthogonal to a subspace T, every vector in S must be orthogonal to every vector in T. | A matrix is effectively a set of vectors. If the columns are orthonormal, then it can be called a '''matrix with orthonormal columns'''. These are usually denoted as '''''Q'''''. |
| Line 37: | Line 27: |
| ---- | Matrices with orthonormal columns have several important properties: * '''''Q'''^T^'''Q''' = '''I''''' * The [[LinearAlgebra/Projection|projection matrix]] is given as '''''P''' = '''QQ'''^T^''. The second follows from the first. Recall that, when projecting ''b'' into ''C('''A''')'', the projection matrix is given as '''''P''' = '''A'''('''A'''^T^'''A'''^-1^)'''A'''^T^''. This comes from the linear system '''''A'''^T^'''A'''x̂ = '''A'''^T^b'' and requiring that ''p = '''P'''b''. For a matrix '''''Q''''' with orthonormal columns, the first property simplifies the linear system to ''x̂ = '''Q'''^T^b''. Therefore, '''''P''' = '''QQ'''^T^''. If and ''only'' if such a matrix is square, it can be called an '''orthogonal matrix'''. These have several further properties: * '''''Q'''^T^'''Q''' = '''QQ'''^T^ = '''I''''' * '''''Q'''^T^ = '''Q'''^-1^'' * The [[LinearAlgebra/Determinant|determinant]] is always 1 or -1 * The projection matrix is given as '''''P''' = '''I''''', indicating that ''b'' must be in ''C('''A''')''. |
| Line 41: | Line 42: |
| == Vectors and Planes == | === Subspaces === |
| Line 43: | Line 44: |
| The [[LinearAlgebra/NullSpaces|null space]] of a matrix '''''A''''' contains the vectors that are not in the row space. These vectors cancel out; they are not a linear combination of the rows; if the row space is a plane, then these vectors are not on that plane. | For two subspaces to be orthogonal, every vector in the span of one should be orthogonal to every vector in the span of the other. |
| Line 45: | Line 46: |
| The null space of '''''A''''' (a.k.a. ''N('''A''')'') is '''orthogonal''' to the row space of '''''A''''' (a.k.a. ''R('''A''')''). The null space of '''''A'''^T^'' (a.k.a. ''N('''A'''^T^)'') is orthogonal to the column space of '''''A''''' (a.k.a. ''C('''A''')''). Commonly this means that the row and column spaces are planes while the null spaces of '''''A''''' and '''''A'''^T^'' are vectors, but that isn't always true. | For example, consider a plane in ''R^3^''. A plane is a subspace spanned by 2 vectors. The subspace that is orthogonal to a plane must be spanned by 1 vector, i.e. it is a line. A plane and a line can be checked for orthogonality by comparing each of the vectors spanning the plane for orthogonality with the single vector spanning the line. |
| Line 47: | Line 48: |
| ---- == Matrices == If a matrix is composed of orthonormal columns, then it is a '''matrix with orthonormal columns'''. These are usually denoted as '''''Q'''''. If a matrix with orthonormal columns is also square, only then can it be called an '''orthogonal matrix'''. Both categories are useful but there are many properties implied by the latter. 1. '''''Q'''^T^'''Q''' = '''I''''' 2. '''''Q'''^T^ = '''Q'''^-1^'' The [[LinearAlgebra/Projections#Matrices|projection]] of '''''A''''' if '''''A''''' is a matrix with orthonormal columns simplifies from '''''P''' = '''A'''('''A'''^T^'''A'''^-1^)'''A'''^T^'' into '''''P''' = '''QQ'''^T^''. Correspondingly, the system of normal equations simplifies from '''''A'''^T^'''A'''x̂ = '''A'''^T^b'' into ''x̂ = '''Q'''^T^b''. But if '''Q''' is also square, then the projection matrix further simplifies to '''''P''' = '''I'''''. |
For another example, [[LinearAlgebra/NullSpace|null spaces]] are orthogonal by definiton. For any subspace '''''A''''', the vectors spanning ''N('''A''')'' are precisely those that are not in ''R('''A''')'', therefore they are orthogonal. Similarly, the vectors spanning ''N('''A'''^T^)'' are precisely those that are not in ''C('''A''')'', therefore they are orthogonal. |
Orthogonality
Orthogonality is a generalization of perpendicularity. Orthonormality is a related concept, requiring that the components be unit vectors.
See also vector orthogonality.
Contents
Description
Orthogonality is an extension of perpendicularity to higher dimensions of Euclidean space, and also to arbitrary inner product spaces. To notate that x is orthogonal to y, use x ⊥ y.
Orthonormality is a further constraint: the orthogonal vectors are also unit vectors.
For a two vectors (which are not the zero vector) to be orthogonal, their inner product should be equal to 0. For a set of vectors to be orthogonal, every possible pair from the set should be orthogonal.
Matrices
A matrix is effectively a set of vectors. If the columns are orthonormal, then it can be called a matrix with orthonormal columns. These are usually denoted as Q.
Matrices with orthonormal columns have several important properties:
QTQ = I
The projection matrix is given as P = QQT.
The second follows from the first. Recall that, when projecting b into C(A), the projection matrix is given as P = A(ATA-1)AT. This comes from the linear system ATAx̂ = ATb and requiring that p = Pb. For a matrix Q with orthonormal columns, the first property simplifies the linear system to x̂ = QTb. Therefore, P = QQT.
If and only if such a matrix is square, it can be called an orthogonal matrix. These have several further properties:
QTQ = QQT = I
QT = Q-1
The determinant is always 1 or -1
The projection matrix is given as P = I, indicating that b must be in C(A).
Subspaces
For two subspaces to be orthogonal, every vector in the span of one should be orthogonal to every vector in the span of the other.
For example, consider a plane in R3. A plane is a subspace spanned by 2 vectors. The subspace that is orthogonal to a plane must be spanned by 1 vector, i.e. it is a line. A plane and a line can be checked for orthogonality by comparing each of the vectors spanning the plane for orthogonality with the single vector spanning the line.
For another example, null spaces are orthogonal by definiton. For any subspace A, the vectors spanning N(A) are precisely those that are not in R(A), therefore they are orthogonal. Similarly, the vectors spanning N(AT) are precisely those that are not in C(A), therefore they are orthogonal.