|
Size: 729
Comment: Initial commit
|
Size: 2861
Comment: Simplifying matrix page names
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Projections = | = Projection = A '''projection''' is an approximation within a column space. A '''projection matrix''' describes the linear transformation of the approximation. See also [[Calculus/Projection|vector projection]]. |
| Line 9: | Line 13: |
| == Definition == | |
| Line 11: | Line 14: |
| Given two vectors ''a'' and ''b'', we can '''project''' ''b'' onto ''a'' to get the best possible estimate of the former as a multiple of the latter. This projection ''p'' has an error term ''e''. | == Vectors == |
| Line 13: | Line 16: |
| Take the multiple as ''x'', so that ''p = ax''. The error term can be characterized as ''b-ax''. | Given vectors ''a'' and ''b'', ''b'' can be [[Calculus/Projection|projected]] into the column space of ''a'' (i.e., ''C(a)''). A '''projection matrix''' describes the linear transformation from vector ''a'' to projection vector ''p''. Such a projection matrix '''''P''''' satisfies ''p = '''P'''a''. |
| Line 15: | Line 18: |
| ''a'' is [[LinearAlgebra/Orthogonality|orthogonal]] to ''e''. Therefore, ''a^T^(b-ax) = 0''. This simplifies to ''x = (a^T^b)/(a^T^a)''. Altogether, the projection is characterized as ''p = a(a^T^b)/(a^T^a)''. | The column space of the projection matrix (i.e., ''C('''P''')'') is equivalent to ''C(b)''. It follows that '''''P''''' is also of [[LinearAlgebra/Rank|rank]] 1. |
| Line 17: | Line 20: |
| A matrix '''''P''''' can be defined such that ''p = '''P'''b''. The projection matrix is ''(aa^T^)/(a^T^a)''. | === Properties === Projection matrices are [[LinearAlgebra/SpecialMatrices#Symmetric_Matrices|symmetric]] (i.e., '''''P'''^T^ = '''P''''') and [[LinearAlgebra/Idempotency|idempotent]] (i.e., '''''P'''^2^ = '''P'''''). ---- == Matrices == Given vector ''b'' and matrix '''''A''''', ''b'' can be projected into the column space of '''''A''''' (i.e., ''C('''A''')''). Given a linear system as '''''A'''x = b'', if ''b'' is in ''C('''A''')'', there are solutions for ''x''. If ''b'' is ''not'' in ''C('''A''')'' however, there is no possible solution. The best ''approximation'' is expressed as '''''A'''x̂ = p'' where projection ''p'' estimates ''b'' with an error term ''e''. [[Statistics/OrdinaryLeastSquares|This should sound familiar.]] The error term can be characterized by ''e = b - p'' or ''e = b - '''A'''x̂''. ''e'' is orthogonal to ''R('''A''')'', the row space of '''''A'''''; equivalently it is orthogonal to ''C('''A'''^T^)''. Orthogonality in this context means that ''e'' is in the [[LinearAlgebra/NullSpace|null space]], so '''''A'''^T^(b - '''A'''x̂) = 0''. The system of '''normal equations''' is '''''A'''^T^'''A'''x̂ = '''A'''^T^b''. This simplifies to ''x̂ = ('''A'''^T^'''A''')^-1^'''A'''^T^b''. Altogether, the projection is characterized by ''p = '''A'''('''A'''^T^'''A''')^-1^'''A'''^T^b''. The projection matrix '''''P''''' satisfies ''p = '''P'''b''. It is calculated as '''''P''' = '''A'''('''A'''^T^'''A''')^-1^'''A'''^T^''. ''b'' can also be projected onto ''e'', which geometrically means projecting into the null space of '''''A'''^T^''. Algebraically, if one projection matrix has been computed as '''''P''''', then the projection matrix for going the other way is ''('''I''' - '''P''')b''. === Properties === As above, the projection matrix '''''P''''' is symmetric and idempotent. If '''''A''''' is square, the above equations simplify rapidly. If ''b'' is in ''C('''A''')'', then '''''P''' = '''I'''''. Conversely, if ''b'' is orthogonal to ''C('''A''')'', then '''''P'''b = 0'' and ''b = e''. |
Projection
A projection is an approximation within a column space. A projection matrix describes the linear transformation of the approximation.
See also vector projection.
Contents
Vectors
Given vectors a and b, b can be projected into the column space of a (i.e., C(a)). A projection matrix describes the linear transformation from vector a to projection vector p. Such a projection matrix P satisfies p = Pa.
The column space of the projection matrix (i.e., C(P)) is equivalent to C(b). It follows that P is also of rank 1.
Properties
Projection matrices are symmetric (i.e., PT = P) and idempotent (i.e., P2 = P).
Matrices
Given vector b and matrix A, b can be projected into the column space of A (i.e., C(A)).
Given a linear system as Ax = b, if b is in C(A), there are solutions for x. If b is not in C(A) however, there is no possible solution. The best approximation is expressed as Ax̂ = p where projection p estimates b with an error term e. This should sound familiar.
The error term can be characterized by e = b - p or e = b - Ax̂. e is orthogonal to R(A), the row space of A; equivalently it is orthogonal to C(AT). Orthogonality in this context means that e is in the null space, so AT(b - Ax̂) = 0.
The system of normal equations is ATAx̂ = ATb. This simplifies to x̂ = (ATA)-1ATb. Altogether, the projection is characterized by p = A(ATA)-1ATb.
The projection matrix P satisfies p = Pb. It is calculated as P = A(ATA)-1AT.
b can also be projected onto e, which geometrically means projecting into the null space of AT. Algebraically, if one projection matrix has been computed as P, then the projection matrix for going the other way is (I - P)b.
Properties
As above, the projection matrix P is symmetric and idempotent.
If A is square, the above equations simplify rapidly.
If b is in C(A), then P = I. Conversely, if b is orthogonal to C(A), then Pb = 0 and b = e.