|
Size: 729
Comment: Initial commit
|
← Revision 20 as of 2026-02-16 16:43:48 ⇥
Size: 2753
Comment: Fixed link
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Projections = | = Projection = A '''projection''' is an approximation within a column space. |
| Line 9: | Line 11: |
| == Definition == | |
| Line 11: | Line 12: |
| Given two vectors ''a'' and ''b'', we can '''project''' ''b'' onto ''a'' to get the best possible estimate of the former as a multiple of the latter. This projection ''p'' has an error term ''e''. | == Vectors == |
| Line 13: | Line 14: |
| Take the multiple as ''x'', so that ''p = ax''. The error term can be characterized as ''b-ax''. | Given two vectors ''a'' and ''b'', ''b'' can be [[Calculus/Projection|projected]] into ''C(a)'', the column space of ''a''. |
| Line 15: | Line 16: |
| ''a'' is [[LinearAlgebra/Orthogonality|orthogonal]] to ''e''. Therefore, ''a^T^(b-ax) = 0''. This simplifies to ''x = (a^T^b)/(a^T^a)''. Altogether, the projection is characterized as ''p = a(a^T^b)/(a^T^a)''. | Furthermore, the projection vector with the least [[Calculus/Distance#Euclidean_distance|error]] as compared to the true vector ''b'' is characterized by orthogonality. Let ''e'' be the error vector; it is orthogonal to ''a''. |
| Line 17: | Line 18: |
| A matrix '''''P''''' can be defined such that ''p = '''P'''b''. The projection matrix is ''(aa^T^)/(a^T^a)''. | The [[LinearAlgebra/LinearMapping|transformation]] of vector ''b'' into projection vector ''p'' can be described by a '''projection matrix'''. It is notated '''''P''''' as in ''p = '''P'''b''. === Properties === The projection matrix '''''P''''' satisfying ''p = '''P'''b'' is [[LinearAlgebra/Rank|rank]] 1. ''C('''P''')'', the column space of the projection matrix, is equivalent to ''C(b)''. Projection matrices are [[LinearAlgebra/SpecialMatrices#Symmetric_Matrices|symmetric]] (i.e., '''''P'''^T^ = '''P''''') and [[LinearAlgebra/Idempotency|idempotent]] (i.e., '''''P'''^2^ = '''P'''''). ---- == Matrices == For all the same reasons, a vector ''b'' can be projected into ''C('''A''')'', the column space of '''''A'''''. The [[Calculus/Distance#Euclidean_distance|error]] vector ''e'' is orthogonal to ''R('''A''')'', the row space of '''''A''''', and is therefore in the [[LinearAlgebra/NullSpace|null space]]. The projection matrix is now notated '''''P''''' as in ''p = '''P'''b''. === Least Squares === Given a consistent system as '''''A'''x = b'', i.e. ''b'' is in ''C('''A''')'', there are solutions for ''x''. If the system is inconsistent, then there is no solution. The best approximation is expressed as '''''A'''x̂ = p'' where projection ''p'' estimates ''b'' with an error term ''e''. [[Statistics/OrdinaryLeastSquares|This should sound familiar.]] The error term can be generally characterized by ''e = b - p''. An expression for ''p'' is known, so ''e = b - '''A'''x̂''. ''e'' is orthogonal to ''R('''A''')'', so '''''A'''^T^e = 0''. Substituting in the above expression gives '''''A'''^T^(b - '''A'''x̂) = 0'' Altogether, '''''A'''^T^'''A'''x̂ = '''A'''^T^b'' which simplifies to ''x̂ = ('''A'''^T^'''A''')^-1^'''A'''^T^b''. The projection matrix is calculated as '''''P''' = '''A'''('''A'''^T^'''A''')^-1^'''A'''^T^''. The projection is calculated as ''p = '''A'''('''A'''^T^'''A''')^-1^'''A'''^T^b''. === Properties === Projection matrices are still symmetric and idempotent. If ''b'' is in ''C('''A''')'', then '''''P''' = '''I'''''.. Conversely, if ''b'' is orthogonal to ''C('''A''')'', then '''''P'''b = 0'' and ''b = e''. |
Projection
A projection is an approximation within a column space.
Vectors
Given two vectors a and b, b can be projected into C(a), the column space of a.
Furthermore, the projection vector with the least error as compared to the true vector b is characterized by orthogonality. Let e be the error vector; it is orthogonal to a.
The transformation of vector b into projection vector p can be described by a projection matrix. It is notated P as in p = Pb.
Properties
The projection matrix P satisfying p = Pb is rank 1.
C(P), the column space of the projection matrix, is equivalent to C(b).
Projection matrices are symmetric (i.e., PT = P) and idempotent (i.e., P2 = P).
Matrices
For all the same reasons, a vector b can be projected into C(A), the column space of A. The error vector e is orthogonal to R(A), the row space of A, and is therefore in the null space.
The projection matrix is now notated P as in p = Pb.
Least Squares
Given a consistent system as Ax = b, i.e. b is in C(A), there are solutions for x.
If the system is inconsistent, then there is no solution. The best approximation is expressed as Ax̂ = p where projection p estimates b with an error term e. This should sound familiar.
The error term can be generally characterized by e = b - p. An expression for p is known, so e = b - Ax̂.
e is orthogonal to R(A), so ATe = 0. Substituting in the above expression gives AT(b - Ax̂) = 0
Altogether, ATAx̂ = ATb which simplifies to x̂ = (ATA)-1ATb.
The projection matrix is calculated as P = A(ATA)-1AT.
The projection is calculated as p = A(ATA)-1ATb.
Properties
Projection matrices are still symmetric and idempotent.
If b is in C(A), then P = I.. Conversely, if b is orthogonal to C(A), then Pb = 0 and b = e.