|
Size: 3077
Comment: Rewrite
|
Size: 2861
Comment: Simplifying matrix page names
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Projections = | = Projection = |
| Line 3: | Line 3: |
| When a vector does not exist in a column space, the '''projection''' is the best approximation of it in linear combinations of that column space. | A '''projection''' is an approximation within a column space. A '''projection matrix''' describes the linear transformation of the approximation. See also [[Calculus/Projection|vector projection]]. |
| Line 11: | Line 13: |
| Line 13: | Line 16: |
| Given vectors ''a'' and ''b'', ''b'' can be projected into ''C(a)'', the column space of ''a''. This projection ''p'' has an error term ''e''. | Given vectors ''a'' and ''b'', ''b'' can be [[Calculus/Projection|projected]] into the column space of ''a'' (i.e., ''C(a)''). A '''projection matrix''' describes the linear transformation from vector ''a'' to projection vector ''p''. Such a projection matrix '''''P''''' satisfies ''p = '''P'''a''. |
| Line 15: | Line 18: |
| The factor which converts ''a'' into an estimate is notated as ''x̂'', so that ''p = ax̂''. The '''error term''' can be characterized by ''e = b - p'' or ''e = b - ax̂''. ''a'' is [[LinearAlgebra/Orthogonality|orthogonal]] to ''e'', so ''a^T^(b - ax̂) = 0''. This simplifies to ''x̂ = (a^T^b)/(a^T^a)''. Altogether, the projection is ''p = a(a^T^b)/(a^T^a)''. The '''projection matrix''' '''''P''''' satisfies ''p = '''P'''b''. It is calculated ''(aa^T^)/(a^T^a)''. ''C('''P''')'', the column space of '''''P''''', is equivalent to the column space of ''a''. (It follows that '''''P''''' is also of [[LinearAlgebra/Rank|rank]] 1.) |
The column space of the projection matrix (i.e., ''C('''P''')'') is equivalent to ''C(b)''. It follows that '''''P''''' is also of [[LinearAlgebra/Rank|rank]] 1. |
| Line 23: | Line 24: |
| The projection matrix '''''P''''' is [[LinearAlgebra/MatrixProperties#Symmetry|symmetric]] (i.e. '''''P'''^T^ = '''P''''') and [[LinearAlgebra/MatrixProperties#Idempotency|idempotent]] (i.e. '''''P'''^2^ = '''P'''''). | Projection matrices are [[LinearAlgebra/SpecialMatrices#Symmetric_Matrices|symmetric]] (i.e., '''''P'''^T^ = '''P''''') and [[LinearAlgebra/Idempotency|idempotent]] (i.e., '''''P'''^2^ = '''P'''''). |
| Line 31: | Line 32: |
| Given a system as '''''A'''x = b'', if ''b'' is not in ''C('''A''')'', the column space of '''''A''''', then there is no possible solution for ''x''. The best approximation is expressed as '''''A'''x̂ = p'' where projection ''p'' estimates ''b'' with an error term ''e''. | Given vector ''b'' and matrix '''''A''''', ''b'' can be projected into the column space of '''''A''''' (i.e., ''C('''A''')''). |
| Line 33: | Line 34: |
| The error term can be characterized by ''e = b - p'' or ''e = b - '''A'''x̂''. ''e'' is orthogonal to ''R('''A''')'', the row space of '''''A'''''; equivalently it is orthogonal to ''C('''A'''^T^)''. Orthogonality in this context means that ''e'' is in the [[LinearAlgebra/NullSpaces|null space]], so '''''A'''^T^(b - '''A'''x̂) = 0''. | Given a linear system as '''''A'''x = b'', if ''b'' is in ''C('''A''')'', there are solutions for ''x''. If ''b'' is ''not'' in ''C('''A''')'' however, there is no possible solution. The best ''approximation'' is expressed as '''''A'''x̂ = p'' where projection ''p'' estimates ''b'' with an error term ''e''. [[Statistics/OrdinaryLeastSquares|This should sound familiar.]] The error term can be characterized by ''e = b - p'' or ''e = b - '''A'''x̂''. ''e'' is orthogonal to ''R('''A''')'', the row space of '''''A'''''; equivalently it is orthogonal to ''C('''A'''^T^)''. Orthogonality in this context means that ''e'' is in the [[LinearAlgebra/NullSpace|null space]], so '''''A'''^T^(b - '''A'''x̂) = 0''. |
| Line 49: | Line 52: |
| If ''b'' actually ''was'' in ''C('''A''')'', then '''''P''' = '''I'''''. Conversely, if ''b'' is orthogonal to ''C('''A''')'', then '''''P'''b = 0'' and ''b = e''. === Usage === [[Statistics/OrdinaryLeastSquares|This should look familiar.]] A projection is inherently the minimization of the error term. |
If ''b'' is in ''C('''A''')'', then '''''P''' = '''I'''''. Conversely, if ''b'' is orthogonal to ''C('''A''')'', then '''''P'''b = 0'' and ''b = e''. |
Projection
A projection is an approximation within a column space. A projection matrix describes the linear transformation of the approximation.
See also vector projection.
Contents
Vectors
Given vectors a and b, b can be projected into the column space of a (i.e., C(a)). A projection matrix describes the linear transformation from vector a to projection vector p. Such a projection matrix P satisfies p = Pa.
The column space of the projection matrix (i.e., C(P)) is equivalent to C(b). It follows that P is also of rank 1.
Properties
Projection matrices are symmetric (i.e., PT = P) and idempotent (i.e., P2 = P).
Matrices
Given vector b and matrix A, b can be projected into the column space of A (i.e., C(A)).
Given a linear system as Ax = b, if b is in C(A), there are solutions for x. If b is not in C(A) however, there is no possible solution. The best approximation is expressed as Ax̂ = p where projection p estimates b with an error term e. This should sound familiar.
The error term can be characterized by e = b - p or e = b - Ax̂. e is orthogonal to R(A), the row space of A; equivalently it is orthogonal to C(AT). Orthogonality in this context means that e is in the null space, so AT(b - Ax̂) = 0.
The system of normal equations is ATAx̂ = ATb. This simplifies to x̂ = (ATA)-1ATb. Altogether, the projection is characterized by p = A(ATA)-1ATb.
The projection matrix P satisfies p = Pb. It is calculated as P = A(ATA)-1AT.
b can also be projected onto e, which geometrically means projecting into the null space of AT. Algebraically, if one projection matrix has been computed as P, then the projection matrix for going the other way is (I - P)b.
Properties
As above, the projection matrix P is symmetric and idempotent.
If A is square, the above equations simplify rapidly.
If b is in C(A), then P = I. Conversely, if b is orthogonal to C(A), then Pb = 0 and b = e.