= Projection = A '''projection''' is an approximation within a column space. <> ---- == Vectors == Given two vectors ''a'' and ''b'', ''b'' can be [[Calculus/Projection|projected]] into ''C(a)'', the column space of ''a''. Furthermore, the projection vector with the least [[Calculus/Distance#Euclidean_distance|error]] as compared to the true vector ''b'' is characterized by orthogonality. Let ''e'' be the error vector; it is orthogonal to ''a''. The [[LinearAlgebra/LinearMapping|transformation]] of vector ''b'' into projection vector ''p'' can be described by a '''projection matrix'''. It is notated '''''P''''' as in ''p = '''P'''b''. === Properties === The projection matrix '''''P''''' satisfying ''p = '''P'''b'' is [[LinearAlgebra/Rank|rank]] 1. ''C('''P''')'', the column space of the projection matrix, is equivalent to ''C(b)''. Projection matrices are [[LinearAlgebra/SpecialMatrices#Symmetric_Matrices|symmetric]] (i.e., '''''P'''^T^ = '''P''''') and [[LinearAlgebra/Idempotency|idempotent]] (i.e., '''''P'''^2^ = '''P'''''). ---- == Matrices == For all the same reasons, a vector ''b'' can be projected into ''C('''A''')'', the column space of '''''A'''''. The [[Calculus/Distance#Euclidean_distance|error]] vector ''e'' is orthogonal to ''R('''A''')'', the row space of '''''A''''', and is therefore in the [[LinearAlgebra/NullSpace|null space]]. The projection matrix is now notated '''''P''''' as in ''p = '''P'''b''. === Least Squares === Given a consistent system as '''''A'''x = b'', i.e. ''b'' is in ''C('''A''')'', there are solutions for ''x''. If the system is inconsistent, then there is no solution. The best approximation is expressed as '''''A'''x̂ = p'' where projection ''p'' estimates ''b'' with an error term ''e''. [[Statistics/OrdinaryLeastSquares|This should sound familiar.]] The error term can be generally characterized by ''e = b - p''. An expression for ''p'' is known, so ''e = b - '''A'''x̂''. ''e'' is orthogonal to ''R('''A''')'', so '''''A'''^T^e = 0''. Substituting in the above expression gives '''''A'''^T^(b - '''A'''x̂) = 0'' Altogether, '''''A'''^T^'''A'''x̂ = '''A'''^T^b'' which simplifies to ''x̂ = ('''A'''^T^'''A''')^-1^'''A'''^T^b''. The projection matrix is calculated as '''''P''' = '''A'''('''A'''^T^'''A''')^-1^'''A'''^T^''. The projection is calculated as ''p = '''A'''('''A'''^T^'''A''')^-1^'''A'''^T^b''. === Properties === Projection matrices are still symmetric and idempotent. If ''b'' is in ''C('''A''')'', then '''''P''' = '''I'''''.. Conversely, if ''b'' is orthogonal to ''C('''A''')'', then '''''P'''b = 0'' and ''b = e''. ---- CategoryRicottone