|
Size: 4348
Comment: Simplifying matrix page names
|
Size: 2880
Comment: Simplifying matrix page names
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from LinearAlgebra/Projections = Projections = |
= Projection = |
| Line 4: | Line 3: |
| When a vector does not exist in a column space, the '''projection''' is the best approximation of it in linear combinations of that column space. | A '''projection''' is an approximation within a column space. A '''projection matrix''' describes the linear transformation of the approximation. See also [[Calculus/Projection|vector projection]]. |
| Line 12: | Line 13: |
| Line 14: | Line 16: |
| Given vectors ''a'' and ''b'', ''a'' can be projected into ''C(b)'', the column space of ''b''. This projection ''p'' has an error term ''e''. | Given vectors ''a'' and ''b'', ''b'' can be [[Calculus/Projection|projected]] into the column space of ''a'' (i.e., ''C(a)''). A '''projection matrix''' describes the linear transformation from vector ''a'' to projection vector ''p''. Such a projection matrix '''''P''''' satisfies ''p = '''P'''a''. |
| Line 16: | Line 18: |
=== Trigonometric Approach === Projections with vectors can be calculated in terms of ''θ'' is the angle formed by ''a'' and ''b''. A vector in the direction of ''b'' with the magnitude of ''a'' is given by ''||b|| cos(θ)''. This can be called the '''scalar projection'''. However, a '''vector projection''' should have a magnitude based on how much ''a'' moved through ''C(b)''. This is captured by ''â'', the unit vector in the direction of ''a'', which can be calculated as ''a/||a||''. The projection vector is given by ''(||a|| cos(θ)) (a/||a||) = (||b|| cos(θ)) â''. === Algebraic Approach === Projections with vectors can also be calculated in terms of the vectors themselves, as they represent linear transformations. First, the [[Calculus/VectorOperations#Dot_Product|dot product]] can be substituted into the above formulas to give a scalar projection as ''a⋅b/||a||'' and a vector projection as ''(a⋅b/||a||) a/||a|| = (a⋅b/||a||) â''. The vector projection can then be reformulated like: ''p = (a⋅b/||a||) a/||a||'' ''p = (a⋅b/||a||^2^) a'' ''p = (a⋅b/a⋅a) a'' or: ''p = (a⋅b/||a||) â'' ''p = (â⋅b) â'' === Linear Algebraic Approach === The linear transformation from vector ''a'' to projection vector ''p'' is expressed as ''p = ax̂''. The projection carries an '''error term''' that can be characterized by ''e = b - p'' or ''e = b - ax̂''. ''a'' is [[LinearAlgebra/Orthogonality|orthogonal]] to ''e'', so ''a⋅(b - ax̂) = 0''. This simplifies to ''x̂ = (a⋅b)/(a⋅a)''. Altogether, the projection vector is ''p = a (a⋅b)/(a⋅a)''. The '''projection matrix''' '''''P''''' satisfies ''p = '''P'''b''. ''C('''P''')'', the column space of '''''P''''', is equivalent to ''C(a)''. It follows that '''''P''''' is also of [[LinearAlgebra/Rank|rank]] 1. |
The column space of the projection matrix (i.e., ''C('''P''')'') is equivalent to ''C(b)''. It follows that '''''P''''' is also of [[LinearAlgebra/Rank|rank]] 1. |
| Line 68: | Line 32: |
| Given a system as '''''A'''x = b'', if ''b'' is not in ''C('''A''')'', the column space of '''''A''''', then there is no possible solution for ''x''. The best approximation is expressed as '''''A'''x̂ = p'' where projection ''p'' estimates ''b'' with an error term ''e''. | Given vector ''b'' and matrix '''''A''''', ''b'' can be projected into the column space of '''''A''''' (i.e., ''C('''A''')''). Given a linear system as '''''A'''x = b'', if ''b'' is in ''C('''A''')'', there are solutions for ''x''. If ''b'' is ''not'' in ''C('''A''')'' however, there is no possible solution. The best ''approximation'' is expressed as '''''A'''x̂ = p'' where projection ''p'' estimates ''b'' with an error term ''e''. [[Statistics/OrdinaryLeastSquares|This should sound familiar.]] |
| Line 86: | Line 52: |
| If ''b'' actually ''was'' in ''C('''A''')'', then '''''P''' = '''I'''''. Conversely, if ''b'' is orthogonal to ''C('''A''')'', then '''''P'''b = 0'' and ''b = e''. === Usage === [[Statistics/OrdinaryLeastSquares|This should look familiar.]] A projection is inherently the minimization of the error term. |
If ''b'' is in ''C('''A''')'', then '''''P''' = '''I'''''.Conversely, if ''b'' is orthogonal to ''C('''A''')'', then '''''P'''b = 0'' and ''b = e''. |
Projection
A projection is an approximation within a column space. A projection matrix describes the linear transformation of the approximation.
See also vector projection.
Contents
Vectors
Given vectors a and b, b can be projected into the column space of a (i.e., C(a)). A projection matrix describes the linear transformation from vector a to projection vector p. Such a projection matrix P satisfies p = Pa.
The column space of the projection matrix (i.e., C(P)) is equivalent to C(b). It follows that P is also of rank 1.
Properties
The projection matrix P is symmetric (i.e. PT = P) and idempotent (i.e. P2 = P).
Matrices
Given vector b and matrix A, b can be projected into the column space of A (i.e., C(A)).
Given a linear system as Ax = b, if b is in C(A), there are solutions for x. If b is not in C(A) however, there is no possible solution. The best approximation is expressed as Ax̂ = p where projection p estimates b with an error term e. This should sound familiar.
The error term can be characterized by e = b - p or e = b - Ax̂. e is orthogonal to R(A), the row space of A; equivalently it is orthogonal to C(AT). Orthogonality in this context means that e is in the null space, so AT(b - Ax̂) = 0.
The system of normal equations is ATAx̂ = ATb. This simplifies to x̂ = (ATA)-1ATb. Altogether, the projection is characterized by p = A(ATA)-1ATb.
The projection matrix P satisfies p = Pb. It is calculated as P = A(ATA)-1AT.
b can also be projected onto e, which geometrically means projecting into the null space of AT. Algebraically, if one projection matrix has been computed as P, then the projection matrix for going the other way is (I - P)b.
Properties
As above, the projection matrix P is symmetric and idempotent.
If A is square, the above equations simplify rapidly.
If b is in C(A), then P = I.Conversely, if b is orthogonal to C(A), then Pb = 0 and b = e.