= Eigenvalues and Eigenvectors = Matrices are characterized by their '''eigenvalues''' and '''eigenvectors'''. <> ---- == Introduction == The square matrix '''''A''''' is a [[LinearAlgebra/LinearMapping|mapping]] of vectors. There are certain input vectors that map to a linear combination of themself, i.e. '''''A''': a -> λa''. Another way to think of this is that there are certain input vectors which maintain their direction through a transformation. For vectors in these directions, the transformation only involves some stretching factor ''λ''. This intuition should also make clear that there are infinitely many such vectors. These certain input vector are called '''eigenvectors'''. Each eigenvector has a corresponding scaling factor ''λ'', called an '''eigenvalue'''. They form '''eigenpairs'''. ---- == Description == Eigenvalues and eigenvectors are the pairs of ''λ'' and ''x'' that satisfy '''''A'''x = λx'' and ''|'''A''' - λ'''I'''| = 0''. Only square matrices have eigenvectors. Given a matrix of size ''n x n'', either there are ''n'' unique eigenpairs, or the matrix is '''defective'''. Independent eigenvectors form an '''eigenbasis''', usually notated as '''''S'''''. The set of eigenvalues of a matrix forms its '''spectrum'''. === Properties === Adding ''n'''I''''' to '''''A''''' does not change its eigenvectors and adds ''n'' to the eigenvalues. The [[LinearAlgebra/Trace|trace]] is the sum of eigenvalues. The [[LinearAlgebra/Determinant|determinant]] is the product the eigenvalues. In an upper or lower triangular matrix, the numbers in the diagonal are its eigenvalues. Furthermore, in a [[LinearAlgebra/Diagonalization|diagonal matrix]], the columns are its eigenvectors. This also means that any diagonalizable matrix of size ''n x n'' always has ''n'' unique eigenpairs, and is never defective. If a matrix has an eigenvalue of 0, it is [[LinearAlgebra/Invertibility|singular and non-invertible]]. === Complex Eigenpairs === Eigenvectors and eigenvalues often include [[Calculus/ComplexNumbers|complex numbers]]. {{{ julia> using LinearAlgebra julia> A = [0 0 1; 0 1 0; -1 0 0] 3×3 Matrix{Int64}: 0 0 1 0 1 0 -1 0 0 julia> eigvals(A) 3-element Vector{ComplexF64}: 0.0 - 1.0im 0.0 + 1.0im 1.0 + 0.0im julia> eigvecs(A) 3×3 Matrix{ComplexF64}: 0.707107-0.0im 0.707107+0.0im 0.0+0.0im 0.0-0.0im 0.0+0.0im 1.0+0.0im 0.0-0.707107im 0.0+0.707107im 0.0+0.0im }}} When a matrix has a complex eigenpair, its [[Calculus/ComplexNumbers#Complex_Conjugate|conjugate]] will always be another eigenpair. === Rotation and Eigenvectors === Consider the 2-dimensional [[LinearAlgebra/RotationMatrix|rotation matrix]]: {{attachment:rot2.svg}} In two dimensions, there clearly cannot be ''any'' vectors which do not change direction through rotation. There are eigenvectors and eigenvalues, but they are complex. Consider now the 3-dimensional rotation matrix for rotation around the Z axis. This is fundamentally the same rotation. {{attachment:rot3.svg}} While 2 of the eigenpairs are complex again, 1 is not. All vectors along the axis of rotation itself are unchanged by this transformation. A complex eigenpair means that the transformation involves rotation. === Multiplicity === Consider a spectrum of ''{1, 1, 1, 2, 2, 3}''. The eigenvalue 1 is therefore said to have an '''algebraic multiplicity''' of 3. Even if there is a repeated eigenvalue, there may be multiple independent eigenvectors corresponding to it, forming unique eigenpairs. Consider: {{{ julia> using LinearAlgebra julia> eigen([1 0 0; 0 1 0; 0 0 2]) Eigen{Float64, Float64, Matrix{Float64}, Vector{Float64}} values: 3-element Vector{Float64}: 1.0 1.0 2.0 vectors: 3×3 Matrix{Float64}: 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 }}} There are two unique eigenvalues. Despite 1 having algebraic multiplicity of 2, there are 2 unique eigenvectors corresponding to it. Therefore it has '''geometric multiplicity''' of 2. (2 has algebraic ''and'' geometric multiplicity of 1, incidentally.) Consider instead: {{attachment:mult.svg}} Clearly this upper triangular matrix has only 1 unique eigenvalue: 1 with algebraic multiplicity of 2. There is only one independent eigenvector associated with that eigenvalue however, so it only has geometric multiplicity of 1. For each unique eigenvalue, the geometric multiplicity is always equal to or less than algebraic multiplicity. The algebraic multiplicity is the dimensionality of the '''eigenspace''', while the geometric multiplicity is the dimensions spanned by the eigenbasis. To reiterate, a diagonalizable matrix of size ''n x n'' always has ''n'' unique eigenpairs. This follows from the fact that a diagonalizable matrix must have algebraic multiplicity equal to geometric multiplicity for all eigenvalues. ---- == Solving for Eigenvalues == === Trivial Case === If a matrix is upper triangular, lower triangular, or [[LinearAlgebra/Diagonalization|diagonal]], then the numbers on the diagonal are the eigenvalues. === Simple Case === The eigenvalues of a ''2 x 2'' matrix can sometimes be identified through the [[LinearAlgebra/Determinant|determinant]] and [[LinearAlgebra/Trace|trace]]. Recall that: * the trace is the sum of eigenvalues * the determinant is the product of eigenvalues A little bit of mental math can usually solve the system given by ''|'''A'''| = λ,,1,, * λ,,2,,'' and ''tr('''A''') = λ,,1,, + λ,,2,,''. === Conventional Method === Because eigenvalues are characterized by ''|'''A''' - λ'''I'''| = 0'', they can be solved for by: * subtracting ''λ'' from each value on the diagonal * formulating the determinant for this difference * setting the formulation for 0 * solving for ''λ'' In a simple ''2 x 2'' matrix, this looks like: {{{ | A - λI | = 0 │ ┌ ┐ ┌ ┐ │ │ │ a b│ -│ λ 0│ │ = 0 │ │ c d│ │ 0 λ│ │ │ └ ┘ └ ┘ │ │ ┌ ┐ │ │ │ a-λ b│ │ = 0 │ │ c d-λ│ │ │ └ ┘ │ (a-λ)(d-λ) - bc = 0 }}} This leads to the '''characteristic polynomial''' of '''''A'''''; solving for the roots, as through either factorization or the quadratic formula, gives the eigenvalues. === Algebraic Method === Recall again that: * the trace is the sum of eigenvalues * the determinant is the product of eigenvalues It follows that, for a ''2 x 2'' matrix, 1/2 of the trace is equal to the mean of the eigenvalues. Furthermore, because the characteristic polynomial of a ''R^2^'' matrix is quadratic, the eigenvalues must be evenly spaced from the center (i.e., the mean). The mean (''m'') is known from the above properties. The eigenvectors can be expressed as ''m-d'' and ''m+d''. Recall once more that the determinant is the product of eigenvalues. It must be that ''|'''A'''| = (m-d)(m+d) = m^2^ - d^2^''. Altogether, {{attachment:shortcut.svg}} ---- == Solving for Eigenvectors == === Trivial Cases === If a matrix is [[LinearAlgebra/Diagonalization|diagonal]], then the columns are the eigenvectors. If one of the eigenvectors of a ''2 x 2'' matrix is known ''and'' it is complex, then it [[Calculus/ComplexNumbers#Complex_Conjugate|conjugate]] must be the other eigenvector. === Conventional Method === This assumes that the eigenvalues have already been solved for. Eigenvectors are solved as the [[LinearAlgebra/NullSpace|null space]] of '''''A''' - λ'''I'''''. Which is to say, for a given ''λ'', solve for ''x'' in the system ''('''A''' - λ'''I''')x = 0''. In a simple ''2 x 2'' matrix, this looks like: {{{ ( A - λI ) x = 0 / ┌ ┐ ┌ ┐ \ ┌ ┐ ┌ ┐ │ │ a b│ -│ λ 0│ │ │ u│ = │ 0│ │ │ c d│ │ 0 λ│ │ │ v│ │ 0│ \ └ ┘ └ ┘ / └ ┘ └ ┘ ┌ ┐ ┌ ┐ ┌ ┐ │ a-λ b│ │ u│ = │ 0│ │ c d-λ│ │ v│ │ 0│ └ ┘ └ ┘ └ ┘ ┌ ┐ ┌ ┐ ┌ ┐ │ a-λ b│ │ u│ = │ 0│ │ c d-λ│ │ v│ │ 0│ └ ┘ └ ┘ └ ┘ (a-λ)u + (b)v = 0 (c)u + (d-λ)v = 0 }}} Then proceed with any method for solving for the null space. As an example: {{{ julia> using LinearAlgebra julia> A = [2 1; 1 2] 2×2 Matrix{Int64}: 2 1 1 2 julia> eigvals, eigvecs = eigen(A) Eigen{Float64, Float64, Matrix{Float64}, Vector{Float64}} values: 2-element Vector{Float64}: 1.0 3.0 vectors: 2×2 Matrix{Float64}: -0.707107 0.707107 0.707107 0.707107 julia> nullspace(A - eigvals[1]*I) 2×1 Matrix{Float64}: -0.7071067811865475 0.7071067811865476 }}} ---- CategoryRicottone