Eigenvalues and Eigenvectors
Matrices are characterized by their eigenvalues and eigenvectors.
Contents
Introduction
The square matrix A is a mapping of vectors. There are certain input vectors that map to a linear combination of themself, i.e. A: a -> λa.
Another way to think of this is that there are certain input vectors which maintain their direction through a transformation. For vectors in these directions, the transformation only involves some stretching factor λ. This intuition should also make clear that there are infinitely many such vectors.
These certain input vector are called eigenvectors. Each eigenvector has a corresponding scaling factor λ, called an eigenvalue. They form eigenpairs.
Description
Eigenvalues and eigenvectors are the pairs of λ and x that satisfy Ax = λx and |A - λI| = 0.
Only square matrices have eigenvectors. Given a matrix of size n x n, either there are n unique eigenpairs, or the matrix is defective.
Independent eigenvectors form an eigenbasis, usually notated as S.
The set of eigenvalues of a matrix forms its spectrum.
Properties
Adding nI to A does not change its eigenvectors and adds n to the eigenvalues.
The trace is the sum of eigenvalues. The determinant is the product the eigenvalues.
In an upper or lower triangular matrix, the numbers in the diagonal are its eigenvalues. Furthermore, in a diagonal matrix, the columns are its eigenvectors. This also means that any diagonalizable matrix of size n x n always has n unique eigenpairs, and is never defective.
If a matrix has an eigenvalue of 0, it is singular and non-invertible.
Complex Eigenpairs
Eigenvectors and eigenvalues often include complex numbers.
julia> using LinearAlgebra
julia> A = [0 0 1; 0 1 0; -1 0 0]
3×3 Matrix{Int64}:
0 0 1
0 1 0
-1 0 0
julia> eigvals(A)
3-element Vector{ComplexF64}:
0.0 - 1.0im
0.0 + 1.0im
1.0 + 0.0im
julia> eigvecs(A)
3×3 Matrix{ComplexF64}:
0.707107-0.0im 0.707107+0.0im 0.0+0.0im
0.0-0.0im 0.0+0.0im 1.0+0.0im
0.0-0.707107im 0.0+0.707107im 0.0+0.0imWhen a matrix has a complex eigenpair, its conjugate will always be another eigenpair.
Rotation and Eigenvectors
Consider the 2-dimensional rotation matrix:
In two dimensions, there clearly cannot be any vectors which do not change direction through rotation. There are eigenvectors and eigenvalues, but they are complex.
Consider now the 3-dimensional rotation matrix for rotation around the Z axis. This is fundamentally the same rotation.
While 2 of the eigenpairs are complex again, 1 is not. All vectors along the axis of rotation itself are unchanged by this transformation.
A complex eigenpair means that the transformation involves rotation.
Solving for Eigenvalues
Trivial Case
If a matrix is upper triangular, lower triangular, or diagonal, then the numbers on the diagonal are the eigenvalues.
Simple Case
The eigenvalues of a 2 x 2 matrix can sometimes be identified through the determinant and trace. Recall that:
- the trace is the sum of eigenvalues
- the determinant is the product of eigenvalues
A little bit of mental math can usually solve the system given by |A| = λ1 * λ2 and tr(A) = λ1 + λ2.
Conventional Method
Because eigenvalues are characterized by |A - λI| = 0, they can be solved for by:
subtracting λ from each value on the diagonal
- formulating the determinant for this difference
- setting the formulation for 0
solving for λ
In a simple 2 x 2 matrix, this looks like:
| A - λI | = 0 │ ┌ ┐ ┌ ┐ │ │ │ a b│ -│ λ 0│ │ = 0 │ │ c d│ │ 0 λ│ │ │ └ ┘ └ ┘ │ │ ┌ ┐ │ │ │ a-λ b│ │ = 0 │ │ c d-λ│ │ │ └ ┘ │ (a-λ)(d-λ) - bc = 0
This leads to the characteristic polynomial of A; solving for the roots, as through either factorization or the quadratic formula, gives the eigenvalues.
Algebraic Method
Recall again that:
- the trace is the sum of eigenvalues
- the determinant is the product of eigenvalues
It follows that, for a 2 x 2 matrix, 1/2 of the trace is equal to the mean of the eigenvalues.
Furthermore, because the characteristic polynomial of a R2 matrix is quadratic, the eigenvalues must be evenly spaced from the center (i.e., the mean). The mean (m) is known from the above properties. The eigenvectors can be expressed as m-d and m+d.
Recall once more that the determinant is the product of eigenvalues. It must be that |A| = (m-d)(m+d) = m2 - d2.
Altogether,
Solving for Eigenvectors
Trivial Cases
If a matrix is diagonal, then the columns are the eigenvectors.
If one of the eigenvectors of a 2 x 2 matrix is known and it is complex, then it conjugate must be the other eigenvector.
Conventional Method
This assumes that the eigenvalues have already been solved for.
Eigenvectors are solved as the null space of A - λI. Which is to say, for a given λ, solve for x in the system (A - λI)x = 0.
In a simple 2 x 2 matrix, this looks like:
( A - λI ) x = 0 / ┌ ┐ ┌ ┐ \ ┌ ┐ ┌ ┐ │ │ a b│ -│ λ 0│ │ │ u│ = │ 0│ │ │ c d│ │ 0 λ│ │ │ v│ │ 0│ \ └ ┘ └ ┘ / └ ┘ └ ┘ ┌ ┐ ┌ ┐ ┌ ┐ │ a-λ b│ │ u│ = │ 0│ │ c d-λ│ │ v│ │ 0│ └ ┘ └ ┘ └ ┘ ┌ ┐ ┌ ┐ ┌ ┐ │ a-λ b│ │ u│ = │ 0│ │ c d-λ│ │ v│ │ 0│ └ ┘ └ ┘ └ ┘ (a-λ)u + (b)v = 0 (c)u + (d-λ)v = 0
Then proceed with any method for solving for the null space. As an example:
julia> using LinearAlgebra
julia> A = [2 1; 1 2]
2×2 Matrix{Int64}:
2 1
1 2
julia> eigvals, eigvecs = eigen(A)
Eigen{Float64, Float64, Matrix{Float64}, Vector{Float64}}
values:
2-element Vector{Float64}:
1.0
3.0
vectors:
2×2 Matrix{Float64}:
-0.707107 0.707107
0.707107 0.707107
julia> nullspace(A - eigvals[1]*I)
2×1 Matrix{Float64}:
-0.7071067811865475
0.7071067811865476