Orthonormalization

Gram-Schmidt orthonormalization is a process for transforming a vectors into unit vectors.

Vectors

Two vectors a and b can be orthonormalized into unit vectors A and B. This means...

1. that they are made orthogonal to each other by removing any components of one from the other.

2. that they are normalized to a unit distance of 1.

These are accomplished in discrete steps. The first is to enforce orthogonality. But orthogonality is a property of two vectors, not one. Therefore a needs no transformation to become A.

The process of transforming b into B is simply the subtraction of all components of a from b. This is a linear combination and does not change the column space of a system that includes both a and b. Projections are a complimentary idea; p is the component of a that estimates b. The process of orthonormalization is the same as computing projections but the error term e is the desired result. Recall that e = b - ax̂ and x̂ = (A^Tb)/(A^TA). Therefore, B = b - A (A^Tb)/(A^TA).

To transform another vector c into being orthogonal to both A and B, apply the same process for each component: C = c - A (A^Tc)/(A^TA) - B (B^Tc)/(B^TB).

The final step is to normalize the orthogonal vectors by their own distance, as in A/||A|| and B/||B||.

Matrices

The process applied to vectors is also applicable to the columns in a matrix. Instead of vectors a and b, use v₁ and v₂ in V. The process yields u₁ and u₂ in U. Then the columns are normalized into Q like q₁ = u₁/||u₁||.

To re-emphasize, this is a linear combination generalized as A = QR, and does not change the column space of A.

Note that Q is a matrix with orthonormal columns; it must also be square to be called an orthogonal matrix.

CategoryRicottone