Theory

Introduction

A very important general method for understanding what matrices do when they act upon vectors is to transform them into matrices or products of matrices with standardized form. Each standardized form is designed to make certain aspects of matrix behavior very easy to see, or to make certain calculations that apply the matrix very easy to perform. Typically a matrix is converted to a standardized form using an algorithmic procedure that is run without intuitive or subjective assessment along the way, and the final result is studied afterwards.

In what follows we study a standardized form called ‘LU factorization’ that is produced using the technique of elimination. Given the LU factorization (or a refinement of part of it called row-echelon form or reduced row-echelon form), one can read off qualitative aspects of the matrix such as its rank, which is the dimension of its image subspace, as well as provide a basis for the image and a basis for the kernel, which is the subspace that is reduced to zero by the matrix action.

In addition to this qualitative information about the action of a matrix on some vectors, the LU factorization can be followed by another important technique called back-substitution. The combined process is typically used for inverting matrix equations such as these:

$$A\mathbf{x}=\mathbf{b},AX=B.$$

The vector inverse of $\mathbf{b}$ under the action of $A$ is the vector $\mathbf{x}$ which $A$ sends to $\mathbf{b}$. The matrix inverse of $B$ under the action of $A$ is the matrix $X=A^{-1}B$ which is sent to $B$ by the action of $A$ multiplying it on the left. (In the second case, if $B=I_n$ is the identity, then we are just computing $X=A^{-1}$.) We will see below how the $LU$ factorization process followed by back-substitution allows us to find $\mathbf{x}$ or $X$.