Chapter 02 - Matrices in 2D and 3D

Tutorial: Inventory of Matrices in 2D and 3D

Introduction

Vectors can be considered from an abstract perspective as items in a space of vectors all of which satisfy the rules of the “game” of vectors, as well as from a concrete perspective as lists of numbers upon which the operations of vectors are defined (addition and scaling).

Similarly, matrices can be considered from an abstract perspective as “linear operations” upon an abstract space of vectors, or from a concrete perspective as arrays of numbers which operate upon the lists of numbers in concrete vectors.

Link to original

Matrices acting on vectors

Matrices acting on vectors

An $n\times m$ matrix is an $n\times m$ array of numbers. Given a matrix called $A$, the numbers making it up are called “coefficients” and written $a_{ij}$ for $i=1,\dots ,n$ and $j=1,\dots ,m$. Sometimes we write $A=(a_{ij})$. The first index always specifies the row, and the second index specifies the column. For example, we could have:

$$A=\left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right)\text{or}B=\left(\begin{array}{ccc}{b}_{11}& b_{12}& b_{13}\\ b_{21}& b_{22}& b_{23}\\ b_{31}& b_{32}& b_{33}\end{array}\right).$$

Suppose we have a 2D vector $\mathbf{u}$ and a 3D vector $\mathbf{v}$:

$$\mathbf{u}=\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right)\text{and}\left(\begin{array}{c}{v}_1\\ v_2\\ v_3\end{array}\right).$$

Now the action of $A$ on $\mathbf{u}$ and of $B$ on $\mathbf{v}$ can be written out with these formulas:

$$A\mathbf{u}=\left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right)\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right)=\left(\begin{array}{c}{a}_{11}{u}_1+a_{12}{u}_2\\ a_{21}{u}_1+a_{22}{u}_2\end{array}\right),$$ $$B\mathbf{v}=\left(\begin{array}{ccc}{b}_{11}& b_{12}& b_{13}\\ b_{21}& b_{22}& b_{23}\\ b_{31}& b_{32}& b_{33}\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\\ v_3\end{array}\right)=\left(\begin{array}{c}{b}_{11}{v}_1+b_{12}{v}_2+b_{13}{v}_3\\ b_{21}{v}_1+b_{22}{v}_2+b_{23}{v}_3\\ b_{31}{v}_1+b_{32}{v}_2+b_{33}{v}_3\end{array}\right).$$ Link to original

Example

Matrix acting on a vector

Matrix acting on a vector

$$\left(\begin{array}{cc}1& 4\\ -2& -1\end{array}\right)\left(\begin{array}{c}3\\ 6\end{array}\right)=\left(\begin{array}{c}1\cdot 3+4\cdot 6\\ -2\cdot 3+-1\cdot 6\end{array}\right)=\left(\begin{array}{c}27\\ -12\end{array}\right)$$

Link to original

Matrices acting on vectors (part 2)

Let $C$ be an $n\times m$ matrix and let $\mathbf{w}$ be an $m$D vector. Using summation notation for large numbers of indices, the matrix times vector operation is given by the general formula for the $i^{th}$ row of the output vector, where $i=1,\dots ,n$:

$$(C\mathbf{w}{)}_i=\sum _{j=1}^m{C}_{ij}{w}_j.$$ Link to original

Question

01

Summation notation

Write out the formula for the operation of a $4\times 4$ matrix on a $4$D vector. (Convert the summation notation to ordinary sums.)

Link to original

Matrices acting on vectors (part 3)

The formulas for the action of a matrix on a vector should be understood in two different ways.

• For concrete calculations, it is best to think of each row of the output vector as the dot product of the corresponding matrix row with the input vector.
• For conceptual and abstract understanding, it is better to think of the operation as sending each standard basis vector to the corresponding column vector of the matrix.

Let’s consider now the second way in more detail. A matrix can be considered as a list of column vectors. For example:

$$A=\left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right)=\left(\begin{array}{cc}{\mathbf{a}}_1& {\mathbf{a}}_2\end{array}\right),\text{ with }{\mathbf{a}}_1=\left(\begin{array}{c}{a}_{11}\\ a_{21}\end{array}\right),{\mathbf{a}}_2=\left(\begin{array}{c}{a}_{12}\\ a_{22}\end{array}\right).$$

Then $A\mathbf{u}=u_1{\mathbf{a}}_1+u_2{\mathbf{a}}_2$:

$$\left(\begin{array}{cc}{\mathbf{a}}_1& {\mathbf{a}}_2\end{array}\right)\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right)=u_1\left(\begin{array}{c}{a}_{11}\\ a_{21}\end{array}\right)+u_2\left(\begin{array}{c}{a}_{12}\\ a_{22}\end{array}\right).$$

Recall the standard basis vectors are ${\mathbf{e}}_1=\left(\begin{array}{c}1\\ 0\end{array}\right)$ and ${\mathbf{e}}_2=\left(\begin{array}{c}0\\ 1\end{array}\right)$. So:

$$A{\mathbf{e}}_1=\left(\begin{array}{c}{a}_{11}\\ a_{21}\end{array}\right),A{\mathbf{e}}_2=\left(\begin{array}{c}{a}_{12}\\ a_{22}\end{array}\right).$$

Similarly, in general:

$$C{\mathbf{e}}_i={\mathbf{c}}_i=\left(\begin{array}{c}{c}_{1i}\\ c_{2i}\\ ⋮\\ c_{ni}\end{array}\right).$$ Link to original

Diagonal matrices: stretching

Diagonal matrices - stretching

The main diagonal of a matrix $A=(a_{ij})$ consists of the entries $a_{11},a_{22},\dots ,a_{nn}$. A matrix is called diagonal when all entries are zero except on the main diagonal. The action of a diagonal matrix is to stretch each spatial dimension according to the corresponding diagonal number:

$$\left(\begin{array}{ccc}{a}_1& 0& 0\\ 0& a_2& 0\\ 0& 0& a_3\end{array}\right)\left(\begin{array}{c}x\\ y\\ z\end{array}\right)=\left(\begin{array}{c}{a}_{1}x\\ a_{2}y\\ a_{3}z\end{array}\right).$$

This action is very easy to express algebraically in terms of standard basis vectors:

$$A{\mathbf{e}}_1=a_1{\mathbf{e}}_1,A{\mathbf{e}}_2=a_2{\mathbf{e}}_2,A{\mathbf{e}}_3=a_3{\mathbf{e}}_3.$$

In other words, standard basis vectors are sent to scalar multiples of themselves. This means that standard basis vectors are eigenvectors (“self-vectors”) for a diagonal matrix, and the diagonal numbers (the scalar multiples) are eigenvalues (“self-values”).

When the diagonal entries are all the same, the effect is to scale all of space by equal proportions:

$$\left(\begin{array}{ccc}a& 0& 0\\ 0& a& 0\\ 0& 0& a\end{array}\right)\left(\begin{array}{c}x\\ y\\ z\end{array}\right)=\left(\begin{array}{c}ax\\ ay\\ az\end{array}\right)=a\left(\begin{array}{c}x\\ y\\ z\end{array}\right).$$

In this case, when $a>1$ the transformation is called a dilation because vectors expand away from the origin, and when $a<1$ it is called a contraction because vectors shrink towards the origin.

Link to original

Question

02

Stretching and area

Consider the unit square $[0,1]\times [0,1]$. What is the area of the image of this square under the action of the matrix $\left(\begin{array}{cc}{a}_1& 0\\ 0& a_2\end{array}\right)$?

Link to original

Shearing

Shearing

A simple shear mapping (also shear transformation or transvection) performs a displacement of one coordinate by a proportion of another. The matrix of a 2D shear in $x$ is given by:

$$\left(\begin{array}{cc}1& m\\ 0& 1\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)=\left(\begin{array}{c}x+my\\ y\end{array}\right).$$

Horizontal lines remain horizontal, while vertical lines take on a new slope of $1/m$. Interestingly, shearing preserves area. (We will see a very simple explanation later.)

There is also shear in $y$:

$$\left(\begin{array}{cc}1& 0\\ m& 1\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)=\left(\begin{array}{c}x\\ y+mx\end{array}\right).$$

A shear in 3D might look like this:

$$\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& m\\ 0& 0& 1\end{array}\right)\left(\begin{array}{c}x\\ y\\ z\end{array}\right)=\left(\begin{array}{c}x\\ y+mz\\ z\end{array}\right).$$

The shear takes place in the component corresponding to the row of the matrix with $m$.
There is a general definition of shear mapping in $n$D space, but we do not use it.

Link to original

Rotating

Rotating

A rotation in 2D by the angle $\theta$ (counterclockwise) can be performed by a matrix:

$$\left(\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)=\left(\begin{array}{c}x\cos \theta -y\sin \theta \\ x\sin \theta +y\cos \theta \end{array}\right).$$ Link to original

Exercise

01

Rotation matrix

Using geometry and trig sum identities, verify that this is the correct formula for rotation of the point $(x,y)$ counterclockwise about the origin by the angle $\theta$.

Link to original

Rotating (part 2)

From the matrix for rotation by $\theta$ we can deduce that counterclockwise rotation by $90^{\circ }$ is given in matrix form by $\left(\begin{array}{cc}0& -1\\ 1& 0\end{array}\right)$, which acts by $(x,y)\mapsto (-y,x)$, and clockwise rotation by $90^{\circ }$ is given in matrix form by $\left(\begin{array}{cc}0& 1\\ -1& 0\end{array}\right)$, which acts by $(x,y)\mapsto (y,-x)$. Rotation by $180^{\circ }$ is given by $\left(\begin{array}{cc}-1& 0\\ 0& -1\end{array}\right)$, which acts by $(x,y)\mapsto (-x,-y)$, and this is simply reflection through the origin.

Study of 3D rotations has myriad applications in science and engineering, but rotations in 3D are much more complicated than in 2D, and we do not investigate the theory very much here. Some basic rotations are given by simple 2D rotations in the separate pairs of coordinates:

$$R_{xy}(\theta )=\left(\begin{array}{ccc}\cos \theta & -\sin \theta & 0\\ \sin \theta & \cos \theta & 0\\ 0& 0& 1\end{array}\right),R_{xz}(\theta )=\left(\begin{array}{ccc}\cos \theta & 0& -\sin \theta \\ 0& 1& 0\\ \sin \theta & 0& \cos \theta \end{array}\right),R_{yz}(\theta )=\left(\begin{array}{ccc}1& 0& 0\\ 0& \cos \theta & -\sin \theta \\ 0& \sin \theta & \cos \theta \end{array}\right)$$

These rotations are called yaw, pitch, and roll, respectively. (These terms apply to an airplane flying in the $xy$-plane in the $+x$ axis direction.) Simple rotations like these in two chosen axes are also called Givens rotations (after J. Wallace Givens). They are easily generalized to $n$D space.

Link to original

Question

03

Positive pitch

Verify from the formulas that a positive pitch means an airplane is ascending. Does a positive roll imply an airplane is tilting to the right or to the left?

Link to original

Projecting

Projecting

A projection matrix acts on vectors by sending them to the nearest point in some chosen linear subspace, for example on a line (in 2D space) or on a plane (in 3D space). The chosen subspace has lower dimension than all of space, and the image under projection is like a shadow of the original.

Consider the projection of $\mathbf{u}$ to the direction of $\mathbf{v}$. Let us assume that $\mathbf{v}$ is a unit vector; then the projection formula simplifies to:

$${\mathrm{proj}}_{\mathbf{v}}(\mathbf{u})=\mathbf{v}(\mathbf{v}\cdot \mathbf{u}).$$

We can express this formula using a matrix. In 2D we have:

$$(\mathbf{v}\cdot \mathbf{u})=\left(\begin{array}{cc}{v}_1& v_2\end{array}\right)\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right)=v_1{u}_1+v_2{u}_2,$$

and

$$\mathbf{v}(\mathbf{v}\cdot \mathbf{u})=\left(\begin{array}{c}{v}_1(\mathbf{v}\cdot \mathbf{u})\\ v_2(\mathbf{v}\cdot \mathbf{u})\end{array}\right)=\left(\begin{array}{c}{v}_1(v_1{u}_1+v_2{u}_2)\\ v_2(v_1{u}_1+v_2{u}_2)\end{array}\right)=\left(\begin{array}{c}{v}_1{v}_1{u}_1+v_1{v}_2{u}_2\\ v_2{v}_1{u}_1+v_2{v}_2{u}_2\end{array}\right)=\left(\begin{array}{cc}{v}_1{v}_1& v_1{v}_2\\ v_2{v}_1& v_2{v}_2\end{array}\right)\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right).$$

The matrix $\left(\begin{array}{cc}{v}_1{v}_1& v_1{v}_2\\ v_2{v}_1& v_2{v}_2\end{array}\right)$ actually has a name: it is called the outer product $\mathbf{v}{\mathbf{v}}^{\top }$, where ${\mathbf{v}}^{\top }$ is the matrix $(v_1{v}_2)$. In more detail: ${\mathbf{v}}^{\top }$ is a $1\times 2$ row vector matrix which, when applied to any vector $\mathbf{u}$, gives the scalar quantity $\mathbf{v}\cdot \mathbf{u}$. This horizontal matrix can be embedded inside each row of the vector $\mathbf{v}$ by multiplying ${\mathbf{v}}^{\top }$ by the component in that row of $\mathbf{v}$ and substituting the result into the row, thereby converting the vector $\mathbf{v}$ into a matrix. The action of this matrix on $\mathbf{u}$ is first to take the dot product $\mathbf{v}\cdot \mathbf{u}$, and then to apply this scalar to each row of $\mathbf{v}$, in other words multiplying $\mathbf{v}$ overall by this scalar.

In conclusion, the operation ${\mathrm{proj}}_{\mathbf{v}}(-)$ is a linear mapping, and it can be represented as the action of a matrix:

$$\left[{\mathrm{proj}}_{\mathbf{v}}\right]=\left(\begin{array}{cc}{v}_1{v}_1& v_1{v}_2\\ v_2{v}_1& v_2{v}_2\end{array}\right).$$

Now if $\mathbf{v}$ is not a unit vector, this matrix is incorrect. The matrix can be corrected by multiplying every coefficient by $(\mathbf{v}\cdot \mathbf{v}{)}^{-1}$, which is $1/(v_1{v}_1+v_2{v}_2)$.

Link to original

Exercise

02

Computing a projection matrix

Find the matrix of the projection of ${\mathbb{R}}^2$ in the direction of the vector $\mathbf{v}=(3,-4)$. (First compute the unit vector.)

Link to original

Exercise

03

Double projections

Consider a unit vector $\mathbf{v}$ and another vector $\mathbf{u}$. Show that ${\mathrm{proj}}_{\mathbf{v}}({\mathrm{proj}}_{\mathbf{v}}(\mathbf{u}))={\mathrm{proj}}_{\mathbf{v}}(\mathbf{u})$. Do this by applying the matrix of the projection of $\mathbf{v}$ to the vector $\mathbf{u}$ twice in sequence (meaning you apply the matrix again on the output vector of the first application of the matrix). To simplify the result, remember that $\mathbf{v}\cdot \mathbf{v}=1$ because $\mathbf{v}$ is a unit vector.

Link to original

Projecting (part 2)

The same matrix formula can be extended to 3D and $n$D. In 3D the matrix would be:

$$\left(\begin{array}{c}{v}_1\\ v_2\\ v_3\end{array}\right)\left(\begin{array}{ccc}{v}_1& v_2& v_3\end{array}\right)=\left(\begin{array}{ccc}{v}_1{v}_1& v_1{v}_2& v_1{v}_3\\ v_2{v}_1& v_2{v}_2& v_2{v}_3\\ v_3{v}_1& v_3{v}_2& v_3{v}_3\end{array}\right).$$

For $\mathbf{v}$ a unit vector, this matrix acts on $\mathbf{u}$ by giving ${\mathrm{proj}}_{\mathbf{v}}(\mathbf{u})$, the projection to the direction of $\mathbf{v}$. We can see this because it clearly gives $\mathbf{v}(\mathbf{v}\cdot \mathbf{u})$.

It is good to highlight the matrices of projection to the standard basis vectors:

$$\left[{\mathrm{proj}}_{{\mathbf{e}}_1}\right]=\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right),\left[{\mathrm{proj}}_{{\mathbf{e}}_2}\right]=\left(\begin{array}{cc}0& 0\\ 0& 1\end{array}\right).$$ Link to original

Question

04

Projection to standard basis

Check that ${\mathrm{proj}}_{{\mathbf{e}}_1}$ sends $(x,y)\mapsto (x,0)$ and ${\mathrm{proj}}_{{\mathbf{e}}_2}$ sends $(x,y)\mapsto (0,y)$. What is the matrix of ${\mathrm{proj}}_{{\mathbf{e}}_i}$ in 3D, for $i=1,2,3$?

Link to original

Reflecting

Reflecting

A reflection matrix is a matrix that acts on vectors by reflecting them across a hyperplane that passes through the origin. These operations are called Householder reflections.

(A hyperplane is a plane of one dimension lower than the surrounding space. So, a hyperplane in 2D is a line, and a hyperplane in 3D is a 2D plane. A hyperplane in $n$D would be an $(n-1)$D plane.)

The matrix of a reflection can be derived from the matrix of a certain projection by considering a formula for the reflection in terms of dot products.

If $\mathbf{u}$ is any vector, the reflection of $\mathbf{u}$ in the hyperplane through the origin, perpendicular to the vector $\mathbf{v}$ is given by:

$${\mathrm{refl}}_{\mathbf{v}}(\mathbf{u})=\mathbf{u}-2{\mathrm{proj}}_{\mathbf{v}}(\mathbf{u}).$$

To see how the formula works, consider that $\mathbf{u}-{\mathrm{proj}}_{\mathbf{v}}(\mathbf{u})={\mathbf{u}}_{\perp }$ is the component of $\mathbf{u}$ perpendicular to $\mathbf{v}$, meaning that it lies in the hyperplane of reflection. Adding ${\mathbf{u}}_{\parallel }$ to ${\mathbf{u}}_{\perp }$ recovers $\mathbf{u}$ by bringing back the parallel part on the same side of the hyperplane as $\mathbf{v}$. Subtracting the term ${\mathbf{u}}_{\parallel }$ a second time does the opposite, it restores the part parallel to $\mathbf{v}$ but on the other side of the hyperplane.

Reasoning via the hyperplane is not strictly necessary in 2D, but the same reasoning also works in $n$D. In 3D the same formula gives the reflection across the plane through the origin, perpendicular to the vector $\mathbf{v}$. Here, ${\mathrm{proj}}_{\mathbf{v}}(\mathbf{u})$ gives the component of $\mathbf{u}$ coming out of this plane in along $\mathbf{v}$. Subtracting once gives a vector inside the plane; subtracting twice gives the mirror image vector on the opposite side of the plane.

A 2D reflection in the hyperplane perpendicular to $\mathbf{v}$ can be represented as the action of a matrix:

$$\begin{array}{rl}\left[{\mathrm{refl}}_{\mathbf{v}}\right]& =\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)-2\left[{\mathrm{proj}}_{\mathbf{v}}\right]\\ & =\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)-2\left(\begin{array}{cc}{v}_1{v}_1& v_1{v}_2\\ v_2{v}_1& v_2{v}_2\end{array}\right)\\ & =\left(\begin{array}{cc}1-2v_1{v}_1& -2v_1{v}_2\\ -2v_2{v}_1& 1-2v_2{v}_2\end{array}\right).\end{array}$$

The action of this matrix sends a vector $\mathbf{u}$ to the vector $\mathbf{u}-2{\mathrm{proj}}_{\mathbf{v}}(\mathbf{u})$. (Check this fact yourself!)

Link to original

Exercise

04

Computing a reflection matrix

Compute the matrix representing 3D reflection in the plane perpendicular to the vector $(2,3,6)$. (First find the unit vector!)

Link to original

Permuting

Permuting

A permutation is a way of reassigning some labels to a collection of items. For example, if some marbles numbered $1,\dots ,n$ are placed (out of order) in bins numbered $1,\dots ,n$, then a permutation corresponds to withdrawing the marbles and placing them in the same bins but in a new order.

A permutation matrix is a matrix in which each column and each row has a single $1$, and all other entries are $0$. For example:

$$\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right),\left(\begin{array}{ccc}0& 1& 0\\ 1& 0& 0\\ 0& 0& 1\end{array}\right),\left(\begin{array}{cccc}0& 0& 0& 1\\ 1& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 1& 0& 0\end{array}\right).$$

The action of a permutation matrix on a vector is simply to permute the rows of the vector:

$$\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)=\left(\begin{array}{c}{v}_2\\ v_1\end{array}\right),\left(\begin{array}{ccc}0& 1& 0\\ 1& 0& 0\\ 0& 0& 1\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\\ v_3\end{array}\right)=\left(\begin{array}{c}{v}_2\\ v_1\\ v_3\end{array}\right),$$ $$\left(\begin{array}{cccc}0& 0& 0& 1\\ 1& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 1& 0& 0\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\\ v_3\\ v_4\end{array}\right)=\left(\begin{array}{c}{v}_4\\ v_1\\ v_3\\ v_2\end{array}\right).$$

Such a matrix acts by permuting rows because each row, with its single entry of $1$, “picks out” the entry of the vector whose row corresponds to the column of this $1$ (all others are given times $0$); and since each column of the matrix has only a single $1$, each row of the vector is “picked out” only one time.

The action of a permutation matrix on standard basis vectors is easy to describe: it permutes them.

Link to original

Exercise

05

Finding a permutation matrix

Find the permutation matrix that acts as follows:

$$\left(\begin{array}{c}{a}_2\\ a_4\\ a_1\\ a_5\\ a_3\end{array}\right)⟼\left(\begin{array}{c}{a}_1\\ a_2\\ a_3\\ a_4\\ a_5\end{array}\right).$$

Link to original

Tutorial: Theory of Matrices in 2D and 3D

Matrix multiplication

Matrix multiplication

Suppose we have a matrix $A$ and a vector $u$ that $A$ acts upon. We have written “$Au$” for the output vector given by letting $A$ act on $u$.

Matrices act by sending vectors to other vectors. It is useful to consider another matrix that acts upon the output vector. For example, if $\mathbf{v}:=A\mathbf{u}$, and if $B$ is a matrix that acts upon $v$, we can write $B(A\mathbf{u})=B\mathbf{v}$. (The colon in $\mathbf{v}:=A\mathbf{u}$ means “$\mathbf{v}$ is defined as $A\mathbf{u}$.”)

Suppose we define $\mathbf{w}:=B\mathbf{v}$, so that $\mathbf{w}=B(A\mathbf{u})$. Then we say that $w$ is the output of $\mathbf{u}$ under a composition of actions of the matrices $A$ and $B$. It turns out that if we start with two matrices $A$ and $B$, the composition action (letting the matrices act consecutively) can itself be represented by a single matrix. (The composition of linear transformations is a linear transformation.) Let us find this matrix in the case when $A$ and $B$ are $2\times 2$ matrices.

Suppose the entries of $A$ and $B$ are given by

$$A=\left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right),B=\left(\begin{array}{cc}{b}_{11}& b_{12}\\ b_{21}& b_{22}\end{array}\right).$$

Now compute the action of $A$ followed by the action of $B$ on the vector $\mathbf{u}=\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right)$:

$$\begin{array}{rl}B(A\mathbf{u})& =\left(\begin{array}{cc}{b}_{11}& b_{12}\\ b_{21}& b_{22}\end{array}\right)\left(\left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right)\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right)\right)\\ & =\left(\begin{array}{cc}{b}_{11}& b_{12}\\ b_{21}& b_{22}\end{array}\right)\left(\begin{array}{c}{a}_{11}{u}_1+a_{12}{u}_2\\ a_{21}{u}_1+a_{22}{u}_2\end{array}\right)\\ & =\left(\begin{array}{c}{b}_{11}(a_{11}{u}_1+a_{12}{u}_2)+b_{12}(a_{21}{u}_1+a_{22}{u}_2)\\ b_{21}(a_{11}{u}_1+a_{12}{u}_2)+b_{22}(a_{21}{u}_1+a_{22}{u}_2)\end{array}\right)\\ & =\left(\begin{array}{c}(b_{11}{a}_{11}+b_{12}{a}_{21})u_1+(b_{11}{a}_{12}+b_{12}{a}_{22})u_2\\ (b_{21}{a}_{11}+b_{22}{a}_{21})u_1+(b_{21}{a}_{12}+b_{22}{a}_{22})u_2\end{array}\right)\\ & =\left(\begin{array}{cc}{b}_{11}{a}_{11}+b_{12}{a}_{21}& b_{11}{a}_{12}+b_{12}{a}_{22}\\ b_{21}{a}_{11}+b_{22}{a}_{21}& b_{21}{a}_{12}+b_{22}{a}_{22}\end{array}\right)\left(\begin{array}{c}{u}_1\\ u_2\end{array}\right).\end{array}$$

Therefore, we define the product matrix $BA$ by this formula:

$$BA=\left(\begin{array}{cc}{b}_{11}& b_{12}\\ b_{21}& b_{22}\end{array}\right)\left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right)=\left(\begin{array}{cc}{b}_{11}{a}_{11}+b_{12}{a}_{21}& b_{11}{a}_{12}+b_{12}{a}_{22}\\ b_{21}{a}_{11}+b_{22}{a}_{21}& b_{21}{a}_{12}+b_{22}{a}_{22}\end{array}\right).$$

With this definition and the above reasoning, we have that $B(A\mathbf{u})=(BA)\mathbf{u}$.
(Because of this equality, we can omit the parentheses and write simply $BA\mathbf{u}$, since the result is the same whether you act first by $A$ and then by $B$ or instead you first multiply the matrices and then act by $BA$.)

Just as for the formula for a matrix acting on a vector, we have multiple ways to interpret the formula for the matrix $BA$.

• Each entry of the product matrix $BA$ is given by taking the dot product of a row of $B$ with a column of $A$.
• Each column of the product matrix $BA$ is given by letting $B$ act upon the corresponding column vector of $A$. In other words, $BA=\left(\begin{array}{cc}B{\mathbf{a}}_1& B{\mathbf{a}}_2\end{array}\right)$. (Recall that the action of $B$ upon ${\mathbf{a}}_i$ has two interpretations of its own!)
• Each row of the product matrix $BA$ is given by letting the entries in the corresponding row vector of $B$ provide the coefficients for a linear combination of the rows of $A$.
  The first of these interpretations is best for doing calculations, and the second is best for understanding abstract theory.

Bottom-Right Method for computing $AB$:

E.g.:

Max Born, Nobel Prize lecture:

Link to original

Question

01

Some matrix products

Compute the following matrix products by hand:

• (a) $\left(\begin{array}{cc}2& 3\\ 1& -5\end{array}\right)\left(\begin{array}{ccc}4& 3& 6\\ 1& -2& 3\end{array}\right)$
• (b) $\left(\begin{array}{ccc}1& 3& -5\\ 0& -2& 1\\ 0& 0& 4\end{array}\right)\left(\begin{array}{ccc}2& 1& 1\\ 0& -1& 3\\ 0& 0& 2\end{array}\right)$

In both (a) and (b) there are three matrices: two are given and their product is the third. In (a), what size vectors do these three matrices act upon? (How many components?) In (b), what do you notice about the nature of the two matrices and their resulting product? Can you guess a rule that generalizes this fact?

Link to original

Question

02

Double rotation

Show that ${\left(\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right)}^2=\left(\begin{array}{cc}\cos (2\theta )& -\sin (2\theta )\\ \sin (2\theta )& \cos (2\theta )\end{array}\right)$, in other words two rotations by $\theta$ is equivalent to one rotation by $2\theta$.

Link to original

Exercise

01

Nilpotent matrix

Define the matrix $A_{\lambda }$:

$$A_{\lambda }=\left(\begin{array}{ccccc}0& \lambda & 0& 0& 0\\ 0& 0& \lambda & 0& 0\\ 0& 0& 0& \lambda & 0\\ 0& 0& 0& 0& \lambda \\ 0& 0& 0& 0& 0\end{array}\right).$$

For $\lambda =2$, calculate the powers $A_2$, $A_2^2$, $A_2^3$, $A_2^4$, $A_2^5$.

(What special property does $A_{\lambda }$ have? A matrix with that property is said to be nilpotent.)

Link to original

Matrix multiplication (part 2)

Identity matrix
Notice that the diagonal matrix with $1$’s on the diagonal acts upon another vector or matrix by doing nothing:

$$\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)\left(\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right)=\left(\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right).$$

This matrix is called the identity matrix, since it preserves the identity of whatever it acts upon. It is frequently written “$I$”, regardless of the size of the matrix. So $I\mathbf{u}=\mathbf{u}$ for any $\mathbf{u}$.

Sometimes $I$ is written more specifically as $I_n$, where $n$ is the number of rows or columns. (Identity matrices always have the same number of rows and columns.)

Link to original

Exercise

02

Reflections square to identity

Show that ${\left[{\mathrm{refl}}_{\mathbf{v}}\right]}^2=I_2$ for any $\mathbf{v}=\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)$ with $\mathbf{v}\cdot \mathbf{v}=1$.

This means that reflecting twice across the same line gives back the original vector. (In 2D the hyperplane of reflection is just a line.)

Link to original

Matrices acting on matrices

Matrices acting on matrices

It is useful to think of matrices as acting upon other matrices by multiplying on the left. The effect is the same as the effect of acting on a vector, but it performs this (same) action on each column vector separately. The action is therefore the same across any given row.

Diagonal matrices
When two diagonal matrices are multiplied, the result is a diagonal matrix. The diagonal entries of the product are simply the products of the diagonal entries:

$$\left(\begin{array}{ccc}{a}_1& 0& 0\\ 0& a_2& 0\\ 0& 0& a_3\end{array}\right)\left(\begin{array}{ccc}{b}_1& 0& 0\\ 0& b_2& 0\\ 0& 0& b_3\end{array}\right)=\left(\begin{array}{ccc}{a}_1{b}_1& 0& 0\\ 0& a_2{b}_2& 0\\ 0& 0& a_3{b}_3\end{array}\right).$$

A consequence is that diagonal matrices commute with each other: $AB=BA$ when $A,B$ are both diagonal. (As seen with 3D rotations, frequently this is not true for matrix multiplication.)

Row-scale matrices
The diagonal matrices

$$A_1(\lambda )=\left(\begin{array}{ccc}\lambda & 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right),A_2(\lambda )=\left(\begin{array}{ccc}1& 0& 0\\ 0& \lambda & 0\\ 0& 0& 1\end{array}\right),A_3(\lambda )=\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& \lambda \end{array}\right)$$

have the effect of scaling rows $1,2,3$ (respectively) by the number $\lambda$, without changing other rows, when they are multiplied on a vector or a matrix on the left. These matrices are called row-scale matrices.

Permutation matrices
When two permutation matrices are multiplied, the result is a permutation matrix. For example:

$$\left(\begin{array}{ccc}0& 1& 0\\ 1& 0& 0\\ 0& 0& 1\end{array}\right)\left(\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& 1& 0\end{array}\right)=\left(\begin{array}{ccc}0& 0& 1\\ 1& 0& 0\\ 0& 1& 0\end{array}\right).$$ Link to original

Question

03

Permutation matrices

Explain why the product of two permutation matrices is always a permutation matrix. (Recall the definition: a matrix is a permutation matrix when each row and each column contains exactly one $1$ and the other entries are $0$.)

Link to original

Matrices acting on matrices (part 2)

Row-swap matrices
A permutation matrix acts upon another matrix (multiplying on the left) by permuting the rows of the matrix, in the same way that it would permute the rows of a vector:

$$\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)\left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right)=\left(\begin{array}{cc}{a}_{21}& a_{22}\\ a_{11}& a_{12}\end{array}\right).$$

The most important permutation type is one which simply swaps two rows and leaves the others unchanged. This type of matrix is called a row-swap matrix.

The row-swap action is performed by a permutation matrix which has $1$’s on the diagonal except for two rows. In these two rows, the ones from a diagonal matrix are swapped with each other.

Link to original

Example

Row-swap example

Row-swap example

The $4\times 4$ matrix which swaps rows $2$ and $4$ (of a vector or another matrix) works like this:

$$\left(\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 0& 1\\ 0& 0& 1& 0\\ 0& 1& 0& 0\end{array}\right)\left(\begin{array}{cccc}{a}_{11}& a_{12}& a_{13}& a_{14}\\ a_{21}& a_{22}& a_{23}& a_{24}\\ a_{31}& a_{32}& a_{33}& a_{34}\\ a_{41}& a_{42}& a_{43}& a_{44}\end{array}\right)=\left(\begin{array}{cccc}{a}_{11}& a_{12}& a_{13}& a_{14}\\ a_{41}& a_{42}& a_{43}& a_{44}\\ a_{31}& a_{32}& a_{33}& a_{34}\\ a_{21}& a_{22}& a_{23}& a_{24}\end{array}\right).$$

To obtain this row-swap matrix, start with the diagonal matrix with $1$’s along the diagonal, and then swap rows $2$ and $4$.

Link to original

Matrices acting on matrices (part 3)

Row-add matrices
A matrix that has $1$’s on the diagonal and the other entries $0$’s, except for a single additional entry $\lambda$, is called an elementary shear matrix, or a row-add matrix:

$$\left(\begin{array}{ccc}1& \lambda & 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right),\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ \lambda & 0& 1\end{array}\right),\text{etc.}$$

If the $\lambda$ entry is in the $i^{th}$ row and $j^{th}$ column, this matrix acts by taking the $j^{th}$ row (of a matrix or vector that it’s acting upon) and adding it into the $i^{th}$ row with a $\lambda$ coefficient:

$$\left(\begin{array}{ccc}1& \lambda & 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)\left(\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right)=\left(\begin{array}{ccc}{a}_{11}+\lambda a_{21}& a_{12}+\lambda a_{22}& a_{13}+\lambda a_{23}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right).$$

Remember that when a matrix is multiplied on the left on a vector or a matrix, the top row of the matrix controls the top row of the output.

Link to original

Matrix algebra

Matrix algebra

Matrix addition and scaling
Matrices can be added, subtracted, and multiplied by scalers, just like vectors. These operations are done componentwise:

$$\begin{array}{rl}\left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right)+\left(\begin{array}{cc}{b}_{11}& b_{12}\\ b_{21}& b_{22}\end{array}\right)& =\left(\begin{array}{cc}{a}_{11}+b_{11}& a_{12}+b_{12}\\ a_{21}+b_{21}& a_{22}+b_{22}\end{array}\right),\\ \lambda \left(\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right)& =\left(\begin{array}{cc}\lambda a_{11}& \lambda a_{12}\\ \lambda a_{21}& \lambda a_{22}\end{array}\right).\end{array}$$

In terms of the actions performed by these matrices, addition is like addition of functions: the action of a sum of matrices is to give the sum of the outputs under the actions of the separate matrices. For functions, this would be written $(f+g)(x)=f(x)+g(x)$. For matrices, we write $(A+B)\mathbf{u}=A\mathbf{u}+B\mathbf{u}$, and similarly for subtraction, scalar multiples, and other variations on basic algebra operations.

Matrix distributivity
It is important to notice that matrix multiplication is compatible with matrix addition in the usual sense of distributivity:

$$A(B+C)=AB+AC.$$

One way to prove this formula is to show that it is true after both sides are applied to an arbitrary vector $\mathbf{u}$. (To complete the proof, let $\mathbf{u}$ take the specific value of each standard basis vector ${\mathbf{e}}_i$ in turn, and you find that column $i$ of the matrices on each side must agree.) To show the equality after multiplying both sides on an arbitrary $\mathbf{u}$, we need to check distributivity of matrix actions on vectors in general:

Link to original

Question

04

Linearity / Distributivity of matrix actions

Show that $A(\mathbf{u}+\mathbf{v})=A\mathbf{u}+A\mathbf{v}$ whenever a matrix $A$ acts on vectors $\mathbf{u}$ and $\mathbf{v}$.

Hint: first try this with a $3\times 3$ matrix and 3D vector with arbitrary entries. Then to show it in general, use the “summation notation” for the matrix action on a vector:

$$(A\mathbf{u}{)}_i=\sum _{j=1}^n{a}_{ij}{u}_j.$$

Link to original

Matrix algebra (part 2)

Matrix division
For real numbers, division and multiplication are inverse processes, meaning that division “undoes” the action of multiplication, and vice versa.

Any number can divide another number provided it is not zero. For matrices, the situation is more complex, because “being zero” is more complex. Many matrices have non-zero entries, yet they “act as zero” upon certain other vectors. When this happens, the action cannot be “undone”, because you cannot divide by the zero action performed on those vectors.

Link to original

Example

Projection is not invertible

Projection is not invertible

Consider the projection matrix $\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right)$ which sends a vector $(x,y)$ to the vector $(x,0)$. This matrix acts by zero upon any vector $(0,y)$, sending $(0,y)$ to $(0,0)$. This action cannot be undone! (What specific vector would the inverse action send $(0,0)$ to?)

Link to original

Matrix algebra (part 3)

So, some matrices can be divisors. These are called invertible matrices. Others cannot, and these are called non-invertible matrices. A matrix $A$ is invertible when the inverse matrix $A^{-1}$ exists.

When it exists, an inverse matrix $A^{-1}$ acts by undoing multiplication by $A$. Concretely, this means:

$$A^{-1}A\mathbf{u}=\mathbf{u},A{A}^{-1}\mathbf{u}=\mathbf{u}\text{for any vector }\mathbf{u}.$$

Notice in the second formula that the action of $A$ can be undone before $A$ even acts. (Ordinary division also works like this!) Another way to look at it is that $A$ acts by undoing the effect of $A^{-1}$. (Ordinary division: multiplication by $5$ undoes the effect of multiplying by $1/5$.)

Recall the “identity matrix” $I$ which satisfies $I\mathbf{u}=\mathbf{u}$ for any $\mathbf{u}$. So we can write the above formula using matrices:

$$A^{-1}A=I,A{A}^{-1}=I.$$

Notice that we have to write both formulas because matrix multiplication is not commutative! (Ordinary division: $5\cdot (1/5)=1$ and $(1/5)\cdot 5=1$ are equivalent because $ab=ba$ for any numbers.)

Inverse of $2\times 2$ matrices
There is a general formula for the inverse of any $2\times 2$ matrix that should be memorized:

$$A=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right),A^{-1}=\frac{1}{ad-bc}\left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right).$$ Link to original

Exercise

03

Checking formula for $2\times 2$ inverse

Check the formula for $A^{-1}$ for a $2\times 2$ matrix $A$ by computing by hand $A{A}^{-1}$ and $A^{-1}A$. (You should obtain $\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)$ in both cases.)

Which $2\times 2$ matrices $A$ have an inverse?

(Hint: you just checked a formula that works whenever it makes sense, so there is an inverse whenever it makes sense, given by the formula. Suppose $a,b,c,d$ are such that the formula does not make sense. Can you find a vector $\mathbf{u}\ne 0$ which is sent to $0$ by $A$?)

Link to original

Exercise

04

Inverse of a reflection

Show (in 2D) that the matrix of a reflection is the inverse of itself. Show this first abstractly (using the definition of inverse), and then concretely (using the formulas for reflection and inverse matrices).

Link to original

Exercise

05

Inverse of a rotation

Show (in 2D) that the inverse of the matrix of rotation by $\theta$ is the matrix of rotation by $-\theta$.

Link to original

Matrix algebra (part 4)

The inverse of $A$ is useful for finding a vector $\mathbf{u}$ which is sent to a given vector $\mathbf{v}$ by the action of a given matrix $A$:

$$A\mathbf{u}=\mathbf{v}\text{if and only if}\mathbf{u}=A^{-1}\mathbf{v}.$$

If we know $A$ and $\mathbf{v}$, and we want $\mathbf{u}$, this method will give us $\mathbf{u}$.

Link to original

Example

Inverse matrix to find preimage

Inverse matrix to find preimage

Problem: Define

$$A=\left(\begin{array}{cc}1& 2\\ -2& -3\end{array}\right),\mathbf{v}=\left(\begin{array}{c}1\\ 4\end{array}\right).$$

Find a vector $\mathbf{u}$ with the property that $A\mathbf{u}=\mathbf{v}$, meaning that $A$ sends $\mathbf{u}$ to the given $\mathbf{v}$.

Solution: First we compute $A^{-1}$ using the formula. We have $ad-bc=-3--4=1$, so $A^{-1}=\left(\begin{array}{cc}-3& -2\\ 2& 1\end{array}\right)$. Now compute that $A^{-1}\mathbf{v}=\left(\begin{array}{c}-11\\ 6\end{array}\right)$, and this is our answer for $\mathbf{u}$. Check this answer by computing $A\left(\begin{array}{c}-11\\ 6\end{array}\right)=\left(\begin{array}{c}1\\ 4\end{array}\right)$, as expected.

Link to original

Exercise

06

Inverse matrix to find preimage

Repeat the previous example for this data:

$$A=\left(\begin{array}{cc}1& -3\\ -2& 4\end{array}\right),\mathbf{v}=\left(\begin{array}{c}-4\\ 2\end{array}\right).$$

Link to original

Matrix algebra (part 5)

Determinant of $2\times 2$ matrices
The determinant of a matrix $A$, written $\det A$, is a single number that is associated to the matrix.

For a $2\times 2$ matrix, the determinant is given by the formula in the divisor of the formula for $A^{-1}$:

$$\det A=ad-bc\text{when}A=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right).$$

We will learn many things about determinants. The important thing for now is that $A^{-1}$ exists if and only if $\det A\ne 0$.

Optional: interpretation of $2\times 2$ determinant
Just for fun, here is a visual proof (by S. Golomb) that $\det A$ gives the area of the parallelogram spanned by the column vectors of $A$, namely $\left(\begin{array}{c}a\\ c\end{array}\right)$ and $\left(\begin{array}{c}b\\ d\end{array}\right)$:

Here is another visual for parallelograms from obtuse vectors:

Link to original

Eigenvectors, eigenvalues

Eigenvectors, eigenvalues

We saw that standard basis vectors ${\mathbf{e}}_i$ are eigenvectors for diagonal matrices, and the diagonal entries are the eigenvalues. If $A$ has diagonal entries $a_1,a_2,a_3$, then $A{\mathbf{e}}_i=a_i{\mathbf{e}}_i$.

Now consider the problem of finding eigenvectors and eigenvalues for a non-diagonal matrix $A$. This means finding a vector $\mathbf{v}\ne 0$ and a scalar $\lambda$ such that $A\mathbf{v}=\lambda \mathbf{v}$.

Suppose we start with a hypothetical $\lambda$. Write $\lambda I_2$ for the matrix $\left(\begin{array}{cc}\lambda & 0\\ 0& \lambda \end{array}\right)$. So we wish to solve $A\mathbf{v}=(\lambda I_2)\mathbf{v}$, which is equivalent to $(A-\lambda I_2)\mathbf{v}=0$. Now write $A_{\lambda }:=A-\lambda I_2$. So we wish to find $\mathbf{v}$ such that $A_{\lambda }\mathbf{v}=0$.

Notice that if $A_{\lambda }$ is invertible, then there is no solution. (The only solution would be $\mathbf{v}=0$ and that is not allowed for an eigenvector.) The reason is that if $A_{\lambda }^{-1}$ exists, we can let it act upon both sides of the equation $A_{\lambda }\mathbf{v}=0$ to obtain $\mathbf{v}=A_{\lambda }^{-1}0=0$. Therefore, we only have a chance of solving this if $A_{\lambda }$ is non-invertible. Recall from above that $A_{\lambda }$ is non-invertible precisely when $\det A_{\lambda }=0$.

It turns out we can solve the equation if $\det A_{\lambda }=0$. Let us do this by hand for $2\times 2$ matrices.

Derivation that if $\det M=0$ then $M\mathbf{v}=0$ can be solved with $\mathbf{v}\ne 0$

Let $M=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$. Assume $\det M=0$, which means $ad-bc=0$ or $ad=bc$ or $c/a=d/b$. Write $x$ for the common ratio $c/a=d/b$. Thus $M=\left(\begin{array}{cc}a& b\\ xa& xb\end{array}\right)$, and now we wish to solve the equation:

$$\left(\begin{array}{cc}a& b\\ xa& xb\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right).$$

This means $a{v}_1+b{v}_2=0$ and $xa{v}_1+xb{v}_2=0$. Dividing the latter by $x$ we arrive at the former, so we can take any $(v_1,v_2)$ that satisfy $v_1/v_2=-b/a$. Let us choose $(v_1,v_2)=(-b,a)$. You can now check directly that it works:

$$\left(\begin{array}{cc}a& b\\ xa& xb\end{array}\right)\left(\begin{array}{c}-b\\ a\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right).$$

If $\det A_{\lambda }=0$, there must be a vector $\mathbf{v}\ne 0$ such that $A\mathbf{v}=\lambda \mathbf{v}$ for some scalar $\lambda$

Proof: Define $A=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$, and so

$$A_{\lambda }=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)-\left(\begin{array}{cc}\lambda & 0\\ 0& \lambda \end{array}\right)=\left(\begin{array}{cc}a-\lambda & b\\ c& d-\lambda \end{array}\right).$$

Suppose $\det A_{\lambda }=0$. By the formula for $\det$, this means $(a-\lambda )(d-\lambda )-bc=0$. That is a quadratic equation in $\lambda$ and could be rewritten as ${\lambda }^2+(-a-d)\lambda +(ad-bc)=0$. By the quadratic formula, the solutions are $\lambda =\frac{1}{2}\left(a+d\pm \sqrt{(a-d{)}^2+4bc}\right)$. Write these two solutions as ${\lambda }_1$ and ${\lambda }_2$.

So we have found ${\lambda }_1$ and ${\lambda }_2$ with $\det A_{{\lambda }_1}=0$ and $\det A_{{\lambda }_2}=0$. In either case we can solve for a ${\mathbf{v}}_1\ne 0$ such that $A_{{\lambda }_1}{\mathbf{v}}_1=0$ and a ${\mathbf{v}}_2$ such that $A_{{\lambda }_1}{\mathbf{v}}_2=0$ using the method in the box above.

Link to original

Example

Finding eigenvectors and eigenvalues

Finding eigenvectors and eigenvalues

Problem: Define $A=\left(\begin{array}{cc}1& 6\\ 5& 2\end{array}\right)$. Find the two eigenvalues and some (any) two corresponding eigenvectors of $A$.

Solution: First write $A_{\lambda }=\left(\begin{array}{cc}1-\lambda & 6\\ 5& 2-\lambda \end{array}\right)$. Then $\det A_{\lambda }=0$ means $(1-\lambda )(2-\lambda )-30=0$. Solve this quadratic equation to obtain ${\lambda }_1=7$ and ${\lambda }_2=-4$. We have found the two eigenvalues.

Next, write out the matrices

$$A_{{\lambda }_1}=\left(\begin{array}{cc}-6& 6\\ 5& -5\end{array}\right),A_{{\lambda }_2}=\left(\begin{array}{cc}5& 6\\ 5& 6\end{array}\right).$$

Our goal is to find ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$ such that $A_{{\lambda }_i}{\mathbf{v}}_i=0$. We can guess two solutions: let ${\mathbf{v}}_1=\left(\begin{array}{c}1\\ 1\end{array}\right)$ and let ${\mathbf{v}}_2=\left(\begin{array}{c}-6\\ 5\end{array}\right)$. To check that these are indeed eigenvectors of the original $A$ corresponding to ${\lambda }_1=7$ and ${\lambda }_2=-4$, compute:

$$\left(\begin{array}{cc}1& 6\\ 5& 2\end{array}\right)\left(\begin{array}{c}1\\ 1\end{array}\right)=\left(\begin{array}{c}7\\ 7\end{array}\right)=7\left(\begin{array}{c}1\\ 1\end{array}\right),\left(\begin{array}{cc}1& 6\\ 5& 2\end{array}\right)\left(\begin{array}{c}-6\\ 5\end{array}\right)=\left(\begin{array}{c}24\\ -20\end{array}\right)=-4\left(\begin{array}{c}-6\\ 5\end{array}\right).$$

You could also use the procedure in the gray box “Derivation.” For $A_{{\lambda }_1}$ we have $x=-5/6$, and for $A_{{\lambda }_2}$ we have $x=1$. Then we would get ${\mathbf{v}}_1=\left(\begin{array}{c}-6\\ -6\end{array}\right)$ and ${\mathbf{v}}_2=\left(\begin{array}{c}-6\\ 5\end{array}\right)$. Notice that the first eigenvector is a scalar multiple of ours above: $\left(\begin{array}{c}-6\\ -6\end{array}\right)=-6\left(\begin{array}{c}1\\ 1\end{array}\right)$.

This is true in general: if $A\mathbf{v}=\lambda \mathbf{v}$, then $A(x\mathbf{v})=\lambda (x\mathbf{v})$. In other words, you can scale an eigenvector and the result is still an eigenvector with the same eigenvalue. This is because $Ax=xA$ when $x$ is a scalar, and of course $x\lambda =\lambda x$ when both are scalars. In yet other words, the line $⟨\mathbf{v}⟩$ spanned by an eigenvector $\mathbf{v}$, called an eigenline, is preserved by the matrix: every vector on this line is mapped back to something on this line.

Link to original

Summary

Summary

• Matrices can be multiplied by each other. Express $A=\left(\begin{array}{ccc}{\mathbf{a}}_1& {\mathbf{a}}_2& {\mathbf{a}}_3\end{array}\right)$ as a row of column vectors; then $BA$ is the matrix which is the row of column vectors $\left(\begin{array}{ccc}B{\mathbf{a}}_1& B{\mathbf{a}}_2& B{\mathbf{a}}_3\end{array}\right)$. In other words, $BA$ is computed by letting $B$ act upon the columns of $A$.
• Matrices acting on vectors satisfy $A(\mathbf{u}+\mathbf{v})=A\mathbf{u}+A\mathbf{v}$, and acting on matrices they satisfy $A(B+C)=AB+AC$.
• The identity matrix $I_3=\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)$ has the property that $I_{3}A=A$ and $A{I}_3=A$ for any other matrix $A$.
• Row-scale matrices act on a matrix (or vector) by scaling a chosen row. Row-swap matrices act by swapping two rows. Row-add matrices act by adding one row into another.
• An “invertible” matrix $A$ is one having an inverse $A^{-1}$, which has the property that $A^{-1}A=A{A}^{-1}=I_n$. (Here $A$ is an $n\times n$ matrix.) For a $2\times 2$ matrix, its inverse is given by a formula: $$A=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right),A^{-1}=\frac{1}{ad-bc}\left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right).$$
• The inverse matrix $A^{-1}$ can be used to solve for $\mathbf{u}$ in the equation $A\mathbf{u}=\mathbf{v}$, where $A$ and $\mathbf{v}$ are both given and $\mathbf{u}$ is sought.
• The determinant $\det A$ for the $2\times 2$ matrix $A$ (as above) is the number $ad-bc$. The matrix $A$ is invertible if and only if $\det A\ne 0$.
• Given the matrix $A$, an eigenvector $\mathbf{v}$ is a vector ($\ne 0$) such that $A\mathbf{v}=\lambda \mathbf{v}$ for some scalar $\lambda$ called the eigenvalue corresponding to $\mathbf{v}$.
• We can find eigenvalues and eigenvectors by: setting $\det (A-\lambda I_2)=0$ and solving a quadratic equation to find ${\lambda }_1$, ${\lambda }_2$, then plugging these in for $\lambda$ and writing $A_{{\lambda }_1}=A-{\lambda }_1{I}_2$ and $A_{{\lambda }_1}=A-{\lambda }_1{I}_2$, and then manually solving for vectors ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$ satisfying $A_{{\lambda }_1}{\mathbf{v}}_1=0$ and $A_{{\lambda }_2}{\mathbf{v}}_2=0$. This is possible using the fact that one row of $A_{{\lambda }_i}$ will be a multiple $x$ times the other row.

Link to original