Problems

01

Spans of column vectors

Matrices $M_1,M_2$ are $n\times n$ matrices, and $k<n$. Matrix $M_1$ has ones everywhere on and above the main diagonal, down to the $k^{\text{th}}$ row, and zeros elsewhere, including everything after the $k^{\text{th}}$ row. Matrix $M_2$ has ones on the main diagonal only, down to the $k^{\text{th}}$ row, and zeros everywhere else. Show that the span of the column vectors of $M_1$ is the same subspace as the span of the column vectors of $M_2$. What subspace is it?

(Hint: first try to verify the assertion for small values of $k$ and $n$, for example when $n=2$ and $k=1$, and then $n=3$ and $k=1$ or $k=2$. Then see if you can generalize your method to all $k<n$.)

02

Computing dimensions by hand

• Show that the span $⟨\left(\begin{array}{c}1\\ 1\end{array}\right),\left(\begin{array}{c}1\\ -1\end{array}\right)⟩$ has dimension $2$ as a subspace by using the “exchange” technique.
• Show that the span $⟨\left(\begin{array}{c}1\\ 1\\ 1\end{array}\right),\left(\begin{array}{c}1\\ 1\\ 0\end{array}\right),\left(\begin{array}{c}0\\ 2\\ 2\end{array}\right)⟩$ has dimension $3$ as a subspace by using the “exchange” technique.
• Show that the span $⟨\left(\begin{array}{c}1\\ -2\\ 3\end{array}\right),\left(\begin{array}{c}4\\ 0\\ 2\end{array}\right),\left(\begin{array}{c}3\\ -2\\ 4\end{array}\right)⟩$ has dimension $2$ as a subspace.

03

Suppose we are given a collection of data points $(x_1,y_1),\dots ,(x_n,y_n)$.

Later in the course, we will learn how to compute the linear regression of this data, namely the line of best fit. In this problem, you will learn about the correlation coefficient of the data: a measure of how linear the data is. There is no point in computing a linear regression if the data is not very linear!

First, let $\mathbf{x}=(x_1,x_2,\dots ,x_n)$ and $\mathbf{y}=(y_1,y_2,\dots ,y_n)$. Then let $\overline{x}$ be the average of the $x_i$, and $\overline{y}$ be the average of the $y_i$. Therefore $\mathbf{x}-\overline{x}$ and $\mathbf{y}-\overline{y}$ have zero average. (By subtracting a scalar from a vector, we really mean to subtract the scalar from each component.) Then define the correlation coefficient:

$$\rho (\mathbf{x},\mathbf{y})=\frac{(\mathbf{x}-\overline{x})\cdot (\mathbf{y}-\overline{y})}{|\mathbf{x}-\overline{x}||\mathbf{y}-\overline{y}|}.$$

Correlation coefficient

Problem: Compute the correlation coefficient of the data:

$$(-3,4),(1,-1),(0,-1),(4,-3),(-2,1).$$

The correlation coefficient $\rho (\mathbf{x},\mathbf{y})$ is frequently written ‘$r$’. The formula describes $r$ as a dot product of unit vectors, so $-1\le r\le +1$. When $r=\pm 1$, the data are very linear. When $r=+1$, the data lie on a line with positive slope, meaning that if you increase $x$ then you expect $y$ to increase as well. When $r=-1$, the data lie on a line with negative slope, so if you increase $x$ then you expect $y$ to decrease.

Later, in linear regression as projection, we will see why $r=\rho (\mathbf{x},\mathbf{y})$ measures linearity. For now, consider the following. Suppose the data is linear, so $y_i=m{x}_i+b$ for some $m$ and $b$. Then as vectors, we have $\mathbf{y}=m\mathbf{x}+b$. If we subtract the average of both sides, we have

$$\begin{array}{rl}\mathbf{y}-\overline{y}& =m\mathbf{x}+b-\overline{m\mathbf{x}+b}\\ & =m(\mathbf{x}-\overline{x})+b-\overline{b}\\ & =m(\mathbf{x}-\overline{x}).\end{array}$$

Therefore $\mathbf{y}-\overline{y}$ is a scalar multiple of $\mathbf{x}-\overline{x}$, and the dot product of their unit vectors tells us whether they align or anti-align.

This problem (and its sequel) illustrates the use of $n$D vectors to study data that is presented as $2$D vectors.