Examples

Quadratic form

Quadratic form

$$\begin{array}{rl}Q(\mathbf{x})={\mathbf{x}}^{\mathsf{T}}A\mathbf{x}& =\left(\begin{array}{ccc}{x}_1& x_2& x_3\end{array}\right)\left(\begin{array}{ccc}5& -1/2& 0\\ -1/2& 3& 4\\ 0& 4& 2\end{array}\right)\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)\\ & \\ & =5x_1^2+3x_2^2+2x_3^2-x_1{x}_2+8x_2{x}_3.\end{array}$$

Eliminating cross terms

Eliminating cross terms

Problem: Eliminate the cross terms in the quadratic form $Q(\mathbf{x})=2x_1^2+6x_1{x}_2-6x_2^2$.

Solution: First compute the matrix of $Q$. By reading off the components (dividing in half the cross term), we see

$$A=\left(\begin{array}{cc}2& 3\\ 3& -6\end{array}\right).$$

Now we must write $A=PD{P}^{\mathsf{T}}$ for $P$ an orthogonal matrix. We find the eigenvalues and then seek eigenvectors that are orthonormal. Observe that $\det A_{\lambda }={\lambda }^2+4\lambda -21=(\lambda +7)(\lambda -3)$, so the eigenvalues are $\lambda =-7,3$.

Then we have:

$$A_{-7}=\left(\begin{array}{cc}2--7& 3\\ 3& -6--7\end{array}\right)=\left(\begin{array}{cc}9& 3\\ 3& 1\end{array}\right)$$

so $3x_1+x_2=0$ gives the relation satisfied by the rows of an eigenvector. Choose $\left(\begin{array}{c}1\\ -3\end{array}\right)$ and then normalize this to the unit vector ${\mathbf{v}}_{-7}=\left(\begin{array}{c}1/\sqrt{10}\\ -3/\sqrt{10}\end{array}\right)$.

Then we also have:

$$A_3=\left(\begin{array}{cc}2-3& 3\\ 3& -6-3\end{array}\right)=\left(\begin{array}{rc}-1& 3\\ 3& -9\end{array}\right)$$

so $-x_1+3x_2=0$ and we choose $\left(\begin{array}{c}3\\ 1\end{array}\right)$ as an eigenvector, and normalize to the unit vector ${\mathbf{v}}_3=\left(\begin{array}{c}3/\sqrt{10}\\ 1/\sqrt{10}\end{array}\right)$. Notice that ${\mathbf{v}}_{-7}\cdot {\mathbf{v}}_3=0$.

Now we set ${\mathbf{q}}_1={\mathbf{v}}_{-7}$ and ${\mathbf{q}}_2={\mathbf{v}}_3$ and write our expression for $A$:

$$\begin{array}{rl}A& =\left(\begin{array}{cc}2& 3\\ 3& -6\end{array}\right)=PD{P}^{\mathsf{T}}\\ & =\left(\begin{array}{cc}1/\sqrt{10}& 3/\sqrt{10}\\ -3/\sqrt{10}& 1/\sqrt{10}\end{array}\right)\left(\begin{array}{rc}-7& 0\\ 0& 3\end{array}\right)\left(\begin{array}{cc}1/\sqrt{10}& -3/\sqrt{10}\\ 3/\sqrt{10}& 1/\sqrt{10}\end{array}\right).\end{array}$$

In terms of the $\mathbf{y}$ variables, we have $Q(\mathbf{x})=Q(P\mathbf{y})=-7y_1^2+3y_2^2$. The formulas relating the variables to each other are given by interpreting $P^{\mathsf{T}}$ as the change of variable transfer matrix:

$$\begin{array}{rl}{y}_1& =\frac{1}{\sqrt{10}}{x}_1-\frac{3}{\sqrt{10}}{x}_2\\ y_2& =\frac{3}{\sqrt{10}}{x}_1+\frac{1}{\sqrt{10}}{x}_2.\end{array}$$

Optimizing the quadratic form from the eliminating cross terms example

Optimizing $Q(\mathbf{x})=2x_1^2+6x_1{x}_2-6x_2^2$

In the Example of the previous section on eliminating cross terms, we obtained the formula $-7y_1^2+3y_2^2$ for $Q$ in terms of the variables $y_1,y_2$. We can see directly from this formula that the min eigenvalue is $-7$ and the max eigenvalue is $3$.

The min of $-7$ occurs when $y_1=1,y_2=0$. In other words it is at the first eigenvector ${\mathbf{q}}_1$. In terms of the $x_1,x_2$ variables, this occurs at $x_1=1/\sqrt{10}$ and $x_2=-3/\sqrt{10}$, the components of ${\mathbf{q}}_1={\mathbf{v}}_{-7}$ in the standard basis.

The max of $3$ occurs when $y_1=0,y_2=1$. In other words at ${\mathbf{q}}_2$, which corresponds to $x_1=3/\sqrt{10}$ and $x_2=1/\sqrt{10}$.