W14 - Examples

Mean square error

Minimal MSE estimate given PMF

Minimal MSE estimate given PMF

Suppose $X$ has the following PMF:

$k$	1	2	3	4	5
$P_X(k)$	0.15	0.28	0.26	0.19	0.13

Find the minimal MSE estimate of $X$, given that $X$ is even. What is the error of this estimate?

Solution

The minimal MSE given $A$ is just $E[X\mid A]$ where $A=\{2,4\}$.

First compute the conditional PMF:

$$P_{X|A}(k)=\{\begin{array}{ll}0.19/0.47& k=4\\ 0.28/0.47& k=2\\ 0& k\ne 2,4\end{array}$$

Therefore:

$${\hat{x}}_A=2\frac{0.28}{0.47}+4\frac{0.19}{0.47}\approx 2.80851$$

The error is:

$$\begin{array}{c}{e}_{X|A}=(2-2.81{)}^2\frac{0.28}{0.47}+(4-2.81{)}^2\frac{0.19}{0.47}\\ \\ \gg \gg \approx 0.9633\end{array}$$ Link to original

Minimal MSE estimate from joint PDF

Minimal MSE estimate from joint PDF

Here is the joint PDF of $X$ and $Y$:

$$f_{X,Y}=\{\begin{array}{ll}8xy& 0\le y\le x\le 1\\ 0& \text{otherwise}\end{array}$$

Find the minimal MSE estimate of $X$ in terms of $Y$.

What is the estimate of $X$ when $Y=0.2$? When $Y=0.8$?

Answer

${\hat{x}}_M(y)=\frac{2}{3}\cdot \frac{1-y^3}{1-y^2}$

${\hat{x}}_M(0.2)=0.6889{\hat{x}}_M(0.8)=0.9037$

Link to original

Estimating on a variable interval

Estimating on a variable interval

Suppose that $R\sim \mathrm{Unif}(0,1)$ and suppose $X\sim \mathrm{Unif}(0,R)$.

(a) Find ${\hat{x}}_M(r)$ (b) Find ${\hat{r}}_M(x)$ (c) Find ${\hat{R}}_{L_{\min }}(X)$

Solution

(a) Find ${\hat{x}}_M(r)$:

We know ${\hat{x}}_M(r)=E[X\mid R=r]$.

Given $R=r$, so $X$ is uniform on $(0,r)$, we have $E[X\mid R=r]=\frac{r}{2}$.

$${\hat{x}}_M(r)=\frac{r}{2}$$

---

(b) Find ${\hat{r}}_M(x)$:

We know ${\hat{r}}_M(x)=E[R\mid X=x]$.

We know $f_R$ and $f_{X|R}$. From these we derive the joint distribution $f_{X,R}$:

$$f_R(r)=\{\begin{array}{ll}1& r\in (0,1)\\ 0& \text{otherwise}\end{array}{f}_{X|R}(x|r)=\{\begin{array}{ll}1/r& x\in (0,r)\\ 0& \text{otherwise}\end{array}$$ $$\gg \gg f_{X,R}(x,r)=f_{X|R}\cdot f_R=\{\begin{array}{ll}1/r& 0<x<r<1\\ 0& \text{else}\end{array}$$

Now extract the marginal $f_X$:

$$\begin{array}{c}\gg \gg f_X(x)={\int }_{-\infty }^{\infty }{f}_{X,R}(x,r)dr\\ \\ \gg \gg {\int }_x^1\frac{1}{r}dr\gg \gg -\ln x(0<x<1)\end{array}$$

Now deduce the conditional $f_{R|X}$:

$$f_{R|X}=\frac{f_{X,R}}{f_X}=\{\begin{array}{ll}\frac{-1}{r\ln x}& 0<x<r<1\\ 0& \text{otherwise}\end{array}$$

Then:

$$\begin{array}{c}E[R\mid X=x]\gg \gg {\int }_x^{1}r\frac{-1}{r\ln x}dr\\ \\ \gg \gg {\hat{r}}_M(x)=\frac{x-1}{\ln x}\end{array}$$

---

(c) Find ${\hat{R}}_{L_{\min }}(X)$:

We need all the basic statistics.

$E[R]=1/2$ because $R\sim \mathrm{Unif}((0,1))$.

${\sigma }_R^2=\frac{(b-a{)}^2}{12}=1/12$.

$E[X]=1/4$ using the marginal PDF $f_X(x)=-\ln x$ on $x\in (0,1)$. (IBP and L’Hopital are needed.)

${\sigma }_X=\sqrt{7}/12$ also using the marginal $f_X(x)=-\ln x$.

$E[XR]=1/6$ using $f_{X,R}(x,r)$, namely:

$$\begin{array}{c}E[XR]={\int }_{r=0}^1{\int }_{x=0}^{r}xr\frac{1}{r}dxdr\\ \\ \gg \gg {\int }_0^1\frac{x^2}{2}dx\gg \gg \frac{1}{6}\end{array}$$

From all this we infer $\mathrm{Cov}[X,R]=1/24$ and ${\rho }_{X,R}=\sqrt{3/7}$.

Hence:

$$L_{\min }(x)=\frac{6}{7}x+\frac{2}{7}$$

Thus:

$${\hat{R}}_{L_{\min }}(X)=\frac{6}{7}X+\frac{2}{7}$$

Link to original

Line of minimal MSE given joint PDF

Line of minimal MSE given joint PDF

Here is the joint PDF of $X$ and $Y$:

$$f_{X,Y}=\{\begin{array}{ll}8xy& 0\le y\le x\le 1\\ 0& \text{otherwise}\end{array}$$

Find the line giving the linear MSE estimate of $X$ in terms of $Y$.

What is the expected error of this line, $e_{L_{\min }}$?

What is the estimate of $X$ when $Y=0.2$? When $Y=0.8$?

Answer

${\hat{X}}_{L_{\min }}(Y)=0.3637Y+0.6060$

$e_{L_{\min }}=0.02020$

${\hat{x}}_{L_{\min }}(0.2)=0.67874{\hat{x}}_{L_{\min }}(0.8)=0.89696$

Link to original