W11 - Examples

Summations

Binomial expectation and variance

Binomial expectation and variance

Suppose we have repeated Bernoulli trials $X_1,\dots ,X_n$ with $X_i\sim \mathrm{Ber}(p)$.

The sum is a binomial variable: $S_n=\sum _{i=1}^n{X}_i$.

We know $E[X_i]=p$ and $\mathrm{Var}[X_i]=pq$.

The summation rule for expectation:

$$E[S_n]=\sum _{i=1}^{n}E[X_i]\gg \gg \sum _{i=1}^{n}p\gg \gg np$$

The summation rule for variance:

$$\begin{array}{c}\mathrm{Var}[S_n]=\sum _{i=1}^n\mathrm{Var}[X_i]+2\sum _{i<j}\mathrm{Cov}[X_i,X_j]\\ \\ \gg \gg \sum _{i=1}^{n}pq+2\cdot 0\gg \gg npq\end{array}$$ Link to original

Multinomial covariances

Multinomial covariances

Each trial of an experiment has possible outcomes labeled $1,\dots ,r$ with probabilities of occurrence $p_1,\dots ,p_r$. The experiment is run $n$ times.

Let $X_i$ count the number of occurrences of outcome $i$. So $X_i\sim \mathrm{Bin}(n,p_i)$.

Find $\mathrm{Cov}[X_i,X_j]$.

Solution

Notice that $X_i+X_j$ is also a binomial variable with success probability $p=p_i+p_j$.
(‘Success’ is an outcome of either $i$ or $j$. ‘Failure’ is any other value.)

The variance of a binomial is known to be $npq$.

Compute $\mathrm{Cov}[X_i,X_j]$ by solving:

$$\begin{array}{rl}\mathrm{Var}[X_i+X_j]& =\mathrm{Var}[X_i]+\mathrm{Var}[X_j]+2\mathrm{Cov}[X_i,X_j]\\ & \\ n(p_i+p_j)(1-(p_i+p_j))& =n{p}_i(1-p_i)+n{p}_j(1-p_j)+2\mathrm{Cov}[X_i,X_j]\\ & \\ \gg \gg \mathrm{Cov}[X_i,X_j]& =-n{p}_i{p}_j\end{array}$$ Link to original

Hats in the air

Hats in the air

All $n$ sailors throws their hats in the air, and catch a random hat when they fall back down.

(a) How many sailors do you expect will catch the hat they own?

(b) What is the variance of this number?

Solution

Strangely, the answers are both 1, regardless of the number of sailors. Here is the reasoning:

(a)
Let $X_i=1$ when sailor $i$ catches their own hat, and $X_i=0$ otherwise. Thus $X_i$ is Bernoulli with $p=1/n$.

Now $X={\sum }_{i=1}^n{X}_i$ counts the total number of hats caught by their owners.

Note that $E[X_i]=1/n$. Therefore:

$$\begin{array}{c}E[X]\gg \gg E\left[\sum _{i=1}^n{X}_i\right]\\ \\ \gg \gg \sum _{i=1}^{n}E[X_i]\gg \gg \sum _{i=1}^n\frac{1}{n}\gg \gg 1\end{array}$$

---

(b)
We know:

$$\mathrm{Var}[X]=\sum _{i=1}^n\mathrm{Var}[X_i]+2\sum _{i<j}\mathrm{Cov}[X_i,X_j]$$

---

Now calculate $\mathrm{Var}[X_i]$:

Use $\mathrm{Var}[X_i]=E[X_i^2]-E[X_i{]}^2$. Observe that $X_i^2=X_i$. Therefore:

$$\mathrm{Var}[X_i]\gg \gg \frac{1}{n}-\frac{1}{n^2}\gg \gg \frac{n-1}{n^2}$$

---

Now calculate $\mathrm{Cov}[X_i,X_j]$:

$$\mathrm{Cov}[X_i,X_j]=E[X_i{X}_j]-E[X_i]E[X_j]$$

We need to compute $E[X_i{X}_j]$.

Notice that $X_i{X}_j=1$ when $i$ and $j$ both catch their own hats, and 0 otherwise. So it is Bernoulli. Then:

$$\begin{array}{c}P[X_i=1\text{and}{X}_j=1]\gg \gg \frac{1}{n(n-1)}\\ \\ \gg \gg E[X_i{X}_j]=\frac{1}{n(n-1)}\end{array}$$

Therefore:

$$\begin{array}{c}\mathrm{Cov}[X_i,X_j]\gg \gg \frac{1}{n(n-1)}-\frac{1}{n}\cdot \frac{1}{n}\\ \\ \gg \gg \frac{1}{n^2(n-1)}\end{array}$$

---

Putting everything together:

$$\begin{array}{c}\mathrm{Var}[X]\gg \gg \sum _{i=1}^n\mathrm{Var}[X_i]+2\sum _{i<j}\mathrm{Cov}[X_i,X_j]\\ \\ \gg \gg \sum _{i=1}^n\frac{n-1}{n^2}+2\sum _{i<j}\frac{1}{n^2(n-1)}\\ \\ \gg \gg \frac{n-1}{n}+n(n-1)\frac{1}{n^2(n-1)}\gg \gg 1\end{array}$$ Link to original

Months with a birthday

Months with a birthday

Suppose study groups of 10 are formed from a large population.

For a typical study group, how many months out of the year contain a birthday of a member of the group? (Assume all 12 months have equal duration.)

Solution

(1) Let $X_i$ be 1 if month $i$ contains a birthday, and 0 otherwise.

So we seek $E[X_1+\cdots +X_{12}]$. This equals $E[X_1]+\cdots +E[X_{12}]$.

The answer will be $12E[X_i]$ because all terms are equal.

---

(2) For a given $i$:

$$P[\text{no birthday in month }i]={\left(\frac{11}{12}\right)}^{10}$$

The complement event:

$$P[\text{at least one birthday in month }i]=1-{\left(\frac{11}{12}\right)}^{10}$$

---

(3) Therefore:

$$12E[X_i]=12\left(1-{\left(\frac{11}{12}\right)}^{10}\right)\gg \gg 6.97$$ Link to original

Pascal expectation and variance

Pascal expectation and variance

(1) Let $X\sim \mathrm{Pasc}(\ell ,p)$.

Let $X_1,X_2,\dots$ be independent random variables, where:

• $X_1$ counts the trials until the first success
• $X_2$ counts the trials after the first success until the second success
• $X_i$ counts the trials after the $(i-1{)}^{th}$ success until the $i^{th}$ success

Observe that $X={\sum }_{i=1}^{\ell }{X}_i$.

---

(2) Notice that $X_i\sim \mathrm{Geom}(p)$ for every $i$. Therefore:

$$E[X_i]=\frac{1}{p}\mathrm{Var}[X_i]=\frac{1-p}{p^2}$$

---

(3) Using the summation rule, conclude:

$$\begin{array}{c}E[X]\gg \gg \sum _{i=1}^{\ell }\frac{1}{p}\gg \gg \frac{\ell }{p}\\ \\ \mathrm{Var}[X]\gg \gg \sum _{i=1}^{\ell }\frac{q}{p^2}\gg \gg \frac{\ell q}{p^2}\end{array}$$Link to original

Central Limit Theorem

Binomial estimation: 10,000 flips

Binomial estimation: 10,000 flips

Flip a fair coin 10,000 times. Write $H$ for the number of heads.

Estimate the probability that $4850<H<5100$.

Solution

(1) Check the rule of thumb: $p=q=0.5$ and $n=10,000$, so $npq=2500\gg 10$ and the approximation is effective.

---

(2) Now, calculate needed quantities:

$$\begin{array}{c}\mu =E[X_i]\gg \gg \mu =0.5\gg \gg n\mu =5000\\ \\ {\sigma }^2=\mathrm{Var}[X_i]\gg \gg \sigma =0.5\gg \gg \sigma \sqrt{n}=50\end{array}$$

---

(3) Set up CDF:

$$F_H(h)=\Phi \left(\frac{h-5000}{50}\right)$$

---

(4) Compute desired probability:

$$\begin{array}{c}P[4850<H<5100]=F_H(5100)-F_H(4850)\\ \\ \gg \gg \Phi \left(\frac{100}{50}\right)-\Phi \left(\frac{-150}{50}\right)\gg \gg \Phi (2)-\Phi (-3)\\ \\ \gg \gg \approx 0.9772-(1-0.9987)\gg \gg 0.9759\end{array}$$ Link to original

Summing 1000 dice

Summing 1000 dice

Suppose 1,000 dice are rolled.

Estimate the probability that the total sum of rolled numbers is more than 3,600.

Solution

(1) Let $X_i$ be the number rolled on the $i^{th}$ die.

Let $S={\sum }_{i=1}^n{X}_i$, so $S$ sums up the rolled numbers.

We seek $P[S\ge 3600]$.

---

(2) Now, calculate needed quantities:

$$\begin{array}{c}\mu =E[X_i]\gg \gg \mu =7/2\gg \gg n\mu =3500\\ \\ {\sigma }^2=\mathrm{Var}[X_i]\gg \gg \sigma =\sqrt{\frac{35}{12}}\gg \gg \sigma \sqrt{n}=\sqrt{\frac{35000}{12}}\end{array}$$

---

(3) Set up CDF:

$$F_S(s)\approx \Phi \left(\frac{s-3500}{\sqrt{\frac{35000}{12}}}\right)$$

---

(4) Compute desired probability:

$$\begin{array}{c}P[S\ge 3600]=1-F_S(3600)\\ \\ \gg \gg \approx 1-\Phi \left(\frac{100}{54.01}\right)\gg \gg 1-\Phi (1.852)\approx 0.968\end{array}$$ Link to original

Continuity correction of absurd normal approximation

Continuity correction of absurd normal approximation

Let $S_n$ denote the number of sixes rolled after $n$ rolls of a fair die.

Estimate $P[S_{720}=113]$.

Solution

We have $S_n\sim \mathrm{Bin}(720,1/6)$, and $np=120$ and $\sqrt{npq}=10$.

The usual approximation, since $Z$ is continuous, gives an estimate of 0, which is useless.

Now using the continuity correction:

$$\begin{array}{c}P[113\le S_{720}\le 113]\\ \\ \approx \Phi \left(\frac{113+0.5-120}{10}\right)-\Phi \left(\frac{113-0.5-120}{10}\right)\\ \\ \gg \gg \Phi (-0.65)-\Phi (-0.75)\gg \gg \approx 0.0312\end{array}$$

The exact solution is 0.0318, so this estimate is quite good: the error is 1.9%.

Link to original