Section 3.3
Variance, Covariance, and Correlation
Question 3.3.1
Consider the following joint distribution between random variables \(X\) and \(Y\).
| (X , Y) | 1 | 2 | 3 |
|---|---|---|---|
| 0 | 0.10 | 0.15 | 0.05 |
| 1 | 0.20 | 0.30 | 0.10 |
| 2 | 0.05 | 0.03 | 0.02 |
(and of course, \(\mathbb{P}(X,Y) = 0\) otherwise.)
- Compute the variance of \(X\).
\[ \begin{aligned} \mathbb{E}[X] &= \sum_{x}\sum_{y}xp_{XY}(x,y) \\ &= 0p_X(0) + 1p_X[1] + 2p_X(2) \\ &= 0 + 1(0.20+0.30+0.10) + 2(0.05+0.03+0.02)\\ &= 0.80 \end{aligned} \] \[ \begin{aligned} \mathbb{E}[X^2] &= \sum_{x}\sum_{y}xp_{XY}(x,y) \\ &= 0^2p_X(0) + 1^2p_X[1] + 2^2p_X(2) \\ &= 0 + 1^2(0.20+0.30+0.10) + 2^2(0.05+0.03+0.02)\\ &= 1 \end{aligned} \] Thus,
\[ \begin{aligned} \operatorname{Var}(X) &= \mathbb{E}(X^2) - \left[\mathbb{E}(X)\right]^2\\ &= 1 - 0.8^2\\ &= 0.36 \end{aligned} \]
- Calculate the correlation between \(X\) and \(Y\).
The correlation between \(X\) and \(Y\) is \(\rho_{XY} = \frac{\mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y]}{\sigma_X \sigma_Y}\)
\(\mathbb{E}(X)\) was calculated in part (a), and is 0.80.
\[ \begin{aligned} \mathbb{E}[Y] &= \sum_{x}\sum_{y}yp_{XY}(x,y) \\ &= 1p_Y[1] + 2p_Y(2) + 3p_Y(3) \\ &= 1(0.10+0.20+0.05) + 2(0.15+0.30+0.03) + 3(0.05+0.10+0.02)\\ &= 1.82 \end{aligned} \]
\[ \begin{aligned} \mathbb{E}[XY] &= \sum_{x}\sum_{y}xyp_{XY}(x,y) \\ &= (0)[1](0.10) + (0)(2)(0.15) + (0)(3)(0.05) + [1][1](0.20) + [1](2)(0.30) + [1](3)(0.10)\\ &+ (2)[1](0.05) + (2)(2)(0.03) + (2)(3)(0.02)\\ &= 1.44 \end{aligned} \]
To calculate the standard deviation/variance, we also need:
\[ \begin{aligned} \mathbb{E}[X^2] &= \sum_{x}\sum_{y}x^2p_{XY}(x,y) \\ &= 0^2p_X(0) + 1^2p_X[1] + 2^2p_X(2) \\ &= (0.20+0.30+0.10) + 4(0.05+0.03+0.02)\\ &= 1 \end{aligned} \] Thus, \(\sigma_X = \sqrt{\mathbb{E}[X^2] - \mathbb{E}[X]^2} = \sqrt{1 - 0.8^2} = 0.60\).
\[ \begin{aligned} \mathbb{E}[Y^2] &= \sum_{x}\sum_{y}y^2p_{XY}(x,y) \\ &= 1^2p_Y[1] + 2^2p_Y(2) + 3^2p_Y(3) \\ &= 1(0.10+0.20+0.05) + 4(0.15+0.30+0.03) + 9(0.05+0.10+0.02)\\ &= 3.80 \end{aligned} \]
Thus, \(\sigma_Y = \sqrt{\mathbb{E}[Y^2] - \mathbb{E}[Y]^2} = \sqrt{3.80 - 1.82^2} = 0.6983\).
Then,
\(\rho_{XY} = \frac{1.44 - (0.8)(1.82)}{0.60\times0.6983} = -0.0382\)
Question 3.3.2
Let \(X\) and \(Y\) be continuous random variables with joint pdf \(f_{X,Y}(x,y) = 3xI_{[0,1]}(x)I_{[0, x]}(y)\).
- Compute the variance of \(X\).
\[ \begin{aligned} \mathbb{E}[X] &= \int_{0}^1\int_{0}^x(x)3xdydx\\ &= \int_{0}^1\int_{0}^x3x^2dydx\\ &= \int_{0}^1\left[3x^2y\Big|_{y = 0}^{y=x}\right]dx\\ &= \int_{0}^13x^2xdx\\ &= \int_{0}^13x^3dx\\ &= \frac{3}{4}x^4\Big|_{x=0}^{x=1}\\ &= \frac{3}{4} \end{aligned} \]
\[ \begin{aligned} \mathbb{E}[X^2] &= \int_{0}^1\int_{0}^x(x^2)3xdydx\\ &= \int_{0}^1\int_{0}^x3x^4dydx\\ &= \int_{0}^1\left[3x^3y\Big|_{y = 0}^{y=x}\right]dx\\ &= \int_{0}^13x^3xdx\\ &= \int_{0}^13x^4dx\\ &= \frac{3}{5}x^5\Big|_{x=0}^{x=1}\\ &= \frac{3}{5} \end{aligned} \]
Thus,
\[ \begin{aligned} \operatorname{Var}(X) &= \mathbb{E}[X^2] - \mathbb{E}[X]^2\\ &= \frac{3}{5} - \left(\frac{3}{4}\right)^2\\ &= 0.0375 \end{aligned} \]
- Compute the covariance between \(X\) and \(Y\).
\(E(X)\) was solved in part (a), and is 3/4.
\[ \begin{aligned} \mathbb{E}[Y] &= \int_{0}^1\int_{0}^x(y)3xdydx\\ &= \int_{0}^1\int_{0}^x3xydydx\\ &= \int_{0}^1\left[\frac{3}{2}xy^2\right]\Big|_{y = 0}^{y=x}dx\\ &= \int_{0}^1\frac{3}{2}x^3dx\\ &= \frac{3}{2\times4}x^4\big|_{x =0}^{x = 1}\\ &= \frac{3}{8}\\ \end{aligned} \]
\[ \begin{aligned} \mathbb{E}[XY] &= \int_{0}^1\int_{0}^x(xy)3xdydx\\ &= \int_{0}^1\int_{0}^x3x^2ydydx\\ &= \int_{0}^1\left[\frac{3}{2}x^2y^2\right]\Big|_{y= 0}^{y = x}dx\\ &= \int_{0}^1\frac{3}{2}x^4dx\\ &= \frac{3}{2\times5}x^5\Big|_{x = 0}^{x = 1}\\ &= \frac{3}{10} \end{aligned} \]
Thus,
\[ \begin{aligned} \operatorname{Cov}(X, Y) &= \mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y]\\ &= \frac{3}{10} - \frac{3}{8} \frac{3}{4}\\ &= \frac{3}{160} \text{ or } 0.01875 \end{aligned} \]
Question 3.3.3
Suppose you roll a die. Let \(X\) be the number on the die and let \(Y = X^2\).
Compute \(\operatorname{Cov}(X, Y)\).
\[ \begin{aligned} \operatorname{Cov}(X, Y) &= \operatorname{Cov}(X, X^2)\\ &= \mathbb{E}(X\cdot X^2) - \mathbb{E}(X)\mathbb{E}(X^2)\\ &= \mathbb{E}(X^3) -\mathbb{E}(X)\mathbb{E}(X^2). \end{aligned} \] We have: \[ \begin{aligned} \mathbb{E}(X) = (1/6)(1) + (1/6)(2) + ... + (1/6)(6) = 7/2 \end{aligned} \] \[ \begin{aligned} \mathbb{E}(X^2) = (1/6)(1^2) + (1/6)(2^2) + ... + (1/6)(6^2) = 91/6 \end{aligned} \] \[ \begin{aligned} \mathbb{E}(X^3) = (1/6)(1^3) + (1/6)(2^3) + ... + (1/6)(6^3) = 147/2 \end{aligned} \]
Thus:
\[ \begin{aligned} \operatorname{Cov}(X, Y) &= \mathbb{E}(X^3) -\mathbb{E}(X)\mathbb{E}(X^2)\\ &= 147/2 - (7/2)(91/6)\\ &\approx 20.42 \end{aligned} \]
Question 3.3.4
Show that the variance of the \({\mathrm{Unif}}(L, R)\) distribution is given by the expression \(\frac{(R-L)^2}{12}\).
\(\operatorname{Var}(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2\)
\[ \begin{aligned} \mathbb{E}(X) &= \int_{L}^{R}x\cdot \frac{1}{R-L}dx\\ &= \frac{1}{2(R-L)}x^2\Bigg|_{x = L}^{X = R}\\ &= \frac{1}{2(R-L)}(R^2 - L^2)\\ &= \frac{1}{2(R-L)}(R + L)(R - L)\\ &= \frac{R+L}{2}\\ \end{aligned} \] \[ \begin{aligned} \mathbb{E}(X^2) &= \int_{L}^{R}x^2\cdot \frac{1}{R-L}dx\\ &= \frac{1}{3(R-L)}x^3\Bigg|_{x = L}^{X = R}\\ &= \frac{1}{2(R-L)}(R^3 - L^3)\\ &= \frac{1}{3(R-L)}(R - L)(R^2 + RL + L^2)\\ &= \frac{R^2 + RL + L^2}{3}\\ \end{aligned} \] Thus,
\[ \begin{aligned} \operatorname{Var}(X) &= \mathbb{E}(X^2) - \mathbb{E}(X)^2\\ &= \frac{R^2 + RL + L^2}{3} - \left( \frac{R+L}{2}\right)^2\\ &= \frac{R^2 + RL + L^2}{3} - \frac{R^2 + 2RL + L^2}{4}\\ &= \frac{4R^2 + 4RL + 4L^2}{12} - \frac{3R^2 + 6RL + 3L^2}{12}\\ &= \frac{R^2 - 2RL + L^2}{12} \\ &= \frac{(R - L)^2}{12} \end{aligned} \]
Question 3.3.5
Let \(X \sim {\mathrm{Exp}}(3)\) and \(Y \sim {\mathrm{Poiss}}(5)\). Assume \(X\) and \(Y\) are independent and let \(Z = X + Y\).
- Compute \(\operatorname{Cov}(X, Z)\).
\[ \begin{aligned} \operatorname{Cov}(X, Z) &= \operatorname{Cov}(X, X + Y) \\ &= \operatorname{Cov}(X, X) + \operatorname{Cov}(X, Y)\\ &= \operatorname{Var}(X) + 0 \text{ since X, Y are independent}\\ &= (1/3)^2\\ &= 1/9 \end{aligned} \]
- Compute \(\operatorname{Corr}(X, Z)\).
\[ \begin{aligned} \operatorname{Corr}(X, Z) &= \frac{\operatorname{Cov}(X, Z)}{\sigma_X\sigma_Z} \\ &= \frac{1/9}{(1/9)(1.9 + 5)} \text{ since independent}\\ &= 0.147 \end{aligned} \]