Conditional Distributions and Independence
Last modified — 21 Jun 2026
By the end of this lecture, students are anticipated to be able to:
Let \(Y\) be a random variable, where \(X\) is a discrete random variable. Suppose that \(\mathbb{P}(X = x) >0\) for some \(x\). Then the conditional distribution of \(Y\) given \(X = x\) assigns to each set \(A \subset {\mathbb{R}}\) the probability \[\mathbb{P}(Y \in A \ \vert\ X = x) = \frac{\mathbb{P}(Y \in A, X = x)}{\mathbb{P}(X = x)}.\]
Here, we do not specify whether \(Y\) is continuous or discrete.
If \(X\) and \(Y\) are both discrete random variables, then the conditional PMF of \(Y\) given \(X = x\) is defined by \[p_{Y|X}(y \ \vert\ x) = \frac{\mathbb{P}(Y = y, X = x)}{\mathbb{P}(X = x)} = \frac{p_{X,Y}(x,y)}{p_X(x)}.\]
The joint PMF of \(W\) and \(V\) is
| \(W\ \backslash\ V\) | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1/36 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2 | 0 | 2/36 | 1/36 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 0 | 0 | 2/36 | 2/36 | 1/36 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | 0 | 0 | 0 | 2/36 | 2/36 | 2/36 | 1/36 | 0 | 0 | 0 | 0 |
| 5 | 0 | 0 | 0 | 0 | 2/36 | 2/36 | 2/36 | 2/36 | 1/36 | 0 | 0 |
| 6 | 0 | 0 | 0 | 0 | 0 | 2/36 | 2/36 | 2/36 | 2/36 | 2/36 | 1/36 |
To condition on \(W\), we look at a particular row, while to condition on \(V\), we look at a particular column. Then renormalize by the sum of the row/column.
| \(W\ \backslash\ V\) | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1/36 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2 | 0 | 2/36 | 1/36 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 0 | 0 | 2/36 | 2/36 | 1/36 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | 0 | 0 | 0 | 2/36 | 2/36 | 2/36 | 1/36 | 0 | 0 | 0 | 0 |
| 5 | 0 | 0 | 0 | 0 | 2/36 | 2/36 | 2/36 | 2/36 | 1/36 | 0 | 0 |
| 6 | 0 | 0 | 0 | 0 | 0 | 2/36 | 2/36 | 2/36 | 2/36 | 2/36 | 1/36 |
Given the above table, what is \(\mathbb{P}(W = 5 \ \vert\ V = 8)\)
If \(X\) and \(Y\) are jointly absolutely continuous random variables, then the conditional density of \(Y\) given \(X = x\), is the function \[f_{Y|X}(y \ \vert\ x) = \frac{f_{X,Y}(x,y)}{f_X(x)},\] valid for any \(y\in{\mathbb{R}}\), and all \(x\) such that \(f_X(x) > 0\).
Let \(X\) and \(Y\) be jointly absolutely continuous random variables with joint PDF \(f_{X,Y}\). The conditional distribution of \(Y\) given \(X = x\) assigns to each set \(A \subset {\mathbb{R}}\) the probability \[\mathbb{P}(a \le Y \le b \ \vert\ X = x) = \int_a^b f_{Y|X}(y\ \vert\ x)\, \mathsf{d}y,\] valid for all \(x\) such that \(f_X(x) > 0\).
Let \(X\) and \(Y\) be jointly continuous random variables with joint PDF \[f_{X,Y}(x,y) = \frac{1}{x} e^{-x} I_{\{0 \le y \le x\}}(x,y).\]
Let \(X\) and \(Y\) be independent with \(X \sim {\mathrm{Exp}}(1)\) and \(Y\sim {\mathrm{Exp}}(2)\). Let \(S = X + Y\). Find the conditional density \(f_{X|S}(X = x | S = s)\)
\(X\) and \(Y\) are independent if and only if for any sets \(A\) and \(B\) we have \[\mathbb{P}( X \in A,\ Y \in B ) = \mathbb{P}( X \in A )\ \mathbb{P}( Y \in B ).\]
This is the definition of independence. It says that the joint distribution factors into the product of the marginals.
But this has immediate consequences for the joint CDF, PMF, and PDF.
Choosing \(A = (-\infty, x]\) and \(B = (-\infty, y]\), we have that, if \(X\) and \(Y\) are independent, then \[\begin{aligned} F_{X,Y}(x,y) &= \mathbb{P}(X \le x, Y \le y) = \mathbb{P}(X \le x) \mathbb{P}(Y \le y) = F_X(x) F_Y(y). \end{aligned}\]
The converse is also true: if \(F_{X,Y}(x,y) = F_X(x) F_Y(y)\) for all \(x, y\), then \(X\) and \(Y\) are independent.
If \(X\) and \(Y\) are discrete random variables, then \(X\) and \(Y\) are independent if and only if \[p_{X,Y}(x,y) = p_X(x) p_Y(y),\] for all \(x, y \in {\mathbb{R}}\).
If \(X\) and \(Y\) are jointly continuous random variables, then \(X\) and \(Y\) are independent if and only if their joint density can be chosen such that \[f_{X,Y}(x,y) = f_X(x) f_Y(y),\] for all \(x, y \in {\mathbb{R}}\).
Let \(X\) and \(Y\) have joint pdf \[f_{X,Y}(x, y) = \begin{cases}8xy & \text{if } 0 \le x < y < 1 \\ 0 & \text{else.} \end{cases}\] Find the marginal PDFs of \(X\) and \(Y\). Are \(X\) and \(Y\) independent?
Let \(X\) and \(Y\) be independent random variables with CDFs \(F_X\) and \(F_Y\).
Let \(W = \max\{X, Y\}\).
Find the CDF of \(W\).
The previous example also extends to the maximum of \(n\) independent random variables.
Suppose that \(X_1, X_2, \dots, X_n\) are independent random variables with common CDF \(F_X\). Let \(W = \max\{X_1, X_2, \dots, X_n\}\). Then \[\begin{aligned} F_W(w) &= \mathbb{P}(W \le w) = \mathbb{P}(X_1 \le w, X_2 \le w, \dots, X_n \le w) = (F_X(w))^n,\\ \Longrightarrow f_W(w) &= \frac{\mathsf{d}}{\mathsf{d}w} F_W(w) = n (F_X(w))^{n-1} f_X(w). \quad\quad\text{(by the chain rule if $X$ is continuous)} \end{aligned}\]
Let \(X\sim {\mathrm{Exp}}(\lambda)\) and \(Y\sim{\mathrm{Exp}}(\mu)\) be independent random variables. Find the distribution of \(U = \min\{X, Y\}\).
Hints:
To find the distribution of a sum of independent random variables, we could use the distribution method or the Jacobian method.
For this specific case where RVs are independent, there’s also a third method called convolution.
Let \(X\) and \(Y\) be independent random variables.
If \(X\) and \(Y\) are both discrete random variables, then the PMF of \(U = X + Y\) is given by \[p_U(u) = \sum_{w} p_X(u-w) p_Y(w) = \sum_w p_Y(u - w) p_X(w).\]
If \(X\) and \(Y\) are both continuous random variables, then the PDF of \(U = X + Y\) is given by \[f_U(u) = \int_{-\infty}^\infty f_X(u - w) f_Y(w) \mathsf{d}w = \int_{-\infty}^\infty f_Y(u - w) f_X(w) \mathsf{d}w.\]
Let \(X\) and \(Y\) be independent \({\mathrm{Unif}}(0, 1)\) random variables. Let \(S = X +Y\). Find the distribution of \(S\).
Let \(X\) and \(Y\) be independent \({\mathrm{Exp}}(\lambda)\) random variables,. Find the PDF of \(U = X + Y\) using the convolution method.
Hint: be careful with the limits of integration. Recall that if \(Z\sim {\mathrm{Exp}}(\theta)\), then \(f_Z(z) = \theta e^{-\theta z}I_{(0,\infty)}(z)\).
Stat 302 - Winter 2025/26