Module 04

Random variables and distributions

TC and DJM

Last modified — 04 Feb 2026

1 Continuation of independence

Review of conditional probability and independence

Conditional probability is a probability (a function from \(\Omega\) to \([0,1]\) that satisfies the three axioms)
The conditioning event restricts the space of possible events.
This provides additional information.
Under appropriate conditions: \[\begin{aligned} \mathbb{P}(A \ \vert\ B) &= \frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)}\\ \\ \mathbb{P}(A) &= \mathbb{P}(A\ \vert\ B)\mathbb{P}(B) + \mathbb{P}(A \ \vert\ B^c)\mathbb{P}(B^c)\\ \\ \mathbb{P}(A \ \vert\ B) &= \frac{\mathbb{P}(B \ \vert\ A) \mathbb{P}(A)}{\mathbb{P}(B)}. \end{aligned}\]
If \(A\) and \(B\) are independent, then \(\mathbb{P}(A \cap B) = \mathbb{P}(A)\mathbb{P}(B)\) and \(\mathbb{P}(A\ \vert\ B) = \mathbb{P}(A)\).

Independence and complements

Exercise 1

Show the following:

If \(A\) and \(B\) are independent then so are \(A^{c}\) and \(B\).
If \(A\) and \(B\) are independent then so are \(A\) and \(B^{c}\).
if \(A\) and \(B\) are independent then so are \(A^{c}\) and \(B^{c}\)

Independence depends on \(\mathbb{P}\)

Let \(\Omega =\left\{ 1, 2, 3, 4, 5, 6, 7, 8 \right\}\)
Let \(A=\{1, 2, 3, 4\}\) and \(B=\left\{ 4, 8\right\}\)

Case 1

If \(\mathbb{P}(\{i\}) = 1/8 \qquad \forall i\), then

\[\begin{aligned} \mathbb{P}\left( A\cap B\right) &= \mathbb{P}\left( \left\{ 4 \right\} \right) = 1/8, \qquad \text{ and } \qquad \mathbb{P}\left( A\right) \mathbb{P}\left( B\right) = 4/ 8 \times 2/8 = 1/8. \end{aligned}\]

Case 2

If \(\mathbb{P}( \{ i \}) = i / 36 \qquad 1 \le i \le 8\), then

\[\begin{aligned} \mathbb{P}\left( A\cap B\right) &= \mathbb{P}\left( \left\{ 4\right\} \right) =4/36, \qquad \text{ and } \qquad \mathbb{P}\left( A\right) \, \mathbb{P}\left( B\right) = 10 / 36 \times 12/36. \end{aligned}\]

More than 2 independent events

Definition

We say that the events \(A_{1},A_{2},\dots\) are independent if, for any finite collection \(K = \{(i_1,\dots,i_k)\}\), \[\mathbb{P}\left( \bigcap_{i \in K} A_{i} \right) = \prod_{i \in K} \mathbb{P}(A_i).\]

For example, if \(n=3,\) then, \(A_1\), \(A_2\), and \(A_3\) are independent if and only if all of the following hold:

\[\begin{aligned} \mathbb{P}\left( A_{1}\cap A_{2}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{2}\right), \\ \mathbb{P}\left( A_{1}\cap A_{3}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{3}\right),\\ \mathbb{P}\left( A_{2}\cap A_{3}\right) &= \mathbb{P}\left( A_{2}\right) \, \mathbb{P}\left( A_{3}\right),\\ \mathbb{P}\left( A_{1}\cap A_{2}\cap A_{3}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{2}\right) \, \mathbb{P}\left( A_{3}\right). \end{aligned}\]

Coin flipping

We flip a fair coin twice. Define three events:

\(A = \{\text{first flip is H}\}\).
\(B = \{\text{second flip is H}\}\).
\(C = \{\text{flips show the same result}\}\).

Exercise 2

Show that \(A,B,C\) are pairwise independent, but not independent.

2 Continuity of probabilities

Continuity

Theorem

Suppose events \(A_1 \subseteq A_2 \subseteq A_3 \dots\) and \(\bigcup_{n=1}^\infty A_n= A\) for some set \(A\). Then \(\lim_{n \to \infty} \mathbb{P}(A_n) = \mathbb{P}(A)\).

Theorem

Suppose events \(A_1 \supseteq A_2 \supseteq A_3 \dots\) and \(\bigcap_{n=1}^\infty A_n= A\) for some set \(A\). Then \(\lim_{n \to \infty} \mathbb{P}(A_n) = \mathbb{P}(A)\).

Exercise 3

Recall the exercise from the first lecture:

A die is rolled repeatedly until we see a 6.

Let \(Z\) be the event that you eventually stop rolling. Show that \(\mathbb{P}(Z) = 1\).

3 Random variables

Review

So far, we have discussed how probability measures (functions) operate on events (subsets) of the sample space.
For any event \(A \subseteq \Omega\), \(\mathbb{P}(A)\) is a number between 0 and 1.
But events are not always the most natural way to talk about possible outcomes of an experiment.
It is often easier to describe an event \(A \subseteq \Omega\) with a single number.

Example

Recall our experiment “roll a die until you see ⚅ 6”.
There are many sequences of rolls (elements of \(\Omega\)) in the event “first 6 on roll 12”: \(E_{12}\).
What if we summarize these with “12”?

Random variables

A random variable is a way of summarizing events mathematically.
We use capital letters (usually, near the end of alphabet) to denote random variables (\(X\), \(Y\), \(Z\), etc.)

Definition

A random variable is a function from the sample space to a subset of the real numbers.

Therefore, a the random variable called \(X\) is any function

\[X \, : \, \Omega \ \rightarrow {\mathbb{R}}.\]

That is, for each \(\omega \in \Omega\),
\[X( \omega ) \in {\mathbb{R}}.\]

Example

Experiment: Toss a coin 5 times.
Sample space:

\[\Omega \, = \, \Bigl\{ ( x_{1}, x_{2}, \ldots, x_{5} ) \, : \ x_{i} \in \{ H, T \} \ \Bigr\}\]

Let \(X( \omega )=\) number of heads in \(\omega\).

\[\text{For example: } \qquad X \bigl( \, \{T, H, T, H, H\} \, \bigr) = X \bigl( \, \{H, T, T, H, H\} \, \bigr)= 3\]

Let \(Y( \omega )=\) longest run of heads in \(\omega\)

\[\text{For example: } \qquad Y \bigl( \, \{T, H, T, H, H\} \, \bigr) = 2\]

Canucks hockey

The Canucks play 2 games next week.
Each game can be won (W), lost (L), or lost in overtime (O).
The Canucks get points for wins (2), overtime losses (1), and losses (0).

Exercise 4

What is the sample space \(\Omega\)?
Define \(Z = \text{\# games played}\). Is \(Z\) a random variable? If so, enumerate how it operates on each \(\omega \in \Omega\)?
Define \(X = \text{\# points}\). Enumerate how it operates on each \(\omega \in \Omega\).
Which outcomes are in the event \(A = \{X \geq 3\}\)?

Random variables and events

Random variables are naturally used to describe events of interest.
For example \(\Bigl\{ X > 3 \Bigr\}\) or \(\Bigl\{ Y \le 2 \Bigr\}\)
Formally, this notation means:

\[\Bigl\{ X > 3 \Bigr\} \, = \, \Bigl\{ \omega \in \Omega \, : \, X \left( \omega \right) > 3 \Bigr\},\]

and

\[\Bigl\{ Y \le 2 \Bigr\} \, = \, \Bigl\{ \omega \in \Omega \, : \, Y \left( \omega \right) \le 2 \Bigr\}.\]

In other words, \(\Bigl\{ X > 3 \Bigr\}\) and \(\Bigl\{ Y \le 2 \Bigr\}\) are events.

The indicator function

The indicator function is a useful mathematical shorthand.

It is also a random variable: it is a function from \(\Omega \to {\mathbb{R}}\).

Definition

The indicator of the event \(A\) is the random variable given by \[ I_A(\omega) = \begin{cases} 1 & \omega \in A\\ 0 & \text{else}.\end{cases} \]

Exercise 5

Let \(A\) and \(B\) be events, and let \(X = I_A \times I_B\). Is \(X\) an indicator function? If so, of what event?

4 Distributions

Distributions

A probability associates each \(\omega \in \Omega\) with a number between 0 and 1 and satisfies the Probability axioms.
Random variables also operate on \(\Omega\). Taking these together induces a probability on \({\mathbb{R}}\).

Example

Suppose we flip a fair coin with \(\Omega = \{H, T\}\), and we know that \(\mathbb{P}(H) = \mathbb{P}(T) = 0.5\).

Then, considering the RV \(I_H\), we see \(\mathbb{P}(I_H =1) = \mathbb{P}(\{\omega : I_H(\omega) = 1\}) = \mathbb{P}(H)\)

Definition

If \(X\) is a random variable, then the distribution of \(X\) is the collection of probabilities \(\mathbb{P}(X \in B)\) for all subsets \(B\) of \({\mathbb{R}}\).

Some technical details

\(\Omega = \{H, T\}\), and \(\mathbb{P}(H) = \mathbb{P}(T) = 0.5\)

This defines \(\mathbb{P}\) on all subsets \(2^\Omega = \{\varnothing, \{T\}, \{H\}, \Omega\}\).
But considering a random variable \(X\), it isn’t really enough to know it’s probability only on it’s range.
We need to know more than that: we need to know, for every \(B \subset {\mathbb{R}}\), what is \(\mathbb{P}(X \in B) = \mathbb{P}(\{s\in\Omega : X(s) \in B\})\).
To fully specify the Distribution, we need to do this for every (nice) subset \(B\). The collection of these subsets is denoted \(\mathcal{B}\).

The mathematical details are at least at the level of Math 420.

More hockey

The Canucks play 2 games next week. Each game can be won (W), lost (L), or lost in overtime (O).
The Canucks get points for wins (2), overtime losses (1), and losses (0).
The Canucks are not very good. Suppose for each game (independently), \[\mathbb{P}(\omega) = \begin{cases} 0.3 & \omega = W\\ 0.2 & \omega = O \\ 0.5 &\omega = L.\end{cases}\]

Exercise 6

Let \(X\) be the number of wins. Compute \(\mathbb{P}(X = x)\) for all \(x\in{\mathbb{R}}\).
Write a formula for \(\mathbb{P}(X\in B)\) for any \(B\subset{\mathbb{R}}\).
Let \(Y\) be the number of points. Compute \(\mathbb{P}(Y = y)\) for all \(y\in{\mathbb{R}}\).
Write a formula for \(\mathbb{P}(Y\in B)\) for any \(B\subset{\mathbb{R}}\).