Module 2
Matias Salibian Barrera
Last modified — 05 Sep 2025
Finite Sample Spaces
In some cases, the sample space contains finite number of possible outcomes, and they are equally likely (they should receive equal probabilities).
Example
- Experiment: roll a fair die;
- Sample space: \(\Omega = \{1, 2, 3, 4, 5, 6\}\).
- If the die is fair we have \[\mathbb{P}(\{1\}) = \mathbb{P}(\{2\}) = \cdots = \mathbb{P}(\{6\})\]
Finite and equally likely outcomes
When there are finitely many outcomes, and they are equal likely, calculating probabilities involves counting outcomes in events/sets, which can be challenging
This chapter goes over some generic counting techniques, which can be used to calculate probabilities.
Equally likely outcomes
Let \(A\) be a subset (event) of a sample space \(\Omega\).
\[ \mathbb{P}\left( A\right) = \frac{\text{number of elements in }A}{\text{number of elements in }\Omega } \]
Basic principle
Suppose that a random experiment has 2 steps, where
- Step 1 has \(n_1\) possible outcomes, and
- Step 2 has \(n_2\) possible outcomes.
Then, \[\mbox{total number of possible outcomes} \ = \ n_1 \times n_2\]
We are implicitly assuming that the outcome of the second step does not depend on the outcome of the first step.
Extension to “k-step” experiments
If a random experiment has k steps.
- Step 1 has \(n_1\) possible outcomes,
- Step 2 has \(n_2\) possible outcomes,
\(\quad\quad\quad\vdots\quad\quad\quad\)
- Step k has \(n_k\) possible outcomes.
Then, \[\mbox{total number of outcomes} \ = \ n_1 \times n_2 \times n_3 \times \cdots \times n_k\]
Note
Implicit assumption: the outcomes of each step do not depend on each other.
License plates
Example
26 choices for the 1st, 2nd and 3rd entry, and
10 choices for each of the last 4 entries.
\[ \begin{aligned} \# \mbox{of plates} \ &= \ 26 \times 26\times 26\times 10\times 10\times 10\times 10 \\ & = \ 26^3 \times 10^4\\ & = 175,760,000 \end{aligned} \]
Note
Any letter can follow any other letter, any digit can appear with any other digit in a different slot. The outcomes of each slot do not affect each other.
Special license plates
Example
- 26 choices for the 1st entry, 25 choices for the 2nd entry, 24 choices for the 3rd entry,
- 10 choices for the 4th entry, 9 choices for the 5th entry, 8 choices for the 6th entry, and 7 choices for the 7th entry.
\[\begin{aligned} \# \mbox{of plates w/o rep } &= 26 \times 25 \times 24 \times 10 \times 9 \times 8 \times 7 \\ & = 78,624,000 \end{aligned} \]
Note
What answer would you get if you started counting “backwards”, options for the 7th place, then the 6th, etc?
Shuffling license plates
Example
We are randomly selecting a license plate from all possible ones, without any explicitly stated preference, so we will assume that each outcome (license plate) is equally likely.
Shuffling license plates (solution)
The sample space \((\Omega)\) is all possible plates.
The event of interest (\(A\)) consists of those plates without repetitions.
The probability that a randomly chosen plate has no repetitions is given by \[\begin{align*} \mathbb{P}(A) &= \frac{\# A}{\# \Omega} \\ & \\ &= \frac{ 26 \times 25 \times 24 \times 10 \times 9 \times 8 \times 7}{ 26^3 \times 10^4} \\ & \\ &= 0.44734 \approx 0.45 \end{align*}\]
Permutation
Definition
Example
- Some permutations of the elements of A are:
\[\left( 3, 5, 2, 4, 1 \right), \, \left( 2, 1, 3, 4, 5 \right), \, \left( 5, 4, 2, 1, 3 \right), \quad \mbox{ etc.} \]
- Those are all different permutations:
\[\left( 2, 1, 3, 4, 5 \right) \quad \neq \quad \left( 1, 2, 3, 4, 5 \right) \quad \neq \quad \left( 1, 3, 2, 4, 5 \right)\]
Counting permutations
How many permutations of the set \(\left\{ 1, 2, \ldots, n \right\}\) are there?
There are
- \(n\) choices for the 1st entry,
- \((n-1)\) choices for the 2nd entry,
- \((n-2)\) choices for the 3rd entry,
\(\quad\quad\quad\vdots\quad\quad\quad\)
- \(2\) choices for the \((n-1)^{\text{th}}\) entry, and
- \(1\) choice for the last \(n^{\text{th}}\) entry.
Thus, there are \[n \times (n-1) \times (n-2) \times \cdots \times 2 \times 1 \, = \, n! \] possible permutations of the set \(\left\{ 1, 2, \ldots, n \right\}\)
Counting permutations
Key assumption in counting permutations this way:
all \(n\) objects are distinct.
When some of them are not distinct, the number of possible distinct outcomes by re-arrangement (“permutation”) is not given by this formula.
We need a different formula.
Example
In how many different ways can the letters of “pepper” be arranged? Answer:
- If the letters were all different, we would have \(6!=720\) arrangements.
- But each arrangement does not change if:
- we permute the 3 p’s (3! ways of doing this)
- we permute the 2 e’s (2! ways of doing this)
- Thus, in the list of 6! arrangements, each “p/e” pattern appears \(3! \times 2!\) times: \[\text{number of arrangements}=\frac{6!}{3!2!}=60.\]
International chess
A chess tournament with 10 players:
| # of players | Country of origin |
|---|---|
| 4 | Russia |
| 3 | USA |
| 2 | Canada |
| 1 | Brazil |
Example: Counting outcomes
Example: Go Canada
Counting outcomes
If we see only nationalities in the final rank, how many outcomes are possible?
- There are \(10!\) permutations for the players.
- Each country-ranking pattern appears \(4! \times 3! \times 2!\) times (Russian players can be permuted in \(4!\) ways, USA players in \(3!\) ways, etc.).
- If we see only nationalities, there are \[\frac{10!}{4! \times 3! \times 2!} = \frac{3628800}{288}= 12600\] possible outcomes.
Go Canada
Count favourable outcomes: a Canadian player is ranked first.
- There are 2 choices for the top ranked player,
- There are \(9!\) ways of arranging (ranking) the other 9 players.
- Thus, we have \[2 \times 9! = 725760\] rankings where a Canadian player is first.
But if we see only nationalities, these outcomes are not distinct!
Go Canada
Each country pattern appears \(4! \times 3! \times 2!\) times.
The number of country rankings where Canada appears 1st is \[\frac{725760}{4! \times 3! \times 2!}= 2520.\]
Therefore, \[\begin{aligned} \mathbb{P}\left( \left\{ \text{Canada wins}\right\} \right) &=\frac{\#\text{of favorable outcomes}}{\#\text{of possible outcomes}} \\ &=\frac{2520}{12600}\\ &=0.20. \end{aligned} \]
In general, keep only 3 significant digits in your answers, unless there is an obvious simpler expression (e.g. “1/7”).
Combinations
Definition
Consider the set \[S=\left\{ a, b, c, d, e\right\}\]
The following are all the possible subsets of \(S\) of size 3:
| \(\{ a, b, c\}\) | \(\left\{ a, d, e\right\}\) |
| \(\{ a, b, d\}\) | \(\left\{ b, c, d\right\}\) |
| \(\{ a, b, e\}\) | \(\left\{ b, c, e\right\}\) |
| \(\{ a, c, d\}\) | \(\left\{ b, d, e\right\}\) |
| \(\{ a, c, e\}\) | \(\left\{ c, d, e\right\}\) |
- We call these sets “combinations” when we only care which elements are in the set.
Combinations
- Note that the order of the elements does not matter (these are sets, not sequences).
\[\Bigl\{ a, b, d \Bigr\} \, = \, \Bigl\{ d, a, b \Bigr\} \, = \, \Bigl\{ b, d, a \Bigr\} \] (and other possible rearrangements).
- Given a set of \(n\) distinct items \[S = \{s_1,\ s_2,\ \dots, s_n\}\] how many different combinations of size \(m\) can be formed? (where, of course, \(m \le n\))
Number of combinations
The number of combinations of size \(m\) out of a set of size \(n \ge m\) has various notations:
\[\binom{n}{m} = \left._{n} C_{m}\right. = C_m^n\]
- \(n\): size of the set from which combinations are drawn
- \(m\): size of the combinations
- we read it as “n choose m”
Tip
In this course we use \(\binom{n}{m}\).
Example
For example, when \(n=5\) and \(m=3\) we have
| \(\{ 1,2,3\}\) | \(\left\{ 1,4,5\right\}\) |
| \(\{ 1,2,4\}\) | \(\left\{ 2,3,4\right\}\) |
| \(\{ 1,2,5\}\) | \(\left\{ 2,3,5\right\}\) |
| \(\{ 1,3,4\}\) | \(\left\{ 2,4,5\right\}\) |
| \(\{ 1,3,5\}\) | \(\left\{ 3,4,5\right\}\) |
hence, the number of combinations must be
\[\binom{5}{3}= {10}\]
General formula
\[\binom{n}{m} = \frac{n!}{m!(n-m)!}\]
These are also called binomial coefficients.
They have many beautiful interpretations.
Intuition for the combination formula (first version)
Note that combinations can be constructed using permutations as follows:
- pick \(m\) items out of the \(n\) items, in order;
- each of these \(m\)-long permutations generates a subset;
- but many (how many?) of these permutations correspond to the same subset
Given a set of \(n\) distinct items
\[S = \Bigl\{ s_1, s_2, \ldots, s_n \Bigr\}\]
we can form
\[n \times (n-1) \times (n-2) \times \cdots \times (n-m+1)\]
different \(m\)-tuples by choosing from elements of \(S\)
Intuition for the combination formula (first version)
- However, any permutation of these \(m\) elements corresponds to the same subset
- There are \(m!\) ways to reorder these \(m\) elements
- In other words, each different subset appears \(m!\) times in the list of ordered \(m\)-tuples
- Hence, the number of combinations is: \[\begin{multline*} \frac{n \, (n-1) \, (n-2) \, \cdots \, (n - m + 1)}{m!} = \frac{n \, (n-1) \, (n-2) \, \cdots \, (n - m + 1)}{m!} \\ = \frac{n!}{m! \, (n - m)!}. \end{multline*}\]
Intuition for the combination formula (second version)
Another way to construct combinations using permutations uses the intuition:
- take a permutation of all \(n\) items in the set
- keep the first \(m\) terms of the permutation, and discard the remaining \((n-m)\) terms.
Intuition for the combination formula (second version)
\[\overset{\text{permutation}} {\overbrace{ ( i_{1},\ldots, i_{m},i_{m+1}, \ldots, i_{n} ) }} \to \overset{m}{\overbrace{i_{1},\ldots,i_{m}}} \text{ } \overset{n-m}{\overbrace{i_{m+1}, \ldots, i_{n}}} \to \overset{\text{combination}}{\overbrace{ \{ i_{1},\ldots, i_{m} \} }}.\]
Concrete example — \(m=3\) and \(n=5\).
\[\begin{aligned} \overset{\text{permutation}} {\overbrace{ (5,1,4,2,3 ) }} &\longrightarrow \overbrace{5,1,4} \overbrace{2,3} &\longrightarrow \overset{\text{combination}} {\overbrace {\{ 1,4,5 \}}} \\ \overset{\text{permutation}} {\overbrace{ ( 5,1,4,3,2 ) }} &\longrightarrow \overbrace{5,1,4} \overbrace{3,2} &\longrightarrow \overset{\text{combination}}{\overbrace{\{ 1,4,5 \} }} \\ \overset{\text{permutation}} {\overbrace{ ( 1,5,4,3,2 ) }} &\longrightarrow \overbrace{1,5,4} \overbrace{3,2} &\longrightarrow \overset{\text{combination}}{\overbrace{\{ 1,4,5 \} }} \end{aligned}\]
Permutations sharing the same first 3 and last 2 elements give the same combination.
Intuition for the combination formula (second version)
Each permutation generates a combination
But many permutations generate the same combination
The number of permutation needs to be corrected to remove repetitions
Two permutations lead to the same combination if and only if their first \(m\) elements are identical, and they need not to be in the same order.
Intuition for the combination formula (second version)
- For a permutation of length \(n\),
- there are \(m!\) ways to re-order its first \(m\) elements and
- \((n-m)!\) ways to re-order its last \(n-m\) elements:
- The total number of repetitions is \[m!\times \left(n-m\right) !\]
- There are \(n!\) distinct length-\(n\) permutations.
- Therefore, the number of distinct combinations is \[\binom{n}{m} = \, \frac{n!}{m!\left( n-m\right) !}.\]
Conventions
We define \[0!=1\]
So, for example \[ \begin{aligned} \binom{5}{5} &=\frac{5!}{5!\left( 5-5\right) !}=\frac{5!}{5!\text{ }0!}=1 &\text{(as you would guess)}\\ \binom{5}{0} &=\frac{5!}{0!\text{ }5!\text{ }}=1 &\text{(convention, but sensible)}. \end{aligned} \]
Lotto 6/49
Player chooses 6 distinct integers between 0 and 49.
Dealer randomly selects 6 distinct integers between 0 and 49.
The more matches, the bigger the prize.
What is the probability that the player matches \(k\) numbers? (for different values of \(k \in \{0, 1, ..., 6 \}\))
Lotto 6/49 solution
Lotto 6/49 solution: numerical values
| \(k\) | \(P\left( k \mbox{ matches } \right)\) |
|---|---|
| 0 | 0.444 |
| 1 | 0.410 |
| 2 | 0.128 |
| 3 | 0.017 |
| 4 | 0.001 |
| 5 | 1.66\(\times\) 10\(^{-5}\) |
| 6 | 6.29\(\times\) 10\(^{-8}\) |
Summary
We exclusively considered experiments with a finite number of possible outcomes, where [every outcome] is equally likely.
We count the number of outcomes in an event of interest (“favourable outcomes”), and the number of outcomes in the sample space (all possible outcomes).
The ratio of those two counts is the probability of the event.
1 Exploiting symmetry
General formula with proportional probability of intersection
Theorem
\[\mathbb{P}\left( A_{1}\cup A_{2}\cup \cdots \cup A_{n}\right) = \sum_{h=1}^{n}\left( -1\right) ^{h-1}\binom{n}{h}\text{ }p_{h}.\]
Every intersection of \(h\) subsets has probability \(p_h\): \[\begin{aligned} p_{1} &= \mathbb{P}\left( A_{1}\right) = \mathbb{P}\left( A_{2}\right) =\cdots = \mathbb{P}\left( A_{n}\right) \\ p_{2} &= \mathbb{P}\left( A_{1}\cap A_{2}\right) = \mathbb{P}\left( A_{1}\cap A_{3}\right) =\cdots = \mathbb{P}\left( A_{n-1}\cap A_{n}\right) \\ p_{3} &= \mathbb{P}\left( A_{1}\cap A_{2}\cap A_{3}\right) =\cdots = \mathbb{P}\left( A_{n-2}\cap A_{n-3}\cap A_{n}\right) \\ &\quad\quad \vdots \\ p_{n} &= \mathbb{P}\left( A_{1}\cap A_{2}\cap \cdots \cap A_{n}\right)\end{aligned} \]
Gentle proof
Let’s build some intuition first.
The probability of event \(A\), or \(B\), or \(C\), or \(\dots\)
\[\mathbb{P}\Bigl( A_{1}\cup A_{2}\cup \cdots \cup A_{n}\Bigr)\]
Suppose \(n=2\):
\[\begin{aligned} \mathbb{P}\left( A_{1}\cup A_{2}\right) & = \mathbb{P}\left( A_{1}\right) + \mathbb{P}\left( A_{2}\right) -\mathbb{P}\left( A_{1}\cap A_{2}\right) & \text{rule} \\ & \\ &= \mathbb{P}\left( A_{1}\right) + \mathbb{P}\left( A_{2}\right) & \mbox{desired events (inclusion)} \\ & \\ & \quad - \mathbb{P}\left( A_{1}\cap A_{2}\right) & \mbox{double counting (exclusion)} \end{aligned}\]
What about \(n=3\)?
Claim:
\[\begin{aligned} & \mathbb{P}\left( A_{1}\cup A_{2}\cup A_{3}\right)\\ \\ &= \mathbb{P}\left( A_{1}\right) + \mathbb{P}\left( A_{2}\right) + \mathbb{P}\left( A_{3}\right) & \mbox{desired events (inclusion)} \\ \\ &\quad -\ \mathbb{P}\left( A_{1}\cap A_{2}\right) -\mathbb{P}\left( A_{1}\cap A_{3}\right) - P\left(A_{2}\cap A_{3}\right) & \mbox{double counted (exclusion)} \\ \\ &\quad+\ \mathbb{P}\left( A_{1}\cap A_{2}\cap A_{3}\right) & \mbox{removed too much (inclusion)} \end{aligned}\]
Proof of claim: A union of \(n=3\) sets
Proof
General formula
\[\begin{aligned} \mathbb{P}\left( \mathop{\mathrm{\mathchoice{\bigcup}{\cup}{\cup}{\cup}}}_{i=1}^{n}A_{i}\right) & =\sum_{1\leq i\leq n} \mathbb{P}\left( A_{i}\right) & \text{inclusion} \\ & \quad-\ \sum_{i<j} \mathbb{P}\left( A_{i}\cap A_{j}\right) &\text{exclusion} \\ & \quad+\ \sum_{i<j<k} \mathbb{P}\left( A_{i}\cap A_{j}\cap A_{k}\right) & \text{inclusion} \\ & \quad -\ \sum_{i<j<k<h} \mathbb{P}\left( A_{i}\cap A_{j}\cap A_{k}\cap A_{h}\right) & \text{exclusion} \\ & \qquad \qquad \vdots \\ & \quad +\ (-1)^{n-1}\mathbb{P}\left( A_{1}\cap A_{2}\cdots \cap A_{n}\right) &\text{inclusion} \end{aligned}\]
Proportional probability of intersection
Recall that the intersection of any \(h\) subsets has the same probability \(p_h\)
In other words, we have:
\[\begin{aligned} \sum_{i=1}^n \mathbb{P}\left( A_{i}\right) &= \sum_{i} p_{1}= np_{1}=\binom{n}{1} p_{1}\\ \sum_{i M j} \mathbb{P}\left( A_{i}\cap A_{j}\right) &= \sum_{i < j} p_{2}=\binom{n}{2} p_{2}\\ \sum_{i < j < k} \mathbb{P}\left( A_{i}\cap A_{j}\cap A_{k}\right) &= \sum_{i < j < k} p_{3}= \binom{n}{3} p_{3}\\ &\quad\quad\quad\vdots \end{aligned}\]
Plug this into the previous formula and you’re done!
\[\mathbb{P}\left( A_{1}\cup A_{2}\cup \cdots \cup A_{n}\right) = \sum_{h=1}^{n}\left( -1\right) ^{h-1}\binom{n}{h}\text{ }p_{h}.\]
More problems
General comments
Events can often be expressed as unions, intersections, or complements of simpler events.
When the probabilities of these simpler events are known, or easy to calculate, we can then use of rules of probabilities for unions, complements, etc. to compute the probability of the event of interest.
Problem:
Suppose that the probability that a student passes this class is 50%. Moreover, out of a group of 4 students, suppose that \[\begin{aligned} \mathbb{P}(\text{one student passes}) &= 1/2, \text{ for any student}\\ \mathbb{P}(\text{two students pass}) &= (1/2)^2, \text{ for any 2 students}\\ \mathbb{P}(\text{three students pass}) &= (1/2)^3, \text{ for any 3 students}\\ \mathbb{P}(\text{four students pass}) &= (1/2)^4. \end{aligned}\]
Calculate the probability that
- at least one student passes
- no student passes
- only student number 4 passes
Problem setup
a. at least one student passes (solution)
b. no student passes (solution)
c. only student number 4 passes (solution)
c. only student number 4 passes (solution)
Problem: Couples at a party
\(n\) couples attend a dance party.
At one moment, each lady randomly picks a gentleman to dance.
What is the probability that none of them pick their partner?
Couples at a party (solution)
The number of failed processors
A cell phone has 4 GPUs to run ChatGPT.
The more working GPUs, the faster you can get answers to your homework.
Let \(G_i\) be the event that processor \(i\) fails.
The probability of failures is known: \[\begin{aligned} &\mathbb{P}(G_i) = 0.1 \quad \forall i\\ &\mathbb{P}( G_{i}\cap G_{j}) =0.01 \quad\forall i\neq j\\ &\mathbb{P}( G_{i}\cap G_{j}\cap G_{h}) =0.001 \quad\forall i\neq j\neq h\\ &\mathbb{P}( G_{1}\cap G_{2}\cap G_{3}\cap G_{4}) =0.0001 \end{aligned}\]
What is the probability that
- The phone has at least one failed GPU?
- The phone has all GPUs working?
- The phone has exactly one failed GPU?
- Say, the phone can solve your homework if at most one GPU fails. What is the probability that your homework is solved?
a. The phone has at least one failed GPU? (solution)
b. The phone has all GPUs working?
c. The phone has exactly one failed GPU? (solution)
d. Homework solved. (solution)
\[\begin{aligned} B &=\left\{ \text{Item works}\right\}\\ &=\left\{ \text{Zero failed components}\right\} \mathop{\mathrm{\mathchoice{\bigcup}{\cup}{\cup}{\cup}}}\left\{ \text{One failed component}\right\} \\ &=B_{0}\cup B_{1}\\ \Rightarrow \mathbb{P}(B) &= \mathbb{P}(B_0) + \mathbb{P}(B_1) \quad\quad \text{(disjoint)} \\ &= 0.6561 + 0.2916 = 0.9477 \end{aligned}\]
Note
For further practice, try to verify that
- the probability of two failed GPUs is 0.0486;
- and the probability of three failed GPUs is 0.0036.
Stat 302 - Winter 2025/26