Conditional Probability and Independence
Last modified — 23 May 2026
By the end of this lecture, students are anticipated to be able to:
We will continue our discussion of conditional probability.
If \(\mathbb{P}(A_1) > 0\): \[ \mathbb{P}\left( A_{1} \cap A_{2}\right) = \mathbb{P}\left(A_{2}\ \vert\ A_{1}\right) \, \mathbb{P}\left( A_{1}\right). \]
If \(\mathbb{P}(A_1),\ \mathbb{P}(A_1 \cap A_2),\dots,\ P(A_1 \cap A_2 \cap \dots \cap A_{n-1}) > 0\), then \[\begin{aligned} \mathbb{P}\left( A_{1}\cap A_{2}\cap \cdots \cap A_{n}\right) &= \mathbb{P}\left( A_{n}\ \vert\ A_{1}\cap A_{2}\cap \cdots \cap A_{n-1}\right) \\ &\quad \times \mathbb{P}\left( A_{n-1}\ \vert\ A_{1}\cap A_{2}\cap \cdots \cap A_{n-2}\right) \\ & \quad \times \cdots \times\\ & \quad \times \mathbb{P}\left( A_{3}\ \vert\ A_{1}\cap A_{2}\right) \times \mathbb{P}\left( A_{2}\ \vert\ A_{1}\right) \times \mathbb{P}\left( A_{1}\right) \end{aligned}\]
An urn has 10 red balls and 40 black balls.
Three balls are randomly drawn without replacement.
Calculate the probability that:
The 3rd ball is red given that the 1st is red and the 2nd is black.
The first drawn ball is red, the 2nd is black and the 3rd is red.
We say that \(B_{1}, \ldots, B_{n}\) is a partition of \(\Omega\) if
They are disjoint \[ B_{i}\cap B_{j} \, = \, \varnothing \quad \mbox{ for } i \ne j \, , \]
They cover the whole sample space: \(\bigcup_{i=1}^{n} B_{i} \, = \, \Omega\)
A simple partition is any event \(A\) and its complement \(A^c\).
If \(B_{1}, \ldots, B_{n}\) is a partition of \(\Omega\), then, for any \(A \in \Omega\), \[\mathbb{P}\left( A\right) =\sum_{i=1}^{n} \mathbb{P}\left( A\ \vert\ B_{i}\right) \, \mathbb{P}\left( B_{i}\right).\]
\(A = A \cap \Omega = A \cap \left( \bigcup _{i=1}^{n}B_{i}\right) = \bigcup_{i=1}^{n}\left( A \cap B_{i}\right)\)
The events \(\left( A \cap B_{i}\right)\) are disjoint.
Therefore, by Axiom 3, we have \[\begin{aligned} \mathbb{P}\left( A\right) & = \mathbb{P}\left( \bigcup_{i=1}^{n} A \cap B_{i} \right) \\ &=\sum_{i=1}^{n} \mathbb{P}\left( A\cap B_{i}\right) \\ &=\sum_{i=1}^{n} \mathbb{P}\left( A\ \vert\ B_{i}\right) \, \mathbb{P}\left( B_{i}\right). \end{aligned}\]
If a new patient walks into the ER, what is the probability that they test positive for the flu?
Sometimes we have information about \(\mathbb{P}(A\ \vert\ B)\) but require \(\mathbb{P}(B\ \vert\ A)\).
Bayes’ Theorem allows us to relate these conditional probabilities.
Bayes’ Theorem
Let \(A\) and \(B\) be arbitrary sets with \(\mathbb{P}(A)>0\). we have \[\mathbb{P}\left( B \ \vert\ A\right) \, = \, \frac{\mathbb{P}\left( A\ \vert\ B \right) \, \mathbb{P}\left( B\right) }{\mathbb{P}(A) }\]
You can also consider \(B_{1}\), \(B_{2}\), …, \(B_{n}\) is a partition of \(\Omega\), then for each \(i=1,\dots,n\), so that:
\[\mathbb{P}\left( B \ \vert\ A\right) \, = \, \frac{\mathbb{P}\left( A\ \vert\ B \right) \, \mathbb{P}\left( B\right) }{\sum_{j=1}^{n} \mathbb{P}\left( A\ \vert\ B_{j}\right) \, \mathbb{P}\left( B_{j}\right)}\]
\[ \begin{aligned} \mathbb{P}\left( B \ \vert\ A\right) &=\frac{\mathbb{P}\left( A\cap B\right)}{\mathbb{P}\left( A\right) } & \text{(Definition of conditional prob)} \\ &=\frac{ \mathbb{P}\left( A\ \vert\ B \right) \, \mathbb{P}\left( B \right)}{\mathbb{P}\left( A\right) } & \text{(Multiplication Rule)} \\ &=\frac{\mathbb{P}\left( A\ \vert\ B \right) \, \mathbb{P}\left( B \right) }{\sum_{j=1}^{n} \mathbb{P}\left( A\ \vert\ B_{j}\right) \, \mathbb{P}\left( B_{j}\right) } & \text{(Rule of Total Prob)} \end{aligned} \]
In general, \(\mathbb{P}(A\ \vert\ B) \ne \mathbb{P}(B\ \vert\ A)\). The assumption that these two probabilities are equivalent is referred to as the “conditional probability fallacy” or “confusion of the inverse”
Suppose you bring a friend to the ER.
You select a door, the host then opens one of the 2 remaining doors, revealing a goat 🐐. The host asks
Would you like to switch to the remaining closed door?
Show that the probability of winning the car if you switch doors is 2/3.
Independence:
We say that events \(A\) and \(B\) are independent if \[\mathbb{P}\left( A\cap B\right) \, = \, \mathbb{P}\left( A \right) \, \mathbb{P}\left( B\right).\]
If \(\mathbb{P}\left( B\right) >0\) and \(A\) and \(B\) are independent events, then:
\[\mathbb{P}\left( A\ \vert\ B\right) = \frac{\mathbb{P}\left( A\cap B\right) }{\mathbb{P}\left( B\right) } = \frac{\mathbb{P}\left( A\right) \mathbb{P}\left( B\right) }{\mathbb{P}\left( B\right) } = \mathbb{P}\left( A\right).\]
We say that an event \(A\) is non-trivial if \(0<P\left( A\right) <1\).
If \(A\) and \(B\) are non-trivial events. Then,
\[ \mathbb{P}\left( A\ \vert\ B\right) =\frac{\mathbb{P}\left( A\cap B\right) }{\mathbb{P}\left( B\right) } = \frac{0}{\mathbb{P}\left( B\right)}= 0 \neq \mathbb{P}\left( A\right) \]
\[ \mathbb{P}\left( A \ \vert\ B\right) =\frac{\mathbb{P}\left( A\cap B\right) }{\mathbb{P}\left( B\right) } =\frac{\mathbb{P}\left( A\right)}{\mathbb{P}\left( B\right) } \neq \mathbb{P}\left( A \right) \]
Show the following:
We say that the events \(A_{1},A_{2},\dots\) are independent if, for any finite collection \(K = \{(i_1,\dots,i_k)\}\), \[\mathbb{P}\left( \bigcap_{i \in K} A_{i} \right) = \prod_{i \in K} \mathbb{P}(A_i).\]
For example, if \(n=3,\) then, \(A_1\), \(A_2\), and \(A_3\) are independent if and only if all of the following hold:
\[\begin{aligned} \mathbb{P}\left( A_{1}\cap A_{2}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{2}\right), \\ \mathbb{P}\left( A_{1}\cap A_{3}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{3}\right),\\ \mathbb{P}\left( A_{2}\cap A_{3}\right) &= \mathbb{P}\left( A_{2}\right) \, \mathbb{P}\left( A_{3}\right),\\ \mathbb{P}\left( A_{1}\cap A_{2}\cap A_{3}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{2}\right) \, \mathbb{P}\left( A_{3}\right). \end{aligned}\]
We flip a fair coin twice. Define the following three events:
Show that \(A,B,C\) are pairwise independent, but not independent.
Stat 302 - Winter 2025/26