Lecture 5

Conditional Probability (Part 2)

Grace Tompkins

Last modified — 02 Oct 2026

Learning Outcomes

By the end of this lecture, students are anticipated to be able to:

Apply Bayes theorem
Determine whether events are independence based on conditional probabilities
Prove results where events are independent

Bayes’ Theorem

Sometimes we have information about \(\mathbb{P}(A\ \vert\ B)\) but require \(\mathbb{P}(B\ \vert\ A)\).
Bayes’ Theorem allows us to relate these conditional probabilities.

Bayes’ Theorem

Let \(A\) and \(B\) be arbitrary sets with \(\mathbb{P}(A)>0\). we have \[\mathbb{P}\left( B \ \vert\ A\right) \, = \, \frac{\mathbb{P}\left( A\ \vert\ B \right) \, \mathbb{P}\left( B\right) }{\mathbb{P}(A) }\]

You can also consider \(B_{1}\), \(B_{2}\), …, \(B_{n}\) is a partition of \(\Omega\), then for each \(i=1,\dots,n\), so that:

\[\mathbb{P}\left( B \ \vert\ A\right) \, = \, \frac{\mathbb{P}\left( A\ \vert\ B \right) \, \mathbb{P}\left( B\right) }{\sum_{j=1}^{n} \mathbb{P}\left( A\ \vert\ B_{j}\right) \, \mathbb{P}\left( B_{j}\right)}\]

Proof of Bayes Formula

\[ \begin{aligned} \mathbb{P}\left( B \ \vert\ A\right) &=\frac{\mathbb{P}\left( A\cap B\right)}{\mathbb{P}\left( A\right) } & \text{(Definition of conditional prob)} \\ &=\frac{ \mathbb{P}\left( A\ \vert\ B \right) \, \mathbb{P}\left( B \right)}{\mathbb{P}\left( A\right) } & \text{(Multiplication Rule)} \\ &=\frac{\mathbb{P}\left( A\ \vert\ B \right) \, \mathbb{P}\left( B \right) }{\sum_{j=1}^{n} \mathbb{P}\left( A\ \vert\ B_{j}\right) \, \mathbb{P}\left( B_{j}\right) } & \text{(Rule of Total Prob)} \end{aligned} \]

Bayes’ Theorem

In general, \(\mathbb{P}(A\ \vert\ B) \ne \mathbb{P}(B\ \vert\ A)\). The assumption that these two probabilities are equivalent is referred to as the “conditional probability fallacy” or “confusion of the inverse”

In some court systems, presenting evidence as a conditional probability has been banned due to the frequent confusion of the inverse
For example: \(\mathbb{P}(\text{DNA found at the crime scene} \ \vert\ \text{Guilty}) \ne \mathbb{P}(\text{Guilty} \ \vert\ \text{DNA found at the crime scene})\)

Flu Prevalence

Suppose that every patient who visits the ER is given a flu test.
Suppose that 30% of tests are positive.
A patient with flu tests positive 90% of the time.
A patient without flu tests negative 80% of the time.

Suppose you bring a friend to the ER.

What is the probability that your friend has the flu if they test positive?
What is the probability that your friend has the flu if they test negative?

Flu Prevalence

1 Independence

Independence

Independence:

We say that events \(A\) and \(B\) are independent if \[\mathbb{P}\left( A\cap B\right) \, = \, \mathbb{P}\left( A \right) \, \mathbb{P}\left( B\right).\]

If \(\mathbb{P}\left( B\right) >0\) and \(A\) and \(B\) are independent events, then:

\[\mathbb{P}\left( A\ \vert\ B\right) = \frac{\mathbb{P}\left( A\cap B\right) }{\mathbb{P}\left( B\right) } = \frac{\mathbb{P}\left( A\right) \mathbb{P}\left( B\right) }{\mathbb{P}\left( B\right) } = \mathbb{P}\left( A\right).\]

Knowledge about \(B\) occurring does not change the probability of \(A\) and vice versa.

Independence

We say that an event \(A\) is non-trivial if \(0<P\left( A\right) <1\).

If \(A\) and \(B\) are non-trivial events. Then,

If \(A\cap B=\varnothing\) then \(A\) and \(B\) are not independent
If \(A\subset B\) then \(A\) and \(B\) are not independent.

Independence

Proof:

Independence and Complements

Show the following:

If \(A\) and \(B\) are independent then so are \(A^{c}\) and \(B\).
If \(A\) and \(B\) are independent then so are \(A\) and \(B^{c}\).
If \(A\) and \(B\) are independent then so are \(A^{c}\) and \(B^{c}\)

Independence and Complements

More than 2 Independent Events

We say that the events \(A_{1},A_{2},\dots\) are independent if, for any finite collection \(K = \{(i_1,\dots,i_k)\}\), \[\mathbb{P}\left( \bigcap_{i \in K} A_{i} \right) = \prod_{i \in K} \mathbb{P}(A_i).\]

For example, if \(n=3,\) then, \(A_1\), \(A_2\), and \(A_3\) are independent if and only if all of the following hold:

\[\begin{aligned} \mathbb{P}\left( A_{1}\cap A_{2}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{2}\right), \\ \mathbb{P}\left( A_{1}\cap A_{3}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{3}\right),\\ \mathbb{P}\left( A_{2}\cap A_{3}\right) &= \mathbb{P}\left( A_{2}\right) \, \mathbb{P}\left( A_{3}\right),\\ \mathbb{P}\left( A_{1}\cap A_{2}\cap A_{3}\right) &= \mathbb{P}\left( A_{1}\right) \, \mathbb{P}\left( A_{2}\right) \, \mathbb{P}\left( A_{3}\right). \end{aligned}\]

Coin Flipping

We flip a fair coin twice. Define the following three events:

\(A = \{\text{first flip is H}\}\).
\(B = \{\text{second flip is H}\}\).
\(C = \{\text{flips show the same result}\}\).

Show that \(A,B,C\) are pairwise independent, but not independent.

Coin Flipping

Checkpoint 1

You may have:

HANDWRITTEN Cheat Sheet
Non-graphing calculator
Writing materials

Phones and laptops put away, please! You can have 15 minutes.

. . .

Checkpoint 2 (on Section 1.5: Conditional Probability and Independence) is next Friday!

Same rules as Checkpoint 1!