Lecture 15

Convergence, Part I


Grace Tompkins

Last modified — 21 Jun 2026

Learning Outcomes

By the end of this lecture, students are anticipated to be able to:

  • Define convergence in probability
  • Determine when a sequence converges in probability
  • Define and apply the Weak Law of Large Numbers

1 Convergence in Probability

Convergence (Calculus)

  • In calculus, you looked at the limit of a sequence of numbers, \(a_n\), as \(n\) goes to infinity.
  • \(\lim_{n \to \infty} 1/n = 0;\)
  • \(\lim_{n \to \infty} (1 + 1/n)^n = e;\)
  • \(\lim_{n \to \infty} (1 + a/n)^n = e^a \text{ for all } a \in {\mathbb{R}};\)
  • \(\lim_{n \to \infty} (1 + a_n/n)^n = e^a \text{ if } a_n \to a.\)

  • In probability, we look at the limit of a sequence of random variables, \(X_n\), as \(n\) goes to infinity.

  • This turns out to be more complicated, because there are different modes of convergence.

We will discuss 3 types of convergence.

Convergence in Probability

A sequence of random variables \(X_1,X_2,\ldots,X_n,\ldots\) converges in probability to a random variable \(X\) if for all \(\epsilon > 0\), \[\lim_{n \to \infty} \mathbb{P}(|X_n - X| < \epsilon) = 1.\]

  • We can think of \(a_n = \mathbb{P}(|X_n - X| < \epsilon)\) as a sequence of numbers that goes to one as \(n\) goes to infinity.
  • This is the most similar to limits of sequences of numbers.
  • Can also be written as \(\mathbb{P}(|X_n - X| \geq \epsilon) \to 0\).
  • Common notation: \(X_n \overset{p}{\to}X\).

Convergence in Probability

Suppose \(\mathbb{P}(X_n = 1 - 1/n) = 1\) and \(\mathbb{P}(Y=1)=1\). Show that the sequence \(\{X_n\}\) converges in probability to \(Y\).

Convergence in Probability

Convergence in Probability

Let \(U \sim {\mathrm{Unif}}(0,1)\) and define

\[X_n = U + B_n,\]

where \(B_n\sim {\mathrm{Bern}}(1/n)\) are independent Bernoulli random variables, also independent of \(U\).

Show \(X_n\overset{p}{\to}U\).

Convergence in Probability

Convergence in Probability

Let \(U_1, U_2, \ldots\) be i.i.d. \({\mathrm{Unif}}(0,1)\) random variables. Define \(Y_n = \max\{U_1, \ldots, U_n\}\).

Show that \(Y_n \overset{p}{\to}1\).

Hint: \(|Y_n - 1| > \epsilon\) if and only if \(Y_n < 1 - \epsilon\).

Convergence in Probability

Weak Law of Large Numbers (WLLN)

Let \(X_{1},X_{2}, \ldots, X_{n} \, \ldots\) be independent and identically distributed (i.i.d) random variables with finite mean \(\mu\). Then,

\[\overline{X}_n = \frac{1}{n} \, \sum_{i=1}^n X_i \overset{p}{\to}\mu.\]

Interpretation

The distribution of \(\overline{X}_n\) gets more and more concentrated around \(\mu\) as \(n\) increases.

Weak Law of Large Numbers (WLLN)

Let \(X_n\) be the sum of the squares of \(n\) independent rolls of a fair six-sided die.

That is \[X_n = \sum_{i=1}^n X_{n,i}^2,\] where \(X_{n,i}\) is the result of the \(i\)-th die roll.

Show that \(n^{-1}X_n \overset{p}{\to}m\) for some \(m\) (find \(m\) explicitly).

Weak Law of Large Numbers (WLLN)

Proof of WLLN (with an extra condition)

  • Assume that \(\operatorname{Var}( X_{i}) =\sigma ^{2} < \infty\) (same for all \(i\)). This is not required for the WLLN, but it makes the proof easier.

Then, by Chebyshev’s inequality, for all \(\epsilon > 0\), \[\begin{aligned} \mathbb{P}\left( \left\vert \overline{X}_n-\mu \right\vert \geq \epsilon \right) &\leq \frac{\operatorname{Var}(\overline{X}_n)}{\epsilon^2} = \frac{\sigma^2/n}{\epsilon^2} \to 0. \end{aligned}\]

2 Convergence Almost Surely (with Probability One)

Convergence Almost Surely (with Probability One)

A sequence of random variables \(X_1,X_2,\ldots,X_n,\ldots\) converges almost surely (or w.p. 1) to a random variable \(X\) if for all \(\epsilon > 0\), \[\mathbb{P}\left(\lim_{n \to \infty} |X_n - X| < \epsilon\right) = 1.\]

  • This is a stronger notion of convergence than convergence in probability.
  • This is equivalent to saying that \(\mathbb{P}(\lim_{n \to \infty} X_n = X) = 1\).
  • Writing this statement more explicitly, we are really demanding that \[\mathbb{P}\left(\{\omega : \lim_{n \to \infty} X_n(\omega) = X(\omega)\}\right) = 1.\]
  • Common notation: \(X_n \overset{a.s.}{\to}X\).

Convergence Almost Surely (with Probability One)

Let \(U \sim {\mathrm{Unif}}(0, 1)\) and

\[ X_n = \begin{cases} 3 & U \le \frac{2}{3} - \frac{1}{n}\\ 8 & \text{otherwise}\end{cases} \]

\[ Y = \begin{cases} 3 & U \le \frac{2}{3}\\ 8 & \text{otherwise}\end{cases} \]

Does \(X_n \overset{a.s.}{\to}Y\)?

Convergence Almost Surely (with Probability One)

Convergence Almost Surely (with Probability One)

Let \(Y \sim {\mathrm{Unif}}(0, 1)\) and \(X_n = Y^n\). Prove that \(X_n \overset{a.s.}{\to}0\).

Convergence Almost Surely (with Probability One)

Convergence Almost Surely (with Probability One)

Let \(U \sim \text{Uniform}(0,1)\). Define the sequence \(X_1, X_2, X_3, \ldots\) by partitioning \([0,1]\) into successive blocks:

\[ B_1 = [0,1]; \quad B_2 = \bigl[0,\tfrac{1}{2}\bigr]; \quad B_3 = \bigl[\tfrac{1}{2},1\bigr]; \quad B_4 = \bigl[0,\tfrac{1}{3}\bigr]; \quad B_5 = \bigl[\tfrac{1}{3},\tfrac{2}{3}\bigr]; \quad \ldots \]

In general, row \(m\) contains \(m\) blocks each of length \(1/m\), tiling \([0,1]\) completely. The blocks are indexed \(n = 1, 2, 3, \ldots\) by reading left to right across rows. Set

\[ X_n(\omega) = \mathbf{1}[\omega \in B_n]. \]

Show that \(X_n \xrightarrow{P} 0\) but \(X_n \not\to 0\) almost surely.

Convergence Almost Surely (with Probability One)

Convergence Almost Surely (with Probability One)

To Do

  • Work on Assignment 5, due Wednesday June 17, 11:59pm on Gradescope.
  • Read Chapter 4.4 before next class.