Prerequisites

Random variables, expected value

Summary

Integration can be seen as a kind of limit operation - we approximate a given function by a sequence of step functions, etc. This section will treat the topic of interchanging integration with other limit operations. The centerpiece of this section is Lebesegue's Dominated Convergence Theorem, which has been called the swiss army knife for integration problems. Fatou's Lemma and the monotone convergence theorem are also quite useful, and they are proved in this section as well.

Integration and Limit

Define $X_n$ on $[0,1]$ as $X_n = n {\mathbf 1}_{(0,1/n)}$. That is, $X_n$ is $n$ with probability $1/n$ and 0 otherwise. Then

$$\lim_{n \rightarrow \infty} {\mathbb{E}}(X_n) = \lim_{n \rightarr... ... 0 = {\mathbb{E}}(0) = {\mathbb{E}}\left(\lim_{n \rightarrow \infty} X_n\right)$$ (1)

This example shows that integration and limit cannot always be exchanged. However, there are circumstances which allow one to interchange limits.

Theorem 1 (Monotone Convergence Theorem)   If $0\leq X_n \uparrow X$ then ${\mathbb{E}}(X_n) \uparrow {\mathbb{E}}(X)$.

Proof. Since ${\mathbb{E}}(X_n) \leq {\mathbb{E}}(X_{n+1})$, there is $\alpha \in [0, \infty]$ such that ${\mathbb{E}}(X_n) \to \alpha$ as $n \to \infty$. Furthermore, since $X_n \leq X$ we have ${\mathbb{E}}(X_n) \leq {\mathbb{E}}(X)$, and thus $\alpha \leq {\mathbb{E}}(X)$. Let $S$ be any simple random variable such that $0 \leq S \leq X$ and let $c$ be a constant $0<c<1$. Define $B_n = \{\omega:X_n(\omega) \geq cS(\omega)\}$ for $n = 1,2,3\ldots$. Note that $B_n$ is measurable, $B_n \subset B_{n+1}$, and since $X_n(\omega) \to X(\omega)>S(\omega)$, $\Omega = \bigcup B_n$. We have

$$\int_\Omega X_n \geq \int_{B_n} X_n \geq c \int_{B_n} S.$$

Let $n\to \infty$. The left hand side goes to $\alpha$ by definition, and the right hand side goes to $\int_\Omega S$. (To check the last statement, recall the definition of integral of a simple random variable and apply continuity of probability, that is, $A_n \uparrow A \implies {\mathbb{P}}A_n \uparrow {\mathbb{P}}A$).

Thus $\alpha \geq c {\mathbb{E}}(S)$ for any $c$ in $(0,1)$. If follows that $\alpha \geq {\mathbb{E}}(S)$. Since $S$ was an arbitrary simple random variable less than $X$, it follows from the definition of integral that $\alpha \geq {\mathbb{E}}(X)$. Since we already have $\alpha \leq {\mathbb{E}}(X)$, the proof is complete. $\qedsymbol$

Theorem 2 (Fatou's Lemma)   If $X_n \geq 0$ then $\liminf_{n \rightarrow \infty} {\mathbb{E}}X_n \geq {\mathbb{E}}(\liminf_{n \rightarrow \infty}X_n)$.

Proof. Let $Y_n(\omega) = \inf_{m\geq n} X_m(\omega).$ $X_n \geq Y_n$, and as $n\uparrow \infty$, $Y_n \uparrow Y= \liminf_{n \to \infty} X_n$. $\int X_n \geq \int Y_n$, so take a lim inf on both sides. Note that since $Y_n$ is increasing, so is $\int Y_n$, and thus $\lim_{n \to \infty} \int Y_n = \liminf_{n \to \infty} \int Y_n$. By the monotone convergence theorem, $\lim_{n \to \infty} \int Y_n = \int (\lim_{n \to \infty}Y_n) = \int( \liminf_{n \to \infty} X_n)$. $\qedsymbol$

Theorem 3 (Dominated Convergence Theorem)   If $X_n \rightarrow X$ a.s., $\vert X_n\vert\leq Y$ for all $n$, and ${\mathbb{E}}\vert Y\vert<\infty$, then ${\mathbb{E}}\vert X_n - X \vert \to 0$, and ${\mathbb{E}}(X_n) \rightarrow {\mathbb{E}}(X)$.

Proof. Since $\vert X_n - X\vert<2Y$, Fatou's Lemma applies to the functions $2Y - \vert X_n -X\vert$ and yields

$$\begin{multline*} {\mathbb{E}}(2Y) \leq \liminf_{n\to \infty} {\mathbb{E}}(2Y - ... ...{E}}(2Y) - \limsup_{n \to \infty} {\mathbb{E}}\vert X_n -X\vert. \end{multline*}$$

${\mathbb{E}}(2Y)$ is finite, so we can subtract to get $\limsup_{n \to \infty} {\mathbb{E}}\vert X_n - X\vert \leq 0$. Thus

$$\lim_{n \to \infty} E\vert X_n - X\vert = 0.$$

Since $\vert{\mathbb{E}}(X_n) - {\mathbb{E}}(X)\vert = \vert{\mathbb{E}}(X_n - X)\vert \leq {\mathbb{E}}\vert X_n - X\vert)$, we have ${\mathbb{E}}(X_n) \to {\mathbb{E}}(X)$. $\qedsymbol$

The simplest bound $Y$ in the dominated convergence theorem is a constant. This works because we are in a finite measure space - the situation is a little more delicate when we work with space with infinite measure, such as Lebesgue measure on ${\mathbb{R}}$.

References

Durrett, Probability: Theory and Examples, Section 1.3
Rudin, Real and Complex Analysis.