Prerequisites

Random variable, expected value, monotone convergence theorem

Summary

You will be presented the change of variables formula, its proof, and a simple example.

Change of Variables

Let $(\Omega, {\cal F},{\mathbb{P}})$ be a probability space and $X:\Omega \to S$ a $({\cal F}\backslash \mathcal{S})$-measurable random variable. $X$ induces a new probability measure ${\mathbb{P}}_X$ on $(S,\mathcal{S})$.

Definition 1   ${\mathbb{P}}_X(A) = {\mathbb{P}}(\omega : X(\omega) \in A) = {\mathbb{P}}(\omega : \omega=X^{-1}(A))$ is called the ${\mathbb{P}}$ law of $X$ or the ${\mathbb{P}}$ distribution of $X$.

Theorem 2 (Change of variable formula)   Let $f$ be measurable function from $(S,\mathcal{S})$ to $(R,\mathcal{R})$. If $f\geq 0$ or ${\mathbb{E}}\vert f(X) \vert<\infty$ then

$${\mathbb{E}}f(X) = \int_{\Omega} f(X(\omega)) d{\mathbb{P}}= \int_S f(x) d{\mathbb{P}}_X$$

Proof. We will approach this proof in four different cases, each of which will build on the previous. First, for indicator functions, take $A \in \mathcal{S}$.

$${\mathbb{E}}{\mathbf 1}_A(X) = {\mathbb{P}}(X \in A) = {\mathbb{P}}_X(A) = \int_{S} {\mathbf 1}_A(x) d{\mathbb{P}}_X$$

Second, look at simple functions. $f$ is of the form $f(x) = \sum_{i=1}^n k_i {\mathbf 1}_{A_i}$. Where $k_i \in {\mathbb{R}}$ and $A_i \in \mathcal{S}$ for $i = 1,2,3,...,n$. From the linearity of integration, the case for indicator functions, and the linearity of integration, we arrive at the following equalities:

$${\mathbb{E}}f(X) = \sum_{i=1}^n k_i {\mathbb{E}}{\mathbf 1}_{A_i}... ...\int_{S} {\mathbf 1}_{A_i}(x) d{\mathbb{P}}_X ] = \int_{S} f(x) d{\mathbb{P}}_X$$

Next, look at nonnegative functions $f$. Note: we will use $\lfloor x \rfloor$ to denote the greatest integer less than $x$. E.g. $\lfloor 5.3 \rfloor = 5$. Also, take $x \wedge y =$ min$(x,y)$. Let $f_n(x) = (\lfloor 2^n f(x) \rfloor /2^n) \wedge n$ and notice that it is a simple function. So, ${\mathbb{E}}f_n(X) = \int_{S} f_n(x) d{\mathbb{P}}_X$. Additionally, $f_n \uparrow f$ and $\int_{S} f_n(x) d{\mathbb{P}}_X \uparrow \int_{S} f(x) d{\mathbb{P}}_X$ So, we can use the monotone convergence theorem.

$${\mathbb{E}}f(X) = \lim_n {\mathbb{E}}f_n(X) = \lim_n \int_{S} f_n(x)d{\mathbb{P}}_X = \int_{S} f(x)d{\mathbb{P}}_X$$

Lastly, let's observe integrable functions. Split $f$ into the difference between its positive and negative parts, $f(x) = f^+(x) - f^-(x)$. The condition ${\mathbb{E}}\vert f(X) \vert<\infty$ ensures ${\mathbb{E}}f^+(X)<\infty$ and ${\mathbb{E}}f^-(X)<\infty$. So, we can proceed by using the case of nonnegative functions and the linearity of integration.

$${\mathbb{E}}f(X) = {\mathbb{E}}f^+(X) - {\mathbb{E}}f^-(X) = \int... ...d{\mathbb{P}}_X - \int_{S} f^-(x)d{\mathbb{P}}_X = \int_{S} f(x)d{\mathbb{P}}_X$$

$\qedsymbol$

References

Durrett, Probability: Theory and Examples, Section 1.3.