Sagar

Measure Theoretic Probability

$(\Omega, \mathcal{A}, \mathbb{P})$ is a probability space if $(\Omega, \mathcal{A})$ is a measurable space and $\mathbb{P}$ is a measure on $\mathcal{A}$ such that $\mathbb{P}(\Omega) =1$.

Our goal is to understand what a probability space is, which in turn consists of a measurable space and a measure. So in the following we will define them.

We will first define what an algebra on a set $\Omega$ is and then we will define what a $\sigma-$algebra is, $\Omega$ and a $\sigma-$algebra on $\Omega$ define a measurable space. A measure is a function from a $\sigma-$algebra to the reals. A measure space is a triplet consisting of a set $\Omega$, a sigma algebra $A$ on it and a measure $\mu$ on the sigma algebra.

Algebra: A collection of subsets of a set $\Omega$ is called an algebra $A$ if :

  • $\Omega \in A$
  • $C,B \in A \rightarrow C\cup B, C \cap B \in A$
  • $B \in A \rightarrow \Omega \backslash B \in A$ [Closure under finite unions]

    $\sigma-$Algebra: A collection of subsets of a set $\Omega$ is called sigma algebra $A$ if :

  • $\Omega \in A$
  • $C,B \in A \rightarrow C\cup B, C \cap B \in A$
  • $B \in A \rightarrow \Omega \backslash B \in A$
  • If $C_i \in A$ $\rightarrow$ $\cup_{i \geq 1} C_i \in A$ [Closure under countably infinite unions]
    Hence, $\sigma-$Algebra additionally requires closure not only under pairwise union but also under countably infinite union.

    Note: $\{A_i\}_{i\in I}$ are some sigma algebras on $X$, then $\bigcap_{i\in I} A_i$ is also a sigma algebra. Note that $\bigcap_{i\in I} A_i$ is the smallest sigma algebra with all the properties in $\{A_i\}_{i\in I}$

    Example of algebras that is not a $\sigma$-algebra:

  • Let $A$ be the set of all finite subsets of $\mathbb{N}$ or the subsets with finite complement. Clearly, $A$ does not contain the set of odd numbers. Now, the reason this is not a $\sigma-$ algebra is that it is not closed under infinite unions as then it would need to contain the set of all odd numbers.
  • Let $X$ be an infinite set, and $A$ be the collection of all subsets of $X$ which are finite or have finite complement. Then $A$ is an algebra of sets which is not a sigma algebra.

    $\sigma(M)$ or sigma algebra generated by $M$, where $M \subseteq P(X)$ is some set is the smallest sigma algebra containing $M$. i.e. : $$\sigma(M):= \bigcap_{A \subseteq M }A$$ where $A$ are all possible sigma algebras on $X$.

    Topological Space [via Open sets]: A topology on a set $X$ may be defined as a collection $\tau$ of subsets of $X$, called open sets and satisfying the following axioms:

  • $\emptyset$ and $X$ are in $\tau$
  • Any arbitrary union (finite or infinite) belongs in $\tau$
  • The intersection of any finite number of members of $\tau$ belong in $\tau$ Any element of a topology is known as an open set. The collection $\tau$ is called a topology on $X$.

    Borel-Sigma Algebra: The Borel-sigma algebra denoted by $B$, of the Topological space $(X,\tau)$ is the sigma algebra generated by the family $\tau$ of open sets. It's elements are called Borel Sets.

    Lemma: Let $C = \{(a,b): a < b\}$. Then $\sigma(C) = B_R$ is the borel field generated by the family of all open intervals in $C$.

    Measurable Space: The pair of a set $\Omega$ and a $\sigma-$ algebra on it.

    Measure $\mu$ is a function defined on a measurable space $(\Omega, A)$ from it's $\sigma-$ algebra to reals. Such that:
  • Non-negativity: For all $E \in A$, we have that $\mu(E) \geq 0$
  • Null empty set: $\mu(\emptyset) = 0$
  • Countable additivity or sigma additivity: For all countable collections $\{E_k\}^{\infty}_{k=1}$ of pair-wise disjoint sets in $A$, $$\mu(\bigcup_{k=1}^{\infty} E_k) = \sum_{k=1}^{\infty} \mu(E_k)$$

    Measure Space is a triplet $(\Omega,A,\mu )$

    Probability Measure $\mathbb{P}$ is a measure such that $\mu(\Omega) = 1$

    Probability Space is a measure space $(\Omega,A,\mathbb{P} )$

    Continuity An equivalent formulation of countable additivity is continuity. $\mathbb{P}$ is a finitely additive and continuous if for any decreasing sequence $B_n \in A$ such that $B_n \subseteq B_{n+1}$, $$B = \bigcap_{n \geq 1 } B_n \rightarrow \lim_{n \rightarrow \infty} \mathbb{P}(B_n) = \mathbb{P}(B)$$

    Continuity $\leftrightarrow$ Countable additivity.

    Lebesgue Measure: There is a unique measure $\lambda$ on $(\mathbb{R},B_{\mathbb{R}})$ that satisifes: $$\lambda([a,b]) = b-a$$ for every finite interval $[a,b]$, $-\infty < a \leq b < \infty$. This is called the Lebesgue measure.
    If we restrict $\lambda$ to the measure space $([0,1], B_{[0,1]})$, then $\lambda$ is a probability measure.

    Zero Measure: A $\mu$-measurable set $E$ is said to have $\mu$-measure zero if $\mu(E) = 0$

    Almost everywhere:

    A particular property is said to hold "almost everywhere" if the set of points for which the property does not hold has a measure zero.
    Example: "A function vanishes almost everywhere", "$f=g$ almost everywhere"

    Kolmogorov's Axioms define a list of axioms for a probability measure. Let $P:E\rightarrow [0,1]$ be a probability measure and $E$ be some sigma algebra generated by X.

  • Axiom 1: $P[A] \leq 1$ for all $A \in E$
  • Axiom 2: $P[X]=1$
  • Axiom 3: $P[A_1\cup A_2,....,\cup A_n] = \sum_{i}P[A_i]$, where $A_i$ are disjoint sets.