Measure¶
Measurable spaces (\(\sigma\)-algebras)¶
Before we define a measure recall that a measure has for domain a collection of sets. For a measure to have some desirable properties this collection of sets cannot be left unrestricted. It turns out that the appropriate family of sets to be consider is that of \(\sigma\)-algebra.
Definition 22.1: \(\sigma\)-algebra
Let \(S\) be a set and \(\mathcal{A}\subseteq2^{S}\) a family of its subsets. \(\mathcal{A}\) is a \(\sigma\)-algebra if and only if:
- \(\emptyset,S\in\mathcal{A}\).
- \(A\in\mathcal{A}\) implies \(A^{c}=S\backslash A\in\mathcal{A}\). We say that \(\mathcal{A}\) is closed under complement.
-
\(A_{n}\in\mathcal{A}\) for \(n=1,\ldots\) implies \(\cup A_{n}\in\mathcal{A}\). We say that \(\mathcal{A}\) is closed under countable union.
- \(\mathcal{A}\) is closed under countable intersection because \(\cap A_{n}=\left(\cup A_{n}^{c}\right)^{c}\).
If \(\mathcal{A}\) is only closed under finite union (or intersection) then \(\mathcal{A}\) is an algebra.
A \(\sigma\)-algebra imposes certain consistency to the family of sets under consideration. The way to interpret it is that only subsets of the \(\sigma\)-algebra can be known, hence measured. Because of property (i) it is possible to know when none or all of the outcomes occurred. Also if there is an outcome that occurred it must be possible to determine if it didn't. Finally if it is possible to determine that some outcomes occurred individually it can also be determined if at least one or all of them were realized.
It is instructive to consider two simple examples of \(\sigma\)-algebras that arise from throwing a 4 sided dice, then \(S=\left\lbrace 1,2,3,4\right\rbrace\). One (trivial) \(\sigma\)-algebra is:
Another one is the \(\sigma\)-algebra generated by the collection \(\left\lbrace \left\lbrace 1\right\rbrace ,\left\lbrace 2\right\rbrace ,\left\lbrace 3\right\rbrace ,\left\lbrace 4\right\rbrace \right\rbrace\), then:
In this case \(\mathcal{A}=2^{S}\), but this is not necessarily true, imagine that one can only determine if an even number was thrown, then the outcomes are \(\left\lbrace \left\lbrace 1,3\right\rbrace ,\left\lbrace 2,4\right\rbrace \right\rbrace\), the \(\sigma\)-algebra is:
When \(S\) has uncountably many elements this process cannot be exemplified as easily but one can always define the \(\sigma\)-algebra generated by a subset \(\mathcal{A}\subseteq2^{S}\) as the intersection of all \(\sigma\)-algebras that contain \(\mathcal{A}\). Clearly the arbitrary intersection of \(\sigma\)-algebras is again a \(\sigma\)-algebra.
Now that we have defined a \(\sigma\)-algebra its possible to say what a measurable set and a measurable space are:
Definition 22.2: Measurable Space
A pair \(\left(S,\mathcal{A}\right)\) where \(S\) is any set and \(\mathcal{A}\) is a \(\sigma\)-algebra is called a measurable space. A set \(A\in\mathcal{A}\) is called \(\mathcal{A}\)-measurable.
We say that \(A\subseteq S\) is measurable with respect to a \(\sigma\)-algebra \(\mathcal{A}\) if its elements are identifiable, that is, if the outcomes represented in \(A\) can be told apart from other outcomes given the information in \(\mathcal{A}\). For example the set \(A=\left\lbrace 4\right\rbrace\) is not measurable in the last example above, because its impossible to know if a \(4\) was the outcome of the throw.
A \(\sigma\)-algebra of special importance is the Borel \(\sigma\)-algebra.
Definition 22.3: Borel \(\sigma\)-algebra
Let \(S=\mathbb{R}\) and \(\mathcal{A}\) be the set of open and half open intervals. The Borel algebra, noted by \(\mathcal{B}\), is the \(\sigma\)-algebra generated by \(\mathcal{A}\). A set \(B\in\mathcal{B}\) is called a Borel set.
The Borel algebra could have been defined equivalently with the closed and half closed intervals (use complement). In general one can define the Borel algebra for any metric space \(\left(S,\rho\right)\) as the smallest \(\sigma\)-algebra containing all the open balls. In the case of the Euclidean spaces it can also be generated with open rectangles.
What follows is to define the measure of a measurable set.
Measures¶
Measures in \(\sigma\)-algebras¶
Given a measurable space \(\left(S,\mathcal{A}\right)\) a measure is nothing but a function \(\mu:\mathcal{A}\to\overline{\mathbb{R}}\) with certain restrictions that guarantee its consistency:
Definition 22.4: Measure
Let \(\left(S,\mathcal{A}\right)\) be a measurable space. A measure is an extended real-valued function \(\mu:\mathcal{A}\to\overline{\mathbb{R}}\) such that:
- \(\mu\left(\emptyset\right)=0\)
- \(\mu\left(A\right)\geq0\) for all \(A\in\mathcal{A}\).
-
\(\mu\) is countably additive. If \(\left\lbrace A_{n}\right\rbrace _{n=1}^{\infty}\) is a countable, disjoint sequence in \(\mathcal{A}\), then:
\[ \mu\left(\cup A_{n}\right)=\sum\mu\left(A_{n}\right) \]
If furthermore \(\mu\left(S\right)<\infty\) then \(\mu\) is said to be a finite measure, and if \(\mu\left(S\right)=1\) then \(\mu\) is said to be a probability measure.
Definition 22.5: Measure Space
A triple \(\left(S,\mathcal{A},\mu\right)\) where \(S\) is a set, \(\mathcal{A}\) is a \(\sigma\)-algebra of its subsets and \(\mu\) is a measure on \(\mathcal{A}\) is called a measure space. The triple is called a probability space if \(\mu\) is a probability measure.
An important concept is that of almost everywhere and almost surely. These are qualifiers for a given proposition that can be evaluated in sets of \(\mathcal{A}\).
Definition 22.6: Almost Everywhere and Almost Surely
Let \(\left(S,\mathcal{A},\mu\right)\) be a measure space. A proposition is said to hold almost everywhere (a.e.) or almost surely (a.s.) if there exists a set \(A\in\mathcal{A}\) such that \(\mu\left(A\right)=0\) and the proposition holds in \(A^{c}\).
An example of the use of a.e. or a.s. is when treating functions that are similar to each other. One can say that two functions are equivalent a.e. or that a function is continuous a.e. Then the functions \(f\) and \(g\) satisfy \(f\left(x\right)=g\left(x\right)\) and \(A=\left\lbrace x|f\left(x\right)\neq f\left(y\right)\right\rbrace\) satisfies \(\mu\left(A\right)=0\). In measure theory the behavior of functions a.e. is all that matters, then we can treat functions that have anomalies as long as those anomalies occur only in sets of measure zero.
There are some properties of a measure that are useful to keep in mind, a crucial one is used for Bayes law and the definition of conditional probability.
Proposition 22.1
Let \(\left(S,\mathcal{A},\mu\right)\) be a measure space and \(B\in\mathcal{A}\) a set. Define \(\lambda:\mathcal{A}\to\overline{\mathbb{R}}\) as \(\lambda\left(A\right)=\mu\left(A\cap B\right)\). Then \(\lambda\) is a measure on \(\left(S,\mathcal{A}\right)\). If in addition \(\mu\left(B\right)<\infty\) then \(\tilde{\lambda}\) defined as \(\tilde{\lambda}\left(A\right)=\frac{\mu\left(A\cap B\right)}{\mu\left(B\right)}\) is a probability measure on \(\left(S,\mathcal{A}\right)\).
Proof
First, if \(A,B\in\mathcal{A}\) then \(A\cap B\in\mathcal{A}\), this follows from a \(\sigma\)-algebra being closed under countable intersection, by letting \(A_{1}=A\) and \(A_{n}=B\) for \(n\geq2\) the result obtains. It is left to check the three properties of a measure:
- \(\lambda\left(\emptyset\right)=\mu\left(\emptyset\cap B\right)=\mu\left(\emptyset\right)=0\).
- \(\lambda\left(A\right)=\mu\left(A\cap B\right)\geq0\).
-
Let \(\left\lbrace A_{n}\right\rbrace _{n=1}^{\infty}\) be a countable, disjoint sequence in \(\mathcal{A}\), then the sequence \(\left\lbrace A_{n}\cap B\right\rbrace _{n=1}^{\infty}\) is also disjoint and that:
\[ \lambda\left(\cup A_{n}\right)=\mu\left(\left(\cup A_{n}\right)\cap B\right)=\mu\left(\cup\left(A_{n}\cap B\right)\right)=\sum\mu\left(A_{n}\cap B\right)=\sum\lambda\left(A_{n}\right) \] -
If \(\mu\left(B\right)<\infty\) then all the previous results hold for \(\tilde{\lambda}\) by dividing everything by \(\mu\left(B\right)\). Furthermore \(\tilde{\lambda}\left(S\right)=\frac{\mu\left(S\cap B\right)}{\mu\left(B\right)}=\frac{\mu\left(B\right)}{\mu\left(B\right)}=1\).
Another useful property is given by the following proposition, it reflects the intuitive property of measures being 'increasing':
Proposition 22.2
Let \(\left(S,\mathcal{A},\mu\right)\) be a measure space and \(A,B\in\mathcal{A}\) sets. If \(A\subseteq B\) then \(\mu\left(A\right)\leq\mu\left(B\right)\), if in addition \(\mu\) is finite then \(\mu\left(B\backslash A\right)=\mu\left(B\right)-\mu\left(A\right)\).
Proof
Because \(A\subseteq B\), there exits \(C=B\backslash A=B\cap A^{c}\) such that \(A\cup C=B\) and \(A\cap C=\emptyset\). Then
Moreover, \(\mu\left(A\right)\leq\mu\left(B\right)\) because \(\mu\left(C\right)\geq0\). If \(\mu\) is finite then all elements above are well defined and: \(\mu\left(B\backslash A\right)=\mu\left(B\right)-\mu\left(A\right)\).
The following property is widely used to establish properties of limits of functions, and of the Lebesgue integral:
Proposition 22.3
Let \(\left(S,\mathcal{A},\mu\right)\) be a measure space:
-
If \(\left\lbrace A_{n}\right\rbrace\) is an increasing sequence in \(\mathcal{A}\), that is, if \(A_{n}\subseteq A_{n+1}\) for all \(n\), then:
\[ \mu\left(\cup A_{n}\right)=\lim\mu\left(A_{n}\right) \] -
If \(\left\lbrace B_{n}\right\rbrace\) is an decreasing sequence in \(\mathcal{A}\), that is, if \(B_{n}\supseteq B_{n+1}\) for all \(n\), then:
\[ \mu\left(\cap B_{n}\right)=\lim\mu\left(B_{n}\right) \]
Proof
Stokey et al. (1989, Sec. 7.2). Satisfying these two properties makes a measure continuous.
Measures in algebras and extensions [Optional]¶
So far we have defined a measure on an \(\sigma\)-algebra, but a \(\sigma\)-algebra is usually a large collection of sets and defining a function on such a set while preserving the consistency required for a measure is not an easy task. An alternative is given by defining measures on algebras, which are smaller and less complicated collections of sets. It can be shown that these measures preserve all the desirable properties of the more complicated spaces, and also allow for an extension to \(\sigma\)-algebras, once the measure is properly constructed.
We start by defining a measure on an algebra.
Definition 22.7: Measure
Let \(\left(S,\mathcal{A}\right)\) be a measurable space. A measure is an extended real-valued function \(\mu:\mathcal{A}\to\overline{\mathbb{R}}\) such that:
- \(\mu\left(\emptyset\right)=0\)
- \(\mu\left(A\right)\geq0\) for all \(A\in\mathcal{A}\).
-
If \(\left\lbrace A_{n}\right\rbrace _{n=1}^{\infty}\) is a countable, disjoint sequence in \(\mathcal{A}\), and \(\cup A_{n}\in\mathcal{A}\), then:
\[ \mu\left(\cup A_{n}\right)=\sum\mu\left(A_{n}\right) \]
If furthermore \(\mu\left(S\right)<\infty\) then \(\mu\) is said to be a finite measure, and if \(\mu\left(S\right)=1\) then \(\mu\) is said to be a probability measure.
Condition (iii) also includes finite union of disjoint sets as a special case.
Definition 22.8: \(\sigma\)-finite measure
Let \(S\) be a set, \(\mathcal{A}\) an algebra of its subsets and \(\mu\) a measure defined on \(\mathcal{A}\). If there is a countable sequence of sets in \(\mathcal{A}\), \(\left\lbrace A_{n}\right\rbrace\), such that \(\mu\left(A_{n}\right)<\infty\) and \(S=\cup A_{n}\) then \(\mu\) is \(\sigma\)-finite
It is now possible to extend the notion of this measure to a \(\sigma\)-algebra.
Theorem 22.1
\textbf{\textup{(Caratheodory extension theorem) }}\textup{Let \(S\) be a set, \(\mathcal{A}\) an algebra of its subsets and \(\mu\) a measure defined on \(\mathcal{A}\). Let \(\mathcal{A}^{\star}\) be the smallest \(\sigma\)-algebra containing \(\mathcal{A}\). There exists a measure \(\mu^{\star}\) on \(\mathcal{A}^{\star}\) such that \(\mu^{\star}\left(A\right)=\mu\left(A\right)\) for all \(A\in\mathcal{A}\).}
The problem of uniqueness is also solved.
Theorem 22.2
\textbf{\textup{(Hahn extension theorem) }}\textup{Let \(S\) be a set, \(\mathcal{A}\) an algebra of its subsets, \(\mu\) a measure defined on \(\mathcal{A}\) and \(\mathcal{A}^{\star}\) the minimal \(\sigma\)-algebra of \(\mathcal{A}\). If \(\mu\) is \(\sigma\)-finite then the extension \(\mu^{\star}\) is unique. }
To see how these theorems and the extension of a measure are used consider defining a measure on the Borel \(\sigma\)-algebra. It seems logical to define the measure of an interval \(A=\left(a,b\right)\) as \(\mu\left(A\right)=b-a\) if \(b\geq a\) and \(\mu\left(A\right)=0\) otherwise (because the interval would be empty). Yet the Borel \(\sigma\)-algebra contains sets beyond simple intervals, and the countable union of intervals can give rise to weird sets. An answer to this problem is given by defining a measure on the Borel algebra, formed by all types of intervals and their finite unions. Defining a measure on this set seems straightforward:
- \(\mu\left(\emptyset\right)=0\)
- \(\mu\left(\left(a,b\right)\right)=\mu\left(\left[a,b\right]\right)=\mu\left(\left(a,b\right]\right)=\mu\left(\left[a,b\right)\right)=b-a\)
- \(\mu\left(\left(-\infty,\infty\right)\right)=\mu\left(\left(-\infty,b\right]\right)=\mu\left(\left[a,\infty\right)\right)=\infty\)
- \(\mu\left(\cup\left(a_{n},b_{n}\right)\right)=\sum\left(b_{n}-a_{n}\right)\) if the intervals are disjoint.
The function \(\mu\) can be verified to be a measure on the Borel algebra, and hence an extension to the Borel \(\sigma\)-algebra exists. If we restrict our attention to \(S=\left[a,b\right]\) and the intervals contained in it we can define a \(\sigma\)-finite measure, obtaining uniqueness of the extension. This is how we can deal with complicated environments.
Once the measure is extended to the \(\sigma\)-algebra all the results obtained above apply.
Completion of a measure [Optional]¶
One small detail is left to be checked. Sometimes there is a set \(B\subseteq S\) such that \(B\subseteq A\in\mathcal{A}\) and \(\mu\left(A\right)=0\), but if \(B\notin\mathcal{A}\) then its measure is undefined, while it should be clearly zero. The completion of a \(\sigma\)-algebra to include these type of 'harmless' sets is what follows. As before, including sets or behaviors of measure zero is of no consequence.
Definition 22.9: Completion of a \(\sigma\)-algebra
Let \(\left(S,\mathcal{A},\mu\right)\) be a measure space. Define a collection \(\mathcal{C}\) as:
The completion of \(\sigma\)-algebra \(\mathcal{A}\) is:
By letting \(C_{1}=C_{2}=\emptyset\) we get \(\mathcal{A}\subseteq\mathcal{A}^{\prime}\), \(\mathcal{A}^{\prime}\) includes all sets in \(2^{S}\) that differ from a set in \(\mathcal{A}\) by a set of measure \(0\).
Definition 22.10: Completion of a measure
Let \(\left(S,\mathcal{A},\mu\right)\) be a measure space and \(\mathcal{A}^{\prime}\) the completion of \(\mathcal{A}\). \(\mu\left(B^{\prime}\right)=\mu\left(B\right)\) for any \(B^{\prime}\in\mathcal{A}^{\prime}\) that differs from \(B\in\mathcal{A}\) by a set of measure \(0\).
The Caratheodory and Hahn extension theorems also apply for completions.