Regular conditional probability

Regular conditional probability is a concept that has developed to overcome certain difficulties in formally defining conditional probabilities for continuous probability distributions. It is defined as an alternative probability measure conditioned on a particular value of a random variable.

Motivation

Normally we define the conditional probability of an event A given an event B as:

$P(A|B)={\frac {P(A\cap B)}{P(B)}}.$

The difficulty with this arises when the event B is too small to have a non-zero probability. For example, suppose we have a random variable X with a uniform distribution on $[0,1],$ and B is the event that $X=2/3.$ Clearly, the probability of B, in this case, is $P(B)=0,$ but nonetheless we would still like to assign meaning to a conditional probability such as $P(A|X=2/3).$ To do so rigorously requires the definition of a regular conditional probability.

Definition

Let $(\Omega ,{\mathcal {F}},P)$ be a probability space, and let $T:\Omega \rightarrow E$ be a random variable, defined as a Borel-measurable function from $\Omega$ to its state space $(E,{\mathcal {E}}).$. One should think of $T$ as a way to "disintegrate" the sample space $\Omega$ into $\{T^{-1}(x)\}_{x\in E}$. Using the disintegration theorem from the measure theory, it allows us to "disintegrate" the measure $P$ into a collection of measures, one for each $x\in E$. Formally, a regular conditional probability is defined as a function $\nu :E\times {\mathcal {F}}\rightarrow [0,1],$ called a "transition probability", where:

• For every $x\in E$, $\nu (x,\cdot )$ is a probability measure on ${\mathcal {F}}$. Thus we provide one measure for each $x\in E$.
• For all $A\in {\mathcal {F}}$, $\nu (\cdot ,A)$ (a mapping $E\mapsto [0,1]$) is ${\mathcal {E}}$-measurable, and
• For all $A\in {\mathcal {F}}$ and all $B\in {\mathcal {E}}$

$P{\big (}A\cap T^{-1}(B){\big )}=\int _{B}\nu (x,A)\,P{\big (}T^{-1}(dx){\big )}.$

where $P\circ T^{-1}$ is the pushforward measure $T_{*}P$ of the distribution of the random element $T$, $x\in \mathrm {supp} \,T,$ i.e. the topological support of the $T_{*}P$. Specifically, if we take $B=E$, then $A\cap T^{-1}(E)=A$, and so

$P(A)=\int _{E}\nu (x,A)\,P{\big (}T^{-1}(dx){\big )}$,

where $\nu (x,A)$ can be denoted, using more familiar terms $P(A\ |\ T=x)$ (this is "defined" to be conditional probability of $A$ given $x$, which can be undefined in elementary constructions of conditional probability). As can be seen from the integral above, the value of $\nu$ for points x outside the support of the random variable is meaningless; its significance as a conditional probability is strictly limited to the support of T.

The measurable space $(\Omega ,{\mathcal {F}})$ is said to have the regular conditional probability property if for all probability measures $P$ on $(\Omega ,{\mathcal {F}}),$ all random variables on $(\Omega ,{\mathcal {F}},P)$ admit a regular conditional probability. A Radon space, in particular, has this property.

See also conditional probability and conditional probability distribution.

Example

To continue with our motivating example above, we consider a real-valued random variable X and write

$P(A|X=x_{0})=\nu (x_{0},A)=\lim _{\epsilon \rightarrow 0+}{\frac {P(A\cap \{x_{0}-\epsilon <X<x_{0}+\epsilon \})}{P(\{x_{0}-\epsilon <X<x_{0}+\epsilon \})}},$

(where $x_{0}=2/3$ for the example given.) This limit, if it exists, is a regular conditional probability for X, restricted to $\mathrm {supp} \,X.$

In any case, it is easy to see that this limit fails to exist for $x_{0}$ outside the support of X: since the support of a random variable is defined as the set of all points in its state space whose every neighborhood has positive probability, for every point $x_{0}$ outside the support of X (by definition) there will be an $\epsilon >0$ such that $P(\{x_{0}-\epsilon <X<x_{0}+\epsilon \})=0.$

Thus if X is distributed uniformly on $[0,1],$ it is truly meaningless to condition a probability on "$X=3/2$".

See also

• Conditioning (probability)
• Disintegration theorem
• Adherent point
• Limit point

External links

• Regular Conditional Probability on PlanetMath