Misconceptions about the normal distribution

In probability theory, although simple examples illustrate that linear uncorrelatedness of two random variables does not in general imply their independence, it is sometimes mistakenly thought that it does imply that when the two random variables are normally distributed. This article demonstrates that assumption of normal distributions does not have that consequence, although the multivariate normal distribution, including the bivariate normal distribution, does.

To say that the pair $(X,Y)$ of random variables has a bivariate normal distribution means that every constant (i.e. not random) linear combination $aX+bY$ of $X$ and $Y$ has a univariate normal distribution. In that case, if $X$ and $Y$ are uncorrelated then they are independent.^[1] However, it is possible for two random variables $X$ and $Y$ to be so distributed jointly that each one alone is marginally normally distributed, and they are uncorrelated, but they are not independent; examples are given below.

Examples

A symmetric example

Suppose $X$ has a normal distribution with expected value 0 and variance 1. Let $W$ have the Rademacher distribution, so that $W=1$ or $W=-1$ , each with probability 1/2, and assume $W$ is independent of $X$ . Let $Y=WX$ . Then

$X$ and $Y$ are uncorrelated;
both have the same normal distribution; and
$X$ and $Y$ are not independent.^[2]

To see that $X$ and $Y$ are uncorrelated, one may consider the covariance $\operatorname {cov} (X,Y)$ : by definition, it is

\operatorname {cov} (X,Y)=\operatorname {E} (XY)-\operatorname {E} (X)\operatorname {E} (Y).

Then by definition of the random variables $X$ , $Y$ , and $W$ , and the independence of $W$ from $X$ , one has

\operatorname {cov} (X,Y)=\operatorname {E} (XY)-0=\operatorname {E} (X^{2}W)=\operatorname {E} (X^{2})\operatorname {E} (W)=\operatorname {E} (X^{2})\cdot 0=0.

To see that $Y$ has the same normal distribution as $X$ , consider

{\begin{aligned}\Pr(Y\leq x)&{}=\operatorname {E} (\Pr(Y\leq x\mid W))\\&{}=\Pr(X\leq x)\Pr(W=1)+\Pr(-X\leq x)\Pr(W=-1)\\&{}=\Phi (x)\cdot {\frac {1}{2}}+\Phi (x)\cdot {\frac {1}{2}}\end{aligned}}

(since $X$ and $-X$ both have the same normal distribution), where $\Phi (x)$ is the cumulative distribution function of the normal distribution..

To see that $X$ and $Y$ are not independent, observe that $|Y|=|X|$ or that $\operatorname {Pr} (Y>1|X=1/2)=\operatorname {Pr} (X>1|X=1/2)=0$ .

Finally, the distribution of the simple linear combination $X+Y$ concentrates positive probability at 0: $\operatorname {Pr} (X+Y=0)=1/2$ . Therefore, the random variable $X+Y$ is not normally distributed, and so also $X$ and $Y$ are not jointly normally distributed (by the definition above).

An asymmetric example

Suppose $X$ has a normal distribution with expected value 0 and variance 1. Let

Y=\left\{{\begin{matrix}X&{\text{if }}\left|X\right|\leq c\\-X&{\text{if }}\left|X\right|>c\end{matrix}}\right.

where $c$ is a positive number to be specified below. If $c$ is very small, then the correlation $\operatorname {corr} (X,Y)$ is near $-1$ if $c$ is very large, then $\operatorname {corr} (X,Y)$ is near 1. Since the correlation is a continuous function of $c$ , the intermediate value theorem implies there is some particular value of $c$ that makes the correlation 0. That value is approximately 1.54. In that case, $X$ and $Y$ are uncorrelated, but they are clearly not independent, since $X$ completely determines $Y$ .

To see that $Y$ is normally distributed—indeed, that its distribution is the same as that of $X$ —one may compute its cumulative distribution function:

{\begin{aligned}\Pr(Y\leq x)&=\Pr(\{|X|\leq c{\text{ and }}X\leq x\}{\text{ or }}\{|X|>c{\text{ and }}-X\leq x\})\\&=\Pr(|X|\leq c{\text{ and }}X\leq x)+\Pr(|X|>c{\text{ and }}-X\leq x)\\&=\Pr(|X|\leq c{\text{ and }}X\leq x)+\Pr(|X|>c{\text{ and }}X\leq x)\\&=\Pr(X\leq x),\end{aligned}}

where the next-to-last equality follows from the symmetry of the distribution of $X$ and the symmetry of the condition that $|X|\leq c$ .

In this example, the difference $X-Y$ is nowhere near being normally distributed, since it has a substantial probability (about 0.88) of it being equal to 0. By contrast, the normal distribution, being a continuous distribution, has no discrete part—that is, it does not concentrate more than zero probability at any single point. Consequently $X$ and $Y$ are not jointly normally distributed, even though they are separately normally distributed.^[3]

References

^ Hogg, Robert; Tanis, Elliot (2001). "Chapter 5.4 The Bivariate Normal Distribution". Probability and Statistical Inference (6th ed.). pp. 258–259. ISBN 0130272949.
^ UIUC, Lecture 21. The Multivariate Normal Distribution, 21.6:"Individually Gaussian Versus Jointly Gaussian".
^ Edward L. Melnick and Aaron Tenenbein, "Misspecifications of the Normal Distribution", The American Statistician, volume 36, number 4 November 1982, pages 372–373

[1] Hogg, Robert; Tanis, Elliot (2001). "Chapter 5.4 The Bivariate Normal Distribution". Probability and Statistical Inference (6th ed.). pp. 258–259. ISBN 0130272949.

[2] UIUC, Lecture 21. The Multivariate Normal Distribution, 21.6:"Individually Gaussian Versus Jointly Gaussian".

[3] Edward L. Melnick and Aaron Tenenbein, "Misspecifications of the Normal Distribution", The American Statistician, volume 36, number 4 November 1982, pages 372–373

[1]

[2]

[3]

Examples

A symmetric example

An asymmetric example

See also

References