21.2 - Joint P.D.F. of X and Y

We previously assumed that:

  1. \(Y\) follows a normal distribution,
  2. \(E(Y|x)\), the conditional mean of \(Y\) given \(x\) is linear in \(x\), and
  3. \(\text{Var}(Y|x)\), the conditional variance of \(Y\) given \(x\) is constant.

    Based on these three stated assumptions, we found the conditional distribution of \(Y\) given \(X=x\). Now, we'll add a fourth assumption, namely that:

  4. \(X\) follows a normal distribution.

Based on the four stated assumptions, we will now define the joint probability density function of \(X\) and \(Y\).

Definition. Assume \(X\) is normal, so that the p.d.f. of \(X\) is:

\(f_X(x)=\dfrac{1}{\sigma_X \sqrt{2\pi}} \text{exp}\left[-\dfrac{(x-\mu_X)^2}{2\sigma^2_X}\right]\)

for \(-\infty<x<\infty\). And, assume that the conditional distribution of \(Y\) given \(X=x\) is normal with conditional mean:

\(E(Y|x)=\mu_Y+\rho \dfrac{\sigma_Y}{\sigma_X}(x-\mu_X)\)

and conditional variance:

\(\sigma^2_{Y|X}= \sigma^2_Y(1-\rho^2)\)

That is, the conditional distribution of \(Y\) given \(X=x\) is:

\begin{align} h(y|x) &= \dfrac{1}{\sigma_{Y|X} \sqrt{2\pi}} \text{exp}\left[-\dfrac{(Y-\mu_{Y|X})^2}{2\sigma^2_{Y|X}}\right]\\ &= \dfrac{1}{\sigma_Y \sqrt{1-\rho^2} \sqrt{2\pi}}\text{exp}\left[-\dfrac{[y-\mu_Y-\rho \dfrac{\sigma_Y}{\sigma_X}(X-\mu_X)]^2}{2\sigma^2_Y(1-\rho^2)}\right],\quad -\infty<x<\infty\\ \end{align}

Therefore, the joint probability density function of \(X\) and \(Y\) is:

\(f(x,y)=f_X(x) \cdot h(y|x)=\dfrac{1}{2\pi \sigma_X \sigma_Y \sqrt{1-\rho^2}} \text{exp}\left[-\dfrac{q(x,y)}{2}\right]\)

where:

\(q(x,y)=\left(\dfrac{1}{1-\rho^2}\right) \left[\left(\dfrac{X-\mu_X}{\sigma_X}\right)^2-2\rho \left(\dfrac{X-\mu_X}{\sigma_X}\right) \left(\dfrac{Y-\mu_Y}{\sigma_Y}\right)+\left(\dfrac{Y-\mu_Y}{\sigma_Y}\right)^2\right]\)

This joint p.d.f. is called the bivariate normal distribution.

Our textbook has a nice three-dimensional graph of a bivariate normal distribution. You might want to take a look at it to get a feel for the shape of the distribution. Now, let's turn our attention to an important property of the correlation coefficient if \(X\) and \(Y\) have a bivariate normal distribution.

Theorem

If \(X\) and \(Y\) have a bivariate normal distribution with correlation coefficient \(\rho_{XY}\), then \(X\) and \(Y\) are independent if and only if \(\rho_{XY}=0\). That "if and only if" means:

  1. If \(X\) and \(Y\) are independent, then \(\rho_{XY}=0\)
  2. If \(\rho_{XY}=0\), then \(X\) and \(Y\) are independent
 

Recall that the first item is always true. We proved it back in the lesson that addresses the correlation coefficient. We also looked at a counterexample i that lesson that illustrated that item (2) was not necessarily true! Well, now we've just learned a situation in which it is true, that is, when \(X\) and \(Y\) have a bivariate normal distribution. Let's see why item (2) must be true in that case.

Proof

Since we previously proved item (1), our focus here will be in proving item (2). In order to prove that \(X\) and \(Y\) are independent when \(X\) and \(Y\) have the bivariate normal distribution and with zero correlation, we need to show that the bivariate normal density function:

\(f(x,y)=f_X(x)\cdot h(y|x)=\dfrac{1}{2\pi \sigma_X \sigma_Y \sqrt{1-\rho^2}} \text{exp}\left[-\dfrac{q(x,y)}{2}\right]\)

factors into the normal p.d.f of \(X\) and the normal p.d.f. of \(Y\). Well, when \(\rho_{XY}=0\):

\(q(x,y)=\left(\dfrac{1}{1-0^2}\right) \left[\left(\dfrac{X-\mu_X}{\sigma_X}\right)^2+0+\left(\dfrac{Y-\mu_Y}{\sigma_Y}\right)^2 \right]\)

which simplifies to:

\(q(x,y)=\left(\dfrac{X-\mu_X}{\sigma_X}\right)^2+\left(\dfrac{Y-\mu_Y}{\sigma_Y}\right)^2 \)

Substituting this simplified \(q(x,y)\) into the joint p.d.f. of \(X\) and \(Y\), and simplifying, we see that \(f(x,y)\) does indeed factor into the product of \(f(x)\) and \(f(y)\):

\begin{align} f(x,y) &= \dfrac{1}{2\pi \sigma_X \sigma_Y \sqrt{1-\rho^2}} \text{exp}\left[-\dfrac{1}{2}\left(\dfrac{X-\mu_X}{\sigma_X}\right)^2--\dfrac{1}{2}\left(\dfrac{Y-\mu_Y}{\sigma_Y}\right)^2\right]\\ &= \dfrac{1}{\sigma_X \sqrt{2\pi} \sigma_Y \sqrt{2\pi}}\text{exp}\left[-\dfrac{(x-\mu_X)^2}{2\sigma_X^2}\right] \text{exp}\left[-\dfrac{(y-\mu_Y)^2}{2\sigma_Y^2}\right]\\ &= \dfrac{1}{\sigma_X \sqrt{2\pi}}\text{exp}\left[-\dfrac{(x-\mu_X)^2}{2\sigma_X^2}\right]\cdot \dfrac{1}{\sigma_Y \sqrt{2\pi}}\text{exp}\left[-\dfrac{(y-\mu_Y)^2}{2\sigma_Y^2}\right]\\ &=f_X(x)\cdot f_Y(y)\\ \end{align}

Because we have shown that:

\(f(x,y)=f_X(x)\cdot f_Y(y)\)

we can conclude, by the definition of independence, that \(X\) and \(Y\) are independent. Our proof is complete.