Printer-friendly versionPrinter-friendly version

On this page, we'll begin our investigation of what the correlation coefficient tells us. All we'll be doing here is getting a handle on what we can expect of the correlation coefficient if X and Y are independent, and what we can expect of the correlation coefficient if X and Y are dependent. On the next page, we'll take a more in depth look at understanding the correlation coefficient. Let's start with the following theorem.

Theorem.  If X and Y are independent random variables (discrete or continuous!), then:

\(Corr(X,Y)=Cov(X,Y)=0\)

 Proof. For the sake of this proof, let us assume that X and Y are discrete. (The proof that follows can be easily modified if X and Y are continuous.) Let's start with the expected value of XY. That is, let's see what we can say about the expected value of XY if X and Y are independent:

That is, we have shown that if X and Y are independent, then E(XY) = E(XE(Y).  Now the rest of the proof follows.  If X and Y are independent, then:

\begin{align}
Cov(X,Y) &=E(XY)-\mu_X\mu_Y\\
&= E(X)E(Y)-\mu_X\mu_Y\\
&= \mu_X\mu_Y-\mu_X\mu_Y=0
\end{align}

and therefore:

\(Corr(X,Y)=\dfrac{Cov(X,Y)}{\sigma_X \sigma_Y}=\dfrac{0}{\sigma_X \sigma_Y}=0\)

Let's take a look at an example of the theorem in action. That is, in the example that follows, we see a case in which X and Y are independent and the correlation between X and Y is 0.

colored diceExample

Let X = outcome of a fair, black, 6-sided die. Because the die is fair, we'd expect each of the six possible outcomes to be equally likely. That is, the p.m.f. of X is:

\(f_X(x)=\dfrac{1}{6},\quad x=1,\ldots,6.\)

Let Y = outcome of a fair, red, 4-sided die. Again, because the die is fair, we'd expect each of the four possible outcomes to be equally likely. That is, the p.m.f. of Y is:

\(f_Y(y)=\dfrac{1}{4},\quad y=1,\ldots,4.\)

If we toss the pair of dice, the 24 possible outcomes are (1, 1) (1, 2) ... (1, 4) ... (6, 1) ... (6, 4), with each of the 24 outcomes being equally likely. That is, the joint p.m.f. of X and Y is:

 \(f(x,y)=\dfrac{1}{24},\quad x=1,2,\ldots,6,\quad y=1,\ldots,4.\)

Although we intuitively feel that the outcome of the black die is independent of the outcome of the red die, we can formally show that X and are independent:

\(f(x,y)=\dfrac{1}{24}f_X(x)f_Y(y)=\dfrac{1}{6} \cdot \dfrac{1}{4} \qquad \forall x,y\)

What is the covariance of X and Y ?  What the correlation of X and Y ?

Solution. Well, the mean of X is:

\(\mu_X=E(X)=\sum\limits_x xf(x)=1\left(\dfrac{1}{6}\right)+\cdots+6\left(\dfrac{1}{6}\right)=\dfrac{21}{6}=3.5\)

And, the mean of Y is:

\(\mu_Y=E(Y)=\sum\limits_y yf(y)=1\left(\dfrac{1}{4}\right)+\cdots+4\left(\dfrac{1}{4}\right)=\dfrac{10}{4}=2.5\)

The expected value of the product XY is:

\(E(XY)=\sum\limits_x\sum\limits_y xyf(x,y)=(1)(1)\left(\dfrac{1}{24}\right)+(1)(2)\left(\dfrac{1}{24}\right)+\cdots+(6)(4)\left(\dfrac{1}{24}\right)=\dfrac{210}{24}=8.75\)

Therefore, the covariance of X and Y is:

\(Cov(X,Y)=E(XY)-\mu_X\mu_Y=8.75-(3.5)(2.5)=8.75-8.75=0\)

and therefore, the correlation between X and Y is 0:

\(Corr(X,Y)=\dfrac{Cov(X,Y)}{\sigma_X \sigma_Y}=\dfrac{0}{\sigma_X \sigma_Y}=0\)

Again, this example illustrates a situation in which X and Y are independent, and the correlation between X and Y is 0, just as the theorem states it should be.

Note, however, that the converse of the theorem is not neccessarily true!  That is, zero correlation and zero covariance do not imply independence. Let's take a look at an example that illustrates this claim.

Example

Let X and Y be discrete random variables with the following joint probability mass function:

table

What is the correlation between X and Y? And, are X and Y independent?

Solution. The mean of X is:

\(\mu_X=E(X)=\sum xf(x)=(-1)\left(\dfrac{2}{5}\right)+(0)\left(\dfrac{1}{5}\right)+(1)\left(\dfrac{2}{5}\right)=0\)

And the mean of Y is:

\(\mu_Y=E(Y)=\sum yf(y)=(-1)\left(\dfrac{2}{5}\right)+(0)\left(\dfrac{1}{5}\right)+(1)\left(\dfrac{2}{5}\right)=0\)

The expected value of the product XY is also 0:

\(E(XY)=(-1)(-1)\left(\dfrac{1}{5}\right)+(-1)(1)\left(\dfrac{1}{5}\right)+(0)(0)\left(\dfrac{1}{5}\right)+(1)(-1)\left(\dfrac{1}{5}\right)+(1)(1)\left(\dfrac{1}{5}\right)\)

\(E(XY)=\dfrac{1}{5}-\dfrac{1}{5}+0-\dfrac{1}{5}+\dfrac{1}{5}=0\)

Therefore, the covariance of X and Y is 0:

\(Cov(X,Y)=E(XY)-\mu_X\mu_Y=0-(0)(0)=0\)

and therefore the correlation between X and Y is necessarily 0.

Yet,and Y are not independent, since the product space is not rectangular!  That is, we can find an x and a y for which the joint probability mass function f(x,y) can't be written as the product of f(x), the probability mass function of X, and f(y), the probability mass function of Y. For example, when x = 0 and y = −1:

\(f(0,-1)=0 \neq f_X(0)f_Y(-1)=(1/5)(2/5)=2/25\)

In summary, again, this example illustrates that if the correlation between X and Y is 0, it does not necessarily mean that X and Y are independent. On the contrary, we've shown a case here in which the correlation between X and Y is 0, and yet X and Y are dependent!

The contrapositive of the theorem is always true!  That is, if the correlation is not zero, then X and Y are dependent. Let's take a look at an example that illustrates this claim.

tshirtsExample

A quality control inspector for a t-shirt manufacturer inspects t-shirts for defects. She labels each t-shirt she inspects as either:

  • "good" 
  • a "second" which could be sold at a reduced price, or 
  • "defective," in which the t-shirt could not be sold at all

The quality control inspector inspects n = 2 t-shirts:

  • Let X = # of good t-shirts. Historically, the probability that a t-shirt is good is p1 = 0.6.
  • Let Y = # of second t-shirts. Historically, the probability that a t-shirt is labeled as a second is p2 = 0.2.
  • Let 2-X-Y = # of defective t-shirts. Historically, the probability that a t-shirt is labeled as defective is 1− p1− p2 = 1−0.6−0.2 = 0.2

Then, the joint probability mass function of X and Y is the trinomial distribution. That is:

\(f(x,y)=\dfrac{2!}{x!y!(2-x-y)!} 0.6^x 0.2^y 0.2^{2-x-y},\qquad 0 \leq x+y \leq 2\)

Are X and Y independent? And, what is the correlation between X and Y

Solution. First, and are indeed dependent, since the support is triangular. Now, for calculating the correlation between X and Y. The random variable X is binomial with n = 2 and p1 = 0.6. Therefore, the mean and standard deviation of X are 1.2 and 0.69, respectively:

\begin{array}{lcll}
X  \sim b(2,0.6) & \qquad & \mu_X &=np_1=2(0.6)=1.2\\
                 & \qquad & \sigma_X &=\sqrt{np_1(1-p_1)}=\sqrt{2(0.6)(0.4)}=0.69
\end{array}

The random variable Y is binomial with n = 2 and p2 = 0.2. Therefore, the mean and standard deviation of Y are 0.4 and 0.57, respectively:

\begin{array}{lcll}
Y \sim b(2,0.2) & \qquad & \mu_Y &=np_2=2(0.2)=0.4\\
                 & \qquad & \sigma_Y &=\sqrt{np_2(1-p_2)}=\sqrt{2(0.2)(0.8)}=0.57
\end{array}

The expected value of the product XY is:

\begin{align}
E(XY)&= \sum\limits_x \sum\limits_y xyf(x,y)\\
     &= (1)(1)\dfrac{2!}{1!1!0!} 0.6^1 0.2^1 0.2^0=2(0.6)(0.2)=0.24\\
     \end{align}

Therefore, the covariance of X and Y is −0.24:

\(Cov(X,Y)=E(XY)-\mu_X\mu_Y=0.24-(1.2)(0.4)=0.24-0.48=-0.24\)

and the correlation between X and Y is −0.61:

\(Corr(X,Y)=\dfrac{Cov(X,Y)}{\sigma_X \sigma_Y}=\dfrac{-0.24}{(0.69)(0.57)}=-0.61\)

In summary, again, this is an example in which the correlation between X and Y is not 0, and and Y are dependent.