18.3 - Understanding Rho

On this page, we'll begin our investigation of what the correlation coefficient tells us. All we'll be doing here is getting a handle on what we can expect of the correlation coefficient if \(X\) and \(Y\) are independent, and what we can expect of the correlation coefficient if \(X\) and \(Y\) are dependent. On the next page, we'll take a more in depth look at understanding the correlation coefficient. Let's start with the following theorem.

Theorem

If \(X\) and \(Y\) are independent random variables (discrete or continuous!), then:

\(Corr(X,Y)=Cov(X,Y)=0\)

Proof

For the sake of this proof, let us assume that \(X\) and \(Y\) are discrete. (The proof that follows can be easily modified if \(X\) and \(Y\) are continuous.) Let's start with the expected value of \(XY\). That is, let's see what we can say about the expected value of \(XY\) if \(X\) and \(Y\) are independent:

That is, we have shown that if \(X\) and \(Y\) are independent, then \(E(XY)=E(X)E(Y)\). Now the rest of the proof follows. If \(X\) and \(Y\) are independent, then:

\begin{align} Cov(X,Y) &=E(XY)-\mu_X\mu_Y\\ &= E(X)E(Y)-\mu_X\mu_Y\\ &= \mu_X\mu_Y-\mu_X\mu_Y=0 \end{align}

and therefore:

\(Corr(X,Y)=\dfrac{Cov(X,Y)}{\sigma_X \sigma_Y}=\dfrac{0}{\sigma_X \sigma_Y}=0\)

Let's take a look at an example of the theorem in action. That is, in the example that follows, we see a case in which \(X\) and \(Y\) are independent and the correlation between \(X\) and \(Y\) is 0.

Example 18-2 Section

colored dice

Let \(X\) = outcome of a fair, black, 6-sided die. Because the die is fair, we'd expect each of the six possible outcomes to be equally likely. That is, the p.m.f. of \(X\) is:

\(f_X(x)=\dfrac{1}{6},\quad x=1,\ldots,6.\)

Let \(Y\) = outcome of a fair, red, 4-sided die. Again, because the die is fair, we'd expect each of the four possible outcomes to be equally likely. That is, the p.m.f. of \(Y\) is:

\(f_Y(y)=\dfrac{1}{4},\quad y=1,\ldots,4.\)

If we toss the pair of dice, the 24 possible outcomes are (1, 1) (1, 2) ... (1, 4) ... (6, 1) ... (6, 4), with each of the 24 outcomes being equally likely. That is, the joint p.m.f. of \(X\) and \(Y\) is:

\(f(x,y)=\dfrac{1}{24},\quad x=1,2,\ldots,6,\quad y=1,\ldots,4.\)

Although we intuitively feel that the outcome of the black die is independent of the outcome of the red die, we can formally show that \(X\) and \(Y\) are independent:

\(f(x,y)=\dfrac{1}{24}f_X(x)f_Y(y)=\dfrac{1}{6} \cdot \dfrac{1}{4} \qquad \forall x,y\)

What is the covariance of \(X\) and \(Y\) ? What the correlation of \(X\) and \(Y\) ?

Solution

Well, the mean of \(X\) is:

\(\mu_X=E(X)=\sum\limits_x xf(x)=1\left(\dfrac{1}{6}\right)+\cdots+6\left(\dfrac{1}{6}\right)=\dfrac{21}{6}=3.5\)

And, the mean of \(Y\) is:

\(\mu_Y=E(Y)=\sum\limits_y yf(y)=1\left(\dfrac{1}{4}\right)+\cdots+4\left(\dfrac{1}{4}\right)=\dfrac{10}{4}=2.5\)

The expected value of the product \(XY\) is:

\(E(XY)=\sum\limits_x\sum\limits_y xyf(x,y)=(1)(1)\left(\dfrac{1}{24}\right)+(1)(2)\left(\dfrac{1}{24}\right)+\cdots+(6)(4)\left(\dfrac{1}{24}\right)=\dfrac{210}{24}=8.75\)

Therefore, the covariance of \(X\) and \(Y\) is:

\(Cov(X,Y)=E(XY)-\mu_X\mu_Y=8.75-(3.5)(2.5)=8.75-8.75=0\)

and therefore, the correlation between \(X\) and \(Y\) is 0:

\(Corr(X,Y)=\dfrac{Cov(X,Y)}{\sigma_X \sigma_Y}=\dfrac{0}{\sigma_X \sigma_Y}=0\)

Again, this example illustrates a situation in which \(X\) and \(Y\) are independent, and the correlation between \(X\) and \(Y\) is 0, just as the theorem states it should be.

NOTE! the converse of the theorem is not necessarily true! That is, zero correlation and zero covariance do not imply independence. Let's take a look at an example that illustrates this claim.

Example 18-3 Section

Let \(X\) and \(Y\) be discrete random variables with the following joint probability mass function:

-1 0 1f (x,y)yx-1010.2000.2000.2000.2000.200.40.20.40.40.20.4fY (y)fX (x)1

What is the correlation between \(X\) and \(Y\)? And, are \(X\) and \(Y\) independent?

Solution

The mean of \(X\) is:

\(\mu_X=E(X)=\sum xf(x)=(-1)\left(\dfrac{2}{5}\right)+(0)\left(\dfrac{1}{5}\right)+(1)\left(\dfrac{2}{5}\right)=0\)

And the mean of \(Y\) is:

\(\mu_Y=E(Y)=\sum yf(y)=(-1)\left(\dfrac{2}{5}\right)+(0)\left(\dfrac{1}{5}\right)+(1)\left(\dfrac{2}{5}\right)=0\)

The expected value of the product \(XY\) is also 0:

\(E(XY)=(-1)(-1)\left(\dfrac{1}{5}\right)+(-1)(1)\left(\dfrac{1}{5}\right)+(0)(0)\left(\dfrac{1}{5}\right)+(1)(-1)\left(\dfrac{1}{5}\right)+(1)(1)\left(\dfrac{1}{5}\right)\)

\(E(XY)=\dfrac{1}{5}-\dfrac{1}{5}+0-\dfrac{1}{5}+\dfrac{1}{5}=0\)

Therefore, the covariance of \(X\) and \(Y\) is 0:

\(Cov(X,Y)=E(XY)-\mu_X\mu_Y=0-(0)(0)=0\)

and therefore the correlation between \(X\) and \(Y\) is necessarily 0.

Yet, \(X\) and \(Y\) are not independent, since the product space is not rectangular! That is, we can find an \(x\) and a \(y\) for which the joint probability mass function \(f(x,y)\) can't be written as the product of \(f(x)\), the probability mass function of \(X\), and \(f(y)\), the probability mass function of \(Y\). For example, when \(x=0\) and \(y=-1\):

\(f(0,-1)=0 \neq f_X(0)f_Y(-1)=(1/5)(2/5)=2/25\)

In summary, again, this example illustrates that if the correlation between \(X\) and \(Y\) is 0, it does not necessarily mean that \(X\) and \(Y\) are independent. On the contrary, we've shown a case here in which the correlation between \(X\) and \(Y\) is 0, and yet \(X\) and \(Y\) are dependent!

The contrapositive of the theorem is always true! That is, if the correlation is not zero, then \(X\) and \(Y\) are dependent. Let's take a look at an example that illustrates this claim.

Example 18-4 Section

tshirts

A quality control inspector for a t-shirt manufacturer inspects t-shirts for defects. She labels each t-shirt she inspects as either:

  • "good"
  • a "second" which could be sold at a reduced price, or
  • "defective," in which the t-shirt could not be sold at all

The quality control inspector inspects \(n=2\) t-shirts:

  • Let \(X\) = # of good t-shirts. Historically, the probability that a t-shirt is good is \(p_1=0.6\).
  • Let \(Y\) = # of second t-shirts. Historically, the probability that a t-shirt is labeled as a second is \(p_2=0.2\).
  • Let \(2-X-Y\)= # of defective t-shirts. Historically, the probability that a t-shirt is labeled as defective is \(1-p_1-p_2=1-0.6-0.2=0.2\)

Then, the joint probability mass function of \(X\) and \(Y\) is the trinomial distribution. That is:

\(f(x,y)=\dfrac{2!}{x!y!(2-x-y)!} 0.6^x 0.2^y 0.2^{2-x-y},\qquad 0 \leq x+y \leq 2\)

Are \(X\) and \(Y\) independent? And, what is the correlation between \(X\) and \(Y\)?

Solution

First, \(X\) and \(Y\) are indeed dependent, since the support is triangular. Now, for calculating the correlation between \(X\) and \(Y\). The random variable \(X\) is binomial with \(n=2\) and \(p_1=0.6\). Therefore, the mean and standard deviation of \(X\) are 1.2 and 0.69, respectively:

\begin{array}{lcll} X \sim b(2,0.6) & \qquad & \mu_X &=np_1=2(0.6)=1.2\\ & \qquad & \sigma_X &=\sqrt{np_1(1-p_1)}=\sqrt{2(0.6)(0.4)}=0.69 \end{array}

The random variable \(Y\) is binomial with \(n=2\) and \(p_2=0.2\). Therefore, the mean and standard deviation of \(Y\) are 0.4 and 0.57, respectively:

\begin{array}{lcll} Y \sim b(2,0.2) & \qquad & \mu_Y &=np_2=2(0.2)=0.4\\ & \qquad & \sigma_Y &=\sqrt{np_2(1-p_2)}=\sqrt{2(0.2)(0.8)}=0.57 \end{array}

The expected value of the product \(XY\) is:

\begin{align} E(XY)&= \sum\limits_x \sum\limits_y xyf(x,y)\\ &= (1)(1)\dfrac{2!}{1!1!0!} 0.6^1 0.2^1 0.2^0=2(0.6)(0.2)=0.24\\ \end{align}

Therefore, the covariance of \(X\) and \(Y\) is −0.24:

\(Cov(X,Y)=E(XY)-\mu_X\mu_Y=0.24-(1.2)(0.4)=0.24-0.48=-0.24\)

and the correlation between \(X\) and \(Y\) is −0.61:

\(Corr(X,Y)=\dfrac{Cov(X,Y)}{\sigma_X \sigma_Y}=\dfrac{-0.24}{(0.69)(0.57)}=-0.61\)

In summary, again, this is an example in which the correlation between \(X\) and \(Y\) is not 0, and \(X\) and \(Y\) are dependent.