Covariance of X and Y

Printer-friendly versionPrinter-friendly version

Here, we'll begin our attempt to quantify the dependence between two random variables X and Y by investigating what is called the covariance between the two random variables.  We'll jump right in with a formal definition of the covariance.

Definition. Let X and Y be random variables (discrete or continuous!) with means μX and μY. The covariance of X and Y, denoted Cov(X,Y) or σXY, is defined as:

\(Cov(X,Y)=\sigma_{XY}=E[(X-\mu_X)(Y-\mu_Y)]\)

That is, if X and Y are discrete random variables with joint support S, then the covariance of X and Y is:

\(Cov(X,Y)=\mathop{\sum\sum}\limits_{(x,y)\in S} (x-\mu_X)(y-\mu_Y) f(x,y)\)

 And, if X and Y are continuous random variables with supports S1 and S2, respectively, then the covariance of X and Y is:

\(Cov(X,Y)=\int_{S_2} \int_{S_1} (x-\mu_X)(y-\mu_Y) f(x,y)dxdy\)

Example

Suppose that X and Y have the following joint probability mass function:

table

What is the covariance of X and Y?

Solution. 

Two questions you might have right now: 1) What does the covariance mean? That is, what does it tell us? and 2) Is there a shortcut formula for the covariance just as there is for the variance? We'll be answering the first question in the pages that follow.  Well, sort of! In reality, we'll use the covariance as a stepping stone to yet another statistical measure known as the correlation coefficient. And, we'll certainly spend some time learning what the correlation coefficient tells us. In regards to the second question, let's answer that one now by way of the following theorem.

Theorem. For any random variables X and Y (discrete or continuous!) with means μX and μY, the covariance of X and Y can be calculated as:

\(Cov(X,Y)=E(XY)-\mu_X\mu_Y\)

Proof. In order to prove this theorem, we'll need to use the fact (which you are asked to prove in your homework) that, even in the bivariate situation, expectation is still a linear or distributive operator: 

Example (continued)

Suppose again that X and Y have the following joint probability mass function:

table

Use the theorem we just proved to calculate the covariance of X and Y.

Solution.

Now that we know how to calculate the covariance between two random variables, X and Y, let's turn our attention to seeing how the covariance helps us calculate what is called the correlation coefficient.