Here, we'll begin our attempt to quantify the dependence between two random variables \(X\) and \(Y\) by investigating what is called the covariance between the two random variables. We'll jump right in with a formal definition of the covariance.
- Covariance
-
Let \(X\) and \(Y\) be random variables (discrete or continuous!) with means \(\mu_X\) and \(\mu_Y\). The covariance of \(X\) and \(Y\), denoted \(\text{Cov}(X,Y)\) or \(\sigma_{XY}\), is defined as:
\(Cov(X,Y)=\sigma_{XY}=E[(X-\mu_X)(Y-\mu_Y)]\)
That is, if \(X\) and \(Y\) are discrete random variables with joint support \(S\), then the covariance of \(X\) and \(Y\) is:
\(Cov(X,Y)=\mathop{\sum\sum}\limits_{(x,y)\in S} (x-\mu_X)(y-\mu_Y) f(x,y)\)
And, if \(X\) and \(Y\) are continuous random variables with supports \(S_1\) and \(S_2\), respectively, then the covariance of \(X\) and \(Y\) is:
\(Cov(X,Y)=\int_{S_2} \int_{S_1} (x-\mu_X)(y-\mu_Y) f(x,y)dxdy\)
Example 18-1 Section
Suppose that \(X\) and \(Y\) have the following joint probability mass function:
\( \begin{array}{cc|ccc|c} & f(x, y) & 1 & 2 & 3 & f_{X}(x) \\ \hline x & 1 & 0.25 & 0.25 & 0 & 0.5 \\ & 2 & 0 & 0.25 & 0.25 & 0.5 \\ \hline & f_{Y}(y) & 0.25 & 0.5 & 0.25 & 1 \end{array} \)
so that \(\mu_{\mathrm{x}}=3 / 2\), \(\mu_{\mathrm{Y}}=2, \sigma_{\mathrm{X}}=1 / 2\), and \(\sigma_{\mathrm{Y}}=\sqrt{1/2}\)
What is the covariance of \(X\) and \(Y\)?
Solution
Two questions you might have right now: 1) What does the covariance mean? That is, what does it tell us? and 2) Is there a shortcut formula for the covariance just as there is for the variance? We'll be answering the first question in the pages that follow. Well, sort of! In reality, we'll use the covariance as a stepping stone to yet another statistical measure known as the correlation coefficient. And, we'll certainly spend some time learning what the correlation coefficient tells us. In regards to the second question, let's answer that one now by way of the following theorem.
For any random variables \(X\) and \(Y\) (discrete or continuous!) with means \(\mu_X\) and \(\mu_Y\), the covariance of \(X\) and \(Y\) can be calculated as:
\(Cov(X,Y)=E(XY)-\mu_X\mu_Y\)
Proof
In order to prove this theorem, we'll need to use the fact (which you are asked to prove in your homework) that, even in the bivariate situation, expectation is still a linear or distributive operator:
Example 18.1 continued Section
Suppose again that \(X\) and \(Y\) have the following joint probability mass function:
\( \begin{array}{cc|ccc|c} & f(x, y) & 1 & 2 & 3 & f_{X}(x) \\ \hline x & 1 & 0.25 & 0.25 & 0 & 0.5 \\ & 2 & 0 & 0.25 & 0.25 & 0.5 \\ \hline & f_{Y}(y) & 0.25 & 0.5 & 0.25 & 1 \end{array} \)
Use the theorem we just proved to calculate the covariance of \(X\) and \(Y\).
Solution
Now that we know how to calculate the covariance between two random variables, \(X\) and \(Y\), let's turn our attention to seeing how the covariance helps us calculate what is called the correlation coefficient.