20 Distributions of Two Continuous Random Variables
Overview
In some cases, \(X\) and \(Y\) may both be continuous random variables. For example, suppose \(X\) denotes the duration of an eruption (in second) of Old Faithful Geyser, and \(Y\) denotes the time (in minutes) until the next eruption. We might want to know if there is a relationship between \(X\) and \(Y\). Or, we might want to know the probability that \(X\) falls between two particular values \(a\) and \(b\), and \(Y\) falls between two particular values \(c\) and \(d\). That is, we might want to know \(P(a<x<b, c<Y<d)\).
Objectives
Upon completion of this lesson, you should be able to:
- understand the formal definition of a joint probability density function of two continuous random variables.
- use a joint probability density function to find the probability of a specific event.
- find a marginal probability density function of a continuous random variable \(X\) from the joint probability density function of \(X\) and \(Y\).
- find the means and variances of the continuous random variables \(X\) and \(Y\) using their joint probability density function.
- understand the formal definition of a conditional probability density function of a continuous RV \(Y\) given a continuous RV \(X\).
- calculate the conditional mean and conditional variance of a continuous RV \(Y\) given a continuous RV \(X\).
- apply the methods learned in the lesson to new problems.
20.1 Two Continuous Random Variables
So far, our attention in this lesson has been directed towards the joint probability distribution of two or more discrete random variables. Now, we’ll turn our attention to continuous random variables. Along the way, always in the context of continuous random variables, we’ll look at formal definitions of joint probability density functions, marginal probability density functions, expectation and independence. We’ll also apply each definition to a particular example.
Def. 20.1 (Joint probability density function) Let \(X\) and \(Y\) be two continuous random variables, and let \(S\) denote the two-dimensional support of \(X\) and \(X\). Then, the function \(f(x,y)\) is a joint probability density function (abbreviated PDF) if it satisfies the following three conditions:
- \(f(x,y)\geq 0\)
- \(\int_{-\infty}^\infty \int_{-\infty}^\infty f(x,y)dxdy=1\)
- \(P[(X,Y) \in A]=\int\int_A f(x,y)dxdy\) where \([(X,Y) \in A]\) is an event in the \(xy\)-plane.
The first condition, of course, just tells us that the function must be nonnegative. Keeping in mind that \(f(x,y)\) is some two-dimensional surface floating above the \(xy\)-plane, the second condition tells us that, the volume defined by the support, the surface and the \(xy\)-plane must be 1. The third condition tells us that in order to determine the probability of an event \(A\), you must integrate the function \(f(x,y)\) over the space defined by the event \(A\). That is, just as finding probabilities associated with one continuous random variable involved finding areas under curves, finding probabilities associated with two continuous random variables involves finding volumes of solids that are defined by the event \(A\) in the \(xy\)-plane and the two-dimensional surface \(f(x,y)\).
Example 20.1 Let \(X\) and \(Y\) have joint probability density function:
\[ f(x,y)=4xy \]
for \(0<x<1\) and \(0<y<1\).
Is \(f(x,y)\) a valid PDF?
Solution
Before trying to verify that \(f(x,y)\) is a valid PDF, it might help to get a feel for what the function looks like.
The red square is the joint support of \(X\) and \(Y\) that lies in the \(xy\)-plane. The blue tent-shaped surface is my rendition of the \(f(x,y)\) surface. Now, in order to verify that \(f(x,y)\) is a valid PDF, we first need to show that \(f(x,y)\) is always non-negative. Clearly, that’s the case, as it lies completely above the \(xy\)-plane. If you’re still not convinced, you can see that in substituting any \(x\) and \(y\) value in the joint support into the function \(f(x,y)\), you always get a positive value.
Now, we just need to show that the volume of the solid defined by the support, the \(xy\)-plane and the surface is 1:
What is \(P(Y<X)\)?
Solution
In order to find the desired probability, we again need to find a volume of a solid as defined by the surface, the \(xy\)-plane, and the support. This time, however, the volume is not defined in the \(xy\)-plane by the unit square. Instead, the region in the \(xy\)-plane is constrained to be just that portion of the unit square for which \(y<x\). If we start with the support \(0<x<1\) and \(0<y<1\) (the red square), and find just the portion of the red square for which \(y<x\), we get the blue triangle:
So, it’s the volume of the solid between the \(f(x,y)\) surface and the blue triangle that we need to find. That is, to find the desired volume, that is, the desired probability, we need to integrate from \(y=0\) to \(x\), and then from \(x=0\) to 1:
Given the symmetry of the solid about the plane \(y=x\), perhaps we shouldn’t be surprised to discover that our calculated probability equals \(\frac{1}{2}\).
Def. 20.2 (Marginal Probability Density Functions) The marginal probability density functions of the continuous random variables \(X\) and \(Y\) are given, respectively, by:
\[ f_X(x)=\int_{-\infty}^\infty f(x,y)dy,\qquad x\in S_1 \]
and:
\[ f_Y(y)=\int_{-\infty}^\infty f(x,y)dx,\qquad y\in S_2 \]
where \(S_1\) and \(S_2\) are the respective supports of \(X\) and \(Y\).
Example (continued)
Let \(X\) and \(Y\) have joint probability density function:
\[ f(x,y)=4xy \]
for \(0<x<1\) and \(0<y<1\).
What is \(f_X(x)\), the marginal PDF of \(X\), and \(f_Y(y)\), the marginal PDF of \(Y\)?
Solution
In order to find the marginal PDF of \(X\), we need to integrate the joint PDF \(f(x,y)\) over \(0<y<1\), that is, over the support of \(Y\). Doing so, we get:
\[ f_X(x)=\int_0^1 4xy dy=4x\left[\dfrac{y^2}{2}\right]_{y=0}^{y=1}=2x, \qquad 0<x<1 \]
In order to find the marginal PDF of \(Y\), we need to integrate the joint PDF \(f(x,y)\) over \(0<x<1\), that is, over the support of \(X\). Doing so, we get:
\[ f_Y(y)=\int_0^1 4xy dx=4y\left[\dfrac{x^2}{2}\right]_{x=0}^{x=1}=2y, \qquad 0<y<1 \]
Def. 20.3 (Expected Value of a Continuous Random Variable) The expected value of a continuous random variable \(X\) can be found from the joint PDF of \(X\) and \(Y\) by:
\[ E(X)=\int_{-\infty}^\infty \int_{-\infty}^\infty xf(x,y)dxdy \]
Similarly, the expected value of a continuous random variable \(Y\) can be found from the joint PDF of \(X\) and \(Y\) by:
\[ E(Y)=\int_{-\infty}^\infty \int_{-\infty}^\infty yf(x,y)dydx \]
Example (continued)
Let \(X\) and \(Y\) have joint probability density function:
\[ f(x,y)=4xy \]
for \(0<x<1\) and \(0<y<1\).
What is the expected value of \(X\)? What is the expected value of \(Y\)?
Solution
The expected value of \(X\) is \(\frac{2}{3}\) as is found here:
We’ll leave it to you to show, not surprisingly, that the expected value of \(Y\) is also \(\frac{2}{3}\).
Def. 20.4 (Independence of Continuous Random Variables) The continuous random variables \(X\) and \(Y\) are independent if and only if the joint PDF of \(X\) and \(Y\) factors into the product of their marginal PDFs, namely:
\[ f(x,y)=f_X(x)f_Y(y), \qquad x\in S_1, \qquad y\in S_2 \]
Example (continued)
Let \(X\) and \(Y\) have joint probability density function:
\[ f(x,y)=4xy \]
for \(0<x<1\) and \(0<y<1\).
Are \(X\) and \(Y\) independent?
Solution
The random variables \(X\) and \(Y\) are indeed independent, because:
\[ f(x,y)=4xy=f_X(x) f_Y(y)=(2x)(2y)=4xy \]
So, this is an example in which the support is “rectangular” and \(X\) and \(Y\) are independent.
Note that, as is true in the discrete case, if the support \(S\) of \(X\) and \(Y\) is “triangular,” then \(X\) and \(Y\) cannot be independent. On the other hand, if the support is “rectangular” (that is, a product space), then \(X\) and \(Y\) may or may not be independent. Let’s take a look first at an example in which we have a triangular support, and then at an example in which the support is rectangular, and, unlike the previous example, \(X\) and \(Y\) are dependent.
Example 20.2 Let \(X\) and \(Y\) have joint probability density function:
\[ f(x,y)=x+y \]
for \(0<x<1\) and \(0<y<1\).
Are \(X\) and \(Y\) independent?
Solution
Again, in order to show that \(X\) and \(Y\) are independent, we need to be able to show that the joint PDF of \(X\) and \(Y\) factors into the product of the marginal PDFs. The marginal PDF of \(X\) is:
\[ f_X(x)=\int_0^1(x+y)dy=\left[xy+\dfrac{y^2}{2}\right]^{y=1}_{y=0}=x+\dfrac{1}{2},\qquad 0<x<1 \]
And, the marginal PDF of \(Y\) is:
\[ f_Y(y)=\int_0^1(x+y)dx=\left[xy+\dfrac{x^2}{2}\right]^{x=1}_{x=0}=y+\dfrac{1}{2},\qquad 0<y<1 \]
Clearly, \(X\) and \(Y\) are dependent, because:
\[ f(x,y)=x+y\neq f_X(x) f_Y(y)=\left(x+\dfrac{1}{2}\right) \left(y+\dfrac{1}{2}\right) \]
This is an example in which the support is rectangular:
and \(X\) and \(Y\) are dependent, as we just illustrated. Again, a rectangular support may or may not lead to independent random variables.
20.2 Conditional Distributions for Continuous Random Variables
Thus far, all of our definitions and examples concerned discrete random variables, but the definitions and examples can be easily modified for continuous random variables. That’s what we’ll do now!
Def. 20.5 (Conditional Probability Density Function of \(Y\) given \(X=x\)) Suppose \(X\) and \(Y\) are continuous random variables with joint probability density function \(f(x,y)\) and marginal probability density functions \(f_X(x)\) and \(f_Y(y)\), respectively. Then, the conditional probability density function of \(Y\) given \(X=x\) is defined as:
\[ h(y|x)=\dfrac{f(x,y)}{f_X(x)} \]
provided \(f_X(x)>0\). The conditional mean of \(Y\) given \(X=x\) is defined as:
\[ E(Y|x)=\int_{-\infty}^\infty yh(y|x)dy \]
The conditional variance of \(Y\) given \(X=x\) is defined as:
\[ \begin{align} \mathrm{Var}(Y|x)&=E\{[Y-E(Y|x)]^2|x\}\\&=\int_{-\infty}^\infty [y-E(Y|x)]^2 h(y|x)dy \end{align} \]
or, alternatively, using the usual shortcut:
\[ \begin{align}\mathrm{Var}(Y|x)&=E[Y^2|x]-[E(Y|x)]^2\\&=\left[\int_{-\infty}^\infty y^2h(y|x)dy\right]-\mu^2_{Y|x} \end{align} \]
Although the conditional PDF, mean, and variance of \(X\), given that \(Y=y\), is not given, their definitions follow directly from those above with the necessary modifications. Let’s take a look at an example involving continuous random variables.
Example 20.3 Suppose the continuous random variables \(X\) and \(Y\) have the following joint probability density function:
\[ f(x,y)=\dfrac{3}{2} \]
for \(x^2\le y\le 1\) and \(0<x<1\).
What is the conditional distribution of \(Y\) given \(X=x\)?
Solution
We can use the formula:
\[ h(y|x)=\dfrac{f(x,y)}{f_X(x)} \]
to find the conditional PDF of \(Y\) given \(X\). But, to do so, we clearly have to find \(f_X(x)\), the marginal PDF of \(X\) first. Recall that we can do that by integrating the joint PDF \(f(x,y)\) over \(S_2\), the support of \(Y\). Here’s what the joint support \(S\) looks like:
So, we basically have a plane, shaped like the support, floating at a constant \(\frac{3}{2}\) units above the \(xy\)-plane. To find \(f_X(x)\) then, we have to integrate:
\[ f(x,y)=\dfrac{3}{2} \]
over the support \(x^2\le y\le 1\). That is:
\[ f_X(x)=\int_{S_2}f(x,y)dy=\int^1_{x^2} 3/2dy=\left[\dfrac{3}{2}y\right]^{y=1}_{y=x^2}=\dfrac{3}{2}(1-x^2) \]
for \(0<x<1\). Now, we can use the joint PDF \(f(x,y)\) that we were given and the marginal PDF \(f_X(x)\) that we just calculated to get the conditional PDF of \(Y\) given \(X=x\):
\[ h(y|x)=\dfrac{f(x,y)}{f_X(x)}=\dfrac{\frac{3}{2}}{\frac{3}{2}(1-x^2)}=\dfrac{1}{(1-x^2)},\quad 0<x<1,\quad x^2 \leq y \leq 1 \]
That is, given \(x\), the continuous random variable \(Y\) is uniform on the interval \((x^2, 1)\). For example, if \(x=\frac{1}{4}\), then the conditional PDF of \(Y\) is:
\[ h(y|1/4)=\dfrac{1}{1-(1/4)^2}=\dfrac{1}{(15/16)}=\dfrac{16}{15} \]
for \(\frac{1}{16}\le y\le 1\). And, if \(x=\frac{1}{2}\), then the conditional PDF of \(Y\) is:
\[ h(y|1/2)=\dfrac{1}{1-(1/2)^2}=\dfrac{1}{1-(1/4)}=\dfrac{4}{3} \]
for \(\frac{1}{4}\le y\le 1\).
What is the conditional mean of \(Y\) given \(X=x\)?
Solution
We can find the conditional mean of \(Y\) given \(X=x\) just by using the definition in the continuous case. That is:
Note that given that the conditional distribution of \(Y\) given \(X=x\) is the uniform distribution on the interval \((x^2,1)\), we shouldn’t be surprised that the expected value looks like the expected value of a uniform random variable!
Let’s take the case where \(x=\frac{1}{2}\). We previously showed that the conditional distribution of \(Y\) given \(X=\frac{1}{2}\) is
\[ h(y|1/2)=\dfrac{1}{1-(1/2)^2}=\dfrac{1}{1-(1/4)}=\dfrac{4}{3} \]
for \(\frac{1}{4}\le y\le 1\). Now, we know that the conditional mean of \(Y\) given \(X=\frac{1}{2}\) is:
\[ E(Y|\dfrac{1}{2})=\dfrac{1+(1/2)^2}{2}=\dfrac{1+(1/4)}{2}=\dfrac{5}{8} \]
If we think again of the expected value as the fulcrum at which the probability mass is balanced, our results here make perfect sense:
