23 Transformations of Two Random Variables
Beta
F-distribution
Introduction
In this lesson, we consider the situation where we have two random variables and we are interested in the joint distribution of two new random variables which are a transformation of the original one. Such a transformation is called a bivariate transformation. We use a generalization of the change of variables technique which we learned in Lesson 22. We provide examples of random variables whose density functions can be derived through a bivariate transformation.
Objectives
Upon completion of this lesson, you should be able to:
- use the change-of-variable technique to find the probability distribution of \(Y_1 = u_1(X_1, X_2), Y_2 = u_2(X_1, X_2)\), a one-to-one transformation of the two random variables \(X_1\) and \(X_2\).
23.1 Change-of-Variables Technique
Recall, that for the univariate (one random variable) situation: Given \(X\) with PDF \(f(x)\) and the transformation \(Y=u(X)\) with the single-valued inverse \(X=v(Y)\), then the PDF of \(Y\) is given by
\[ g(y) = |v^\prime(y)| f\left[ v(y) \right] \]
Now, suppose \((X_1, X_2)\) has joint density \(f(x_1, x_2)\). and support \(S_X\).
Let \((Y_1, Y_2)\) be some function of \((X_1, X_2)\) defined by \(Y_1 = u_1(X_1, X_2)\) and \(Y_2 = u_2(X_1, X_2)\) with the single-valued inverse given by \(X_1 = v_1(Y_1, Y_2)\) and \(X_2 = v_2(Y_1, Y_2)\). Let \(S_Y\) be the support of \(Y_1, Y_2\).
Then, we usually find \(S_Y\) by considering the image of \(S_X\) under the transformation \((Y_1, Y_2)\). Say, given \(x_1, x_2 \in S_X\), we can find \((y_1, y_2) \in S_Y\) by
\[ x_1 = v_1(y_1, y_2), \hspace{1cm} x_2 = v_2(y_1, y_2) \]
The joint PDF \(Y_1\) and \(Y_2\) is
\[ g(y_1, y_2) = |J| f\left[ v_1(y_1, y_2), v_2(y_1, y_2) \right] \]
In the above expression, \(|J|\) refers to the absolute value of the Jacobian, \(J\). The Jacobian, \(J\), is given by
\[ \begin{align*} \left| \begin{array}{cc} \dfrac{\partial v_1(y_1, y_2)}{\partial y_1} & \dfrac{\partial v_1(y_1, y_2)}{\partial y_2} \\ \dfrac{\partial v_2(y_1, y_2)}{\partial y_1} & \dfrac{\partial v_2(y_1, y_2)}{\partial y_2} \end{array} \right| \end{align*} \]
i.e. it is the determinant of the matrix:
\[ \begin{align*} \left( \begin{array}{cc} \dfrac{\partial v_1(y_1, y_2)}{\partial y_1} & \dfrac{\partial v_1(y_1, y_2)}{\partial y_2} \\ \dfrac{\partial v_2(y_1, y_2)}{\partial y_1} & \dfrac{\partial v_2(y_1, y_2)}{\partial y_2} \end{array} \right) \end{align*} \]
Example 23.1 Suppose \(X_1\) and \(X_2\) are independent exponential random variables with parameter \(\lambda = 1\) so that
\[ \begin{align*} &f_{X_1}(x_1) = e^{-x_1} \hspace{1.5 cm} 0< x_1 < \infty \\&f_{X_2}(x_2) = e^{-x_2} \hspace{1.5 cm} 0< x_2 < \infty \end{align*} \]
The joint PDF is given by
\[ f(x_1, x_2) = f_{X_1}(x_1)f_{X_2}(x_2) = e^{-x_1-x_2} \hspace{1.5 cm} 0< x_1 < \infty, 0< x_2 < \infty \]
Consider the transformation: \(Y_1 = X_1-X_2, Y_2 = X_1+X_2\). We wish to find the joint distribution of \(Y_1\) and \(Y_2\).
We have
\[ x_1 = \frac{y_1+y_2}{2}, x_2=\frac{y_2-y_1}{2} \]
OR
\[ v_1(y_1, y_2) = \frac{y_1+y_2}{2}, v_2(y_1, y_2)=\frac{y_2-y_1}{2} \]
The Jacobian, \(J\) is
\[ \begin{align*} \left| \begin{array}{cc} \dfrac{\partial \left( \frac{y_1+y_2}{2} \right) }{\partial y_1} & \dfrac{\partial \left( \frac{y_1+y_2}{2} \right)}{\partial y_2} \\ \dfrac{\partial \left( \frac{y_2-y_1}{2} \right)}{\partial y_1} & \dfrac{\partial \left( \frac{y_2-y_1}{2} \right)}{\partial y_2} \end{array} \right| \end{align*} \]
\[ \begin{align*} =\left| \begin{array}{cc} \frac{1}{2} & \frac{1}{2} \\ -\frac{1}{2} & \frac{1}{2} \end{array} \right| = \frac{1}{2} \end{align*} \]
So,
\[ \begin{align*} g(y_1, y_2) &= e^{-v_1(y_1, y_2) - v_2(y_1, y_2) }|\frac{1}{2}| \\ & = e^{- \left[\frac{y_1+y_2}{2}\right] - \left[\frac{y_2-y_1}{2}\right] }|\frac{1}{2}| \\ &= \frac{e^{-y_2}}{2} \end{align*} \]
Now, we determine the support of \((Y_1, Y_2)\). Since \(0< x_1 < \infty, 0< x_2 < \infty\), we have \(0< \frac{y_1+y_2}{2} < \infty, 0< \frac{y_2-y_1}{2} < \infty\) or \(0< y_1+y_2 < \infty, 0< y_2-y_1 < \infty\). This may be rewritten as \(-y_2< y_1 < y_2, 0< y_2 < \infty\).
Using the joint PDF, we may find the marginal PDF of \(Y_2\) as
\[ \begin{align*} g(y_2) &= \int_{-\infty}^{\infty} g(y_1, y_2) dy_1 \\ &= \int_{-y_2}^{y_2}\frac{1}{2}e^{-y_2} dy_1 \\ &= \left. \frac{1}{2} \left[ e^{-y_2} y_1 \right|_{y_1=-y_2}^{y_1=y_2} \right] \\ &= \frac{1}{2} e^{-y_2} (y_2 + y_2) \\ &= y_2 e^{-y_2}, \hspace{1cm} 0< y_2 < \infty \end{align*} \]
Similarly, we may find the marginal PDF of \(Y_1\) as
\[ \begin{align*} g(y_1)=\begin{cases} \int_{-y_1}^{\infty} \frac{1}{2}e^{-y_2} dy_2 = \frac{1}{2} e^{y_1} & -\infty < y_1 < 0 \\ \int_{y_1}^{\infty} \frac{1}{2}e^{-y_2} dy_2 = \frac{1}{2} e^{-y_1} & 0 < y_1 < \infty \\ \end{cases} \end{align*} \]
Equivalently,
\[ g(y_1) = \frac{1}{2} e^{-|y_1|}, 0 < y_1 < \infty \]
This PDF is known as the double exponential or Laplace PDF.
23.2 Beta Distribution
Let \(X_1\) and \(X_2\) have independent gamma distributions with parameters \(\alpha, \theta\) and \(\beta\) respectively. Therefore, the joint PDF of \(X_1\) and \(X_2\) is given by
\[ f(x_1, x_2) = \frac{1}{\Gamma(\alpha) \Gamma(\beta)\theta^{\alpha + \beta}} x_1^{\alpha-1}x_2^{\beta-1}\text{ exp }\left( -\frac{x_1 + x_2}{\theta} \right), 0 <x_1 <\infty, 0 <x_2 <\infty \]
We make the following transformation:
\[ Y_1 = \frac{X_1}{X_1+X_2}, Y_2 = X_1+X_2 \]
The inverse transformation is given by
\[ \begin{align*} &X_1=Y_1Y_2, \\& X_2=Y_2-Y_1Y_2 \end{align*} \]
The Jacobian is
\[ \begin{align*} \left| \begin{array}{cc} y_2 & y_1 \\ -y_2 & 1-y_1 \end{array} \right| = y_2(1-y_1) + y_1y_2 = y_2 \end{align*} \]
The joint PDF \(g(y_1, y_2)\) is
\[ \begin{align*} g(y_1, y_2) = |y_2| \frac{1}{\Gamma(\alpha) \Gamma(\beta)\theta^{\alpha + \beta}} (y_1y_2)^{\alpha - 1}(y_2 - y_1y_2)^{\beta - 1}e^{-y_2/\theta} \end{align*} \]
with support is \(0<y_1<1, 0<y_2<\infty\)
It may be shown that the marginal PDF of \(Y_1\) is
\[ \begin{align*} g(y_1) & = \frac{y_1^{\alpha - 1}(1 - y_1)^{\beta - 1}}{\Gamma(\alpha) \Gamma(\beta) } \int_0^{\infty} \frac{y_2^{\alpha + \beta -1}}{\theta^{\alpha + \beta}} e^{-y_2/\theta} dy_2 g(y_1) \\& = \frac{ \Gamma(\alpha + \beta) }{\Gamma(\alpha) \Gamma(\beta) } y_1^{\alpha - 1}(1 - y_1)^{\beta - 1}, \hspace{1cm} 0<y_1<1. \end{align*} \]
\(Y_1\) is said to have a beta PDF with parameters \(\alpha\) and \(\beta\).
23.3 F Distribution
We describe a very useful distribution in Statistics known as the F distribution.
Let \(U\) and \(V\) be independent chi-square variables with \(r_1\) and \(r_2\) degrees of freedom, respectively. The joint PDF is:
\[ g(u, v) = \frac{ u^{r_1/2-1}e^{-u/2} v^{r_2/2-1}e^{-v/2} } { \Gamma (r_1/2) 2^{r_1/2} \Gamma(r_2/2) 2^{r_2/2} } , \hspace{1cm} 0<u<\infty, 0<v<\infty \]
Define the random variable \(W = \frac{U/r_1}{V/r_2}\)
This time we use the distribution function technique described in lesson 22,
\[ F(w) = P(W \leq w)= P \left( \frac{U/r_1}{V/r_2} \leq w \right) = P(U \leq \frac{r_1}{r_2} wV) = \int_0^\infty \int_0^{(r_1/r_2)wv} g(u, v) du dv \]
\[ F(w) =\frac{1}{ \Gamma (r_1/2) \Gamma (r_2/2) } \int_0^\infty \left[ \int_0^{(r_1/r_2)wv} \frac{ u^{r_1/2-1}e^{-u/2}}{2^{(r_1+r_2)/2}} du \right] v^{r_1/2-1}e^{-v/2} dv \]
By differentiating the CDF, it can be shown that \(f(w) = F^\prime(w)\) is given by
\[ f(w) = \frac{ \left( r_1/r_2 \right)^{r_1/2} \Gamma \left[ \left(r_1+r_2\right)/2 \right]w^{r_1/2-1} }{\Gamma(r_1/2)\Gamma(r_2/2) \left[1+(r_1w/r_2)\right]^{(r_1+r_2)/2}}, \hspace{1cm} w>0 \]
A random variable with the PDF \(f(w)\) is said to have an F distribution with \(r_1\) and \(r_2\) degrees of freedom. We write this as \(F(r_1, r_2)\). Table VII in Appendix B of the textbook can be used to find probabilities for a random variable with the \(F(r_1, r_2)\) distribution.
It contains the F-values for various cumulative probabilities \((0.95, 0.975, 0.99)\) (or the equivalent upper − \(\alpha\) th probabilities \((0.05, 0.025, 0.01)\)) of various \(F (r1, r2)\) distributions.
When using this table, it is helpful to note that if a random variable (say, \(W\)) has the \(F(r_1, r_2)\) distribution, then its inverse \(\dfrac{1}{W}\) has the \(F(r_2, r_1)\) distribution.
Illustration
The shape of the F distribution is determined by the degrees of freedom \(r_1\) and \(r_2\). The histogram below shows how an F random variable is generated using 1000 observations each from two chi-square random variables (\(U\) and \(V\)) with degrees of freedom 4 and 8 respectively and forming the ratio \(\dfrac{U/4}{V/8}\).
The plot below illustrates how the shape of an F distribution changes with the degrees of freedom \(r_1\) and \(r_2\).
