First, you have to group the data into a set of classes, typically of equal length. There are many, many sets of rules for defining the classes. For our purposes, we'll just rely on our common sense — having too few classes is as bad as having too many.

1. Determine the number, \(n\), in the sample.
2. Define \(k\) class intervals \((c_0, c_1], (c_1, c_2], \ldots, (c_{k-1}, c_k]\) .
3. Determine the frequency, \(f_i\), of each class \(i\).
4. Calculate the relative frequency (proportion) of each class by dividing the class frequency by the total number in the sample — that is, \(\frac{f_i}{n}\).
5. For a frequency histogram: draw a rectangle for each class with the class interval as the base and the height equal to the frequency of the class.
6. For a relative frequency histogram: draw a rectangle for each class with the class interval as the base and the height equal to the relative frequency of the class.
7. For a density histogram: draw a rectangle for each class with the class interval as the base and the height equal to \(h(x)=\dfrac{f_i}{n(c_i-c_{i-1})}\)

We've previously learned that the sample mean can be thought of as the "center" of a set of data, while the sample standard deviation indicates "how spread out" the data are from the sample mean. Now, if a histogram is "mound-shaped" or "bell-shaped," then we can use the sample mean, sample standard deviation, and what is called the Empirical Rule to determine three intervals for which we would expect approximately 68%, 95%, and 99.7% of the data to fall.

The Empirical Rule tells us that if a histogram is at least approximately bell-shaped, then:

1. Approximately 68% of the data are in the interval:

   \((\bar{x}-s,\bar{x}+s)\)

2. Approximately 95% of the data are in the interval:

   \((\bar{x}-2s,\bar{x}+2s)\)

3. Approximately 99.7% of the data are in the interval:

   \((\bar{x}-3s,\bar{x}+3s)\)

The primary advantage of creating an ordered stem-and-leaf plot is that you can readily read what are called the order statistics right off of the plot. If we have a sample of \(n\) observations represented as:

\(x_1,x_2,x_3,\cdots,x_n\)

then when the observations are ordered from smallest to largest, the resulting ordered data are called the order statistics of the sample, and are represented as:

\(y_1 \leq y_2 \leq y_3 \leq \cdots \leq y_n\)

That is, \(y_1\), the smallest data point is the first order statistic. The second smallest data point, \(y_2\), is the second order statistic. And so on, until we reach the largest data point and \(n^{th}\) order statistic, \(y_n\). From the order statistics, it is rather easy to find the sample percentiles.

Here's the typical method used for finding a particular sample percentile:

1. Arrange the sample data in increasing order. That is, determine the order statistics:

   \(y_1 \leq y_2 \leq y_3 \leq \cdots \leq y_n\)

2. If \((n+1)p\) is an integer, then the \((100p)^{th}\) sample percentile is the \((n+1)p^{th}\) order statistic.

3. If \((n+1)p\) is not an integer, but rather equals \(r\) plus some proper fraction, \(a/b\) say, then use a weighted average of the \(r^{th}\) and \((r+1)^{st}\) order statistics. That is, define the \((100p)^{th}\) sample percentile as:

   \(\tilde{\pi}_p=y_r+\left(\dfrac{a}{b}\right)(y_{r+1}-y_r)\)

Let's try this method out on an example or two.

On the last page, we learned how to determine the first quartile, the median, and the third quartile for a sample of data. These three percentiles, along with a data set's minimum and maximum values, make up what is called the five-number summary. One nice way of graphically depicting a data set's five-number summary is by way of a box plot (or box-and-whisker plot).

Here are some general guidelines for drawing a box plot:

1. Draw a horizontal axis scaled to the data.
2. Above the axis, draw a rectangular box with the left side of the box at the first quartile \(q_1\) and the right side of the box at the third quartile \(q_3\).
3. Draw a vertical line connecting the lower and upper horizontal lines of the box at the median \(m\).
4. For the left whisker, draw a horizontal line from the minimum value to the midpoint of the left side of the box.
5. For the right whisker, draw a horizontal line from the maximum value to the midpoint of the right side of the box.

Drawn as such, a box plot does a nice job of dividing the data graphically into fourths. Note, for example, that the horizontal length of the box is the interquartile range IQR, the left whisker represents the first quarter of the data, and the right whisker represents the fourth quarter of the data.

Histograms and box plots can be quite useful in suggesting the shape of a probability distribution. Here, we'll concern ourselves with three possible shapes: symmetric, skewed left, or skewed right.

The following examples probably illustrate symmetry and skewness of distributions better than any formal definitions can.

A continuous random variable takes on an uncountably infinite number of possible values. For a discrete random variable \(X\) that takes on a finite or countably infinite number of possible values, we determined \(P(X=x)\) for all of the possible values of \(X\), and called it the probability mass function ("p.m.f."). For continuous random variables, as we shall soon see, the probability that \(X\) takes on any particular value \(x\) is 0. That is, finding \(P(X=x)\) for a continuous random variable \(X\) is not going to work. Instead, we'll need to find the probability that \(X\) falls in some interval \((a, b)\), that is, we'll need to find \(P(a<X<b)\). We'll do that using a probability density function ("p.d.f."). We'll first motivate a p.d.f. with an example, and then we'll formally define it.

You might recall that the cumulative distribution function is defined for discrete random variables as:

\(F(x)=P(X\leq x)=\sum\limits_{t \leq x} f(t)\)

Again, \(F(x)\) accumulates all of the probability less than or equal to \(x\). The cumulative distribution function for continuous random variables is just a straightforward extension of that of the discrete case. All we need to do is replace the summation with an integral.

At some point in your life, you have most likely been told that you fall in the something-something percentile with regards to some measure. For example, if you are tall, you might have been told that you are in the 95th percentile in height, meaning that you are taller than 95% of the population. When you took the SAT Exams, you might have been told that you are in the 80th percentile in math ability, meaning that you scored better than 80% of the population on the math portion of the SAT Exams. We'll now formally define what a percentile is within the framework of probability theory.

The special expectations, such as the mean, variance, and moment generating function, for continuous random variables are just a straightforward extension of those of the discrete case. Again, all we need to do is replace the summations with integrals.

\(E(X)=M'(0)\)

and the variance:

\(\text{Var}(X)=M^{\prime\prime}(0)-\left(M^\prime(0)\right)^2\)

Some distributions are split into parts. They are not necessarily continuous, but they are continuous over particular intervals. These types of distributions are known as Piecewise distributions. Below is an example of this type of distribution

\(\begin{align*} f(x)=\begin{cases} 2-4x, & x< 1/2\\ 4x-2, & x\ge 1/2 \end{cases} \end{align*}\)

for \(0<x<1\). The pdf of \(X\) is shown below.

The first step is to show this is a valid pdf. To show it is a valid pdf, we have to show the following:

1. \(f(x)>0\). We can see that \(f(x)\) is greater than or equal to 0 for all values of \(X\).

2. \(\int_S f(x)dx=1\).

   \(\begin{align*} & \int_0^{1/2}2-4xdx+\int_{1/2}^1 4x-2dx\\ & = 2\left(\frac{1}{2}\right)-2\left(\frac{1}{4}\right)+2-2-\left[2\left(\frac{1}{4}\right)-1\right]\\ & = 1-\left(\frac{1}{2}\right)+2-2-\left(\frac{1}{2}\right)+1=2-1=1 \end{align*}\)

3. If \((a, b)\subset S\), then \(P(a<X<b)=\int_a^bf(x)dx\). Lets find the probability that \(X\) is between 0 and \(2/3\).

   \(P(X<2/3)=\int_0^{1/2} 2-4xdx+\int_{1/2}^{2/3} 4x-2dx=\frac{5}{9}\)

The next step is to know how to find expectations of piecewise distributions. If we know how to do this, we can find the mean, variance, etc of a random variable with this type of distribution. Suppose we want to find the expected value, \(E(X)\).

\(\begin{align*}& E(X)=\int_0^{1/2} x(2-4x)dx+\int_{1/2}^1 x(4x-2)dx\\& =\left(x^2-\frac{4}{3}x^3\right)|_0^{1/2}+\left(\frac{4}{3}x^3-x^2\right)|_{1/2}^1=\frac{1}{2}\end{align*}\)

The variance and other expectations can be found similarly.

The final step is to find the cumulative distribution function. cdf. Recall the cdf of \(X\) is \(F_X(t)=P(X\le t)\). Therefore, for \(t<\frac{1}{2}\), we have

\(F_X(t)=\int_0^t 2-4xdx=2x-x^2|_0^t=2t-2t^2\)

and for \(t\ge\frac{1}{2}\) we have

\(\begin{align*} & F_X(t)=\int_0^{1/2}2-4xdx+\int_{1/2}^t 4x-2dx=\frac{1}{2}+\left(2x^2-2x\right)|_{1/2}^t\\ & =2t^2-2t+1 \end{align*}\)

Thus, the cdf of \(X\) is

\(\begin{equation*} F_X(t)=\begin{cases} 2t-2t^2 & 0<t<1/2\\ 2t^2-2t+1 & 1/2\le t<1 \end{cases} \end{equation*}\)

Note that the length of the base of the rectangle is \((b-a)\), while the length of the height of the rectangle is \(\dfrac{1}{b-a}\). Therefore, as should be expected, the area under \(f(x)\) and between the endpoints \(a\) and \(b\) is 1. Additionally, \(f(x)>0\) over the support \(a<x<b\). Therefore, \(f(x)\) is a valid probability density function.

Because there are an infinite number of possible constants \(a\) and \(b\), there are an infinite number of possible uniform distributions. That's why this page is called Uniform Distributions (with an s!) and not Uniform Distribution (with no s!). That said, the continuous uniform distribution most commonly used is the one in which \(a=0\) and \(b=1\).

As the picture illustrates, \(F(x)=0\) when \(x\) is less than the lower endpoint of the support (\(a\), in this case) and \(F(x)=1\) when \(x\) is greater than the upper endpoint of the support (\(b\), in this case). The slope of the line between \(a\) and \(b\) is, of course, \(\dfrac{1}{b-a}\).

Here, we present and prove three key properties of a uniform random variable.

Perhaps not surprisingly, the uniform distribution is not particularly useful in describing much of the randomness we see in the natural world. Its claim to fame is instead its usefulness in random number generation. That is, approximate values of the \(U(0,1)\) distribution can be simulated on most computers using a random number generator. The generated numbers can then be used to randomly assign people to treatments in experimental studies, or to randomly select individuals for participation in a survey.

Before we explore the above-mentioned applications of the \(U(0,1)\) distribution, it should be noted that the random numbers generated from a computer are not technically truly random, because they are generated from some starting value (called the seed). If the same seed is used again and again, the same sequence of random numbers will be generated. It is for this reason that such random number generation is sometimes referred to as pseudo-random number generation. Yet, despite a sequence of random numbers being pre-determined by a seed number, the numbers do behave as if they are truly randomly generated, and are therefore very useful in the above-mentioned applications. They would probably not be particularly useful in the applications of cryptography or internet security, however!

Before we jump in and use a computer and a \(U(0,1)\) distribution to make random assignments and random selections, it would be useful to discuss how we might evaluate if a particular set of data follow a particular probability distribution. One possibility is to compare the theoretical mean (\(\mu\)) and variance (\(\sigma^2\)) with the sample mean ( \(\bar{x}\)) and sample variance (\(s^2\)). It shouldn't be surprising that such a comparison is hardly sufficient. Another technique used frequently is the creation of what is called a quantile-quantile plot (or a q-q plot, for short. The basic idea behind a q-q plot is a two-step process: 1) first determine the theoretical quantiles (from the supposed probability distribution) and the sample quantiles (from the data), and then 2) compare them on a plot. If the theoretical and sample quantiles "match," there is good evidence that the data follow the supposed probability distribution. Here are the specific details of how to create a q-q plot:

1. Determine the theoretical quantile of order \(p\), that is, the \((100p)^{th}\) percentile \(\pi_p\).

2. Determine the sample quantile, \(y_r\), of order \(\dfrac{r}{n+1}\), that is the \(100\dfrac{r}{n+1}\) percentile. While that might sound complicated, it amounts to just ordering the data \(x_1, x_2, \ldots, x_n\) to get the order statistics \(y_1\le y_2\le \ldots \le y_n\).

3. Plot the theoretical quantile on the y-axis against the sample quantile on the x-axis. If the sample data follow the theoretical probability distribution, we would expect the points \((y_r, \pi_p)\) to lie close to a line through the origin with slope equal to one.

In the case of the \(U(0,1)\) distribution, the cumulative distribution function is \(F(x)=x\). Now, recall that to find the \((100p)^{th}\) percentile \(\pi_p\), we set \(p\) equal to \(F(\pi_p)\) and solve for \(\pi_p\). That means in the case of the \(U(0,1)\) distribution, we set \(F(\pi_p)=\pi_p\) equal to \(p\) and solve for \(\pi_p\). Ahhhhaaa! In the case of the \(U(0,1)\) distribution, \(\pi_p=p\). That is, \(\pi_{0.05}=0.05\), \(\pi_{0.35}=0.35\), and so on. Let's take a look at an example!

As suggested earlier, the \(U(0,1)\) distribution can be quite useful in randomly assigning experimental units to the treatments in an experiment. First, let's review why randomization is a useful venture when conducting an experiment. Suppose we were interested in measuring how high a person could reach after "taking" an experimental treatment. It would be awfully hard to draw a strong conclusion about the effectiveness of the experimental treatment if the people in one treatment group were, to begin with, significantly taller than the people in the other treatment group. Randomly assigning people to the treatments in an experiment minimize the chances that such important differences exist in the treatment groups. That way if differences exist in the two groups at the conclusion of the study with respect to the primary variable of interest, we can feel confident in attributing the difference strongly to the treatment of interest rather than due to some other fundamental difference in the groups.

Okay, now let's talk about how the \(U(0,1)\) distribution can help us randomly assign the experimental units to the treatments in a completely randomized experiment. For the sake of concreteness, suppose we wanted to randomly assign 20 students to one group (those who complete a blue data collection form, say) and 20 students to a second group (those who complete a green data collection form, say). This is what the procedure might look like:

1. Assign the pool of 40 potential students each one number from 1 to 40. It doesn't matter how you assign these numbers.

2. Generate 40 \(U(0,1)\) numbers in one column of a spreadsheet. Enter the numbers 1 to 40 in a second column of a spreadsheet.

3. Sort the 40 \(U(0,1)\) numbers in sorted increasing order, so that the numbers in the second column follow along during the sorting process. For example, if the 13th generated \(U(0,1)\) number was the smallest number generated, then the number 13 should appear, after sorting, in the first row of the second column. If the 24th generated \(U(0,1)\) number was the second smallest number generated, the number 24 should appear, after sorting, in the second row of the second column. And so on.

4. The students whose numbers appear in the first 20 rows of the second column should be assigned to complete the blue data collection form. The students whose numbers appear in the second 20 rows of the second column should be assigned to complete the green data collection form.

One semester, I conducted the above experiment exactly as described. Twenty students were randomly assigned to complete a blue version of the following form, and the remaining twenty students were randomly assigned to complete a green version of the form:

After administering the forms to the 40 students, here's what the resulting data looked like:

And, here's a portion of a basic descriptive analysis of six of the variables on the form:

The analysis suggests that my randomization worked quite well. For example, the mean grade-point average of those students completing the blue form was 3.40, while the mean g.p.a. for those students completing the green form was 3.46. And, the mean height of those students completing the blue form was 66.8 inches, while the mean height for those students completing the green form was 67.3 inches. The two groups appear to be similar, on average, with respect to the other collected data as well. It should be noted that there is no guarantee that any particular randomization will be as successful as the one I illustrated here. The only thing that the randomization ensures is that the chance that the groups will differ with respect to key measurements will be small.

Just as you should always randomly assign experimental units to treatments when conducting an experiment, you should always randomly select your participants when conducting a survey. If you don't, you might very well end up with biased survey results. (The people who choose to take the time to complete a survey in a magazine or on a web site typically have quite strong opinions!) The procedure we can use to randomly select participants for a survey is quite similar to that used for randomly assigning people to treatments in a completely randomized experiment. This is what the procedure would look like if you wanted to randomly select, say, 1000 students to participate in a survey from a potential pool of, say, 40000 students:

1. Assign the pool of 40000 potential participants each one number from 1 to 40000. It doesn't matter how you assign these numbers.

2. Generate 40000 \(U(0,1)\) numbers in one column of a spreadsheet. Enter the numbers 1 to 40000 in a second column of a spreadsheet.

3. Sort the 40000 \(U(0,1)\) numbers in sorted increasing order, so that the numbers in the second column follow along during the sorting process. For example, if the 23rd generated \(U(0,1)\) number was the smallest number generated, the number 23 should appear, after sorting, in the first row of the second column. If the 102nd generated \(U(0,1)\) number was the second smallest number generated, the number 102 should appear, after sorting, in the second row of the second column. And so on.

4. The students whose numbers appear in the first 1000 rows of the second column should be selected to participate in the survey.

Following the procedure as described, the 1000 selected students represent a random sample from the population of 40000 students.

Here, we present and prove four key properties of an exponential random variable.

In the previous lesson, we learned that in an approximate Poisson process with mean \(\lambda\), the waiting time \(X\) until the first event occurs follows an exponential distribution with mean \(\theta=\frac{1}{\lambda}\). We now let \(W\) denote the waiting time until the \(\alpha^{th}\) event occurs and find the distribution of \(W\). We could represent the situation as follows:

Just as we did in our work with deriving the exponential distribution, our strategy here is going to be to first find the cumulative distribution function \(F(w)\) and then differentiate it to get the probability density function \(f(w)\). Now, for \(w>0\) and \(\lambda>0\), the definition of the cumulative distribution function gives us:

\(F(w)=P(W\le w)\)

The rule of complementary events tells us then that:

\(F(w)=1-P(W> w)\)

Now, the waiting time \(W\) is greater than some value \(w\) only if there are fewer than \(\alpha\) events in the interval \([0,w]\). That is:

\(F(w)=1-P(\text{fewer than }\alpha\text{ events in } [0,w]) \)

A more specific way of writing that is:

\(F(w)=1-P(\text{0 events or 1 event or ... or }(\alpha-1)\text{ events in } [0,w]) \)

Those mutually exclusive "ors" mean that we need to add up the probabilities of having 0 events occurring in the interval \([0,w]\), 1 event occurring in the interval \([0,w]\), ..., up to \((\alpha-1)\) events in \([0,w]\). Well, that just involves using the probability mass function of a Poisson random variable with mean \(\lambda w\). That is:

\(F(w)=1-\sum\limits_{k=0}^{\alpha-1} \dfrac{(\lambda w)^k e^{-\lambda w}}{k!}\)

Now, we could leave \(F(w)\) well enough alone and begin the process of differentiating it, but it turns out that the differentiation goes much smoother if we rewrite \(F(w)\) as follows:

\(F(w)=1-e^{-\lambda w}-\sum\limits_{k=1}^{\alpha-1} \dfrac{1}{k!} \left[(\lambda w)^k e^{-\lambda w}\right]\)

As you can see, we merely pulled the \(k=0\) out of the summation and rewrote the probability mass function so that it would be easier to administer the product rule for differentiation.

Now, let's do that differentiation! We need to differentiate \(F(w)\) with respect to \(w\) to get the probability density function \(f(w)\). Using the product rule, and what we know about the derivative of \(e^{\lambda w}\) and \((\lambda w)^k\), we get:

\(f(w)=F'(w)=\lambda e^{-\lambda w} -\sum\limits_{k=1}^{\alpha-1} \dfrac{1}{k!} \left[(\lambda w)^k \cdot (-\lambda e^{-\lambda w})+ e^{-\lambda w} \cdot k(\lambda w)^{k-1} \cdot \lambda \right]\)

Pulling \(\lambda e^{-\lambda w}\) out of the summation, and dividing \(k\) by \(k!\) (to get \( \frac{1}{(k-1)!}\)) in the second term in the summation, we get that \(f(w)\) equals:

\(=\lambda e^{-\lambda w}+\lambda e^{-\lambda w}\left[\sum\limits_{k=1}^{\alpha-1} \left\{ \dfrac{(\lambda w)^k}{k!}-\dfrac{(\lambda w)^{k-1}}{(k-1)!} \right\}\right]\)

Evaluating the terms in the summation at \(k=1, k=2\), up to \(k=\alpha-1\), we get that \(f(w)\) equals:

\(=\lambda e^{-\lambda w}+\lambda e^{-\lambda w}\left[(\lambda w-1)+\left(\dfrac{(\lambda w)^2}{2!}-\lambda w\right)+\cdots+\left(\dfrac{(\lambda w)^{\alpha-1}}{(\alpha-1)!}-\dfrac{(\lambda w)^{\alpha-2}}{(\alpha-2)!}\right)\right]\)

Do some (lots of!) crossing out (\(\lambda w -\lambda w =0\), for example), and a bit more simplifying to get that \(f(w)\) equals:

\(=\lambda e^{-\lambda w}+\lambda e^{-\lambda w}\left[-1+\dfrac{(\lambda w)^{\alpha-1}}{(\alpha-1)!}\right]=\lambda e^{-\lambda w}-\lambda e^{-\lambda w}+\dfrac{\lambda e^{-\lambda w} (\lambda w)^{\alpha-1}}{(\alpha-1)!}\)

And since \(\lambda e^{-\lambda w}=\lambda e^{-\lambda w}=0\), we get that \(f(w)\) equals:

\(=\dfrac{\lambda e^{-\lambda w} (\lambda w)^{\alpha-1}}{(\alpha-1)!}\)

Are we there yet? Almost! We just need to reparameterize (if \(\theta=\frac{1}{\lambda}\), then \(\lambda=\frac{1}{\theta}\)). Doing so, we get that the probability density function of \(W\), the waiting time until the \(\alpha^{th}\) event occurs, is:

\(f(w)=\dfrac{1}{(\alpha-1)! \theta^\alpha} e^{-w/\theta} w^{\alpha-1}\)

for \(w>0, \theta>0\), and \(\alpha>0\).

Recall that \(\theta\) is the mean waiting time until the first event, and \(\alpha\) is the number of events for which you are waiting to occur. It makes sense then that for fixed \(\alpha\), as \(\theta\) increases, the probability "moves to the right," as illustrated here with \(\alpha\)fixed at 3, and \(\theta\) increasing from 1 to 2 to 3:

The plots illustrate, for example, that if we are waiting for \(\alpha=3\) events to occur, we have a greater probability of our waiting time \(X\) being large if our mean waiting time until the first event is large (\(\theta=3\), say) than if it is small (\(\theta=1\), say).

It also makes sense that for fixed \(\theta\), as \(\alpha\) increases, the probability "moves to the right," as illustrated here with \(\theta\)fixed at 3, and \(\alpha\) increasing from 1 to 2 to 3:

The plots illustrate, for example, that if the mean waiting time until the first event is \(\theta=3\), then we have a greater probability of our waiting time \(X\) being large if we are waiting for more events to occur (\(\alpha=3\), say) than fewer (\(\alpha=1\), say).

We'll primarily use the definition in order to help us prove the two theorems that follow.

Here, after formally defining the gamma distribution (we haven't done that yet?!), we present and prove (well, sort of!) three key properties of the gamma distribution.

Before we get to the three theorems and proofs, two notes:

1. We consider \(\alpha>0\) a positive integer if the derivation of the p.d.f. is motivated by waiting times until α events. But the p.d.f. is actually a valid p.d.f. for any \(\alpha>0\) (since \(\Gamma(\alpha)\) is defined for all positive \(\alpha\)).

2. The gamma p.d.f. reaffirms that the exponential distribution is just a special case of the gamma distribution. That is, when you put \(\alpha=1\) into the gamma p.d.f., you get the exponential p.d.f.

Chi-squared distributions are very important distributions in the field of statistics. As such, if you go on to take the sequel course, Stat 415, you will encounter the chi-squared distributions quite regularly. In this course, we'll focus just on introducing the basics of the distributions to you. In Stat 415, you'll see its many applications.

As it turns out, the chi-square distribution is just a special case of the gamma distribution! Let's take a look.

As the following theorems illustrate, the moment generating function, mean and variance of the chi-square distributions are just straightforward extensions of those for the gamma distributions.

One of the primary ways that you will find yourself interacting with the chi-square distribution, primarily later in Stat 415, is by needing to know either a chi-square value or a chi-square probability in order to complete a statistical analysis. For that reason, we'll now explore how to use a typical chi-square table to look up chi-square values and/or chi-square probabilities. Let's start with two definitions.

Definition. Let \(\alpha\) be some probability between 0 and 1 (most often, a small probability less than 0.10). The upper \(100\alpha^{th}\) percentile of a chi-square distribution with \(r\) degrees of freedom is the value \(\chi^2_\alpha (r)\) such that the area under the curve and to the right of \(\chi^2_\alpha (r)\) is \(\alpha\):

The above definition is used, as is the one that follows, in Table IV, the chi-square distribution table in the back of your textbook.

Definition. Let \(\alpha\) be some probability between 0 and 1 (most often, a small probability less than 0.10). The \(100\alpha^{th}\) percentile of a chi-square distribution with \(r\) degrees of freedom is the value \(\chi^2_{1-\alpha} (r)\) such that the area under the curve and to the right of \(\chi^2_{1-\alpha} (r)\) is \(1-\alpha\):

With these definitions behind us, let's now take a look at the chi-square table in the back of your textbook.

In summary, here are the steps you should use in using the chi-square table to find a chi-square value:

1. Find the row that corresponds to the relevant degrees of freedom, \(r\) .
2. Find the column headed by the probability of interest... whether it's 0.01, 0.025, 0.05, 0.10, 0.90, 0.95, 0.975, or 0.99.
3. Determine the chi-square value where the \(r\) row and the probability column intersect.

Now, at least theoretically, you could also use the chi-square table to find the probability associated with a particular chi-square value. But, as you can see, the table is pretty limited in that direction. For example, if you have a chi-square random variable with 5 degrees of freedom, you could only find the probabilities associated with the chi-square values of 0.554, 0.831, 1.145, 1.610, 9.236, 11.07, 12.83, and 15.09:

What would you do if you wanted to find the probability that a chi-square random variable with 5 degrees of freedom was less than 6.2, say? Well, the answer is, of course... statistical software, such as SAS or Minitab! For what we'll be doing in Stat 414 and 415, the chi-square table will (mostly) serve our purpose. Let's get a bit more practice now using the chi-square table.

Sometimes taking the integral is not an easy task. We do have some tools, however, to help avoid some of them. Let's take a look at an example.

Suppose we have a random variable, \(X\), that has a Gamma distribution and we want to find the Moment Generating function of \(X\), \(M_X(t)\). There is an example of how to compute this in the notes but let's try it another way.

\(\begin{align*}  M_X(t)&=\int_0^\infty \frac{1}{\Gamma(\alpha)\beta^\alpha} x^{\alpha-1}e^{-x/\beta}e^{tx}dx\\ & = \int_0^\infty \frac{1}{\Gamma(\alpha)\beta^\alpha} x^{\alpha-1}e^{-x\left(\frac{1}{\beta}-t\right)}dx \end{align*}\)

Let's rewrite this integral, taking out the constants for now and rewriting the exponent term.

\(\begin{align*}  M_X(t)&= \frac{1}{\Gamma(\alpha)\beta^\alpha}\int_0^\infty x^{\alpha-1}e^{-x\left(\frac{1}{\beta}-t\right)}dx\\ & =\frac{1}{\Gamma(\alpha)\beta^\alpha}\int_0^\infty x^{\alpha-1}e^{-x\left(\frac{1-\beta t}{\beta}\right)}dx\\ & = \frac{1}{\Gamma(\alpha)\beta^\alpha}\int_0^\infty x^{\alpha-1}e^{-x/\left(\frac{\beta}{1-\beta t}\right)}dx\\ & = \frac{1}{\Gamma(\alpha)\beta^\alpha}\int_0^\infty g(x)dx \end{align*}\)

When we rewrite it this way, the term under the integral, \(g(x)\), looks almost like a Gamma density function with parameters \(\alpha\) and \(\beta^*=\dfrac{\beta}{1-\beta t}\). The only issues that we need to take care of are the constants in front of the integral and the constants to make \(g(x)\) a gamma density function.

To make \(g(x)\) a Gamma density function, we need the constants. Therefore, we need a \(\dfrac{1}{\Gamma(\alpha)}\) term and a \(\dfrac{1}{(\beta^*)^\alpha}\) term. We already have the first term, so lets rewrite the function.

\(M_X(t)=\frac{1}{\beta^\alpha}\int_0^\infty \frac{1}{\Gamma(\alpha)}x^{\alpha-1}e^{-x/\left(\frac{\beta}{1-\beta t}\right)}dx\)

All that is left is the \(\dfrac{1}{(\beta^*)^\alpha}\).

\(\frac{1}{(\beta^*)^\alpha}=\frac{1}{\left(\frac{\beta}{1-\beta t}\right)^\alpha}=\frac{\left(1-\beta t\right)^\alpha}{\beta^\alpha}\)

We have the denominator term already. Lets rewrite just for clarity.

\(M_X(t)=\int_0^\infty \frac{1}{\Gamma(\alpha)\beta^\alpha}x^{\alpha-1}e^{-x/\left(\frac{\beta}{1-\beta t}\right)}dx\)

Since \(\dfrac{1}{(\beta^*)^\alpha}=\dfrac{\left(1-\beta t\right)^\alpha}{\beta^\alpha}\), we need only to include the \((1-\beta t) ^\alpha\) term. If we include the term in the integral, we have to multiply by one. Therefore,

\( \begin{align*} & M_X(t)=\left(\frac{(1-\beta t)^\alpha}{(1-\beta t)^\alpha}\right)\int_0^\infty \frac{1}{\Gamma(\alpha)\beta^\alpha}x^{\alpha-1}e^{-x/\left(\frac{\beta}{1-\beta t}\right)}dx\\ & = \left(\frac{1}{(1-\beta t)^\alpha}\right)\int_0^\infty \frac{(1-\beta t)^\alpha}{\Gamma(\alpha)\beta^\alpha}x^{\alpha-1}e^{-x/\left(\frac{\beta}{1-\beta t}\right)}dx\\ & \left(\frac{1}{(1-\beta t)^\alpha}\right)\int_0^\infty \frac{1}{\Gamma(\alpha)\left(\frac{\beta}{1-\beta t}\right)^\alpha}x^{\alpha-1}e^{-x/\left(\frac{\beta}{1-\beta t}\right)}dx\\ & \left(\frac{1}{(1-\beta t)^\alpha}\right)\int_0^\infty h(x) dx \end{align*}\)

Therefore, \(h(x)\) is now a Gamma density function with parameters \(\alpha\) and \(\beta^*=\dfrac{\beta}{1-\beta t}\). And, since \(h(x)\) is a pdf and we are integrating over the whole space, \(x\ge 0\), then \(\int_0^\infty h(x)dx=1\). If the integral is equal to 1 based on the definition of a pdf, we are left with:

\(M_X(t)=\dfrac{1}{(1-\beta t)^\alpha}\left(1\right)=\dfrac{1}{(1-\beta t)^\alpha}\)

From the notes and the text, you can see that the moment generating function calculated above is exactly what we were supposed to get.

Just to summarize what we did here. We did not actually calculate the integral. We used algebra to manipulate the function to use the definition of a pdf. I find that after practice, this method is a lot quicker for me than doing the integrals.

It is the following known characteristics of the normal curve that directed me in drawing the curve as I did so above.

1. All normal curves are bell-shaped with points of inflection at \(\mu\pm \sigma\).

   Proof

   The proof is left for you as an exercise

2. All normal curves are symmetric about the mean \(\mu\).

   Proof

   All normal curves are symmetric about the mean \(\mu\), because \(f(\mu+x)=f(\mu-x)\) for all \(x\). That is:

   \(f(\mu+x)=\dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2} \left(\dfrac{x+\mu-\mu}{\sigma}\right)^2\right\} =\dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2} \left(\dfrac{x}{\sigma}\right)^2\right\}\)

   equals:

   \(f(\mu-x)=\dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2} \left(\dfrac{\mu-x-\mu}{\sigma}\right)^2\right\} =\dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2} \left(\dfrac{-x}{\sigma}\right)^2\right\}=\dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2} \left(\dfrac{x}{\sigma}\right)^2\right\}\)

   Therefore, by the definition of symmetry, the normal curve is symmetric about the mean \(\mu\).

3. The area under an entire normal curve is 1.

   Proof

   We prove this later on the Normal Properties page.

4. All normal curves are positive for all \(x\). That is, \(f(x)>0\) for all \(x\).

   Proof

   The standard deviation \(\sigma\) is defined to be positive. The square root of \(2\pi\) is positive. And, the natural exponential function is positive. When you multiply positive terms together, you, of course, get a positive number.

5. The limit of \(f(x)\) as \(x\) goes to infinity is 0, and the limit of \(f(x)\) as \(x\) goes to negative infinity is 0. That is:

   \(\lim\limits_{x\to \infty} f(x)=0\) and \(\lim\limits_{x\to -\infty} f(x)=0\)

   Proof

   The function \(f(x)\) depends on \(x\) only through the natural exponential function \(\exp[-x^2]\), which is known to approach 0 as \(x\) approaches infinity or negative infinity.

6. The height of any normal curve is maximized at \(x=\mu\).

   Proof

   Using what we know from our calculus studies, to find the point at which the maximum occurs, we must differentiate \(f(x)\) with respect to \(x\) and solve for \(x\) to find the maximum. Because our \(f(x)\) contains the natural exponential function, however, it is easier to take the derivative of the natural log of \(f(x)\) with respect to \(x\) and solve for \(x\) to find the maximum. [The maximum of \(f(x)\) is the same as the maximum of the natural log of \(f(x)\), because \(\log_e(x)\) is an increasing function of \(x\). That is, \(x_1<x_2\) implies that \(\log_e(x_1)<\log_e(x_2)\). Therefore, \(f(x_1)<f(x_2)\) implies \(\log_e(f(x_1))<\log_e(f(x_2))\).] That said, taking the natural log of \(f(x)\), we get:

   \(\text{log}_e (f(x))=\text{log}\left(\dfrac{1}{\sigma \sqrt{2\pi}} \right)-\dfrac{1}{2\sigma^2}(x-\mu)^2\)

   Taking the derivative of \(\log_e(f(x))\) with respect to \(x\), we get:

   \(\dfrac{d\text{log}f(x)}{dx}=-\dfrac{1}{2\sigma^2}\cdot 2(x-\mu)\)

   Now, setting the derivative of \(\log_e(f(x))\) to 0:

   \(\dfrac{d\text{log}f(x)}{dx}=-\dfrac{1}{2\sigma^2}\cdot 2(x-\mu) \stackrel{\equiv}{\scriptscriptstyle{SET}} 0\)

   and solving for \(x\), we get that \(x=\mu\). Taking the second derivative of \(\log_e(f(x))\) with respect to \(x\), we get:

   \(\dfrac{d^2\text{log}f(x)}{dx^2}=-\dfrac{1}{\sigma^2}\)

   Because the second derivative of \(\log_e(f(x))\) is negative (for all \(x\), in fact), the point \(x=\mu\) is deemed a local maximum.

7. The shape of any normal curve depends on its mean \(\mu\) and standard deviation \(\sigma\).

   Proof

   Given that the curve \(f(x)\) depends only on \(x\) and the two parameters \(\mu\) and \(\sigma\), the claimed characteristic is quite obvious. An example is perhaps more interesting than the proof. Here is a picture of three superimposed normal curves —one of a \(N(0,9)\) curve, one of a \(N(0, 16)\) curve, and one of a \(N(1, 9)\) curve:

   As claimed, the shapes of the three curves differ, as the means \(\mu\) and standard deviations \(\sigma\) differ.

Standard normal, or \(Z\)-tables, can take a number of different forms. There are two standard normal tables, Table Va and Table Vb, in the back of our textbook. Table Va gives the cumulative probabilities for \(Z\)-values, to two decimal places, between 0.00 and 3.09. Here's what the top of Table Va looks like:

\begin{aligned} P(Z \leq z)=& \Phi(z)=\int_{-\infty}^{z} \frac{1}{\sqrt{2 \pi}} e^{-w^{2} / 2} d w \\ \Phi(-z) &=1-\Phi(z) \end{aligned}

For example, you could use Table Va to find probabilities such as \(P(Z\le 0.01), P(Z\le 1.23)\), or \(P(Z\le 2.98)\). Table Vb, on the other hand, gives probabilities in the upper tail of the standard normal distribution. Here's what the top of Table Vb looks like:

\(P(Z > z_\alpha) = \alpha \)

\(P(Z > z) = 1 - \Phi(z) = \Phi(-z)\)

That is, for \(Z\)-values, to two decimal places, between 0.00 and 3.49, we can use Table Vb to find probabilities such as \(P(Z>0.12), P(Z>1.96)\), and \(P(Z>3.32)\).

Now, we just need to learn how to read the probabilities off of each of the tables. First Table Va:

And, then Table Vb:

Now we know how to read the given \(Z\)-tables. Now, we just need to work with some real examples to see that finding probabilities associated with a normal random variable usually involves rewriting the problems just a bit in order to get them to "fit" with the available \(Z\)-tables.

On the last page, we learned how to use the standard normal curve N(0, 1) to find probabilities concerning a normal random variable X with mean \(\mu\) and standard deviation \(\sigma\). What happens if it's not the probability that we want to find, but rather the value of X? That's what we'll investigate on this page. That is, we'll consider what I like to call "inside-out" problems, in which we use known probabilities to find the value of the normal random variable X. Let's start with an example.

So far, all of our attention has been focused on learning how to use the normal distribution to answer some practical problems. We'll turn our attention for a bit to some of the theoretical properties of the normal distribution. We'll start by verifying that the normal p.d.f. is indeed a valid probability distribution. Then, we'll derive the moment-generating function \(M(t)\) of a normal random variable \(X\). We'll conclude by using the moment generating function to prove that the mean and standard deviation of a normal random variable \(X\) are indeed, respectively, \(\mu\) and \(\sigma\), something that we thus far have assumed without proof.

Recall that the probability density function of a normal random variable is:

\(f(x)=\dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2} \left(\dfrac{x-\mu}{\sigma}\right)^2\right\}\)

for \(-\infty<x<\infty\), \(-\infty<\mu<\infty\), and \(0<\sigma<\infty\). Also recall that in order to show that the normal p.d.f. is a valid p.d.f, we need to show that, firstly \(f(x)\) is always positive, and, secondly, if we integrate \(f(x)\) over the entire support, we get 1.

Note that the transformation from \(X\) to \(Z\):

\(Z=\dfrac{X-\mu}{\sigma}\)

tells us the number of standard deviations above or below the mean that \(X\) falls. That is, if \(Z=-2\), then we know that \(X\) falls 2 standard deviations below the mean. And if \(Z=+2\), then we know that \(X\) falls 2 standard deviations above the mean. As such, \(Z\)-scores are sometimes used in medical fields to identify whether an individual exhibits extreme values with respect to some biological or physical measurement.

You might recall earlier in this section, when we investigated exploring continuous data, that we learned about the Empirical Rule. Specifically, we learned that if a histogram is at least approximately bell-shaped, then:

1. approximately 68% of the data fall within one standard deviation of the mean
2. approximately 95% of the data fall within two standard deviations of the mean
3. approximately 99.7% of the data fall within three standard deviations of the mean

Where did those numbers come from? Now, that we've got the normal distribution under our belt, we can see why the Empirical Rule holds true. The probability that a randomly selected data value from a normal distribution falls within one standard deviation of the mean is

\(P(-1<Z<1)=P(Z<1)-P(Z>1)=0.8413-0.1587=0.6826\)

That is, we should expect 68.26% (approximately 68%!) of the data values arising from a normal population to be within one standard deviation of the mean, that is, to fall in the interval:

\((\mu-\sigma, \mu+\sigma)\)

The probability that a randomly selected data value from a normal distribution falls within two standard deviations of the mean is

\(P(-2<Z<2)=P(Z<2)-P(Z>2)=0.9772-0.0228=0.9544\)

That is, we should expect 95.44% (approximately 95%!) of the data values arising from a normal population to be within two standard deviations of the mean, that is, to fall in the interval:

\((\mu-2\sigma, \mu+2\sigma)\)

And, the probability that a randomly selected data value from a normal distribution falls within three standard deviations of the mean is:

\(P(-3<Z<3)=P(Z<3)-P(Z>3)=0.9987-0.0013=0.9974\)

That is, we should expect 99.74% (almost all!) of the data values arising from a normal population to be within three standard deviations of the mean, that is, to fall in the interval:

\((\mu-3\sigma, \mu+3\sigma)\)

Let's take a look at an example of the Empirical Rule in action.