11  Geometric and Negative Binomial Distributions

Geometric Distributions
Negative Binomial

Overview

In this lesson, we learn about two more specially named discrete probability distributions, namely the negative binomial distribution and the geometric distribution.

Objectives

Upon completion of this lesson, you should be able to:

  1. understand the derivation of the formula for the geometric probability mass function.
  2. explore the key properties, such as the mean and variance, of a geometric random variable.
  3. calculate probabilities for a geometric random variable.
  4. explore the key properties, such as the moment-generating function, mean and variance, of a negative binomial random variable.
  5. calculate probabilities for a negative binomial random variable.
  6. understand the steps involved in each of the proofs in the lesson.
  7. apply the methods learned in the lesson to new problems.

11.1 Geometric Distributions

Example 11.1 football game

A representative from the National Football League’s Marketing Division randomly selects people on a random street in Kansas City, Missouri until he finds a person who attended the last home football game. Let \(p\), the probability that he succeeds in finding such a person, equal 0.20. And, let \(X\) denote the number of people he selects until he finds his first success. What is the probability mass function of \(X\)?

Solution

Video 11.1: Example: The geometric distribution, Introduction

Def. 11.1 (Geometric Distribution) Assume Bernoulli trials — that is, (1) there are two possible outcomes, (2) the trials are independent, and (3) \(p\), the probability of success, remains the same from trial to trial. Let \(X\) denote the number of trials until the first success. Then, the probability mass function of \(X\) is:

\[ f(x)=P(X=x)=(1-p)^{x-1}p \]

for \(x=1, 2, \ldots\) In this case, we say that \(X\) follows a geometric distribution.

Note that there are (theoretically) an infinite number of geometric distributions. Any specific geometric distribution depends on the value of the parameter \(p\).

11.2 Key Properties of a Geometric Random Variable

On this page, we state and then prove four properties of a geometric random variable. In order to prove the properties, we need to recall the sum of the geometric series. So, we may as well get that out of the way first.

Recall

  1. The sum of a geometric series is:

\[ g(r)=\sum\limits_{k=0}^\infty ar^k=a+ar+ar^2+ar^3+\cdots=\dfrac{a}{1-r}=a(1-r)^{-1} \]

  1. Then, taking the derivatives of both sides, the first derivative with respect to \(r\) must be:

\[ g'(r)=\sum\limits_{k=1}^\infty akr^{k-1}=0+a+2ar+3ar^2+\cdots=\dfrac{a}{(1-r)^2}=a(1-r)^{-2} \]

  1. And, taking the derivatives of both sides again, the second derivative with respect to \(r\) must be:

\[ g''(r)=\sum\limits_{k=2}^\infty ak(k-1)r^{k-2}=0+0+2a+6ar+\cdots=\dfrac{2a}{(1-r)^3}=2a(1-r)^{-3} \]

We’ll use the sum of the geometric series, first point, in proving the first two of the following four properties. And, we’ll use the first derivative, second point, in proving the third property, and the second derivative, third point, in proving the fourth property. Let’s jump right in now!

NoteTheorem

The probability mass function:

\[ f(x)=P(X=x)=(1-p)^{x-1}p \]

\(0<p<1\), \(x=1, 2, \ldots\) for a geometric random variable \(X\) is a valid PMF.

Video 11.2: Proof: Is it a valid PMF?
NoteTheorem

The cumulative distribution function of a geometric random variable \(X\) is:

\[ F(x)=P(X\leq x)=1-(1-p)^x \]

Video 11.3: Proof: The CDF of a geometric random variable X
NoteTheorem

The mean of a geometric random variable \(X\) is:

\[ \mu=E(X)=\dfrac{1}{p} \]

Video 11.4: Proof: The mean of a geometric random variable X
NoteTheorem

The variance of a geometric random variable \(X\) is:

\[ \sigma^2=\mathrm{Var}(X)=\dfrac{1-p}{p^2} \]

Proof

To find the variance, we are going to use that trick of “adding zero” to the shortcut formula for the variance. Recall that the shortcut formula is:

\[ \sigma^2=\mathrm{Var}(X)=E(X^2)-[E(X)]^2 \]

We “add zero” by adding and subtracting \(E(X)\) to get:

\[ \sigma^2=E(X^2)-E(X)+E(X)-[E(X)]^2=E[X(X-1)]+E(X)-[E(X)]^2 \]

Then, here’s how the rest of the proof goes:

Video 11.5: Proof: Finding the variance of a geometric random variable X

11.3 Geometric Examples

Example 11.2

A representative from the National Football League’s Marketing Division randomly selects people on a random street in Kansas City, Kansas until he finds a person who attended the last home football game. Let \(p\), the probability that he succeeds in finding such a person, equal 0.20. And, let \(X\) denote the number of people he selects until he finds his first success. What is the probability that the marketing representative must select 4 people before he finds one who attended the last home football game?

Solution

To find the desired probability, we need to find \(P(X=4)\), which can be determined readily using the PMF of a geometric random variable with \(p=0.20\), \(1-p=0.80\), and \(x=4\):

\[ P(X=4)=0.80^3 \times 0.20=0.1024 \]

There is about a 10% chance that the marketing representative would have to select 4 people before he would find one who attended the last home football game.

What is the probability that the marketing representative must select more than 6 people before he finds one who attended the last home football game?

Solution

To find the desired probability, we need to find \(P(X>6)=1-P(X\le6)\), which can be determined readily using the c.d.f. of a geometric random variable with \(1-p=0.80\), and \(x=6\):

\[ P(X >6)=1-P(X \leq 6)=1-[1-0.8^6]=0.8^6=0.262 \]

There is about a 26% chance that the marketing representative would have to select more than 6 people before he would find one who attended the last home football game.

How many people should we expect (that is, what is the average number) the marketing representative needs to select before he finds one who attended the last home football game? And, while we’re at it, what is the variance?

Solution

The average number is:

\[ \mu=E(X)=\dfrac{1}{p}=\dfrac{1}{0.20}=5 \]

That is, we should expect the marketing representative to have to select 5 people before he finds one who attended the last football game. Of course, on any given try, it may take 1 person or it may take 10, but 5 is the average number. The variance is 20, as determined by:

\[ \sigma^2=\mathrm{Var}(X)=\dfrac{1-p}{p^2}=\dfrac{0.80}{0.20^2}=20 \]

11.4 Negative Binomial Distributions

Example 11.3

(Are you growing weary of this example yet?) A representative from the National Football League’s Marketing Division randomly selects people on a random street in Kansas City, Kansas until he finds a person who attended the last home football game. Let \(p\), the probability that he succeeds in finding such a person, equal 0.20. Now, let \(X\) denote the number of people he selects until he finds \(r=3\) who attended the last home football game. What is the probability that \(X=10\)?

Solution

Video 11.6: Example: What is the probability that X = 10?

Def. 11.2 (Negative Binomial Distribution) Assume Bernoulli trials — that is, (1) there are two possible outcomes, (2) the trials are independent, and (3) \(p\), the probability of success, remains the same from trial to trial. Let \(X\) denote the number of trials until the \(r^{th}\) success. Then, the probability mass function of \(X\) is:

\[ f(x)=P(X=x)=\dbinom{x-1}{r-1} (1-p)^{x-r} p^r \]

for \(x=r, r+1, r+2, \ldots\). In this case, we say that \(X\) follows a negative binomial distribution.

Note!

  1. There are (theoretically) an infinite number of negative binomial distributions. Any specific negative binomial distribution depends on the value of the parameter \(p\).
  2. A geometric distribution is a special case of a negative binomial distribution with \(r=1\).

11.5 Key Properties of a Negative Binomial Random Variable

NoteTheorem

Just as we did for a geometric random variable, on this page, we present and verify four properties of a negative binomial random variable.

The probability mass function:

\[ f(x)=P(X=x)=\dbinom{x-1}{r-1} (1-p)^{x-r} p^r \]

for a negative binomial random variable \(X\) is a valid PMF.

Proof

Before we start the “official” proof, it is helpful to take note of the sum of a negative binomial series:

\[ (1-w)^{-r}=\sum\limits_{k=0}^\infty \dbinom{k+r-1}{r-1} w^k \]

Now, for the proof:

Video 11.7: Proof: The PMF of a negative binomial random variable X
NoteTheorem

The moment generating function of a negative binomial random variable \(X\) is:

\[ M(t)=E(e^{tX})=\dfrac{(pe^t)^r}{[1-(1-p)e^t]^r} \]

for \((1-p)e^t<1\).

Proof

As always, the moment generating function is defined as the expected value of \(e^{tX}\). In the case of a negative binomial random variable, the MGF is then:

\[ M(t)=E(e^{tX})=\sum\limits_{x=r}^\infty e^{tx} \dbinom{x-1}{r-1} (1-p)^{x-r} p^r \]

Now, it’s just a matter of massaging the summation in order to get a working formula. We start by effectively multiplying the summands by 1, and thereby not changing the overall sum:

\[ M(t)=E(e^{tX})=\sum\limits_{x=r}^\infty e^{tx} \dbinom{x-1}{r-1} (1-p)^{x-r} p^r \times \dfrac{(e^t)^r}{(e^t)^r} \]

Now, since \(p^r\) and \((e^t)^r\) do not depend on \(x\), they can be pulled through the summation. And, since the \((e^t)^r\) that remains sits in the denominator, it can get moved into the numerator by writing is as \((e^t)^{-r}\):

\[ M(t)=E(e^{tX})=p^r(e^t)^r \sum\limits_{x=r}^\infty e^{tx} \dbinom{x-1}{r-1} (1-p)^{x-r} (e^t)^{-r} \]

Now, the \(p^r\) and \((e^t)^r\) can be pulled together as \((pe^t)^r\). And, \(e^{tx}\) and \((e^t)^r\) can be pulled together to get \((e^t)^{x-r}\):

\[ M(t)=E(e^{tX})=(pe^t)^r \sum\limits_{x=r}^\infty \dbinom{x-1}{r-1} (1-p)^{x-r} (e^t)^{x-r} \]

And, \((1-p)^{x-r}\) and \((e^t)^{x-r}\) can be pulled together to get \([(1-p)e^t]^{x-r}\):

\[ M(t)=E(e^{tX})=(pe^t)^r \sum\limits_{x=r}^\infty \dbinom{x-1}{r-1} [(1-p)e^t]^{x-r} \]

Now, let \(k=x-r\), so that \(x=k+r\). Changing the index on the summation, we get:

\[ M(t)=E(e^{tX})=(pe^t)^r \sum\limits_{k=0}^\infty \dbinom{k+r-1}{r-1}[(1-p)e^t]^k \]

Now, we should be able to recognize the summation as a negative binomial series with \(w=(1-p)e^t\). Using what we know about the sum of a negative binomial series, the MGF is then:

\[ M(t)=E(e^{tX})=(pe^t)^r [1-(1-p)e^t]^{-r} \]

which can be rewritten as:

\[ M(t)=E(e^{tX})=\dfrac{(pe^t)^r}{[1-(1-p)e^t]^r} \]

Now, recall that the MGF exists only if it is finite. So, all we need to do is note when \(M(t)\) is finite. Well, that happens when \((1-p)e^t<1\), or equivalently when \(t<-\ln (1-p)\). And the proof is complete…whewwww!

NoteTheorem

The mean of a negative binomial random variable \(X\) is:

\[ \mu=E(X)=\dfrac{r}{p} \]

Video 11.8: Proof: The mean of a negative binomial random variable X
NoteTheorem

The variance of a negative binomial random variable \(X\) is:

\[ \sigma^2=\mathrm{Var}(x)=\dfrac{r(1-p)}{p^2} \]

Since we used the MGF to find the mean, let’s use it to find the variance as well. That is, let’s use:

\[ \sigma^2=M''(0)-[M'(0)]^2 \]

The only problem is that finding the second derivative of \(M(t)\) is even messier than the first derivative of \(M(t)\). Let me cheat a bit then. Let me leave it to you to verify that the second derivative of the MGF of the negative binomial is:

\[ M''(t)=r(pe^t)^r(-r-1)[1-(1-p)e^t]^{-r-2}[-(1-p)e^t]+r^2(pe^t)^{r-1}(pe^t)[1-(1-p)e^t]^{-r-1} \]

Now, with my shortcut taken, let’s use it to evaluate the second derivative of the MGF at \(t=0\):

Video 11.9: Proof: Finding the variance, part 1

Now, for the final calculation:

Video 11.10: Proof: Finding the variance, part 2

11.6 Negative Binomial Examples

Example 11.4

An oil company conducts a geological study that indicates that an exploratory oil well should have a 20% chance of striking oil. What is the probability that the first strike comes on the third well drilled?

Solution

To find the requested probability, we need to find \(P(X=3)\). Note that \(X\) is technically a geometric random variable, since we are only looking for one success. Since a geometric random variable is just a special case of a negative binomial random variable, we’ll try finding the probability using the negative binomial PMF In this case, \(p=0.20, 1-p=0.80, r=1, x=3\), and here’s what the calculation looks like:

\[ \begin{align*} P(X=3)&=\dbinom{3-1}{1-1}(1-p)^{3-1}p^1 \\ &=(1-p)^2 p \\ &=0.80^2\times 0.20 \\ &=0.128 \end{align*} \]

It is at the second equal sign that you can see how the general negative binomial problem reduces to a geometric random variable problem. In any case, there is about a 13% chance thathe first strike comes on the third well drilled.

What is the probability that the third strike comes on the seventh well drilled?

Solution

To find the requested probability, we need to find \(P(X=7)\), which can be readily found using the PMF of a negative binomial random variable with \(p=0.20, 1-p=0.80, x=7, r=3\):

\[ \begin{align*} P(X=7)&=\dbinom{7-1}{3-1}(1-p)^{7-3}p^3 \\ &=\dbinom{6}{2}0.80^4\times 0.20^3 \\ &=0.049 \end{align*} \]

That is, there is about a 5% chance that the third strike comes on the seventh well drilled.

What is the mean and variance of the number of wells that must be drilled if the oil company wants to set up three producing wells?

Solution

The mean number of wells is:

\[ \mu=E(X)=\dfrac{r}{p}=\dfrac{3}{0.20}=15 \]

with a variance of:

\[ \sigma^2=\mathrm{Var}(x)=\dfrac{r(1-p)}{p^2}=\dfrac{3(0.80)}{0.20^2}=60 \]