15.4 - Gamma Distributions

The Situation Section

In the previous lesson, we learned that in an approximate Poisson process with mean \(\lambda\), the waiting time \(X\) until the first event occurs follows an exponential distribution with mean \(\theta=\frac{1}{\lambda}\). We now let \(W\) denote the waiting time until the \(\alpha^{th}\) event occurs and find the distribution of \(W\). We could represent the situation as follows:

0 event 1 αth x x x x x x w

Derivation of the Probability Density Function Section

Just as we did in our work with deriving the exponential distribution, our strategy here is going to be to first find the cumulative distribution function \(F(w)\) and then differentiate it to get the probability density function \(f(w)\). Now, for \(w>0\) and \(\lambda>0\), the definition of the cumulative distribution function gives us:

\(F(w)=P(W\le w)\)

The rule of complementary events tells us then that:

\(F(w)=1-P(W> w)\)

Now, the waiting time \(W\) is greater than some value \(w\) only if there are fewer than \(\alpha\) events in the interval \([0,w]\). That is:

\(F(w)=1-P(\text{fewer than }\alpha\text{ events in } [0,w]) \)

A more specific way of writing that is:

\(F(w)=1-P(\text{0 events or 1 event or ... or }(\alpha-1)\text{ events in } [0,w]) \)

Those mutually exclusive "ors" mean that we need to add up the probabilities of having 0 events occurring in the interval \([0,w]\), 1 event occurring in the interval \([0,w]\), ..., up to \((\alpha-1)\) events in \([0,w]\). Well, that just involves using the probability mass function of a Poisson random variable with mean \(\lambda w\). That is:

\(F(w)=1-\sum\limits_{k=0}^{\alpha-1} \dfrac{(\lambda w)^k e^{-\lambda w}}{k!}\)

Now, we could leave \(F(w)\) well enough alone and begin the process of differentiating it, but it turns out that the differentiation goes much smoother if we rewrite \(F(w)\) as follows:

\(F(w)=1-e^{-\lambda w}-\sum\limits_{k=1}^{\alpha-1} \dfrac{1}{k!} \left[(\lambda w)^k e^{-\lambda w}\right]\)

As you can see, we merely pulled the \(k=0\) out of the summation and rewrote the probability mass function so that it would be easier to administer the product rule for differentiation.

Now, let's do that differentiation! We need to differentiate \(F(w)\) with respect to \(w\) to get the probability density function \(f(w)\). Using the product rule, and what we know about the derivative of \(e^{\lambda w}\) and \((\lambda w)^k\), we get:

\(f(w)=F'(w)=\lambda e^{-\lambda w} -\sum\limits_{k=1}^{\alpha-1} \dfrac{1}{k!} \left[(\lambda w)^k \cdot (-\lambda e^{-\lambda w})+ e^{-\lambda w} \cdot k(\lambda w)^{k-1} \cdot \lambda \right]\)

Pulling \(\lambda e^{-\lambda w}\) out of the summation, and dividing \(k\) by \(k!\) (to get \( \frac{1}{(k-1)!}\)) in the second term in the summation, we get that \(f(w)\) equals:

\(=\lambda e^{-\lambda w}+\lambda e^{-\lambda w}\left[\sum\limits_{k=1}^{\alpha-1} \left\{ \dfrac{(\lambda w)^k}{k!}-\dfrac{(\lambda w)^{k-1}}{(k-1)!} \right\}\right]\)

Evaluating the terms in the summation at \(k=1, k=2\), up to \(k=\alpha-1\), we get that \(f(w)\) equals:

\(=\lambda e^{-\lambda w}+\lambda e^{-\lambda w}\left[(\lambda w-1)+\left(\dfrac{(\lambda w)^2}{2!}-\lambda w\right)+\cdots+\left(\dfrac{(\lambda w)^{\alpha-1}}{(\alpha-1)!}-\dfrac{(\lambda w)^{\alpha-2}}{(\alpha-2)!}\right)\right]\)

Do some (lots of!) crossing out (\(\lambda w -\lambda w =0\), for example), and a bit more simplifying to get that \(f(w)\) equals:

\(=\lambda e^{-\lambda w}+\lambda e^{-\lambda w}\left[-1+\dfrac{(\lambda w)^{\alpha-1}}{(\alpha-1)!}\right]=\lambda e^{-\lambda w}-\lambda e^{-\lambda w}+\dfrac{\lambda e^{-\lambda w} (\lambda w)^{\alpha-1}}{(\alpha-1)!}\)

And since \(\lambda e^{-\lambda w}=\lambda e^{-\lambda w}=0\), we get that \(f(w)\) equals:

\(=\dfrac{\lambda e^{-\lambda w} (\lambda w)^{\alpha-1}}{(\alpha-1)!}\)

Are we there yet? Almost! We just need to reparameterize (if \(\theta=\frac{1}{\lambda}\), then \(\lambda=\frac{1}{\theta}\)). Doing so, we get that the probability density function of \(W\), the waiting time until the \(\alpha^{th}\) event occurs, is:

\(f(w)=\dfrac{1}{(\alpha-1)! \theta^\alpha} e^{-w/\theta} w^{\alpha-1}\)

for \(w>0, \theta>0\), and \(\alpha>0\).

NOTE! that, as usual, there are an infinite number of possible gamma distributions because there are an infinite number of possible \(\theta\) and \(\alpha\) values. That's, again, why this page is called Gamma Distributions (with an s) and not Gamma Distribution (with no s). Because each gamma distribution depends on the value of \(\theta\) and \(\alpha\), it shouldn't be surprising that the shape of the probability distribution changes as \(\theta\) and \(\alpha\) change.

Effect of \(\theta\) and \(\alpha\) on the Distribution Section

Recall that \(\theta\) is the mean waiting time until the first event, and \(\alpha\) is the number of events for which you are waiting to occur. It makes sense then that for fixed \(\alpha\), as \(\theta\) increases, the probability "moves to the right," as illustrated here with \(\alpha\)fixed at 3, and \(\theta\) increasing from 1 to 2 to 3:

050.30.20.10.025101520xDensityAlpha = 3Theta = 1Theta = 2Theta = 3

The plots illustrate, for example, that if we are waiting for \(\alpha=3\) events to occur, we have a greater probability of our waiting time \(X\) being large if our mean waiting time until the first event is large (\(\theta=3\), say) than if it is small (\(\theta=1\), say).

It also makes sense that for fixed \(\theta\), as \(\alpha\) increases, the probability "moves to the right," as illustrated here with \(\theta\)fixed at 3, and \(\alpha\) increasing from 1 to 2 to 3:

050.30.20.10.025101520xDensityTheta = 3Alpha = 1Alpha = 2Alpha = 3

The plots illustrate, for example, that if the mean waiting time until the first event is \(\theta=3\), then we have a greater probability of our waiting time \(X\) being large if we are waiting for more events to occur (\(\alpha=3\), say) than fewer (\(\alpha=1\), say).