# 16.4 - Normal Properties

16.4 - Normal Properties

So far, all of our attention has been focused on learning how to use the normal distribution to answer some practical problems. We'll turn our attention for a bit to some of the theoretical properties of the normal distribution. We'll start by verifying that the normal p.d.f. is indeed a valid probability distribution. Then, we'll derive the moment-generating function $$M(t)$$ of a normal random variable $$X$$. We'll conclude by using the moment generating function to prove that the mean and standard deviation of a normal random variable $$X$$ are indeed, respectively, $$\mu$$ and $$\sigma$$, something that we thus far have assumed without proof.

## The Normal P.D.F. is Valid

Recall that the probability density function of a normal random variable is:

$$f(x)=\dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2} \left(\dfrac{x-\mu}{\sigma}\right)^2\right\}$$

for $$-\infty<x<\infty$$, $$-\infty<\mu<\infty$$, and $$0<\sigma<\infty$$. Also recall that in order to show that the normal p.d.f. is a valid p.d.f, we need to show that, firstly $$f(x)$$ is always positive, and, secondly, if we integrate $$f(x)$$ over the entire support, we get 1.

Proof

Let's start with the easy part first, namely, showing that $$f(x)$$ is always positive. The standard deviation $$\sigma$$ is defined to be positive. The square root of $$2\pi$$ is positive. And, the natural exponential function is positive. When you multiply positive terms together, you, of course, get a positive number. Check... the first part is done.

Now, for the second part. Showing that $$f(x)$$ integrates to 1 is a bit messy, so bear with me here. Let's define $$I$$ to be the integral that we are trying to find. That is:

$$I=\int_{-\infty}^\infty \dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2\sigma^2} (x-\mu)^2\right\}dx$$

Our goal is to show that $$I=1$$. Now, if we change variables with:

$$w=\dfrac{x-\mu}{\sigma}$$

our integral $$I$$ becomes:

$$I=\int_{-\infty}^\infty \dfrac{1}{\sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2} w^2\right\}dw$$

Now, squaring both sides, we get:

$$I^2=\left(\int_{-\infty}^\infty \dfrac{1}{\sqrt{2\pi}} \text{exp}\left\{-\dfrac{x^2}{2} \right\}dx\right) \left(\int_{-\infty}^\infty \dfrac{1}{\sqrt{2\pi}} \text{exp}\left\{-\dfrac{y^2}{2} \right\}dy\right)$$

And, pulling the integrals together, we get:

$$I^2=\dfrac{1}{2\pi}\int_{-\infty}^\infty \int_{-\infty}^\infty \text{exp}\left\{-\dfrac{x^2}{2} \right\} \text{exp}\left\{-\dfrac{y^2}{2} \right\}dxdy$$

Now, combining the exponents, we get:

$$I^2=\dfrac{1}{2\pi}\int_{-\infty}^\infty \int_{-\infty}^\infty \text{exp}\left\{-\dfrac{1}{2}(x^2+y^2) \right\} dxdy$$

Converting to polar coordinates with:

$$x=r\cos\theta$$ and $$y=r\sin\theta$$

we get:

$$I^2=\dfrac{1}{2\pi}\int_0^{2\pi}\left(\int_0^\infty \text{exp}\left\{-\dfrac{r^2}{2} \right\} rdr\right)d\theta$$

Now, if we do yet another change of variables with:

$$u=\dfrac{r^2}{2}$$ and $$du=rdr$$

our integral $$I$$ becomes:

$$I^2=\dfrac{1}{2\pi}\int_0^{2\pi}\left(\int_0^\infty e^{-u}du\right)d\theta$$

Evaluating the inside integral, we get:

$$I^2=\dfrac{1}{2\pi}\int_0^{2\pi}\left\{-\lim\limits_{b\to \infty} [e^{-u}]^{u=b}_{u=0}\right\}d\theta$$

And, finally, completing the integration, we get:

$$I^2=\dfrac{1}{2\pi} \int_0^{2\pi} -(0-1) d \theta= \dfrac{1}{2\pi}\int_0^{2\pi} d \theta =\dfrac{1}{2\pi} (2\pi)=1$$

Okay, so we've shown that $$I^2=1$$. Therefore, that means that $$I=+1$$ or $$I=-1$$. But, we know that $$I$$ must be positive, since $$f(x)>0$$. Therefore, $$I$$ must equal 1. Our proof is complete. Finally.

## The Moment Generating Function

### Theorem

The moment generating function of a normal random variable $$X$$ is:

$$M(t)=\text{exp}\left\{\mu t+\dfrac{\sigma^2 t^2}{2}\right\}$$

### Proof

Well, I better start this proof out by saying this one is a bit messy, too. Jumping right into it, using the definition of a moment-generating function, we get:

$$M(t)=E(e^{tX})=\int_{-\infty}^\infty e^{tx}f(x)dx=\int_{-\infty}^\infty e^{tx}\left[\dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2\sigma^2} (x-\mu)^2\right\} \right]dx$$

Simply expanding the term in the second exponent, we get:

$$M(t)=\int_{-\infty}^\infty \dfrac{1}{\sigma \sqrt{2\pi}}\text{exp}\{tx\} \text{exp}\left\{-\dfrac{1}{2\sigma^2} (x^2-2x\mu+\mu^2)\right\} dx$$

And, combining the two exponents, we get:

$$M(t)=\int_{-\infty}^\infty \dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2\sigma^2} (x^2-2x\mu+\mu^2)+tx \right\} dx$$

Pulling the $$tx$$ term into the parentheses in the exponent, we get:

$$M(t)=\int_{-\infty}^\infty \dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2\sigma^2} (x^2-2x\mu-2\sigma^2tx+\mu^2) \right\} dx$$

And, simplifying just a bit more in the exponent, we get:

$$M(t)=\int_{-\infty}^\infty \dfrac{1}{\sigma \sqrt{2\pi}} \text{exp}\left\{-\dfrac{1}{2\sigma^2} (x^2-2x(\mu+\sigma^2 t)+\mu^2) \right\} dx$$

And, simplifying just a bit more in the exponent, we get:

Now, let's take a little bit of an aside by focusing our attention on just this part of the exponent:

$$(x^2-2(\mu+\sigma^2t)x+\mu^2)$$

If we let:

$$a=\mu+\sigma^2t$$ and $$b=\mu^2$$

then that part of our exponent becomes:

$$x^2-2(\mu+\sigma^2t)x+\mu^2=x^2-2ax+b$$

Now, complete the square by effectively adding 0:

$$x^2-2(\mu+\sigma^2t)x+\mu^2=x^2-2ax+a^2-a^2+b$$

And, simplifying, we get:

$$x^2-2(\mu+\sigma^2t)x+\mu^2=(x-a)^2-a^2+b$$

Now, inserting in the values we defined for $$a$$ and $$b$$, we get:

$$x^2-2(\mu+\sigma^2t)x+\mu^2=(x-(\mu+\sigma^2t))^2-(\mu+\sigma^2t)^2+\mu^2$$

Okay, now stick our modified exponent back into where we left off in our calculation of the moment-generating function:

$$M(t)=\int_{-\infty}^\infty \dfrac{1}{\sigma \sqrt{2\pi}}\text{exp}\left\{-\dfrac{1}{2\sigma^2}\left[(x-(\mu+\sigma^2t))^2-(\mu+\sigma^2t)^2+\mu^2\right]\right\}dx$$

We can now pull the part of the exponent that doesn't depend on $$x$$ through the integral getting:

$$M(t)=\text{exp}\left\{-\dfrac{1}{2\sigma^2}\left[-(\mu+\sigma^2t)^2+\mu^2\right]\right\} \int_{-\infty}^\infty \dfrac{1}{\sigma \sqrt{2\pi}}\text{exp}\left\{-\dfrac{1}{2\sigma^2}\left[(x-(\mu+\sigma^2t))^2 \right]\right\}dx$$

Now, we should recognize that the integral integrates to 1 because it is the integral over the entire support of the p.d.f. of a normal random variable $$X$$ with:

mean $$\mu+\sigma^2t$$ and variance $$\sigma^2$$

That is, because the integral is 1: our moment-generating function reduces to this:

$$M(t)=\text{exp}\left\{-\dfrac{1}{2\sigma^2}\left[-\mu^2-2\mu\sigma^2t-\sigma^4t^2+\mu^2\right]\right\}$$

Now, it's just a matter of simplifying:

$$M(t)=\text{exp}\left\{\dfrac{2\mu\sigma^2t+\sigma^4t^2}{2\sigma^2}\right\}$$

and simplifying a bit more:

$$M(t)=\text{exp}\left\{\mu t +\dfrac{\sigma^2t^2}{2}\right\}$$

Our second messy proof is complete!

## The Mean and Variance

### Theorem

The mean and variance of a normal random variable $$X$$ are, respectively, $$\mu$$ and $$\sigma^2$$.

### Proof

We'll use the moment generating function:

$$M(t)=\text{exp}\left\{\mu t +\dfrac{\sigma^2t^2}{2}\right\}$$

to find the mean and variance. Recall that finding the mean involves evaluating the derivative of the moment-generating function with respect to $$t$$ at $$t=0$$:

So, we just found that the first derivative of the moment-generating function with respect to $$t$$ is:

$$M'(t)=\text{exp}\left(\mu t +\dfrac{\sigma^2t^2}{2}\right)\times (\mu+\sigma^2t)$$

We'll use it to help us find the variance:

  Link ↥ Has Tooltip/Popover Toggleable Visibility