Variance and standard deviation are measures of variability. The most common way we measure variability is by using the standard deviation.  However, when working with standard deviations we must first make sure that our data are normally distributed (otherwise we need to modify the way we look at distributions).  We most often think of distributions being "normal" when they look like the classical "bell curve" of quantitative data.  However it is very important for you to understand the idea of normality will apply to categorical data as well, although normality will be assessed in slightly different ways.  

But let's first take a quick look at standard deviations and variability.  We will take a look at how we calculate these by hand.  Subsequently we will discuss the assessment of normality for both quantitative and categorical data. 

The standard deviation is actually the square root of the variance. But wait a minute.. what is the variance? 

Don't get discouraged, we will walk through these calculations for you, and Susie, computing these values by hand. After this lesson, you will always be computing standard deviation using software such as Minitab Express.

Let's start step by step: 

1. Step 1: Compute the sample mean (which we have already done):
2. \(\overline{x} = \dfrac{\sum x}{n}\)
3. Step 2:Calculate the deviance (or how far away each individual observation is away from the mean) by subtracting the sample mean from each individual value:
4. \(x-\overline{x}\), these are the deviations
5. Step 3:For each deviation, multiply it by itself (in other words, you are squaring the deviation):
6. \((x-\overline{x})^{2}\), these are the squared deviations
7. NOTE: This is an important step.  If you are curious you can skip step 3 and jump right to step 4.  What you will find is if you add up all of the deviation scores from step 2, the sum will always be equal to zero!  If you like algebra you can play with the formula for the mean and the total deviation scores and see that they are equivalent!
8. Step 4: Find the sum of squares by summing of all of the squared deviations from Step 3:
9. \(\sum (x-\overline{x})^{2}\), this is the sum of squares
10. Step 5: Divide the sum of squares by \(n-1\) (where n is the total number of observations):
11. \(\dfrac{\sum (x-\overline{x})^{2}}{n-1}\), this is the sample variance \((s^{2})\)
12. Viola!  You have calculated the variance.  That was a lot of math.  Fortunately, you never have to do this by hand (but it is very interesting to see that variance is a simple matter of subtracting, multiplying, adding, and dividing!
13. But this still hasn't got us to the standard deviation.  So we need to add one last step.  
14. Step 6: Take the square root of the sample variance:
15. \(\sqrt{\frac{\sum (x-\overline{x})^{2}}{n-1}}\), this is the sample standard deviation
16. Viola!  The standard deviation! Just as the definition indicates, this is simply the square root of the variance
17. NOTE: The reason this is the square root is because of Step 3 above.  Because of the math involved we had to square the deviation scores.  This results in the magnitude of the variance always being much larger than the original units of measurements in our data.  By taking the square root of the variance, the variability of the data can be expressed in the original measurement units by using the standard deviation.

So to keep track of some of the vocabulary introduced:

While the standard deviation gives us a measure of variability (I.e. a variable with a standard deviation of zero has no variability), there are other important ways we can think about the standard deviation.

First, we start with a normal distribution, symmetrical and bell-shaped. 

The Empirical Rule is a rule telling us about where an observation lies in a normal distribution. The Empirical Rule states that approximately 68% of data will be within one standard deviation of the mean, about 95% will be within two standard deviations of the mean, and about 99.7% will be within three standard deviations of the mean

While the Empirical Rule is a great tool for helping us understand the variability of our data and how extreme any one observation is, as in the example calculating the 68, 95 and 99.7 percentiles is somewhat labor intensive. An easier way to calculate where a score falls is to use a standardized or Z score.

We can convert any normal distribution into the standard normal distribution in order to find probability and apply the properties of the standard normal. In order to do this, we use the z-value.

Let's take a look at the idea of a z-score within context.

For a recent final exam in STAT 800, the mean was 68.55 with a standard deviation of 15.45.

• If you scored an 80%: \(Z = \dfrac{(80 - 68.55)}{15.45} = 0.74\), which means your score of 80 was 0.74 SD above the mean.

• If you scored a 60%: \(Z = \dfrac{(60 - 68.55)}{15.45} = -0.55\), which means your score of 60 was 0.55 SD below the mean.

Is it always good to have a positive Z score? It depends on the question. For exams, you would want a positive Z-score (indicates you scored higher than the mean). However, if one was analyzing days of missed work then a negative Z-score would be more appealing as it would indicate the person missed less than the mean number of days.

Characteristics of Z-scores

• The scores can be positive or negative.

• For data that is symmetric (i.e. bell-shaped) or nearly symmetric, a common application of Z-scores for identifying potential outliers is for any Z-scores that are beyond ± 3.

• Maximum possible Z-score for a set of data is \(\dfrac{(n−1)}{\sqrt{n}}\)

A more frequent application of standard normal curves are expressions of percentiles or probabilities. Because the value of a z score can be aligned with a specific position within a standard normal distribution, it is also possible to find the equivalent percentile for that observations. 

For example, an observation that has a z score of zero would be at the 50th percentile of the data.

We can also use these properties to make statements about probabilities about relationships among observations.  We will use this information extensively as we progress into hypothesis testing in future modules. However in the immediate application, a simple application for Susie illustrates this point. If Susie knows that a hospital has a z score of +4.00, then she can use software or consult a Standard Normal Table to convert this to a probability. For example with a mean of 50 heart attacks and a standard deviation of 10, Susie can calculate the probability of a hospital reporting more than 100 heart attacks is very low (because 100 heart attacks is in the 99th percentile, for a hospital to report more than that would be very UNLIKELY!)

We will not reference the standard normal tables in these notes, and instead rely on software to generate probabilities for us. 

Measures of position give a range where a certain percentage of the data fall. We briefly referred to percentiles in the last section, but we will also add quartiles to this overview.

A common application of percentiles is their use in determining passing or failure cutoffs for standardized exams such as the GRE. If you have a 95th percentile score then you are at or above 95% of all test takers.

The median is the value where fifty percent or the data values fall at or below it. Therefore, the median is the 50th percentile.

We can find any percentile we wish. There are two other important percentiles. The 25th percentile, typically denoted, Q1, and the 75th percentile, typically denoted as Q3. Q1 is commonly called the lower quartile and Q3 is commonly called the upper quartile.

The method we will demonstrate for calculating Q1 and Q3 may differ from the method described in our textbook. The results shown here will always be the same as Minitab's result. The method here is also different than from the method presented in many undergraduate statistics courses. This method is what we require students to use. There are two steps to follow: Find the location of the desired quartile If there are n observations, arranged in increasing order, then the first quartile is at position \(\dfrac{n+1}{4}\), second quartile (i.e. the median) is at position \(\dfrac{2(n+1)}{4}\), and the third quartile is at position \(\dfrac{3(n+1)}{4}\). Find the value in that position for the ordered data. 

Five number summaries are used to describe some of the key features of a distribution. Using the values in a five number summary we can also compute the range and interquartile range.

Some observations within a set of data may fall outside the general scope of the other observations. Such observations are called outliers. Here, you will learn a more objective method for identifying outliers.

We can use the IQR method of identifying outliers to set up a “fence” outside of Q1 and Q3. Any values that fall outside of this fence are considered outliers. To build this fence we take 1.5 times the IQR and then subtract this value from Q1 and add this value to Q3. This gives us the minimum and maximum fence posts that we compare each observation to. Any observations that are more than 1.5 IQR below Q1 or more than 1.5 IQR above Q3 are considered outliers. This is the method that Minitab Express uses to identify outliers by default.

How do researchers know this all of the sudden? Weren’t the original findings based on science and statistics? Well, yes the original findings were based on science and statistics, but with a lot more research around low dose aspirin and heart attacks, researchers can now calculate an effect size to determine the actual impact of the aspirin. What is an effect size? The effect size arose from social science research, where researchers wanted to be able to draw conclusions across studies. Effect sizes measure the impact of a “treatment” on an outcome. We will learn later in the course that this is also what many of our statistical tools do, the advantage of effect sizes are that they parse out the impact of the size of the sample. If this does not make complete sense at this point, that is okay, as we progress the sample size issue will become clearer.

Adapted from Coe, R. (2002). It’s the effect size, stupid. What effect size is and why it is important. Annual Conference of the British Educational Research Association. Exeter: England. Sept 12-14, 2002.

The effect size calculation is similar to the z score calculation, the difference is in the interpretation. Instead of relying on probabilities, the effect size uses industry standards (as summarized in the table) related to the probability of belonging in a “treatment” group compared to a “control” group (in our example, this could be the group receiving low-dose aspirin (treatment) and those who do not (control). An effect size of 0 indicates the outcome of receiving the treatment (low-dose aspirin) does not impact the chance of a heart attack any more than chance. In contrast, an effect size of 2, indicates a strong effect, meaning the outcome is “effected” by the treatment and it is now possible to conclude the group membership can be highly probably based on the outcome (with an effect size of 2, you can predict group membership with an 84% probability).

In recent years, prestigious social science publications require the inclusion of effect sizes to distinguish findings that are impactful, beyond statistically significant. Reporting effectsizes also allows other researchers to aggregate across studies, a technique called meta-analysis, which we will not cover in the course, but may be of interest to students in the social sciences.

It is of the utmost important for any student beginning the study of statistics to understand the concept of variability. As any researcher knows, the process of science seeks to answer questions, from this perspective, science seeks to explain variability. The concept of within group variability helps students understand why measures of “spread” are such a critical building block for all future work in statistical study.