6.2 - Significance Levels

As we saw in the examples on the previous page, the consequences of Type I and Type II errors vary depending on the situation. Researchers take into account the consequences of each when they are setting their \(\alpha\) level before data are even collected.

In many disciplines an \(\alpha\) level of 0.05 is standard, for example in the social sciences. There are some situations when a higher or lower \(\alpha\) level may be desirable. Pilot studies (smaller studies performed before a larger study) often use a higher \(\alpha\) level because their purpose is to gain information about the data that may be collected in a larger study; pilot studies are not typically used to make important decisions.

Studies in which making a Type I error would be more dangerous than making a Type II error may use smaller \(\alpha\) levels. For example, in medical research studies where making a Type I error could mean giving patients ineffective treatments, a smaller \(\alpha\) level may be set in order to reduce the likelihood of such a negative consequence. Lower \(\alpha\) levels mean that smaller p-values are needed to reject the null hypothesis; this makes it more difficult to reject the null hypothesis, but this also reduces the probability of committing a Type I error.