6a.1 - Introduction to Hypothesis Testing

Basic Terms Section

The first step in hypothesis testing is to set up two competing hypotheses. The hypotheses are the most important aspect. If the hypotheses are incorrect, your conclusion will also be incorrect.

The two hypotheses are named the null hypothesis and the alternative hypothesis.

Null hypothesis
The null hypothesis is typically denoted as \(H_0\). The null hypothesis states the "status quo". This hypothesis is assumed to be true until there is evidence to suggest otherwise.
Alternative hypothesis
The alternative hypothesis is typically denoted as \(H_a\) or \(H_1\). This is the statement that one wants to conclude. It is also called the research hypothesis.

The goal of hypothesis testing is to see if there is enough evidence against the null hypothesis. In other words, to see if there is enough evidence to reject the null hypothesis. If there is not enough evidence, then we fail to reject the null hypothesis.

Consider the following example where we set up these hypotheses.

Example 6-1 Section

A man, Mr. Orangejuice, goes to trial and is tried for the murder of his ex-wife. He is either guilty or innocent. Set up the null and alternative hypotheses for this example.

Answer

Putting this in a hypothesis testing framework, the hypotheses being tested are:

  1. The man is guilty
  2. The man is innocent

Let's set up the null and alternative hypotheses.

\(H_0\colon \) Mr. Orangejuice is innocent

\(H_a\colon \) Mr. Orangejuice is guilty

Remember that we assume the null hypothesis is true and try to see if we have evidence against the null. Therefore, it makes sense in this example to assume the man is innocent and test to see if there is evidence that he is guilty.

 

The Logic of Hypothesis Testing Section

We want to know the answer to a research question. We determine our null and alternative hypotheses. Now it is time to make a decision.

The decision is either going to be...

  1. reject the null hypothesis or...
  2. fail to reject the null hypothesis.
Note! Why can’t we say we “accept the null”? The reason is that we are assuming the null hypothesis is true and trying to see if there is evidence against it. Therefore, the conclusion should be in terms of rejecting the null.

Consider the following table. The table shows the decision/conclusion of the hypothesis test and the unknown "reality", or truth. We do not know if the null is true or if it is false. If the null is false and we reject it, then we made the correct decision. If the null hypothesis is true and we fail to reject it, then we made the correct decision.

Decision Reality
\(H_0\) is true \(H_0\) is false
Reject \(H_0\), (conclude \(H_a\))   Correct decision
Fail to reject \(H_0\) Correct decision  

So what happens when we do not make the correct decision?

When doing hypothesis testing, two types of mistakes may be made and we call them Type I error and Type II error. If we reject the null hypothesis when it is true, then we made a type I error. If the null hypothesis is false and we failed to reject it, we made another error called a Type II error.

Decision Reality
\(H_0\) is true \(H_0\) is false
Reject \(H_0\), (conclude \(H_a\)) Type I error Correct decision
Fail to reject \(H_0\) Correct decision Type II error

Types of errors

Type I error
When we reject the null hypothesis when the null hypothesis is true.
Type II error
When we fail to reject the null hypothesis when the null hypothesis is false.

The “reality”, or truth, about the null hypothesis is unknown and therefore we do not know if we have made the correct decision or if we committed an error. We can, however, define the likelihood of these events.

\(\alpha\) ('Alpha')
The probability of committing a Type I error. Also known as the significance level.
\(\beta\) ('Beta')
The probability of committing a Type II error.
Power
Power is the probability the null hypothesis is rejected given that it is false (ie. \(1-\beta\))

\(\alpha\) and \(\beta\) are probabilities of committing an error so we want these values to be low. However, we cannot decrease both. As \(\alpha\) decreases, \(\beta\) increases.

Note! Type I error is also thought of as the event that we reject the null hypothesis GIVEN the null is true. In other words, Type I error is a conditional event and \(\alpha\) is a conditional probability. The same idea applies to Type II error and \(\beta\).

Example 6-1 Cont'd... Section

A man, Mr. Orangejuice, goes to trial and is tried for the murder of his ex-wife. He is either guilty or not guilty. We found before that...

  • \( H_0\colon \) Mr. Orangejuice is innocent
  • \( H_a\colon \) Mr. Orangejuice is guilty

Interpret Type I error, \(\alpha \), Type II error, \(\beta \).

Answer
Type I Error:
Type I error is committed if we reject \(H_0 \) when it is true. In other words, when the man is innocent but found guilty.
 
\( \alpha \):
\( \alpha \) is the probability of a Type I error, or in other words, it is the probability that Mr. Orangejuice is innocent but found guilty.
 
Type II Error:
Type II error is committed if we fail to reject \(H_0 \) when it is false. In other words, when the man is guilty but found not guilty.
 
\(\beta\):
\(\beta\) is the probability of a Type II error, or in other words, it is the probability that Mr. Orangejuice is guilty but found not guilty.

As you can see here, the hypotheses were set up in such a way that the Type I error (putting an innocent man in jail) is the more serious error. Ethically, it is more serious to put an innocent man in jail than to let a guilty man go free. So to minimize the probability of a type I error we would choose a smaller significance level.

Try it! Section

An inspector has to choose between certifying a building as safe or saying that the building is not safe. There are two hypotheses:

  1. Building is safe
  2. Building is not safe

Set up the null and alternative hypotheses. Interpret Type I and Type II error.

\( H_0\colon\) Building is not safe vs \(H_a\colon \) Building is safe

Decision Reality
\(H_0\) is true \(H_0\) is false
Reject \(H_0\), (conclude  \(H_a\)) Reject "building is not safe" when it is not safe (Type I Error) Correct decision
Fail to reject  \(H_0\) Correct decision Failing to reject 'building not is safe' when it is safe (Type II Error)

Power and \(\beta \) are complements of each other. Therefore, they have an inverse relationship, i.e. as one increases, the other decreases.

It makes sense for us to set up the \(H_0\) and \(H_a\) as above (that is, assume building is not safe until proven otherwise), because if we switch \(H_0\) and \(H_a\) (that is, if \(H_0\) was building is safe and \(H_a\) is building is not safe) and if we fail to reject \(H_0\), we cannot quite conclude that building is safe (we can only fail to reject \(H_0\), we cannot accept \(H_0\)).