1.4 - Research Study Design

1.4 - Research Study Design

Experimental and Observational Designs

Research studies are often classified in terms of their designs. Here, we will make the distinction between experimental and observational research designs.

Experimental Research Design

A study in which the researcher manipulates the treatments received by subjects and collects data; also known as a scientific study

Observational Research Design

A study in which the researcher collects data without performing any manipulations; also known as a non-experimental study

Example: Caffeinated Coffee Studies

Coffee cup

An organization wants to know if drinking caffeinated coffee causes hyperactivity in college students. To test their research question, they select a sample of college students and give them a survey concerning their intake of caffeinated coffee and their hyperactivity levels. This is an observational study because the researchers are not making any manipulations. They are observing what is happening without intervening. This is not an experiment because no treatment was imposed by the researchers.

Another organization also wants to know if drinking caffeinated coffee causes hyperactivity in college students. They design a different study. They select a random sample of college students and randomly assign them to drink coffee with or without caffeine. The researchers observe the students' behaviors. This is an experimental study because a treatment is being imposed. The researchers are manipulating the treatment that each participant receives.

On Your Own

A team of researchers want to know if Advil or Tylenol is more effective.

Think about the following data collection methods, then click on the method to compare your answers.

Method 1
Researchers survey a sample of adults and ask if they use Advil or Tylenol. They ask them to rate the effectiveness of the one they use. Is this an observational study or experimental study?
This is an observational study because the researchers observed the difference between two existing groups (Advil and Tylenol users). The researchers did not manipulate the participants' experiences. 
Method 2
Researchers obtain a random sample of adults. They randomly assign half of the participants to take Advil and the other half to take Tylenol. They ask each participant to rate the effectiveness of the one that they were assigned to take. Is this an observational study or experimental study?
This is an experimental study because the researchers assigned participants to groups. 

1.4.1 - Confounding Variables

1.4.1 - Confounding Variables

Experimental studies are typically preferred over observational studies because they allow for more control. A common problem with observational studies is that there may be other variables influencing the results that the researchers were not able to take into account. These are known as confounding variables.

Confounding Variable

Characteristic that varies between cases and is related to both the explanatory and response variables; also known as a lurking variable or a third variable

Example: Ice Cream & Home Invasions

There is a positive relationship between ice cream sales and home invasions (i.e., as ice cream sales increase throughout the year so do home invasions). It is clear that increases in ice cream sales do not cause home invasions to increase, and home invasions do not cause an increase in ice cream sales. There is a third variable at play here: outdoor temperature. When the weather is warmer both ice cream sales and home invasions increase. In this case, outdoor temperature is a confounding variable.


1.4.2 - Causal Conclusions

1.4.2 - Causal Conclusions

In order to control for confounding variables, participants can be randomly assigned to different levels of the explanatory variable. This act of randomly assigning cases to different levels of the explanatory variable is known as randomization. An experiment that involves randomization may be referred to as a randomized experiment or randomized comparative experiment. By randomly assigning cases to different conditions, a causal conclusion can be made; in other words, we can say that differences in the response variable are caused by differences in the explanatory variable. Without randomization, an association can be noted, but a causal conclusion cannot be made.

Note that randomization and random sampling are different concepts. Randomization refers to the random assignment of experimental units to different conditions (e.g., different treatment groups). Random sampling refers to probability-based methods for selecting a sample from a population.

Randomization
The act of randomly assigning cases to different levels of the explanatory variable
Causation
Changes in one variable can be attributed to changes in a second variable
Association
A relationship between variables

1.4.3 - Independent and Paired Samples

1.4.3 - Independent and Paired Samples

In both observational and experimental studies, we often want to compare two or more groups. When comparing two or more groups, cases may be independent or paired.

Independent Groups
Cases in each group are unrelated to one another.
Paired Groups

Cases in each group are meaningfully matched with one another; also known as dependent  samples or matched pairs

Example: Exam Scores

An instructor wants to compare students' scores on the midterm and final exam. This is most often done by obtaining a sample of students and recording each student's midterm exam score and final exam score. In other words, there would be two measurements for each student. This is an example of a matched pairs design because data would be paired by student. 

Example: Shoes

A shoe company is studying how many shoes Italian men and women own. In one research study they take a random sample of 500 Italian adults and ask each individual if they identify as a man or women and how many pairs of shoes they own. The men and women in this study are in two independent groups. 

In a second study the researchers use a different design. This time they take a random sample of 250 heterosexual married couples in Italy (i.e., 250 husbands and 250 wives). They record the number of shoes owned by each husband and each wife. This is an example of a matched pairs design. Data are paired by couple.


1.4.4 - Control and Placebo Groups

1.4.4 - Control and Placebo Groups

control group is an experimental condition that does not receive the actual treatment and may serve as a baseline. A control group may receive a placebo or they may receive no treatment at all. A placebo is something that appears to the participants to be an active treatment, but does not actually contain the active treatment. For example, a placebo pill is a sugar pill that participants may take not knowing that it does not contain any active medicine. This can lead to a psychological phenomena called the placebo effect which occurs when participants who are given a placebo treatment experience a change even though they are not receiving any active treatment. Researchers use placebos in the control group to determine if any differences between groups are due to the active medicine or the participants' perceptions (the placebo effect).

Control Group
A level of the explanatory variable that does not receive an active treatment; they may receive no treatment or a placebo
Placebo Group
A group that receives what, to them, appears to be a treatment, but actually is neutral and does not contain any active treatment (e.g., a sugar pill in a medication study)

Example: Vitamin B Energy Study

Researchers want to know if adult women who consume a drink that is high in vitamin B-12 have increased energy. They obtain a representative sample of adult women. All of the women are given a drink that they are told to consume every morning. They are not told what is in the drink. Half of the women are given a drink that is high in vitamin B-12 while the other half are given a drink that tastes the same but contains no vitamin B-12.

The women who received the drink with no vitamin B-12 are the placebo group. The purpose of the placebo group in this study is to make the two groups equivalent except for the presence of the vitamin B-12. By comparing these two groups, the researchers will be able to determine what impact the vitamin B-12 had on the response variable. We could also say that this served as a control group because this group did not receive any active ingredients. 


1.4.5 - Blinding

1.4.5 - Blinding

Blinding techniques are also used to avoid bias. In a single-blind study the participants do not know what treatment groups they are in, but the researchers interacting with them do know. In a double-blind study, the participants do not know what treatment groups they are in and neither do the researchers who are interacting with them directly. Double-blind studies are used to prevent researcher bias. 

Blinding
Procedure employed in research to prevent bias in which the participants and/or the researchers interacting with the participations do not know which treatment each case is receiving
Single-Blind Study
Research study in which the participants do not know the treatment group that they have been assigned to
Double-Blind Study
Research study in which neither the participants nor the researchers interacting with them know which cases have been assigned to which treatment groups

Example: Yogurt Tasting

Researchers are comparing a low-fat blueberry yogurt to a high-fat blueberry yogurt. Participants are randomly assigned to receive one type of yogurt. After tasting it, they complete an online survey. The researchers know which yogurt containers are low-fat and which are high-fat, but participants are not told. This is an example of a single-blind study because the researchers know which participants are in the low- and high-fat groups but the participants do not know. A double-blind study may not be necessary in this case since the researchers have only minimal contact with the participants. 

Example: Caffeine Energy Study

Researchers want to know if adult males who consume high amounts of caffeine interact more energetically. They obtain a representative sample and randomly assign half of the participants to take a caffeine pill and half to take a placebo pill.  The pills are randomly numbered and coded so at the time the researchers do not know which participants have been given caffeine and which have been given the placebo. All participants are told that they may have been given a caffeine pill. After taking the pill, researchers observe the participants interacting with one another and rate the interactions in terms of level of energy. 

This is a double-blind study because neither the researchers nor the participants know who is in which group at the time the data are collected. After the data are collected, researchers can look at the pill codes to determine which groups the participants were in to conduct their analyses. A double-blind study is necessary here because the researchers are observing and rating the participants. If the researchers know who is in the caffeine group they may be more likely to rate their levels of energy as very high because that is consistent with their hypothesis. 


Legend
[1]Link
Has Tooltip/Popover
 Toggleable Visibility