1.4 - Research Study Design

Experimental and Observational Designs

Research studies are often classified in terms of their designs. Here, we will make the distinction between experimental and observational research designs.

Experimental Research Design: A study in which the researcher manipulates the treatments (i.e., level of the explanatory variable) received by subjects and collects data; also known as a scientific study

Observational Research Design: A study in which the researcher collects data without performing any manipulations; also known as a non-experimental study

Example: Caffeinated Coffee Studies

An organization wants to know if drinking caffeinated coffee causes hyperactivity in college students. To test their research question, they select a sample of college students and give them a survey concerning their intake of caffeinated coffee and their hyperactivity levels. This is an observational study because the researchers are not making any manipulations. They are observing what is happening without intervening. This is not an experiment because no treatment was imposed by the researchers.

Another organization also wants to know if drinking caffeinated coffee causes hyperactivity in college students. They design a different study. They select a random sample of college students and randomly assign them to drink coffee with or without caffeine. The researchers observe the students' behaviors. This is an experimental study because the researchers are manipulating the treatment that each participant receives.

Choosing a Research Study Design

Usually, if there is an option available, experimental studies are preferred over observational studies. Later in this lesson you will learn about randomization, placebos, and blinding, which can all be built into experimental designs to strengthen the conclusions that can be made.

There are times when an experimental design is not possible. If the independent variable is naturally occurring, it may not be possible for a researcher to manipulate it. For example, race, ethnicity, birthplace, age, gender identity, and biological sex are all variables that cannot be randomly assigned to different cases.

On Your Own

A team of researchers want to know if Advil or Tylenol is more effective.

Think about the following data collection methods, then click on the method to compare your answers.

Method 1

Researchers survey a sample of adults and ask if they use Advil or Tylenol. They ask them to rate the effectiveness of the one they use. Is this an observational study or experimental study?

This is an observational study because the researchers observed the difference between two existing groups (Advil and Tylenol users). The researchers did not manipulate the participants' experiences.

Method 2

Researchers obtain a random sample of adults. They randomly assign half of the participants to take Advil and the other half to take Tylenol. They ask each participant to rate the effectiveness of the one that they were assigned to take. Is this an observational study or experimental study?

This is an experimental study because the researchers assigned participants to groups.

1.4.1 - Confounding Variables

Randomized experiments are typically preferred over observational studies or experimental studies that lack randomization because they allow for more control. A common problem in studies without randomization is that there may be other variables influencing the results. These are known as confounding variables. A confounding variable is related to both the explanatory variable and the response variable.

Confounding Variable: Characteristic that varies between cases and is related to both the explanatory and response variables; also known as a lurking variable or a third variable

Example: Ice Cream & Home Invasions

There is a positive relationship between ice cream sales and home invasions (i.e., as ice cream sales increase throughout the year so do home invasions). It is clear that increases in ice cream sales do not cause home invasions to increase, and home invasions do not cause an increase in ice cream sales. There is a third variable at play here: outdoor temperature. When the weather is warmer both ice cream sales and home invasions increase. In this case, outdoor temperature is a confounding variable because it is related to both ice cream sales and home invasions.

Example: Weight & Preferred Beverage

Research question: Do adults who prefer to drink beer, wine, and water differ in terms of their mean weights?

Data were collected from a sample of World Campus students to address the research question above. The researchers found that adults who preferred beer tended to weigh more than those who preferred wine.

A confounding variable in this study was gender identity. Those who identified as men were more likely to prefer beer and those who identified as women were more likely to prefer wine. In the sample, men weighed more than women on average.

1.4.2 - Causal Conclusions

In order to control for confounding variables, participants can be randomly assigned to different levels of the explanatory variable. This act of randomly assigning cases to different levels of the explanatory variable is known as randomization. An experiment that involves randomization may be referred to as a randomized experiment or randomized comparative experiment. By randomly assigning cases to different conditions, a causal conclusion can be made; in other words, we can say that differences in the response variable are caused by differences in the explanatory variable. Without randomization, an association can be noted, but a causal conclusion cannot be made.

Note that randomization and random sampling are different concepts. Randomization refers to the random assignment of experimental units to different conditions (e.g., different treatment groups). Random sampling refers to probability-based methods for selecting a sample from a population.

Randomization: The act of randomly assigning cases to different levels of the explanatory variable

Causation: Changes in one variable can be attributed to changes in a second variable

Association: A relationship between variables

Example: Fitness Programs

Two teams have designed research studies to compare the weight loss of participants in two different fitness programs. Each team used a different research study design.

The first team surveyed people who already participate in each program. This is an observational study, which means there is no randomization. Each group is comprised of participants who made the personal decision to engaged in that fitness program. With this research study design, the researchers can only determine whether or not there is an association between the fitness program and participants' weight loss. A causal conclusion cannot be made because there may be confounding variables. The people in the two groups may be different in some key ways. For example, if the cost of the two programs is different, the two groups may differ in terms of their finances.

The second team of researchers obtained a sample of participants and randomly assigned half to participate in the first fitness program and half to participate in the second fitness program. They measured each participants' weight twice: both at the beginning and end of their study. This is a randomized experiment because the researchers randomly assigned each participant to one of the two programs. Because participants were randomly assigned to groups, the groups should be balanced in terms of any confounding variables and a causal conclusion may be drawn from this study.

1.4.3 - Independent and Paired Samples

In both observational and experimental studies, we often want to compare two or more groups. When comparing two or more groups, cases may be independent or paired.

Independent Groups: Cases in each group are unrelated to one another.

Paired Groups: Cases in each group are meaningfully matched with one another; also known as dependent samples or matched pairs

Example: Exam Scores

An instructor wants to compare students' scores on the midterm and final exam. This is most often done by obtaining a sample of students and recording each student's midterm exam score and final exam score. In other words, there would be two measurements for each student. This is an example of a matched pairs design because data would be paired by student.

Example: Shoes

A shoe company is studying how many shoes Italian men and women own. In one research study they take a random sample of 500 Italian adults and ask each individual if they identify as a man or women and how many pairs of shoes they own. The men and women in this study are in two independent groups.

In a second study the researchers use a different design. This time they take a random sample of 250 heterosexual married couples in Italy (i.e., 250 husbands and 250 wives). They record the number of shoes owned by each husband and each wife. This is an example of a matched pairs design. Data are paired by couple.

1.4.4 - Control and Placebo Groups

A control group is an experimental condition that does not receive the actual treatment and may serve as a baseline. A control group may receive a placebo or they may receive no treatment at all. A placebo is something that appears to the participants to be an active treatment, but does not actually contain the active treatment. For example, a placebo pill is a sugar pill that participants may take not knowing that it does not contain any active medicine. This can lead to a psychological phenomena called the placebo effect which occurs when participants who are given a placebo treatment experience a change even though they are not receiving any active treatment. Researchers use placebos in the control group to determine if any differences between groups are due to the active medicine or the participants' perceptions (the placebo effect).

Control Group: A level of the explanatory variable that does not receive an active treatment; they may receive no treatment or a placebo

Placebo Group: A group that receives what, to them, appears to be a treatment, but actually is neutral and does not contain any active treatment (e.g., a sugar pill in a medication study)

Example: Vitamin B Energy Study

Researchers want to know if adults who consume a drink that is high in vitamin B-12 have increased energy. They obtain a representative sample of adults. All participants are given a drink that they are told to consume every morning. They are not told what is in the drink. Half are given a drink that is high in vitamin B-12 while the other half are given a drink that tastes the same but contains no vitamin B-12.

The participants who received the drink with no vitamin B-12 are the placebo group. The purpose of the placebo group in this study is to make the two groups equivalent except for the presence of the vitamin B-12. By comparing these two groups, the researchers will be able to determine what impact the vitamin B-12 had on the response variable. We could also say that this served as a control group because this group did not receive any active ingredients.

1.4.5 - Blinding

Blinding techniques are also used to avoid bias. In a single-blind study the participants do not know what treatment groups they are in, but the researchers interacting with them do know. In a double-blind study, the participants do not know what treatment groups they are in and neither do the researchers who are interacting with them directly. Double-blind studies are used to prevent researcher bias.

Blinding: Procedure employed in research to prevent bias in which the participants and/or the researchers interacting with the participations do not know which treatment each case is receiving

Single-Blind Study: Research study in which the participants do not know the treatment group that they have been assigned to

Double-Blind Study: Research study in which neither the participants nor the researchers interacting with them know which cases have been assigned to which treatment groups

Example: Yogurt Tasting

Researchers are comparing a low-fat blueberry yogurt to a high-fat blueberry yogurt. Participants are randomly assigned to receive one type of yogurt. After tasting it, they complete an online survey. The researchers know which yogurt containers are low-fat and which are high-fat, but participants are not told. This is an example of a single-blind study because the researchers know which participants are in the low- and high-fat groups but the participants do not know. A double-blind study may not be necessary in this case since the researchers have only minimal contact with the participants.

Example: Caffeine Energy Study

Researchers want to know if adult males who consume high amounts of caffeine interact more energetically. They obtain a representative sample and randomly assign half of the participants to take a caffeine pill and half to take a placebo pill. The pills are randomly numbered and coded so at the time the researchers do not know which participants have been given caffeine and which have been given the placebo. All participants are told that they may have been given a caffeine pill. After taking the pill, researchers observe the participants interacting with one another and rate the interactions in terms of level of energy.

This is a double-blind study because neither the researchers nor the participants know who is in which group at the time the data are collected. After the data are collected, researchers can look at the pill codes to determine which groups the participants were in to conduct their analyses. A double-blind study is necessary here because the researchers are observing and rating the participants. If the researchers know who is in the caffeine group they may be more likely to rate their levels of energy as very high because that is consistent with their hypothesis.

^[1]	Link
↥	Has Tooltip/Popover
	Toggleable Visibility