Experimental and Observational Studies

STAT 120

Bastola

Association and Causation

Two variables are associated if values of one variable tends to be related to values of the other variable

  • e.g. Families with many cars tend to own many television sets


Two variables are causally associated if changing the value of the explanatory variable influences the value of the response variable

  • e.g. Studies show that taking a practice exam increases your score on an exam

College Education and Aging

“Education seems to be an elixir that can bring us a healthy body and mind throughout adulthood and even a longer life,” says Margie E. Lachman, a psychologist at Brandeis University who specializes in aging. “For those in midlife and beyond, a college degree appears to slow the brain’s aging process by up to a decade, adding a new twist to the cost-benefit analysis of higher education - for young students as well as those thinking about returning to school.”

Which of the following is true?

  1. Explanatory Variable = brain aging; Response Variable = elixir
  2. Explanatory Variable = brain aging; Response Variable = college degree
  3. Explanatory Variable = college degree; Response Variable = brain aging

Click for answer

The correct answer is 3.

A Sharper Mind, Middle Age and Beyond, NY Times, 1/19/12

College Education and Aging

Based on this passage, should we conclude that obtaining a college degree causes a slowdown in the brain’s aging process?

  • Yes
  • No

Click for answer

No. There could be other factors, such as socioeconomic status, lifestyle choices, or genetic predispositions, that contribute to both higher education attainment and slower brain aging

Should you buy more TVs to live longer?

TVs and Life Expectancy


Extra lurking variable

Confounding Variable

A third variable that is associated with both the explanatory variable and the response variable is called a confounding variable

  • A confounding variable can offer a plausible explanation for an association between the two variables.


Whenever confounding variables are present (or may be present), a causal association CANNOT be determined!

Experiment Vs Observational Study

An observational study is a study in which the researcher does not actively control the value of any variable, but simply observes the values as they naturally exist


An experiment is a study in which the researcher actively controls one or more of the explanatory variables

Experiment or Observational Study?

To examine whether farm-grown salmon contain more omega-3 oils if water is more acidic, we collect samples of salmon and water from multiple fish farms to see if the two variables are related.

Click for answer

This is an observational study because the researchers do not actively manipulate or control the acidity of the water; they simply observe the naturally occurring conditions at each fish farm.

Observational Studies and Confounding Variable

  • There are almost always confounding variables in observational studies

  • Observational studies can almost never be used to establish causation!!


To avoid confounding variables: Randomly assign values of the explanatory variable!

Randomized Experiments


Source: click here

Randomized Experiments

  • In a randomized experiment, the explanatory variable for each unit is determined randomly, before the response variable is measured

  • The different levels of the explanatory variable are known as treatments

    • Randomly divide the units into groups, and randomly assign a different treatment to each group
    • Observe the response variable after treatments are applied

Elimination of confounding variables


Because the explanatory variable is randomly assigned, it is not associated with any other variables. Confounding variables are eliminated!!!

Exercise and the Brain

  • A sample of mice were divided randomly into two groups. One group was given access to an exercise wheel, the other group was kept sedentary
  • “The brains of mice and rats that were allowed to run on wheels pulsed with vigorous, newly born neurons, and those animals then breezed through mazes and other tests of rodent IQ” compared to the sedentary mice

Reynolds, “Phys Ed: Your Brain on Exercise”, NY Times, July 7, 2010]

Exercise and the Brain

Is this evidence that exercise causes an increase in brain activity and IQ, at least in mice?

  • Randomized experiment can yield causal conclusions!

Reynolds, “Phys Ed: Your Brain on Exercise”, NY Times, July 7, 2010

Knee Surgery, Control Groups

Study Context

  • Researchers investigated the effectiveness of knee surgery for arthritis.
  • Patients who underwent surgery reported less pain.


The Importance of a Control Group

  • To validate if the surgery was effective, a control group received a fake knee surgery.
  • Results were indistinguishable between the control and treatment groups.


Placebo Effect

  • Control groups often get a placebo to measure the effect.
  • It is estimated that 75% of the effectiveness of anti-depressant medication is due to the placebo effect.


Scenario

  • Question: Is this evidence that the surgery causes a decrease in pain?
  • Answer: No, control group results indicate that the placebo effect might be at play.

Double-blinded Experiments

Nobody knows who gets what, not even the researchers.


Example: A new pill for blood pressure

  • Some people get the real pill, others a fake.
  • Both pills look the same so no one can tell.
  • Researchers checking results don’t know either, so it’s fair.

Randomized Experiments Types: randomized comparative experiment

  • Individuals are randomly assigned to a treatment or control group.
  • Most effective when the investigator has no prior knowledge of the individuals who are part of the experiment.

Example: A researcher randomly assigns participants to either a drug treatment group or a control group to measure the effectiveness of the drug.

Randomized Experiments Types: matched pairs experiment

  • Each individual in the experimental group is matched with an individual in the control group based on a specific criterion.
  • Most effective when the investigator has prior knowledge of the individuals involved in the experiment.

Example: A researcher matches participants based on age, gender, and education level, then assigns one participant in the matched pair to the drug treatment group and the other to the control group to measure the effectiveness of the drug.

Variations of matched pairs experiments: Training program effectiveness

Before-After Test

  • Measure everyone’s skill level before any training.
    • Example: Time each runner’s 100m sprint.
  • Train only one group.
  • Measure again and compare.
    • Example: Time the 100m sprint again and see who improved more.

Variations of matched pairs experiments: Training program effectiveness

Two-Times Test

  • Everyone gets both training and no training, but in random order.
    • Example: Flip a coin to decide who gets training first.
  • Measure skill after each.
    • Example: Time each 100m sprint after both.
  • Compare the same person’s times to see which training worked.

 Group Activity 1


20:00