Alan Reifman, Texas Tech University
Return to Main Syllabus

CORRELATION (BY ITSELF)

DOES NOT PROVE

CAUSALITY

 

Correlation is necessary for causality, but must have two other ingredients:

Hypothetical example

Watching a lot of television as a teen (A) is correlated negatively with SAT Verbal scores (B).  Could be...

A ® B TELEVISION CAUSES LOW VERBAL ABILITY
(most television shows do not expose viewer to intelligent dialogue)
B ® A LOW VERBAL ABILITY CAUSES TV WATCHING
(people with lower verbal ability might prefer to spend free time watching TV than reading books)
    A
  
C
  
    B
LACK OF CHILDHOOD READING EXPERIENCE
(A THIRD VARIABLE "C") CAUSES BOTH HEAVY TELEVISION VIEWING AS A TEEN AND LOW SAT SCORES


Recent Research Findings that Raise Questions of Causality...

...As Well as Some Older Findings that Also Raise Causality Questions...

...Plus an April 2009 piece of sports speculation:  Does good team "chemistry" cause winning or does winning cause good chemistry? 


In-Class Exercise

Each group gets a scenario, based on an actual research finding.  For your scenario:

1.  Being allowed by your parents to see R-rated movies (A) correlated with greater likelihood of kids taking up smoking (B) (research finding).

2.  Having a TV set in the bedroom (A) correlated with couples having lower frequency of sexual activity (B) (research finding).

3.  Adolescents' watching of professional wrestling (A) correlated with fighting during dates and elsewhere (B) (research finding).

4.  Listening to certain types of sexual lyrics (A) correlated with teen sexual activity (B) (research finding).

5.  Alcohol consumption (A) correlated with violent behavior (B) (research findings).

6.  When Green Bay Packers' running back Ahman Green rushes for 100 yards or more (A) the Packers virtually always win, going 17-1 as of October 2003 (B) (article).


A Song, Before We Move to the Next Topic...

Correlation
Lyrics by Alan Reifman
(May be sung to “Revolution,” John Lennon)

You say you’ve got a correlation,
Well, you know, it says nothing about cause,
You’ve got to do experimentation,
Well, you know, to fulfill science’s laws,

With a correlation between B and A,
Causal direction could go either way,

A correlation cannot be,
Airtight!
Airtight!!
Airtight!!!

You say you’ve got a correlation,
Well, you know, A and B are only linked,
You need further investigation,
Well, you know, to show the pathway that you think,

There’s another option, and this is key,
There could be a third variable called C,

A correlation cannot be,
Airtight!
Airtight!!
Airtight!!!


EXPERIMENTATION
The Way to Show that "A" Causes "B"

Previous Example:  Does Alcohol Consumption (A) Cause Violent Behavior (B)?

 

Experiment to See if Alcohol Causes Violence

No Alcohol (nonalcoholic beer) – known as "control group" or "placebo"

Low Alcohol Beer –"experimental group"

High Alcohol Beer— "experimental group"

Can have as many groups as you’d like (but at least one experimental and one control group)

NOTE: Any one experiment can only test a single causal direction.  If we wanted to test if involvement in violence caused people to drink, we'd have to conduct another experiment.  How could we test whether violence causes drinking?


REAL-WORLD EXAMPLES

From the Fox Sports show "Sport Science,"  here's an illustrative single-subject design to study the effects of two different types of distraction (mainly visual; mainly audio), plus a control condition, on basketball free-throw accuracy (Part I and Part II).  This is obviously just a small demonstration for TV.  How could this experiment be improved if we really wanted to study this matter?


Experiment suggests that chewing gum while learning math improves later test-score performance (April 2009)

 


What kind of enclosed photo increases the likelihood someone will return a lost wallet?


In the business world, a term for experimentation is A/B testing.  Some simple examples come from Internet commerce.  Companies want to design their websites to look inviting and encourage consumers to buy their products.  How can we learn which website designs will be most effective?

A company can design two (or more) versions of its website (version A and version B) and have one version or the other come up randomly when someone visits the site.  The number of times the visitor clicks to receive further information or to buy the product can be recorded for each version of the page design and compared.  For examples, see here, here, and here.

I learned of these business applications via the book Super Crunchers, on the role of large "number-crunching" data analyses in our society.  


A memory aid:

Randomly 
Assign to conditions of 
 
Independent variable, then observe on 
Dependent variable, for
Experimental
Research
 

And another memory aid:

The 5 E's.  Idea that experimental and control groups should be treated as identically as possible (amount of time spent with them, amount of attention they receive, equal belief in what they're getting; adapted from Eysenck, 1995), except for the one crucial ingredient of the treatment.  Examples... 

Everything Equal Except Essential Element

 


Sign by a restaurant in DFW airport...

How could we design an experiment to see if soup makes people smarter?


 

The major research methods tend to have strengths, as well as weaknesses. 

Ability to Infer Causality
(INTERNAL VALIDITY)

Low

High

Ability to Represent Real-life Situations
(EXTERNAL 
VALIDITY)
Low EXPERIMENTS
High SURVEYS This would be the ideal – strong in both areas

 


Internal Validity

Degree to which study permits conclusion that independent variable has caused differences in dependent variable.

Threats to Internal Validity
(primarily applicable in laboratory setting)

In addition to confounding (as in the free-throw example and hypothetical version of the wallet-returning study above), threats include...

EXPERIMENTER BIAS – Similar to "interviewer bias," where the researcher gives cues (smiles, nods) that encourage behavior (could happen if same person prepares beverages and oversees the "shock game").

Example from student in Spring 2006 3390 class, from when she judged a school science fair:  One contestant had two plants, one of which he had talked to and the other, not.  Sure enough, the plant that was talked to had more robust growth.  Ideally, the experimenter would take great pains to ensure that the two plants were treated as identically as possible, except for the talking (e.g., equal water, equal sunlight).  If the experimenter gave better treatment to the talked-to plant in these other ways, that would be experimenter bias.

 

EXPERIMENTAL DEMAND (or DEMAND CHARACTERISTICS) – Hypothesis of experiment is apparent to subject, who then tries to comply.
 
Example:  An aggression study by Berkowitz and LePage (1967) raised issues of demand characteristics (see p. 21 of this PowerPoint document).  Berkowitz, however, wrote recently (12/21/07) that:

"I can also say I have had too many aggression experiments fail for me to believe in the idea of subjects eager to confirm the researcher's hypothesis.  But more generally, I've seen little evidence that many research participants are quick to grasp the experiment's true purpose and that they believe they are helping science (Orne's assumption) by acting to confirm the researcher's hypothesis."


Remedy

DOUBLE-BLIND PROCEDURE – Neither experimenter nor subject knows the hypothesis or what condition subject is in. Multiple experimenters are used (one to prepare beverages, one to oversee shock game). Placebo represents a good imitation of experimental group(s), so subject doesn’t know what condition he/she is in. Deception may be a further way to hide hypothesis from subject.
 
Double-blind procedures could be applied to the above plant example through the use of two experimenters.  One would talk to the designated "experimental" (E) plant, and do nothing else.  Another experimenter, "blind" to which plant is being talked to and which one is the "control" (C) plant, would do the watering and tending to the plants; without knowing which plant was E and which C, it would seem unlikely the waterer would favor one plant or the other.

 

Double Blindness
Lyrics by Alan Reifman
(May be sung to the tune of “Double Vision,” Jones/Gramm, for Foreigner)

Experimenter bias, you must avoid,
Or else your study, could end up destroyed,
There’s a technique you can use, to make sure all’s well,
Each subject you meet with, you can’t know their cell, 

Do it right, with that double blindness,
Controls are tight, with that double blindness,
You don’t know the treatment, you can’t give off cues to subjects,
Internal validity, you protect; that's what this method will help you do,

The subject’s condition, is in a secret code,
Good experimenters operate in this mode,
The subjects don’t know, either, what treatment they’ve gotten,
Because if they did, your results would be rotten, 

Do it right, with that double blindness,
Controls are tight, with that double blindness,
You don’t know the treatment, you can’t give off cues to subjects,
Internal validity, you protect; that's what this method will help you do...
 


The original source article is available here.
 


 
CAUSALITY IN THE NEWS... 
 
A recent New York Times article (11/18/07) revisits (for the umpteenth time) the question of whether the death penalty causes a reduction in the homicide rate (i.e., has a deterrent effect).  Experts still disagree, although some seem more open to the possibility.  For the purpose of our class, there are some useful passages in the article:
 
"The studies try to explain changes in the murder rate over time, asking whether the use of the death penalty made a difference. They look at the experiences of states or counties, gauging whether executions at a given time seemed to affect the murder rate that year, the year after or at some other later time. And they try to remove the influence of broader social trends like the crime rate generally, the effectiveness of the criminal justice system, economic conditions and demographic changes."
 
Criteria for causality, as referenced in the above passage (see matching color): correlation, time-order, and removing third variables.
 
Professor Justin Wolfers cites the need for random assignment and the experimental method to show causality:
 
“If I was allowed 1,000 executions and 1,000 exonerations, and I was allowed to do it in a random, focused way,” he said, “I could probably give you an answer.”
 
Wolfers is the same researcher who conducted the study of possible racial bias in basketball officiating, which we discussed earlier in the course (see his website).
 
 

ADDITIONAL RESOURCES ON CAUSALITY AND EXPERIMENTATION

Correlation & Causality website, maintained by Dr. Reifman and former TTU colleague Dr. Bo Cleveland (now at Penn St.)

Excellent list of details to follow in conducting experiments

Online Experiments Page

Compendium of Online Studies

Excellent document linking measurement validity, internal validity, and external validity

Examples of Experimental Studies, Allowing Causal Conclusions

Wearing red sports uniforms (as opposed to blue) is associated causally with winning:
(Study takes advantage of fact that in certain Olympic sports (e.g., boxing, wrestling), the determination of who wears red and who wears blue in a match is done randomly.  Whereas internal validity appears strong, what can we say about the external validity or generalizability?)

A little experiment I did with some colleagues on the effects of seeing former presidential candidate Howard Dean's "Scream Speech" from two different angles.

Does having people pray for you improve recovery after coronary bypass surgery?
Note the use of random assignment, blind design, etc. (official abstract)