Chapter 8



Chapter 8

Lesson 14: Anecdotal Evidence and Extraneous Variables

In Lesson 11, we talked briefly about possible effects of the lunar cycle on humans. To give another example of this, some have claimed that automobile accidents are more frequent during certain phases of the moon. In order to test this claim, Templer, Veleber, and Brooner (1982) looked at the number of automobile accidents throughout the lunar cycle (for an animation of the phases of the moon, go to this page). They found that more car accidents occurred on evenings around the full and new moons. During the new moon, there is no moonlight (the moon is between the earth and the sun). Perhaps the reduced moonlight makes it harder for people to see. But why would more car accidents happen around the full moon? Some people have claimed that people act a little "crazier" around the time of the full moon. Perhaps this supposed "lunacy" is associated with more risky driving behavior.

But there is something else to consider in the study by Templer, Veleber, and Brooner (1982). During the time period in which they made their observations, full and new moons occurred more often on weekends than on weekdays. Do you see the problem here? On weekend nights, people are more likely to be out driving than they are on weekday nights; and drunk driving is more common on the weekends. It may be that these (or other) factors associated with weekend driving is the real culprit in their findings, not the phase of the moon (Rotton & Kelly,1985). This example shows how important it is to be aware of the possible effects of "extraneous variables" when interpreting the results of a study. An extraneous variable is any factor, other than the one being studied, that affects a phenomenon. Although the researchers wanted to see if the phase of the moon was related to the frequency of car accidents, the presence of an extraneous variable--the days of the week on which full and new moons tend to occur--makes it difficult for us to interpret their results.

Let's look at another example, this one from the early-twentieth century. You may have heard of the case of "Clever Hans," a horse who seemed to have an amazing ability: he could solve arithmetic problems! Hans' trainer would give Hans a problem in addition, subtraction, or multiplication, and the horse would provide an answer by hitting one hoof on the ground. For instance, when given the problem, 3 x 2, Hans would tap his hoof six times on the ground. According to Stanovich (2001):

Many people were amazed and puzzled by Clever Hans's performance. Was he really demonstrating an ability thus far unknown in his species? ... Compelling testimonials to Hans's unique ability appeared in the German press. One Berlin newspaper reporter wrote that "this thinking horse is going to give men of science a great deal to think about for a long time to come".... A group of "experts" observed Hans and attested to his abilities. Everyone was baffled. (pp. 92-93)

When the bafflement reached its peak, Oskar Pfungst, a German psychologist, began a careful series of investigations of Hans' ability to provide correct solutions to arithmetic problems. Pfungst found that Hans could not provide a correct solution if he was unable to see his trainer. Furthermore, if the questioner did not know the solution, Hans was unable to provide it. It turned out that, as Hans tapped out his answers, he would watch his trainer very carefully. When the number of taps approached the correct answer, his trainer would, without being aware of it, very slightly move his head in a particular way, and Hans would stop at that point. Hans did not know arithmetic, but he did know that he got rewarded whenever he stopped tapping just after his trainer moved his head!

Just as in the car-accident example, there is an important extraneous variable in the Clever-Hans example--visual cues from the trainer--that observers did not consider when they watched the horse perform his amazing feat. When an extraneous variable affects the results, we say that there exists a confound, which means that the effects of the factor being studied are intertwined with those of another factor not being investigated. In the car-accident example, the day of the week was confounded with the phase of the moon. In the Clever-Hans example, the visual cues from the trainer were confounded with the presentation of an arithmetic problem. The goal of a scientific study is to eliminate confounds, as much as is possible, so that the effects of the factor being studied can be determined.

It should be obvious that, in order to eliminate confounds, we need to "control for" the effects of extraneous variables. When we exert control, we regulate the research situation in such a way that the effects of only one factor will explain the results. In other words, when we exert control, we find ways to exclude the effects of all causal factors but one. In the Clever-Hans example, one way that Pfungst controlled for the extraneous variable of visual cues was to make certain that Hans could not see his trainer. If Hans still tapped out a correct answer under this condition, it would have suggested that visual cues were not important for his behavior. Nevertheless, this would not have meant that Hans necessarily had the ability to do arithmetic: other uncontrolled extraneous variables may have been affecting the behavior. Only after the effects of all possible extraneous variables have been taken into account can we be confident in making a conclusion from the results of an investigation.

Anecdotal Evidence

Everyday, we hear claims about various treatments and products that are promoted as helping us to lose weight, cure anxiety, develop satisfying relationships, and so on. Often, these claims are supported with evidence such as the following:

I tried PVQ Therapy on my children. My daughter, Karen, was diagnosed with attention-deficit disorder. We've been using PVQ Therapy on her for almost a year now. She just got her report card and got A's in everything but math. And she got a B in that! Before we started using PVQ, her grades were much worse. Now, she can sit and read a book all the way through without getting bored. She just finished all the "Harry Potter" books! It's great! She doesn't even need her medicine anymore. She seems to be sleeping better and she's much easier to be around.

I also tried PVQ Therapy on my son, Brandon. He used to be anxious and seemed to have trouble making friends at school. After we used PVQ Therapy for five months, Brandon was elected class president and now has all the friends he could want! He looks so much better and happier since he was "PVQed"!

Although this example is fictional, it is similar to the kinds of "testimonials" you probably hear all of the time (a testimonial is a personal endorsement of something). Probably many of us have been persuaded to try an advertised product based on testimonials. They can be very hard to resist, especially when the person giving the testimonial seems absolutely convinced about the value of the product.

But has the discussion of extraneous variables above suggested to you a possible problem with testimonials? What are some extraneous variables that might explain the girl's improvement after receiving PVQ treatments? One important confound is the placebo effect, in which improvement occurs simply because a person believes that he or she is receiving an effective treatment. When people receive a "placebo" (an inert substance), they may feel better because of their expectation that they will feel better. A second confound is the passage of time. For example, illnesses often go away because of the body's ability to heal itself. A psychological problem, such as severe anxiety caused by stress, also may improve on its own over time. A third confound is the increased attention that the children may be receiving from their mother as she gives them the treatment. This increased attention may be causing improvements in their behavior. Furthermore, Karen may have been misdiagnosed and the mother may be misremembering the behavior of her children prior to receiving the treatment. And so on. Testimonials are interesting because they provide information that we might decide to follow up on. But, because testimonials include a number of possible confounds, it is impossible to interpret the information in any particular way.

Testimonials are one type of "anecdotal" evidence. An anecdote is an account of an interesting or unusual incident. In general, anecdotes cannot be used as evidence for or against a particular explanation of a phenomenon because anecdotes leave uncontrolled any extraneous variables that might be affecting the phenomenon. This does not mean that anecdotes are worthless, however: they point to interesting phenomena that we may decide to study with methods that are able to control for important extraneous variables. The Clever-Hans study started as a set of anecdotes reported in newspapers--anecdotes that told of a horse that was able to count. These anecdotes led Oskar Pfungst to perform a series of studies that controlled for confounds; and, almost 100 years later, we still are discussing his investigation.

Critical Thinking Questions

Question 14-1

In Critical Thinking Lesson 5, Question 5-4 (b), Tom presented evidence for his claim that men are "better thinkers" than women. He correctly noted that most of the greatest philosophers who ever lived have been men. And he also correctly noted that men are more likely than women to be successful in careers involving science and engineering--careers that typically require much intelligence. But, as was stated, it also is true that the average IQ of women is equivalent to men. Furthermore, women are at least as likely as men to go to college, which requires intelligence to complete successfully. If you wanted to make the claim that "women are equal in intelligence to men," how would you explain the apparent contradictions in the evidence?

Suggested Answer

Question 14-2

Your textbook noted that verbal reports (such as answers to survey questions) are not necessarily an accurate reflection of how people actually behave. In other words, there are extraneous variables that affect how people respond to questions about their behaviors, their attitudes, their beliefs, and so on. Since much of the research discussed in Chapter 8 that looked at factors influencing the development of romantic relationships made use of such reports, it is important to find ways of controlling for these extraneous variables if we wish to make valid conclusions about romantic relationships.

Think carefully about extraneous variables that may affect the verbal reports of participants in psychological studies. List the extraneous variables you come up with and try to think of ways to control for them.

Suggested Answer

Question 14-3

The following passage appeared in a previous edition of your textbook:

While authentic love certainly helps in tackling marital problems, it does not guarantee success. In fact, how much you like your lover may be more important than how much you love your lover. When researchers correlated a host of variables with a measure of the "successfulness" of romantic relationships, liking for one's partner was more highly correlated (.62) with relationship success than was love (.50) of one's partner.... [This finding] raises the possibility that liking may conquer problems more effectively than love.

With respect to what you have learned in this lesson, what problem or problems might there be with the claim made in the last sentence of the passage?

Suggested Answer

Question 14-4

A new medication for major depression is being studied by researchers. They selected a group of 100 people with major depression and had them take the new medication daily for three months. They found that 75 people in this group felt much better at the end of that time and, in fact, showed few or no symptoms of depression. Can we conclude that the antidepressant medication is an effective treatment for depression?

Suggested Answer

Question 14-5

Why would we not want to estimate the relative importance of loneliness as a cause of psychological problems by counting up the number of complaints of loneliness made by people who call telephone "hotlines" for troubled people (see page 233 of textbook)?

Suggested Answer

Bibliography and References

Carroll, R. T. (2002). Testimonials. Skeptic's Dictionary. Retrieved June 1, 2002, from

Dodes, J. E. (1997, January/February). The Mysterious Placebo. Skeptical Inquirer, 21(1). Retrieved June 6, 2002, from

Laverty, W. H., Kelly, I. W. (1998). Cyclical calendar and lunar patterns in automobile property accidents and injury accidents. Perceptual and Motor Skills, 86, 299-302.

McGuigan, F. J. (1997). Experimental psychology: Methods of research (7th ed.). Upper Saddle River, NJ: Prentice Hall.

Rotton, J., & Kelly, I. W. (1985). Much ado about the full moon: A meta-analysis of lunar-lunacy research. Psychological Bulletin, 97, 286-306.

Schick, Jr., T., & Vaughn, L. (2002). How to think about weird things: Critical thinking for a new age (3rd ed.). Boston: McGraw-Hill.

Templer, D. I., Veleber, D. M., & Brooner, R. K. (1982). Geophysical variables and behavior: VI. Lunar phase and accident injuries: a difference between night and day. Perceptual and Motor Skills, 55, 280-282.

Weiten, W., & Lloyd, M. A. (2000). Psychology applied to modern life: Adjustment at the turn of the century (6th ed.). Belmont, CA: Wadsworth.

RETURN TO TOP

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download