ACTIVITY - Society for the Teaching of Psychology



Appleby, D. C. (2008). A cognitive taxonomy of multiple-choice questions. In L. T. Benjamin (Ed.), Favorite activities for the teaching of psychology (pp. 119-123). Washington, DC: American Psychological Association.

34

A Cognitive Taxonomy of Multiple-Choice Questions

Drew C. Appleby

Indiana University–Purdue University Indianapolis

Abstract

This activity helps students understand that multiple-choice questions (MCQs) can be written to test different levels of cognitive mastery of a single psychological concept. It is ideally suited to the introductory class because many students at this level have not been exposed to MCQs that test higher-level critical-thinking skills. This situation often causes them to perform poorly, and to react negatively to their poor performance, because they perceive the questions they answer incorrectly to be unfair, ambiguous, or tricky. This in-class activity requires approximately 20 minutes, can be used in any size class, and involves all the students in a class.

Concept

This activity describes a cognitive taxonomy (Anderson & Krathwohl, 2001; Bloom, Englehart, Furst, & Krathwohl, 1956) of MCQs that demonstrates how this type of question can measure different levels of cognitive mastery of the same concept. Although the ability of MCQs to measure higher order thinking skills has been challenged, testing experts such as Haladyna (1994, p. 34) have come to their defense with statements such as, “Under most circumstances, the multiple-choice format is more effective than the essay format for measuring knowledge. In some instances, the multiple-choice format might be as effective or even more effective for measuring some mental skills, such as problem solving or clinical judgment.” Although high quality MCQs are often more difficult to create than essay questions, there are many advantages to using this format in the introductory course where MCQs are abundantly available in test banks that accompany introductory texts. Ease and objectivity of scoring, computerized item analysis, higher reliability, and equal or higher validity are advantages that Haladyna identifies. The purpose of this activity is to demonstrate to students that MCQs can be written to measure the mastery of psychological concepts far beyond the mere memorization of definitions. The use of the handout contained in this activity can increase mastery of course content, guide study techniques, decrease student criticism of test questions, lower test anxiety levels, and negate the old myth that MCQs can measure only rote-learned material.

Materials Needed

The only materials needed are copies of the three-page handout that appears at the end of this document. Although the taxonomy can be reproduced verbatim, it is best used as a model for the construction of an original taxonomy. Constructing their own taxonomies will encourage teachers to take a hard look at the quality and type of MCQs they create or select from test banks. Some teachers may discover that many of their test items measure only memorization or surface understanding of the concepts they present to their students. Others may find their questions measure relations so complex that they would be more appropriate for upper-level classes. The process of creating a taxonomy will also help teachers develop an accurate set of student learning outcomes for their class syllabus (e.g., to retain, understand, apply, compare, contrast, and evaluate psychological terms, concepts, and methods). If a teacher decides to create an original taxonomy, it will be beneficial to choose a question topic that is complex enough so that a number of sample MCQs can be written about it at several levels of cognitive complexity.

Instructions

After the three-page taxonomy handout has been distributed, students should be told that its purpose is to help them understand and give them some practice with the different types of MCQs they will encounter on their first test. The teacher should instruct the students to study the first page of the handout as if it were part of the assigned readings for their first test (i.e., read it carefully, underline important information, and/or take notes in the margins). After studying the first page for approximately 5 minutes, instruct students to turn the page and answer the six questions, without referring back to the first page. After approximately 5 minutes, the teacher should read each question aloud, ask the class to answer it, read the explanation following it on the taxonomy, and then solicit questions regarding the nature of the thought processes required to answer the question correctly. During this process, students begin to realize they must not only memorize the material presented in class or in their textbook, but they must also understand, apply, compare, contrast, and evaluate this material to perform well on tests. They should be urged to develop study techniques that will enable them to answer any type of MCQ on the taxonomy that is written about any of the material presented to them in the class. The third page of the handout provides a set of suggestions for studying for MCQ tests, and teachers should be prepared to offer some helpful strategies to students who suddenly realize that their lack of study techniques may limit their ability to perform up to their levels of expectation. A discussion of Craik and Lockhart’s (1972) levels of processing theory can be very beneficial at this point. Students seem to be particularly receptive to this theory when they realize how relevant it is to their academic performance.

Discussion

There are several reasons why most teachers of the introductory class use MCQs to test their students. First, MCQs can be scored easily, quickly, and accurately. Second, MCQs can produce high levels of content validity in subject areas involving large quantities of widely divergent material. Third, MCQs are provided in abundance by the publishers of introductory psychology textbooks. Unfortunately, students do not always share their instructors’ enthusiasm for this type of test item and often feel there is something inherently unfair or tricky about MCQs, especially those they answer incorrectly. This results in one of the least pleasant aspects of teaching the introductory class, the necessity of defending the validity of individual questions challenged by students when their tests are returned. These post-test interchanges can provide valuable feedback to both students and teachers, but this beneficial effect is often overshadowed by the negative emotions produced when students believe they are being tested unfairly and teachers feel their authority is being questioned. Providing students with examples of the questions they can expect on their tests decreases the frequency of their criticisms. It places the responsibility for missed questions on students who have failed to learn concepts on a variety of levels instead of on the teacher whose questions are labeled as tricky, ambiguous, or unfair by the students who miss them.

Perry (1970) described freshman-level students as dualistic in their thinking; that is, they believe that knowledge is either right or wrong in a very absolute and concrete way. Students at this level firmly believe that a teacher’s primary purpose is to provide them with the facts and then ask them to simply recognize or recall these facts—in the original form in which they were presented—on tests. Students operating in this stage feel most comfortable with, study for, and perform best on test items that are fact-oriented (i.e., items that test retention of terms and their definitions as they are presented in textbooks and lectures). Their level of discomfort rises dramatically as they are required to demonstrate increasingly more complex mastery of terms, principles, and theories with questions that require them to comprehend, apply, compare, contrast, or evaluate this information.

Consider these comments. “You never told us you were going to ask this type of question!” “If I had known you were going to ask these types of questions, I would have studied differently and gotten a better grade!” “I can’t figure out what your questions mean, let alone answer them!” If a teacher has ever heard any of these comments after a test, this taxonomy will be very beneficial. It can help students progress in a more informed—and therefore less painful and resentful—manner to more sophisticated stages of cognitive development by introducing them to questions that require a deeper understanding and mastery of psychological concepts in a nonthreatening atmosphere (i.e., during a classroom demonstration instead of a test). Many students have previously experienced only MCQs that measure the ability to memorize facts, and it is disconcerting for them when they encounter questions that require more complex cognitive skills without advance warning. Students appreciate being exposed to the types of questions they will encounter on their first test and often report a correspondingly lower level of test anxiety. For many, it will be the first time a teacher has cared enough to take the time to show them exactly the types of questions they will encounter on their first test and how to prepare for them.

References and Suggested Reading

Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Allyn & Bacon.

Bloom, B. S., Englehart, M. D., Furst, E. J., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: Cognitive domain. New York: McKay.

Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.

Haladyna, T. M. (1994). Developing and validating multiple-choice test items. Hillsdale, NJ: Erlbaum.

Perry, W. G. (1970). Forms of intellectual and ethical development in college. New York: Holt, Rinehart, & Winston.

The Human Memory System

Human memory is composed of three interconnected memory stores. Information from our senses is initially stored in sensory memory (SM) in the same way it processed by our senses (e.g., as images or sounds). Although information is retained in SM for less than 2 seconds, this is long enough for us to interpret it and to decide which parts of it are important enough to pay attention to and transfer to working memory (WM), where we can continue to “work” on it. Information in WM is stored primarily in terms of how it sounds (i.e., acoustically), and only about 7 pieces of information (e.g., numbers, letters, or words) can be held in WM for only about 30 seconds unless we continue to maintain them by repeating them to ourselves. Forgetting occurs in WM when we transfer more information from SM into WM—and therefore exceed our 7-item capacity—or if we stop repeating the information currently stored in WM. This repetition process, known as rehearsal, serves two purposes. First, it allows us to maintain information in WM as long as we continue to repeat it to ourselves. Second, rehearsal enables us to transfer information we wish to remember more permanently to our third and final memory store, known as long-term memory (LTM). Information in LTM is stored predominantly in terms of its meaning (i.e., semantically), and this memory store is often compared to a library whose contents are organized in a number of different meaningful ways (e.g., by subject matter, title, or author’s name). We are not always be able to retrieve information from LTM when we want to—in the same way that a book in a library may be sometimes hard to find—but once it is stored there, it is stored permanently. We do not actually forget information once it is stored in LTM; it simply becomes unavailable because we cannot retrieve it successfully.

1. Which of the following types of memory stores information for approximately 30 seconds?

a. SM b. WM c. LTM d. none of the above

This question tests your ability to recognize a single, defining characteristic of a specific concept (e.g., information in WM lasts for only about 30 seconds). This requires you merely to memorize a term and its definition.

2. Which of the following is true about WM?

a. It can hold only about 7 pieces of information.

b. Information stored in it is mainly encoded acoustically.

c. Information stored in it lasts only about 30 second unless it is rehearsed.

d. all of the above

This question measures your ability to learn a set of characteristics common to a particular concept (e.g., WM can hold only a certain amount of information in a certain form for only a certain period of time). It requires a more thorough knowledge of a concept than does Question 1, but is still based primarily on memorization.

3. Which of the following is the correct sequence through which information passes as it is processed by the human memory system?

a. SM(WM(LTM b. WM(SM(LTM c. SM(LTM(WM d. WM(LTM( SM

This question requires you to learn a chronological relationship among a series of concepts (e.g., SM, WM, and LTM).

4. Joan looked up a phone number, closed the phone book, and then repeated the number to herself as she dialed the phone. This phone number is being stored in __________.

a. SM b. WM c. LTM d. none of the above

This question measures your ability to understand and apply knowledge to a new, real-life situation that was not presented in your textbook or lecture. It requires a deeper comprehension of the concept, which goes beyond mere memorization of its definition or characteristics.

5. WM is to LTM, as __________ is to __________.

a. 7, unlimited b. semantic, acoustic c. relatively permanent, 30 seconds d. all of the above

This is an analogy question, which measures your ability to compare and contrast two concepts (e.g., WM and LTM). This skill is based on—but goes beyond—the simpler ability of memorizing a concept's set of characteristics (from question 2).

6. Which of the following students has given the best explanation of why Tom has already forgotten the name of the person to whom he was introduced only three minutes ago?

a. Chang: "The name was never in Tom's SM."

b. Monique: "The name was lost from Tom's LTM."

c. Fritz: "Tom did not successfully transfer the name from his LTM to his SM."

d. Juanita: "Tom did not successfully transfer the name from his WM to his LTM."

This question involves all the cognitive skills measured by the first five questions. It requires you to evaluate knowledge you have acquired so you can produce a logical decision based on a thorough understanding of a relational concept (e.g., factors that affect the successful transfer of information from WM to LTM) that can be used to explain an example of complex human behavior (e.g., forgetting).

How well did you perform on this quiz? If you answered all the questions correctly, you possess the ability to retain, understand, apply, compare, contrast, and evaluate the information you will encounter in this class. Now all you need is the motivation to work hard so you can actualize your academic potential. If your performance on this quiz was less than spectacular—and you would like to avoid this type of performance on your tests in this class—then you should spend some time determining why you missed the questions you did. If you are like most students who take this quiz and perform below their expectations, you probably answered the first two or three questions correctly, but then began to experience difficulty as you attempted to answer the later questions that required you to apply, compare, contrast, or evaluate your knowledge. If this is the case, give careful consideration to your personal understanding of what it means "to study" a textbook assignment. If studying means making sure that your eyes see every word in a reading assignment and memorizing the important terms and their definitions, then I urge you to change your idea of what it means "to study." To answer the later questions on this quiz, you must assume an active approach to studying in which you ask yourself some of the following questions as you progress through your reading assignment.

1. Do I truly understand this information or have I just memorized it?

2. Can I apply this information to a "real life" situation?

3. Can I relate this information to other information in this assignment?

a. Is it part of some type of sequence?

b. Is it similar to other information and, if so, how is it similar?

c. Is it different from other information and, if so, how is it different?

4. Can I use this information to evaluate the validity (i.e., truthfulness) of a statement?

When you ask yourself these questions as you read an assignment—and read actively to discover the answers—you will be engaging in a method of study that will increase your cognitive mastery of the material you are reading and increase the probability that you will perform well on tests. Take this exercise and its results seriously. As you study your textbook or lecture notes, think about the different kinds of questions that could be asked about the material you are studying. Anticipating these types of questions—and thinking about how you would answer them—is a powerful strategy that could help you master the material in this class and perform well on your tests.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download