Survey Research - Stanford University

[Pages:30]CHAPTER NINE

Survey Research

PBNNY S. VISSBR, JON A. KROSNICK, AND PAUL J. LAVRAWS

Social psychologists have long recognized that every method of scientific inquiry is subject to limitations and that choosing among research methods inherently involves trade-offs. With the control of a laboratory experiment, for example, comes an artificiality that raises questions about the generalizability of results. And yet the 'naturalness" of a fieldstudyoran observational study canjeopardize the validity of causal inferences.Theinevitabilityof suchlimitationshasled many methodologists to advocate the use of multiple methods and to insist that substantive conclusions can be most confidently derived by triangulating across measuresand methodsthat have nonoverlappingstrengths and weaknesses (see, e.g., Brewer, this volume, Ch. 1; Cpmpbell 6 Piske, 1959; Campbell 6 Stanley, 1963; Cook 6 Campbell, 1969; Crano 6 Brewer, 1986; B.

Smith, this volume, Ch. 2). This chapter describes a research methodology that

we believe has much to offer social psychologists interested in a multimethod approach: survey research. Survey research is a specific type of field study that involves the collection of data from a sample of elements (e.g., adult women) drawn from a well-defined population (e.g., all adult women living in the United States) through the use of a questionnaire (for more lengthy discussions, see Babbie, 1990; Fowler, 1988;

This chapter was completed while the second author was a Fellow at the Center for Advanced Study in the Behavioral Sciences, supported by National Science Foundation Grant SBR-9022192. Correspondence should be directed to Penny S. Visxr, Department of Psychology, Princeton University, Princeton, New Jersey 08544 (e-mail:pvisseSPrinceton.edu). or Jon A. Krosnick. Department of Psychology, Ohio State University, 1885 Neil Avenue, Columbus, Ohio 43210 (e-mail:Krosnick @osu.edu).

Frey, 1989; Lavrakas, 1993; Weisberg, Krosnick, 6 Bowen, 1996).We begin the chapterby suggestingwhy survey research may be valuable to sodal psychologists and then outline the utility of various study designs. Next, we review the basics of survey sampling and questionnaire design. F i y , we describe procedures for pretesting questionnairesandfor data collection.

REASONS FOR SOCIAL PSYCHOLOGISTS TO CONDUCT SURVEY RBSEARCH

Social psychologists are interested in understanding how people influence, and are influenced by, their social environment. And to the extent that social psychological phenomena are universal across different types of people, it makes little difference precisely with

whom social psychological research is conducted -

even data collectedfrom samplesthat are decidedly unrepresentative of the general population can be used to draw inferences about that population.

In recent years, however, psychologists have become increasingly sensitive to the impact of dispositional and contextual factors on human thought and socialbehavior. Instead of broad statementsabout universal processes, social psychologists today are far more likely to offer qualified accounts of which people, under which conditions, are likely to exhibit a particular psychological phenomenon or process. And accordingly, socialpsychologists have increasinglyturned their attention to interactions between various social psychological processes and characteristics of the individual, such as personality traits, identification with a social group or category, or membership in a distinct culture. In many cases, the nature of basic social psychological processes has been shown to depend to a large degree on characteristics of the individual.

2U

PENNY S. VISSER, JON A. KROSNICK, AND PAUL J. LAVRAKAS

The process by which attitude change occurs, for 6 Arenberg, 1983; Nesselroade 6 Baltes, 1974), have

example, has been shown to differ for people who are more weakly established self-images(Mortimer,Pinch,

low and high in what Petty and Cacioppo (1986)have 6 Kumka, 1982),and have less well-developed social

termed 'need for cognition," a general enjoyment of identities (Alwinet al., 1991)than-doolder adults.

and preference for effordul thinking. Attitude change Because of these kinds of differences, Sears (1986)

among people high in need for cognition tends to be argued, the field's reliance on participant pools of

mediated by careful scrutiny of the arguments in a per- college-aged adults raises questions about the gener-

suasive appeal, whereas attitude change among people ahability of some findings from social psychological

low in need for cognition tends to be based on cues in laboratoryresearch and may have contributed to a dis-

the persuasive message or context, such as the attrac- torted portrait of 'human nature." However, the evi-

tiveness of the source.

dence Sears (1986) cited largely reveals the prevalence

Similarly,attributionshave been shown to differ de- of certain characteristics (e.g., the frequency of atti-

pending on social group membership (see, e.g., Hew- tude change or the firmness of socialidentities),rather

stone, Bond, 6 Wan, 1983). People tend to attribute than differences in the processes by which these char-

positive behaviors by members of their own social acteristics or others emerge in different age groups. We

groupor categoryto stable, internalcauses. Those same currently know so little about the operation of social

positive behaviors performed by a member of a differ- psychological processes in other subsets of the popula-

ent social group, however, are more likely to be at- tion that it is impossible to assess the extent of bias in

tributed to transitory or externalfactors.

this regard.

According to much recent research, culture may Doing so will require studies of samples that are

also moderate many socialpsychological phenomena. representative of the general population, and induc-

Markus and Kitayama (1991), for example, have ar- ing most members of such samples to visit a labora-

gued that members of different cultures have differ- tory seems practically impossible. Studying a repre-

ent construals of the self and that these differences sentative sample through field research, however, is

can have a profound impact on tlye nature of cogni- relatively easy and surprisingly practical. Using the ba-

tive, emotional, and mo@vationapl mceses. Similarly, sictenets of probabilitytheory, surveyresearchershave

Nisbett and his colleagues'(Coheh,Nisbett, Bowdle, 6 developed a number of effident strategiesfor drawing

Schwan, 1996; Nisbra, 1993; Nisbett 6 Cohen, 1996) representative samples that are easy to contact. And

have exploredwhat they tcnnedthe 'culture of honorw when samples have been selected in such a manner,

of the American Southand have demonstratedmarked socialpsychologists canconfidentlygeneralize findings

differencesin the cognitive, eimotiod, behavioral, and to the entirepopulation. Furthermore, surveyresearch

even physiological reactionsaf southemmen (relative provides ideal conditions for the exploration of Pro-

to their northern counterparts)when confronted with cess x IndividualDifference interactionsbecause care-

insult.

fully selected samples reflect the full heterogeneity of

These kinds of process-by-indi--difference in- the general population.

teractions suggest that precisely whoparticipatesin so- Therearetwoprimarylimitationsof surveyresearch

cial psychological research can have a profound im- for social psychologists. First, surveys are more expen-

pact on what results are obtained. And of course, for sive and time-consumingthan most laboratoryexperi-

the vast majority of social psychological research, that ments using captive participant pools. However, many

'who" hasbeen the infamouscollegesophomore.Sears cost-savingapproachescan be implemented. Second is

(1986) has argued that the field's overwhelming re- the impracticality of executing elaborate scripted sce-

liance on this narrow base of research -st

narios for social interaction, especially ones involving

may represent a serious problem for sodal psychol- deception. Whereas these sorts of events can be a e -

ogy. Pointing to various attributes that are charac- ated in labs with undergraduate participants, they are

teristic of young adults, Sears (1986) suggested that tougher to do in the field. But as .we discuss shortly,

the 'college sophomore" partidpant pool is unrep- many experimentalproceduresand manipulations can

resentative of the general population in a num- be incorporated in surveys.

ber of important ways. Among other things, young Put simply, social psychology can happily proceed

adults are more susceptible to attitude change (Alwin, doing most of our research with college sophomores,

Cohen, 6 Newcomb, 1991; Glenn, 1980; Krosnick 6 assuming that our findings generalize. And we can live

Alwin, 1989; Sears, 1983), exhibit less stable person- with the skepticism of scholars from other disciplines

ality traits (Caspi, Bem, 6 Elder, 1989; Costa, McCrae, who question that generalizabiity, having documented

the profound impact that context and history have on that can be brought about by respondents' own be-

social processes. Or we can accept the challenge and havior (e.g., misreporting true attitudes, failing to pay

explore the replicabiity of our findings in the general close attention to a question), interviewer behavior

population. Either we will confirm our assumptionsof (e.g., misrecording responses, providing cues that lead

generalizabilityor we willreline our theoriesby adding partidpants to respond in one way or another), and

to them new mediators and moderators. The explica- the questionnaire (e.g., ambiguous or confusing ques-

tion of the survey method offered below is intended to tion wording, biased question wording or response

help those who accept the challenge.

options).

The total s w e y error perspective advocates explic-

TOTAL SURVEY ERROR

itly taking into consideration each of these sources of error and making decisions about the allocation of fi-

Even researchers who recognize the value of s w e y nite resources with the goal of reducing the sum of the

methodologyfor socialpsychologicalinquiry aresome- four. In the sections that follow, we consider each of

times reluctant to initiate s w e y research because of these potential sources of survey error and their im-

misconceptions regarding the feasibility of conducting plications for psychologists seeking to balance prag-

a survey on a limited budget. And indeed, the cost of matic budget considerations against concerns about

prominent large-scale national sweys conducted by data quality.

major s w e y organhtions are well outside of the R-

searchbudgets of most social psychologists. But survey methodologists have recently begun to rekindle and

STUDY DESIGNS

expand the early work of Hansen and his colleagues Sweys offer the opportunity to execute studies with

(e.g., Hansen6Madow, 1953)inthinking aboutsurvey various&signs, each of which is suitablefor addressing

designissueswithin an explicitcost-benefit framework particular research questions of long-standinginterest

geared toward helping researchers make design dea- to social psychologists. In this section, we will review

sionsthat maximhe data quality within the constraints several standard designs, including aoss-sectional, re-

of a limited budget. This approach to survey methodol- peated cross-sectional, panel, and mixed designs, and

ogy, known as the 'total s w e y error" perspective (d. discuss when each is appropriatefor socialpsychologi-

Dillman, 1978, Fowler, 1988; Groves, 1989; Lavrakas, calinvestigation.We willalso review the incorporation

1993), can provide social psychologists with a broad of experiments within sweys.

framework and specificguidelinesformakingdecisions

to conductgoodsweyson limitedbudgets while maximizing data quality.

Cross-Sectional Surveys

The total survey error perspective recognizes that Cross-sectional surveys involve the collection of

the ultimate goal of s w e y research is to accurately data at a singlepoint in time froma sampledrawn from

measure particular constructs within a sample of a speci6ed population. This design is most often used

people who representthepopulation of interest.In any to documenttheprevalenceof particularcharacteristics

given survey, the overalldeviationfromthisideal is the in a population. For example, aoss-sectional sweys

cumulative result of several sources of survey error. are routinely conducted to assess the frequency with

Spedfically, the total s w e y error perspective disag- which people perform certain behaviors or the num-

gregates overall error into four components: coverage ber of people who hold particular attitudes or beliefs.

error, sampling error, nonresponse error, and measure- However, documenting prevalence is typically of little

ment error. C w a g e mor refers to the bias that can interest to social psychologists, who are usually more

result when the pool of potential survey partidpants interested in documenting assodations between vari-

from which a sample is selected does not include some ables and the causal processes that give rise to those

portions of the population of interest. Smnpling mor associations.

refers to the random differences that invariably exist Cross-sectional surveys do offer the opportunity to

between any sample and the population from whicb it assess relations between variables and differences be-

was selected. Nonresponse m r is the bias that can re- tween subgroupsin a population. But although many

sult when data are not collected from all of the mem- scholars believe their value ends there, this is not the

bers of a sample. And musumnmt mor refers to all case. Cross-sectional data can be used to teit causal

distortions in the assessment of the construct of inter- hypotheses in a number of ways. For example, using

est, including systematic biases and random variance statistical techniques such as two-stage least squares

226

PENNY S. YISSBR,JON A. KROSNICK,AND1 PAUL J. LAVRAWS

regression, it is possible to estimate the causal impact most when opinions and opinion-formation processes

of variable A on variable B, as well as the effect of vari- are not firmly grounded in past experienceand in sup-

able B on variable A (Blalock, 1972). Such an anal- porting knowledge bases. From a political viewpoint,

ysis rests on important assumptions about causal re- this finding suggests that news media priming may not

lations among variables, but these assumptions can be especially politically consequentialin nations where

be tested and revised as necessary (see, e.g., James 6 political expertise is high throughout the population.

Singh, 1978). Furthermore, path analytic techniques

canbe appliedto test hypothesesabout the mediatorsof causal relations (Baron 6 Kenny, 1986; Kenny, 1979),

Repeated Cross-Sectional Surveys

thereby validating or challenging notions of the psy- Additional evidence consistent with a hypothesized

chological mechanisms involved. And cross-sectional causal relation would be that changes over time in

data can be used to identify the moderators of rela- a dependent variable parallel changes in a proposed

tions between variables, thereby also shedding some independent variable. One way to generate such evi-

light on the causal processes at work (e.g., Krosnick, dence is to conduct repeated aoss-sectional surveys,

1988b).

in which data are collected from independent sam-

A single, cross-sectional survey can even be used to ples drawn from the same population at two or more

assess the impact of a social event. For example, Kros- points in time. If a hypothesized causal relation exists

nick and Kinder (1990)studiedpriming in a real-world between two variables, between-wave changes in the

setting by focusing on the IranIContra scandal. On independent variable should be mirrored by between-

November 25,1986, the American public learned that wave changes in the dependent variable. Fir example,

members of the National Security Council had been if onebelieves that interracialcontactmay reduceinter-

funneling funds (earned through arms sales to Iran) to racial prejudice, an increase in interracial contact over

the Contras fighting to overthrow the Sandinista gov- a period of years in a sodety should be paralleled by a

ernmentin Nicaragua.Although there had been almost reduction in interracial prejudice.

no nationalnews media attentionto Nicaragua and the One study along these lines was reported by Schu-

Contraspreviously, thisrevelation led to a dramaticin- man, Steeh, and Bobo (1985). Using aoss-sectional

crease in the salience of that country in the American surveys conducted between the 1940s and the-1980s

press during the followingweeks. Krosnick and Kinder in the United States, these investigators documented

suspected that this coveragemight have primed Amer- dramatic increases in the prevalence of positive atti-

icans' attitudes toward U.S. involvement in Nicaragua tudes toward principles of equal treatment of Whites

and thereby increased the impact of these attitudes and Blacks. And there was every reason to believe that

on evaluations of President Ronald Reagan's job these general principles might be important detenni-

performance.

nants of people's attitudes toward specific government

To test this hypothesis, Krosnick and Kinder ( 1990) efforts to ensure equality. However, there was almost

took advantage of the fact that data collection for the no shift during these years in public attitudes toward

1986 National Election Study, a' national survey, was specific implementation strategies. This challenges the

underway well before November 25 and continued notion that the latter attitudes were shaped powerfully

well after that date. So these investigatorssimply split - by the general principles.

the survey sample into one group of respondents who Repeated aoss-sectional surveys can also be used to

had been interviewed before November 25 and the studythe impactof socialeventsthat occurredbetween

others, who had been interviewed afterward. As ex- the surveys (e.g., Weisberg, Haynes, 6-Krosnick,1995).

pected, overall assessments of presidentialjob perfor- And repeated cross-sectionalsurveys can be combined

mance were based much more strongly on attitudes into a single data set for statistical analysis, using in-

toward U.S. involvement in Nicaragua in the second formation from one survey to estimate parameters in

group than in the first group.

another survey (e.g., Brehm 6 Rahn, 1997).

Furthermore, Krosnick and Kinder (1990) found

that this priming effect was concentrated primarily among people who were not especially knowledgeable

Panel Surveys

about politics (so-called 'political novices"), a finding In a panel survey, data are collected from the same

permitted by the heterogeneity in political expertise in people at two or more points in time. Perhaps the most

a nationalsample of adults.From a psychologicalview- obvioususe of panel data isto assessthe stabilityof psy-

point, this suggests that news media priming occurs chological constructs and to identify the determinants

SURVEY RESEARCH

227

of stability (e.g., Krosnick, 1988a; Krosnick 6 Alwin, 1989).But with suchdata, one can test causalhypothesesin at least two ways. First, onecan examinewhether individual-levelchanges over time in an independent variable correspond to individual-level changes in a dependent variable over the same period of time. So, for example, one can ask whether people who experienced increasing interracial contact manifested decreasingracialprejudice, whileat the sametime, people who experienced decreasing interracial contact manifested increasing racial prejudice.

Second, one can assess whether changes over time in a dependent variable can be predicted by prior levels of an independent variable. So, for example, do people who had the highest amounts of interracial contact at Time 1 manifest the largest decreases in racial prejudice between Time 1 and Time 2. Such a demonstration providesrelatively strongevidenceconsistentwith a causal hypothesis, because the changesin the dependent variable could not have caused the prior levels of the independent variable (see, e.g., Blalock, 1985; Kessler 6 Greenberg, 1981, on the methods; see Rahn, Krosnick, 6.Breuning, 1994, for an illusuation of its application).

One application of this approach occurred in a study of a long-standing social psychological idea called the projedm hypothesis. Rooted in cognitive consistency theories, it proposes that people may overestimate the extenttowhich they agreewith otherswhom they like, and they may overestimate the extent to which they disagree with others whom they dislike. By the late 19805, a number of cross-sectional studies by political psychologists yielded correlations consistent with the notion that people's perceptions of the policy stands of presidential candidates were distorted to be consistent with attitudes toward the candidates (e.g., Granberg, 1985; Kinder, 1978). However, there were alternative theoretical interpretationsof these correlations,so an analysis using panel survey data seemed in order. Krosnick (199la) did just such an analysis exploring whether attitudes toward candidates measured at one time point could predict subsequent shifts in perceptions of presidential candidates' issue stands. And he found no projection at all to have occurred, thereby suggesting that the previously documented correlations were more likely due to other processes (e.g., decidinghow much to like a candidatebased on agreement with him orher onpolicy issues; seeByrne, 1971; Krosnlck, 1988b).

The impact of social events can be gauged especially powerfully with panel data. For example, Krosnick and Brannon (1993) studied news media priming using

such data. Their interest was in the impact of the Gulf ' War on the ingredients of public evaluations of presidential job performance. For the 1990-1991 National Election Panel Study of the Political Consequences of War, a panel of respondents had been interviewed first in late 1990 (before the Gulf War) and then again in mid-1991 (after the war). The war brought with it tremendous news coverage of events in the Gulf, and Krosnick and Brannon suspected that this coverage might have primed attitudes toward the Gulf War, thereby increasingtheirimpact onpublicevaluationsof President George Bush's job performance. This hypothesis was confirmed by comparing the determinants of presidentialevaluationsin 1990and 1991.Because the same people had been interviewed on both occasions, this demonstration is not vulnerable to a possible alternative explanation of the Krosnick and Kinder (1990) results described above: that different sorts of people were interviewedbefore and after the IranlContrarevelationand their preestablished presidentialevaluation strategiesmay have produced the patternsof regression coefficients that would then have been misdiagnosed as evidence of news media priming.

Panel surveys do have some disadvantages. First, although people are often quite willing to participate in a single cross-sectional survey, fewer may be willing to complete multiple interviews. Furthermore, with each additional wave of panel data collection, it becomes increasingly difficult to locate respondents to reinterview them, because some people move to different locations, some die, and so on. This may threaten the representativenessof panel surve* samplesif the members of the first-wave sample who agree to participate in several waves of data collection differ in meaningful ways from the people who are interviewed initially but do not agree to partidpate in subsequent waves of interviewing.

Also, participation in the initial survey may sensitize respondents to the issues under investigation, thus changing the phenomena being studied. As a result, respondents may give special attention or thought to these issues, which rhay have an impact on subsequent survey responses. For example, Bridge et al. (1977)demonstrated that individualswho participated in a survey interview about health subsequently considered the topic to be more important. And this increased importance of the topic can be translated into changed behavior. For example, people interviewed about politics are subsequently more likely to vote in elections (Granberg 6. Holmberg, 1992; Kraut 6 McConahay, 1973; Yalch, 1976). Even answering a single survey question about one's intention to vote

increasesthe likelihood that an individualwill turn out of affirmative action have characterizedit as entailing

to vote on election day (Greenwald, Carnot, Beach, 6. reverse disaimination against qualified White candi-

Young, 1987).

dates; other times, opponents have characterized affir-

Finally, panel survey respondents may want to ap- mative action as giving unfair advantages to minority

pear consistent in their responses across waves. There- candidates. Did this difference in framing change the

fore, people may be reluctant to report opinions or be- way the general public formed opinions on the issue?

haviors that appear inconsistent with what they had To answer this question, Kinder and Sanders (1990)

reported during earlierinterviews.The desireto appear asked White respondents in a national survey about

consistent could mask genuine changes over time.

whether they favored or opposed affirmative action

Researchers can capitalize on the strengths of each programs in hiring and promotions and in college ad-

of the designs discussed above by incorporating both missions. Some respondents were randomly assigned

cross-sectional and panel surveys into a single study. to receive a description of opposition to affirmative ac-

. . . If, for example, a researcher is interested in conduct- tion as emanating from the belief that it involves re-

ing a two-wave panel survey but is concerned about verse dwmmmtion. Otherrespondents,again selected

carry-over effects, he or she could conduct an addi- randomly, were told that opposition to affirmative ac-

tional cross-sectionalsurvey at the second wave. That tion emanates from the belief that it provides unfair

is, the identical questionnaire could be administered advantages to minorities.

to both the panel respondents and to an independent This experimental manipulation of the framing of

sample drawn from the same population. Significant opposition did not alter the percentages of people who

differences between the data collected from these two said they favored or opposed affirmative action, but it

samples would suggest that carry-over effects were, in did alter the processes by which those opinions were

fact, a problem in the panel survey. In effect, the cross- formed. When affirmativeaction was framed as giving

sectional survey respondents can serve as a 'control unfair advantageto minorities (therebymakingminor-

group" against which panel survey respondents can be ity group members salient), it evoked more anger, dis-

compared.

gust, and fury from respondents, and opinions were

based more on general racial prejudice, on intoler-

Experiments within Surveys

ance of diversity in society, and on belief in general moral decay in society. But when affirmative action

Additional evidence of causal processes can be doc- was framed as reverse discrimination against qualified

umented in surveys by building in experiments. If re- Whites (thereby making Whites more salient), opin-

spondents are randomly assigned to 'treatment' and ions were based more on the perceived material inter-

'control" groups, differences between the two groups ests of the respondent and of Whites as a group.

can then be attributed to the treatment. Every one of Because Kinder and Sanders (1990) analyzed data

the survey designs desaibed above can be modified to from a national survey, respondents varied a great deal

incorporate experimentalmanipulations. Somes u r y q in terms of their political expertise. Capitalizing on

respondents (selected at random) can be exposed to this diversity, Kinder and Sanders found that the im-

one version of a questionnaire, whereas other respon- pact of framing was concentrated nearly exclusively

dentsareexposedto anotherversion. Differencesin re- among political novices. This reinforced the implica-

sponses can then be attributed to the specific elements tion of Krosnick and Kinder's (1990) finding regard-

that were varied.

ing political expertise in their research on news media

Many social psychologists are aware of examples of priming described earlier.

survey research that have incorporated experiments to Sniderman and Tetlock (1986) and Sniderman,

explore effects of question order and question word- Tetlock, 6.Peterson (1993)have also conductedexper-

ing (see, e.g., Schuman 6 Presser, 1981). Less salient iments within surveys to assess whether conservative

are the abundant examples of experiments within sur- values encourage racial prejudice in judgments about

veys that have been conducted to explore other social who is entitled to public assistance and who is not. In

psychological phenomena.

their studies, respondents were told about a hypothet-

ical person in need of public assistance. Different re-

RACISM. One experimental study within a survey spondents were randomly assigned to receive different

was reported by Kinderand Sanders (1990),who werv descriptions of the person, varying in terms of previous

interested in the impact of public debates on public work history, marital andparental stams,age, and race.

opinion on affirmative action. Sometimes, opponents Interestingly, conservatives did not exhibit prejudice

against Blackswhen deadingwhether he or she should who were described as having been model prisoners.

receive public assistance, even when the person was This suggests that stereotypes can have relatively little

said to have violated traditional values (e.g., by being a impact on public policy debates if didcussions focus on

single parent or having a history of being an unreliable counterstereotypic perpetrators.

worker). And in fact, when presented with an individ-

ual who had a history of being a reliable worker, con- M O O D AND LIFE SATISFACTION. SchwaI'Z and

servativeswere substantiallymore supportiveof public Clore (1983)conducted an experiment in a survey to

assistance for Blacks than for Whites. However, con- explore mood and misattribution. They hypothesized

servatives were significantly less supportive of public that general affective states can sometimes influence

policies designed to assist Blacks as a group and were judgments via misattribution. Specifically, these inves-

more likely to believe that Blacks are irresponsibleand tigators presumed that weather conditions (sunnyvs.

lazy. Sniderman and Tetlock (1986)concluded that a cloudy) influence people's moods, which in turn may

key condition for the expression of racial discrimina- influence how happy they say they are with their lives.

tion is therefore a focus on groups rather than individ- This presumably occurs because people misattribute

ual members of the groups and that a generallyconser- their current mood to the general conditions of their

vative orientation does not encourage individual-level lives, rather than to the weather conditions that hap-

discrimination.

pen to be occurring when they are asked to make the

Peffley and Hurwitz (1997; Peffley, Hurwitz, 6 judgment. As a result, when people are in good moods,

Sniderman, 1997)also conducted experiments within they may overstate their happiness with their lives.

surveys to explore the impact of racial stereotypes on To test this hypothesis, Schwam and Clore (1983)

judgments regarding crime. These investigators hy- / conducted telephone interviews with people on either

pothesized that although some Americans may hold sunny or cloudy days. Among respondents who were

negative stereotypes of Blacks, those stereotypes may randomly selected to be asked simply how happy they

only be used to make judgments about criminal acts were with their lives, those interviewed on sunny days

or public policies regarding crime when the perpetra- reported higher satisfaction than people interviewed

tors have characteristics consistent with those negative on cloudy days. But among people randomly selected

stereotypes. Therefore, when debates about govern- to be asked first, 'By the way, how's the weather down

ment policy focuson counterstereotypicAfrican Amer- there?", those interviewed on sunny days reported

icanperpetrators, publicattitudesmay not be especially identical leveb of life satisfactionto those interviewed

driven by stereotypes.

on cloudy days. The question about the weather pre-

In one experiment, respondents in a representative sumably led people to properly attribute some of their

sample survey were told about a man accused of com- current mood to current weather conditions, thereby

mitting a crime and were asked how likely he was insulating subsequent life satisfaction judgments from

to have actually committed it and how likely he was influence.

to commit a similar crime in the future. Respondents

were randomly assigned to be told that the accused aPB BBNBPFFS OF BXPBRlMBNTS WFFHIN SURVEYS.

was either Black or White, and they were randomly What isthebenefit of doingthese experimentsin repre-

assigned to be told that the crime was either violent or sentative sample surveys? Couldn't they instead have

not violent. When the perpetrator was Black and the been done just as well in laboratory settings with col-

crime was violent (and thereby consistent with some lege undergraduates? Certainly, the answer to this lat-

negative stereotypes of Black criminals), respondents ter question is yes; they could have been done as tradi-

who held especiallynegativestereotypesof Blackswere tional social psychological experiments. But the value

more likely than others to say the person had com- of doing the studies within representative sample sur-

mitted the crime and would do so again. But when veys is at least three-fold. First, survey evidence docu-

the crime was not violent or when the perpetrator ments that the phenomena are widespread enough to

was White, stereotypes of Blacks had no impact on be observable in the general population. This bolsters

judgments of guilt or likelihood of recidivism. In an- the apparent value of the findings in the eyes of the

other experiment, respondents with especially nega- many nonpsychologistswho instinctivelyquestion the

tive stereotypes of Blacks were more opposed than generaliabilityof laboratory5ndingsregarding under-

others to furlough programs for Blacks convicted of graduates.

committingviolent crimes, but were not especiallyop- Second, estimates of effect sue from surveys pro-

posed to furlough programs for Whites or for Blacks vide a more accurate basis for assessing the significance

230

PENNY S. MSSBR, JON A. KROSNICK, AND PAUL J. LAVRAKAS

that any social psychological process is likely to have method has been used. When elements have been se-

in the course of daily life. Effects that seem large in lected through other procedures or when portions of

the lab (perhaps because undergraduates are easily the population had no chance of being included in the

influenced) may actually be quite small and thereby sample, there is no way to know whether the sam-

much less socially consequential in the general popu- ple is representative of the population. Generalizations

lation. And third, generalpopulation samplesallow re- beyond the specific elements in the sample are there-

searchers to explore whether attributes of people that fore only warranted when probability sampling meth-

are homogeneous in the lab but vary dramatically in ods have been used.

the general population (e.g., age, educational attain- Thesecondadvantageof probabilitysamplingis that

ment) moderate the magnitudes of effects or the pro- it permits researchersto precisely estimatethe amount

cesses producing them (e.g., Kinder 6. Sanders, 1990). of variancepresent in a givendata setthat isdueto sam-

pling error.That is, researcherscancalculatethe degree

SAMPLING

to which random differences between the sample and the sampling frame are likely to have diminished the

Once a survey design has been specified, the next step predsion of the obtained estimates. Probability sam-

in a surveyinvestigationis selectinga samplingmethod pling also permits researchers to construct confidence

(see, e.g., Henry, 1990; Kalton, 1983; Kish, 1965; Sud- intervals around their parameter estimates, which in-

man, 1976). One need not look far in the social sci- dicate the predsion of the point estimates.

ence literatureto find exampleswhere the conclusions

of studieswere dramatically altered when proper sam- SIMPLE RANDOM SAMPLING. Simplerandom sam-

pling methodswere used (see,e.&, Laurnann, Michael, pling is the most basic form of probability sampling.

Gagnon, 6. Michaels, 1994). In &is section, we ex- With this method, elements are drawn from the pop-

plain a number of sampling methods and discuss their ulation at random, and all elements have the same

strengthsand weaknesses. In this discussion, the term chance of being selected. Simple random samplingcan

elnncnt is used to refer to the individual unit about be done with or without replacement, where replace-

which information is sought. In most studies, elements ment refers to returning selected elements to the pop-

are the people who make up the population of inter- ulation, making them eligible to be selected again. In

est, but elements can also be groups o f ~ p p l es,uch as practice, sampling without replacement (i.e., so that

families, corporations, or departments. k population is each element hasthe potential to be selectedonlyonce)

the complete group of elements to which one wishes is most common.

to generalize findings obtained from a sample.

Although conceptually a very straightforward pro-

cedure, in practice, simple random sampling is rela-

Probability Sampling

tively difficult and costly to execute. Its main disadvantage is that it requires that all members of the

There are two general classes of sampling methods: population be identified so that elements can be in-

nonprobability and probability sampling. NonprAbil- dependentlyand directly selectedfrom the full popula-

ity sampling refers to selection procedures in which el- tion listing (thesamplingframe).Oncethis hasbeen ac-

ements are not randomly selected from the population complished, the simple random sample is drawn from

or someelementshave unknown probabilitiesof being the frame by applying a seriesof random numbers that

selected. Probability sampling refers to selection proce- lead to certain elements being chosen and others not.

dures in which elements are randomly selected from In many cases,it isimpossibleor impracticalto enumer-

the sampling frame and each element has a known, ate every element of the population of interest, which

nonzero chance of being selected.This doesnot require rules out simple random sampling.

that all elements have an equal probability, nor does it

preclude some elements from having a certain (1.00) SYSTBMATIC SAMPLING. Systematic sampling is a

probability of selection. However, it does require that slightvariation of simplerandom samplingthat is more

the selection of each element must be independent of convenient to execute (e.g., see Ahlgren, 1983).Like

the selection of every other element.

simpk random sampling, systematicsamplingrequires

Probability sampling affords two important advan- that all elements be identified and listed. Based on the

tages. First, researcherscan be confidentthat a selected number of elements in the population and the desired

sample is representative of the larger population from sample size, a sampling interval is determined. For ex-

which it was drawn only when a probability sampling ample, if a population contains 20,000 elements and a

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download