ARE MASS NED IATED HEALTH CAMPAIGNS EFFECTIVE



Effectiveness of

Mass-Media Health Campaigns

Mass media health campaigns have been an important strategy for health promotion and disease prevention since the 1940’s. Yet considerable debate surrounds the effectiveness of these campaigns. Mass media health communication campaigns frequently take the form of a series of television and radio public service announcements (PSAs) with collateral print materials such as posters, booklets, and brochures. Other organizations are often involved as intermediaries to help disseminate the messages of the campaigns. Mass media campaigns have been conducted on topics ranging from general health issues to specific diseases, including the following: cardiovascular health; smoking; alcohol and drug abuse; nutrition; safety; family planning; cancer control; immunization; screening; mental health; lung disease; hypertension; SIDS; Reyes syndrome; and AIDS (Flay, 1987; Hornik, 1993; Joseph, 1988; Lau, Kane, Berry, Ware & Roy, 1980; Solomon, 1982; Reardon, 1988).

The purpose of this paper is to briefly describe the literature on the effects of these campaigns focusing on a study I conducted that systematically reviews the empirical evidence of the effects of campaigns on awareness, knowledge, attitude formation, and behavior (Freimuth & Taylor, 1995). The paper will then identify three key issues, which need to be addressed by those evaluating these campaigns. .

REVIEWS OF MASS MEDIATED CAMPAIGNS

Rogers and Storey (1987) summarized the prevailing views on campaign effectiveness over the years, describing three eras of effects. The first, the era of minimal effects, reached its peak in the 1940s and 1950’s and was characterized by studies that refuted the idea that mass media could directly and consistently affect the behavior of individuals.

The second era, campaigns can succeed, followed in the 1960s and 1970s and brought the realization that campaigns could succeed if they were designed and conducted in more strategic ways. These strategies included the use of formative evaluation, audience segmentation, interpersonal communication networks, and the setting of more reasonable campaign goals.

Contemporary campaign research has reached a more balanced view of effects as evidenced in the title Rogers and Storey (1987) give to the third era, moderate effects. Increasing attention is given to small but overt behavior change and distal consequences of such changes. An Australian review of mass media health promotion campaigns in two areas, cardiovascular risk behavior and safety restraints (Redman, Spencer, and Sanson-Fisher, 1990) illustrates these moderate effects. The authors began with 24 studies but determined that only nine met their criteria for adequate evaluation methodologies. These nine were further divided into two models of media effects: media only and media as agenda-setting plus community programming. Not surprisingly, they concluded that media only campaigns had discouraging results but that most studies of media plus intensive community interventions reported significant changes in behavior. The authors, however, challenged these positive results by questioning how important the media component was to the success of such combined programs.

Hornik (1996) offered the most provocative review of the evidence about the effects of health communication campaigns. He asserted that the large community trials such as Minnesota, Stanford and COMMIT, have shown either minimal or no overall effects yet there is substantial evidence from observational studies such as the National High Blood Pressure campaign evaluation of major health effects and plausible associations between the campaign and the effects. His explanation for this contradiction is the lack of adequate exposure in most of the controlled trials. Hornik suggests that if programs work, they likely work because they activate a complex process of change in social norms rather than because they transfer knowledge.

It is probably time to consider a fourth era and that one is characterized by the use of the internet and by paid media rather than relying on public service time. It is too early to have much data from this fourth era but the White Houses’ Office of the Drug Czar’s anti-drug campaign shows some promising results as do many of the state anti-smoking campaigns.

What has been missing from these previous reviews is a systematic analysis of the size of effects achieved for different types of objectives, e.g., awareness, knowledge, attitudes, and behaviors. I addressed this gap by identifying and reviewing the extant empirical data from evaluations of mass mediated health campaigns (Freimuth & Taylor, 1995).

PROCEDURES USED IN CONDUCTING THE REVIEW

Because campaigns may have multiple objectives ranging from awareness to

behavior change, I categorized the evaluations according to McGuire’s hierarchy of effects model. McGuire (1989) offered a comprehensive information—processing model in the form of an input/output matrix. The output or dependent variables are response steps mediating persuasion and include exposure, attending, comprehending, yielding, behaving, and retaining. The independent communication variables include source, message, channel, receiver, and destination characteristics. His input/output matrix provides a framework for organizing much of the research conducted on campaigns. Certain assumptions had to be made in order to conduct such a review. The first assumption is that health campaigns conducted since 1980 have reached a point of standardization. Most health campaigns incorporate some aspects of planning and strategy selection, selecting channels and materials, developing materials and pretesting, implementation, assessing effectiveness, and feedback to refine the program. Therefore, what is important in this review is not discriminating among campaigns as to quality but instead, assuming some basic level of competency, and focusing on the size of effects the campaign generated.

The second assumption is that the evaluation methods used to produce empirical results are adequate. Some evaluations are clearly more rigorous than others and in the longer description of this analysis. I pointed out these differences, even to the extent of categorizing results according to the evaluation methods used.

An attempt was made to identify all quantitative evaluations of U.S. health campaigns published between 1980 and 1997. Standard databases were used to locate these evaluations such as Medline, Psych Abstracts, Social Science Citation Index, and Eric. Calls were also made to key organizations and agencies such as NIH, CDC, and voluntary health organizations, e.g., the American Cancer Society to ask if there were unpublished campaign evaluations available. Even with these methods, it is quite likely that I have missed some.

Each evaluation identified was examined carefully to determine what campaign objectives were measured and the size of effects achieved. If a campaign evaluation measured multiple objectives, it was included in each of the relevant tables.

RESULTS

Presents the summary data for four campaign objectives: awareness, knowledge, attitude, and behavior. In the following sections, the results for each of these objectives are discussed.

Table 1.

Average changes achieved after mass-mediated health campaigns

Campaign Objective Average Size of Change %

Awareness (N=16) 56

Knowledge change (N=15) 22

Attitude change (N-21) 8

Behavior change (N=29) 13

AWARENESS

Theoretically, the mass media are supposed to be most effective in achieving awareness. This review supports that expectation. When measuring awareness as simple recognition of the message, up to 83% levels of awareness have been reported, with a median of 48%. Although, without a pre message measure, some of this (perhaps up to 9%) may be measurement error, e.g., a desire to please the interviewer.

Ceiling effects must also be considered. If awareness is moderately high before the campaign, there are ceilings on the increases possible and probably these increases are harder to achieve. If both pre and post levels of awareness are available, increases can be calculated based on the percent of audience possible to change. For example, if awareness of the seriousness of colon cancer was 11% prior to a campaign and 40% after it, the increase, instead of being 29% would be 29% of the possible change of 89% which is 33%.

KNOWLEDGE

Knowledge gain is clearly achievable using mass mediated health campaigns. When exposure is guaranteed, dramatic increases in knowledge (as large as 60%) have been observed. When exposure is not guaranteed but the campaign can saturate a community as in the Stanford Three Community Study (Maccoby, et al., 1977) knowledge gains around 25% seem feasible. The size of these knowledge gains decrease when the campaigns are national in scope and must compete with numerous other stimuli. Still, most of the campaigns were successful in achieving some knowledge gain, although around 10% appears to be a more achievable increase. Multi—channel campaigns appear to be much more successful than single channel, especially print only campaigns.

ATTITUDES

All but four of the 21 evaluations of these health communication campaigns showed significant attitude change. The actual amount of change varied considerably. These results suggest that if exposure is insured, considerable attitude change is possible. The greatest amount of change (+38% for an AIDS video shown in waiting rooms of STD clinics (Solomon & DeJong, 1986) a case of forced exposure. The ARTA campaign (Woods, Davis, & Stover, 1991) also demonstrated considerable attitude change, an average of 20% across five attitude items, however, it must be remembered that the ARTA camp has received unusually high exposure for a PSA campaign, and was only part of extensive media coverage of HIV/AIDS. Therefore, it is impossible to know how much of that change is attributable to the campaign itself. Some of the evaluations clearly suffered from ceiling effects and the results are difficult to interpret. The surveys measuring outcomes of the Cancer Prevention Awareness campaign (USHHS, 1986), for example, found pre—campaign levels of 90+% on some of the items leaving little room to measure change. In spite of more control over airing than the typical PSA, the single channel campaigns did not achieve as much attitude change. The AD Council (1991) with its one PSA reported an average of 4.5% change on two items and the Partnership for a Drug Free America (1990) reported six percent change.

BEHAVIORS

Although behavior is normally considered one of the most difficult objectives to achieve in mediated health campaigns, the campaigns reviewed here were quite successful. Only six of the 29 behavioral change campaigns identified failed to achieve some level of change. The average change reported was 13% should be noted that these results may be biased by the tendency toward not publishing non—significant findings.

DISCUSSION

Even though mass media health campaigns are used extensively, considerable debate continues over their effectiveness. This review differed from previous ones in that it included only those campaign evaluations that collected quantitative evidence of impact and it organized these data according to campaign objectives. In general, the results confirm Rogers and Storey’s (1987) description of the era of moderate effects. As McGuire’s (1989) hierarchy of effects model would predict, the size of the effects were greater at the earlier steps, i.e., awareness, and knowledge than the later stages of attitude change, and behavior change.

KEY EVALUATION ISSUES

STANDARDS OF EVIDENCE

These results certainly support the use of mass mediated campaigns in health promotion and disease prevention programs. They also provide estimates for campaign planners to use in predicting the impact of their efforts. Reviewing these campaigns as well as examining the conclusions reached by others suggests some key evaluation issues that need to be addressed by this field. Perhaps the most significant one concerns the standards for acceptable evidence demonstrating effectiveness. There are many groups involved in efforts to systematically review research results and apply criteria for success leading to best practice guidelines. Examples include the Centers for Disease Control and Prevention’s Guide to Community Prevention and the Substance Abuse and Mental Health Agency’s PEP series. Generally, these groups set criteria where “strong” evidence equals randomized clinical trial designs. For example, consider the following three criteria of study design for assessing effectiveness for the Guide to Community Preventive Services:

• Greatest -- Concurrent comparison groups and prospective measurement of exposure and outcome

• Moderate -- All retrospective designs or multiple pre or post measurements but not concurrent comparison group

• Least -- Single pre and post measurements and no concurrent comparison group or exposure and outcome measured in a single group at the same point in time

Given the nature of mass media, it is difficult to establish true control groups. Generally, a quasi-experimental alternative is to match communities and designate one as a treatment and one as a control. Hornik (1996) argues that such designs are inappropriate for these efforts. He even suggests that the better designed the evaluation is, the worse the evidence for important effects. Contrasting the Stanford Five Community Project with the High Blood Pressure Education Program illustrates this point. The Stanford study was a tightly controlled quasi-experiment. Theoretically, the two treatment communities received extensive education and mediated messages about cardiovascular risk reduction over a five-year period contrasted to the control communities who were not exposed to these materials but who, undoubtedly received similar messages from other sources. The project estimated that the individuals in the control communities received about five hours of education each year, divided among several major behavioral objectives. The results showed only small treatment city advantages. Similar patterns have been reported for other large matched community trials such as The Minnesota Heart Health Program, the Pawtucket Heart Health Program, and the Community Intervention Trial for Smoking Cessation (COMMIT). In all of these cases, there was a strong secular trend in risk reduction resulting in significant change in the control communities. Hornik contrasts these carefully controlled designs to the High Blood Pressure Program begun in 1972, which he characterizes as a messy social diffusion effort that was highly successful. Underlying these arguments is the need for a clear understanding of how one expects a mass-mediated health communication campaign to work. If we see these campaigns as a discreet treatment such as a pill that can be delivered or withheld, then the controlled randomized design might be appropriate, but if the campaign is seen as a social process which may include deliberate communication messages which, in turn, may diffuse and effect other media and a range of institutions, then other designs are probably more appropriate. We need to turn out attention to some alternative approaches such as natural experiments and correlated time series rather than experimental and quasi-experimental designs.

MEASUREMENT

Another issue that offers many challenges when evaluating health communication campaigns is measurement. Assuming one begins with a sound logic model of how the campaign is supposed to work, what should be measured and what is the unit of measurement? Most evaluations measure individuals in some way, one’s attention, knowledge increase, attitude change, or behavioral change. Yet Hornik’s concept of a messy diffusion effort would lead one to measure many other forms of social change in addition to individual change. One could examine the impact on the media’s agenda, on social and political policies and institutions Because there is so little precedent for these kinds of measures, it might be helpful to begin with a study of a successful effort such as the National High Blood Pressure Education Program and document its impact on different parts of the social system.

Cost effectiveness is one attribute of a mass mediated campaign that is seldom measured yet would be very useful to campaign planners. For example, Hu and colleagues (1995) calculated that in California a $20,000,000 media campaign produced a decline of about 232,000,000 packs of cigarettes smoked over a two year period, or about $1 per 11 packs not smoked. Over time, their estimates suggest that each 10% increase in media time purchased would produce an additional decline of 0.5% in cigarette sales. As more of these campaigns begin to use paid media, these kinds of measures will become even more important.

Another aspect of measurement in any kind of behavior change effort is whether to depend on self-reported behavior or to actually observe behavior in some way. In general, we have probably depended on self-reported behavior too much. There are some behaviors that cannot ethically be observed such as sexual behavior but there are sometimes creative ways to assess the validity of the self-report data. For example, comparing self-reported use of condoms with actual condom sales. In the smoking area, a subset of self-report data are often validated with physiological data or reports from significant others. We need to continue to find ways to establish the credibility of the measures we use to track health status and effectiveness of our interventions.

RESOURCES FOR EVALUATION

A third and very pragmatic issue around evaluation of mass mediated health campaigns is when to evaluate and what proportion of the overall budget to spend on evaluation. Program planners are often reluctant to conduct outcome or impact evaluations because they want to use all of their resources to reach as many people as possible with the intervention. Others decide or are forced to spend a portion of their budgets on evaluation but then must decide how to divide these resources between formative and summative evaluation. It would be very helpful to have some guidance on the recommended portion of program funds to spend on evaluation and how to distribute those funds between formative and summative evaluation.

CONCLUSION

Mass media health campaigns clearly can be an effective tool for health promotion whether the effort is on a national or local scale. We should stop arguing whether they are more or less effective than other strategies or whether one channel is better than another. Instead we should carefully formulate our conceptual model of how we expect an intervention to work and then evaluate it accordingly. Health promotion interventions are not like pills – they are much more complex and indirect in the way they work. Therefore our evaluation designs may be very different allowing us to track a social influence process and document its effects on social and political institutions as well as on individuals.

REFERENCES

Flay, B. R. (1987). Selling the smokeless society: Fifty-six evaluated mass media programs and campaigns worldwide. Washington, D. C.: American Public Health Association.

Freimuth, V. S. & Taylor, M. (1995). Are mass mediated health campaigns effective? A review of the empirical evidence. Paper prepared for the National Heart, Lung and Blood Institute.

Hornik, R. (1993). Public health education and communication as policy instruments for bringing about changes in behavior. Paper prepared for meeting on behavioral and social factors in disease prevention.

Hornik, R. (1996). Public health communication: Making sense of contradictory evidence. Paper prepared for meeting on evaluation in public health communication.

Hu, T. W., Sung, H.Y, & Keeler, T. E. (1995). Reducing cigarette consumption in California: tobacco taxes as an anti-smoking media campaign. American Journal of Public Health, 85, 1218-22.

Joseph, J. G. (1988). HIV infection and the effectiveness of education for the “General Population”. Paper prepared for the Office of Technology Assessment.

Lau, R., Kane, R., Berry, S., Ware, J., & Roy, D. (1980). Channeling health: A review of the evaluation of televised health campaigns. Health Education Quarterly, 7, 56-89.

Maccoby, N., Farquhar, J. W., Wood, P. D., & Alexander, J. (1977). Reducing the risk of cardiovascular disease: Effects of a community based campaign on knowledge and behavior. Journal of Community Health, 3, 100-114.

McGuire, W. J. (1989). Theoretical Foundations of Campaigns. In R.E. Rice & C. K. Atkin (Eds.), Public Communication Campaigns (2nd ed., pp. 43-65). Newbury Park, CA: Sage Publications.

Partnership for Drug-Free America. (1990). What we’ve learned about advertising. American Association of Advertising Agencies.

Reardon, K. K. (1988). The role of persuasion in health promotion and disease prevention: Review and commentary. In J. A. Anderson (Ed.) Communication yearbook 11. (pp. 277-297), Newbury Park, CA: Sage.

Redman, S., Spencer, E. A., & Sanson-Fisher, R. W. (1990). The role of mass media in changing health-related behavior: a critical appraisal of two models. Journal of Health Promotion of Australia 7(2), 91-99.

Rogers, E. M. & Storey, J. D. (1987). Communication campaigns. In C. Berger and S. Chaffee (Eds.) Handbook of communication science. Newbury Park, CA: Sage.

Solomon, D. S. (1982). Health campaigns on television. In D. Pearl, L. Bouthilet, and J. Lazar (Eds.) Television and behavior. Washington, D. C.: NIMH Technical Reviews.

U. S. Department of Health and Human Services. (1986). Technical Report Cancer Prevention Awareness Survey Wave II. Bethesda, MD: National Institutes of Health.

Woods, D.R., Davis, D., & Westover, B.J. (1991). “American Responds to AIDS”: Its content, development process, and outcome. Public Health Reports, 106(6), 616-622.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download