EXAMINING THE CONSTRUCT VALIDITY OF



ON THE USE OF “BORROWED” SCALES

IN CROSS-NATIONAL RESEARCH:

A CAUTIONARY NOTE

Susan P. Douglas*

Stern School of Business

New York University

and

Edwin J. Nijssen

University of Nijmegen

Revised, April 2002

The support of the Unilever Board for this research is gratefully acknowledged.

* contact author

44 West 4th St., KMEC 7-67

New York, NY 10012-1126

Tel: (212) 998-0418

Fax: (212) 995-4221

E-mail: sdouglas@stern.nyu.edu

ON THE USE OF “BORROWED” SCALES IN CROSS-NATIONAL RESEARCH:

A CAUTIONARY NOTE

ABSTRACT

Cross-national studies may be flawed through “borrowing” scales used in domestic studies, without examining their relevance and equivalence in other countries and contexts. Examining construct equivalence is an essential first step in the design of cross-national and multi-country studies. An important aspect is examining the equivalence in meaning and salience of a construct in all countries or contexts studied. Unless this is established, erroneous or misleading conclusions about the nature and significance of the construct in that context are likely to result. The importance of examining construct equivalence in cross-national research is illustrated based on a series of studies applying the CETSCALE in the Netherlands.

Keywords: cross-national research, construct equivalence, CETSCALE

ON THE USE OF “BORROWED” SCALES IN CROSS-NATIONAL RESEARCH:

A CAUTIONARY NOTE

INTRODUCTION

With the globalization of markets and international market expansion of many companies, increasing interest has been shown in conducting cross-national or multi-country research. Surveys of this research, for example Mintu, Calantone and Gassenheimer (1994), and Sojka and Tansuhaj (1995), indicate a growth in the number of such studies. Particularly notable is the increasing diversity of environments and contexts in which marketing and consumption behavior is being studied. Many of these studies are extensions or replications of a study initially conducted in a domestic context. Often the same conceptual framework, constructs, and scales as used in the domestic study, are applied in another country or context. Observed differences between the two countries are then attributed to ‘cultural” or contextual factors.

Such cross-national or multi-country studies involve a comparison between two or more sample populations with regard to certain properties or attributes, for example, consumer attitudes or behavior (van der Vijver and Leung 1997). For example, consumers in the US, Denmark, Greece and India may be compared with regard to attitudes towards advertising (Durvasula, Andrews, Lysonski and Netemeyer 1993). Comparisons can, however, only be made with regard to properties that are equivalent. Apples and oranges, for example, can only be compared with their characteristics as fruit, neither as an orange nor as an apple. Similarly, in marketing studies, the equivalence of the marketing or consumption behavior studied has to be established for the comparison to be meaningful.

In cross-cultural research in psychology, establishing equivalence is viewed as key in making valid cross-cultural comparisons (Poortinga 1989). Equivalence is viewed as specific to a given cross-cultural comparison, and a function of the characteristics of the research instrument, and the cultural groups or contexts compared. Three different levels of equivalence have been identified: construct equivalence, measurement unit equivalence and scalar equivalence (van de Vijver and Leung 1997). Construct equivalence requires that a construct or concept occurs and has the same meaning across countries and contexts. Measurement unit equivalence requires that the measurement unit is equivalent, for example, monetary units will need to be converted to be equivalent, or temperature measured on a Celsius scale to a Kelvin scale or vice versa. Scalar equivalence requires that response to a given scale is equivalent or has the same meaning and interpretation across contexts.

In marketing, attention has been typically focused on measurement and scalar equivalence issues, (Mullen 1995, Singh 1995, Steenkamp and Baumgartner 1998). These can only be examined once data have been collected. Relatively less concern has been shown with assessing conceptual or construct equivalence in terms of the relevance of the construct in a given research context prior to data collection. Unless, however, the salience of constructs and the need for modification of the related instrument in a given context is addressed in the early stages of research, the validity and meaning of research results, is open to question and may result in erroneous or misleading conclusions about the groups compared.

The purpose of the present paper is to highlight the importance of assessing the relevance of a construct and its operationalization in each research context or setting when designing cross-cultural research. This is particularly crucial where the measurement instrument is “borrowed,” i.e., a scale developed and validated in one country or context is used in another country or context that differs with regard to certain characteristics likely to affect that construct. In this case, the researcher needs to examine whether the construct has the same underlying meaning or significance in another country or context and hence the same measurement instrument can be used to effectively tap this construct in both contexts. This issue is illustrated by a study using the CETSCALE, a measure of consumer ethnocentrism, in the Netherlands. Consumer ethnocentrism provides an appropriate construct to examine since it was initially identified in the U.S. The CETSCALE, an instrument to measure consumer ethnocentrism was initially developed and validated in the U.S. (Shimp and Sharma 1987) and has also been examined in other large industrialized countries such as Japan, Germany and France (Netemeyer, Durvasula and Lichtenstein 1991).

The Netherlands provides a markedly different context from these countries with regard to factors related to ethnocentrism, as for example, nationalism (Adorno et al 1950) and national pride. Such feelings are not strong among the Dutch who exhibit lack of concern for national sovereignty. For example, one study found that members of the Dutch Parliament were not familiar with the words of the Dutch national anthem (Scheepers, Felling and Peters 1989). In addition, the Netherlands differs from the US in aspects likely to influence consumer attitudes toward foreign products/brands, such as interaction with other nationalities and exposure to ideas and products from other cultures and nations. In particular, younger Dutch people travel extensively and are exposed to a range of ideas and influences from other countries and backgrounds.

Issues associated with examining construct salience and equivalence in different cultural contexts are first discussed, together with the dangers associated with assuming that a construct is equivalent without prior examination of the issue (Berry 1969, 1989, Leung and Zhang 1996). The construct of consumer ethnocentrism is then examined, together with its applicability in the Netherlands. The key findings of the illustrative study are next reviewed and some directions for conducting future cross-cultural research suggested.

ASSESSING CONSTRUCT EQUIVALENCE CROSS-NATIONALLY

Interest in the cross-cultural equivalence of constructs and measures has long been a central theme in cross-cultural psychology (van de Vijver and Leung 1997, Lonner and Adamopoulos 1997). Cross-cultural psychology is concerned with testing the universality and generality of psychological theories and concepts developed in one single country or context. It examines variations in attitudes and behavior in different countries to broaden and refine existing concepts, and to stimulate more rigorous conceptual and operational definitions of constructs (Poortinga 1989). Consequently, a key focus of the discipline is identifying factors that challenge the validity of cross-cultural comparisons.

In the early stages of research the conceptualization of the theoretical construct relevant to the study and the framing of research questions can give rise to issues of construct non-equivalence and bias. According to Sears (1961) conceptual or construct equivalence is defined as whether a given construct or concept has the same meaning and is equally relevant in all countries and cultures. Even where relevant, the constructs studied may not be expressed in the same ways or behavior in other countries and contexts. For example, the meaning and relevance of the self-concept or being a good son or daughter has been found to differ substantially in the Chinese and North American cultures (Ho 1996, Markus and Kitayama, 1991). In Asian cultures the self-concept encompasses relations of the self to others, as well as individual feelings of self-worth as in Western cultures. Similarly in Asian cultures being a good son or daughter includes a much broader range of duties and obligations than in Western society. This implies that the construct cannot be operationalized in the same way in different countries or cultures and that different measurement instruments will be needed.

In marketing, concern has been expressed with regard to the cross national applicability of constructs (Durvasula, Andrews, Lysonski and Netemeyer 1993, Lee and Green 1991, Parameswaran and Yaprak 1987, van Raaij 1978), but little empirical research to assess these issues has been conducted. In particular, the need to “decenter” studies, i.e. eliminate the dominance of single country or societal context in developing the conceptual framework (Werner and Campbell 1970) has seldom been considered.

Examination of construct equivalence and bias is especially critical where a theoretical construct and a related measurement instrument have been developed in a specific country and socio-cultural setting (Craig and Douglas 2000). Many attitudinal scales used in marketing such as the CETSCALE, or MARKOR (a measure of marketing orientation developed by Kohli and Jaworski 1993) have been developed in the U.S. Their equivalence in other contexts needs to be established, particularly where these contexts differ with regard to key parameters. This is important for any scale where the construct is strongly socially or culturally embedded, as, for example, shopping orientation, materialism, marketing orientation, etc.

Once data have been collected, construct non-equivalence across cultures and contexts can be assessed, based on the structural or configural equivalence of a measure, as, for example, based on internal consistency, exploratory factor analysis, structural equation modeling and other statistical techniques (Labouvie and Ruetsch 1995, Steenkamp and Baumgartner 1998). If, however, the researcher is concerned with developing a reliable and valid measure of the construct in a given context, a preliminary phase of research needs to be conducted. This should focus on examining differences in definition, in relevant domains or inappropriateness of item content across cultures (Cronbach and Meehl 1995, van de Vijver and Leung 1997). Nomological networks or convergent and discriminant validity of the construct should be studied to assess non-equivalence (Embretson 1983). These issues are next examined in relation to the CETSCALE in the Netherlands.

THE CONCEPT OF CONSUMER ETHNOCENTRISM

The concept of consumer ethnocentrism originated in the more general concept of ethnocentrism. This is defined by Webster’s dictionary as “an attitude that one’s own group (race or people) is superior.” According to LeVine and Campbell (1972), “the symbols and values of one’s own ethnic or national group become objects of pride and attachment, whereas symbols of other groups may become objects of contempt.” The concept is thus closely linked to patriotism and political-economic conservatism and has been studied in that context (Adorno, Frenkel-Brunswik, Levinson and Nevitt Sanford 1950).

As applied to consumer behavior, ethnocentrism has been viewed as beliefs held by U.S. consumers about the appropriateness, indeed morality, of purchasing foreign-made products (Shimp and Sharma 1987). Purchasing imported products is held to be wrong in that it potentially harms the domestic economy, causes loss of jobs, and is unpatriotic. In addition, to the extent that domestic products are viewed as superior, products from other countries (i.e. from outgroups) are objects of contempt to highly ethnocentric consumers.

A seventeen-item scale termed the CETSCALE was designed to measure this concept (Shimp and Sharma 1987). The scale was developed from a pool of 180 items elicited from over 800 consumers in the United States and subsequently refined in two successive studies on large samples. The reliability of the resultant seventeen-item scale was examined extensively in four large-scale studies, and showed high internal validity. It also exhibited discriminant validity relative to Adorno’s patriotism, political economic conservatism and dogmatism scales (1950). Shimp and Sharma (1987) also found a shortened ten-item version to have high internal consistency and external validity (Table 1). In addition, the CETSCALE showed predictive validity relative to general attitudes towards foreign-made products and purchase intentions of foreign-made cars.

The CETSCALE has been translated and used to assess consumer ethnocentrism in a number of other countries outside the U.S. (Sharma, Shimp and Shin 1995, Good and Huddleston 1995). The results of these studies have been somewhat conflicting. One study based on student samples in the U.S., France, Japan and West Germany found evidence of internal consistency of the scale across all four countries, and a similar unidimensional factor structure (Netemeyer, Durvasula and Lichtenstein 1991). Correlation with attitudes towards the home country as well as the importance of buying domestic products was also found. However, correlation with attitudes to buying foreign products was less strong in France, Japan and West Germany than in the U.S. Equally, Good and Huddleston (1995) found consumer ethnocentrism to predict buying intentions for domestic or foreign products in Poland, but not in Russia.

In addition, the countries where the CETSCALE has been found to be reliable, i.e. France, West Germany and Japan, are typically large highly developed countries with low levels of foreign imports, and a large internal market. This is likely to affect attitudes towards foreign products and perceptions that they are from “outgroups” (LeVine and Campbell 1972). In other countries, such as developing markets, or countries with high levels of foreign imports, conditions surrounding attitudes towards foreign products will differ (Balabanis, Diamantopoulos, Mueller and Melewar 2001). For example, in some product categories, foreign brands are positioned as high-end brands and are perceived as superior to domestic brands. Consequently, attitudes towards foreign products are positive.

Equally, in small open societies, the salience of the consumer ethnocentrism construct is open to question for several reasons. In the first place, ethnocentrism tends to be associated with feelings of nationalism and the superiority of the group to which one belongs (Adorno et al 1950). It is thus less likely to be salient in countries where nationalist feelings are less strong. Secondly, in small economies with high levels of foreign trade, imports of foreign goods and few major domestic manufacturers, feelings of consumer ethnocentrism and negative attitudes towards foreign products may be less pronounced, and have less impact on purchase behavior. Equally, they may take a different form or be expressed in different ways. For example, such attitudes may vary according to the product category. This is particularly likely to occur where domestic made or branded products are not available in various consumer product categories, and foreign brands are often market leaders.

The Netherlands provides an interesting test ground to examine this issue. The country is relatively small in terms of geographic area, i.e. 37,330 square kilometers, has a population of 14 million. It has extensive borders to the east with Germany and with Belgium to the south. In 1999, exports accounted for 55% of GDP, while foreign direct investment, a key measure of global integration, was the third highest of any country in the world (World Bank 2000). Furthermore, the major consumer goods companies, such as Unilever, Heineken, Douwe Egbert/Sara Lee and Philips are large multi-nationals that depend on markets outside the Netherlands for the majority of their sales. Over 70% of the assets of these multinationals are outside the Netherlands. As a result, they are regarded by the Dutch as ‘international” rather than national companies. Even among small companies the export sales ratio averages 45% (Central Bureau of Statistics, Netherlands).

The small size of the country and its central location also results in high levels of communication and flows of people, goods and ideas across neighboring borders. This results in awareness and openness to foreign brands and products. In numerous product markets, as, for example, cars, computers, there are no local manufacturers. Consequently, the range of consumer choice consists exclusively of foreign makes and brands. Hence, consumers readily acquire knowledge about and are familiar with different foreign brands and manufacturers.

The Dutch, particularly the younger generation, are well known for their lack of nationalist sentiments and concern for national sovereignty. The Dutch welcomed the euro, pointing out that it would make travel much easier, and enthusiastically embrace the concept of a fully integrated Europe. Patriotic groups such as the Daughters of the American Revolution are notably absent. This absence of nationalistic feelings is reinforced by a propensity to travel and ability to speak several languages. Most Dutch are fluent in English; many also speak German and some French (Wall Street Journal April 27 1998). Well over half the population take their vacations abroad, and an increasing number travel outside Europe. These factors suggest that consumer ethnocentrism, particularly insofar as it reflects attitudes about the morality of purchasing foreign products may not be salient and have limited applicability in the Netherlands. This was examined in two surveys conducted in the Netherlands.

USING THE CETSCALE IN THE NETHERLANDS

Research Approach

Some issues with regard to administering the CETSCALE in the Netherlands were initially uncovered in a consumer survey of brand preferences. The purpose of the study was to assess perceptions and associations for six international and domestic brands in two product categories, beer and soft drinks. The CETSCALE was included in the study and hypothesized to be related to preferences for domestic rather than international brands. Initial translation and pretesting of the CETSCALE raised some questions with regard to its use in the Netherlands. Subsequent analysis suggested that the construct was not structurally equivalent to that found in previous studies in the U.S.

This triggered another study focusing more explicitly on examining the applicability and dimensionality of the CETSCALE in the Netherlands. Two versions of a questionnaire were drawn up, one with a direct or literal translation of the scale and the second, a modified version adapting the statements to the Dutch context. The direct translation was intended to provide a base against which the modified questionnaire could be compared. Each questionnaire also contained a number of questions on related constructs such as patriotism, attitudes towards purchasing domestic (Dutch) products, and reluctance to buy foreign (German) products. These questionnaires were then administered to samples of consumers in the Netherlands. The findings are next discussed in more detail.

Preliminary Findings

A survey of over 1,000 Dutch consumers in three cities in the Netherlands, Nijmegen, Utrecht and Rotterdam provided some initial insights into the construct equivalence of the CETSCALE in the Netherlands. The questionnaire included a number of questions relating to preferences and associations for the six brands of beer and soft drinks, as well as some questions relating to foreign travel and involvement, and the ten-item version of the CETSCALE, using five-point Likert scales.

Some questions concerning the meaning of the CETSCALE in the Netherlands began to emerge when translating it into Dutch (Appendix A). Analysis of several iterations of pretesting revealed that the means on eight items were extremely skewed (i.e. below 2 on a 5-point Likert scale). The only exceptions were two items relating to purchase of foreign made products when domestic products were not available. The scale showed a high level of internal consistency (Cronbach’s alpha of 0.86). However, examination of the factor structure of the items suggested the existence of a two-dimensional structure rather than the unidimensional structure found by Shimp and Sharma. A principal components analysis with varimax rotation resulted in the identification of two factors (Table 2). The first factor clearly related to the core element of consumer ethnocentrism- reflecting preference for domestic products and a belief that the purchase of foreign made products would be potentially damaging to the Dutch economy. The second factor appeared to reflect a more nuanced attitude towards foreign products, i.e. that only products which were not available domestically, should be imported or purchased. Confirmatory factor analysis (LISREL 7) also showed that a two dimensional model provided a better fit for the data than a one-dimensional model. Another interesting finding was the relation of the factor structure to preferences for domestic soft drink brands. When the scale was constrained to a single factor, there was no relation to preferences for domestic soft drink brands. In the two-factor model both sets of factor scores were positively associated with preference for domestic brands.

These findings suggested that the construct of consumer ethnocentrism underlying the CETSCALE was two-dimensional in the Netherlands – one dimension consisting of core ethnocentrism items, the other of items relating to the availability of domestic products. Some differences were also observed based on age. Older consumers, especially over 40 were more likely to have negative attitudes to foreign products, though this was less marked among those with a college education. Overall, negative attitudes were less marked in the Netherlands, as compared with other countries, implying that it may be of less significance in purchase decisions, or only characterize a relatively limited number of people.

The Study

A study further probing potential construct bias in using the CETSCALE

in the Netherlands was then conducted. First, a preliminary phase of qualitative research was conducted to examine the significance and meaning of consumer ethnocentrism among Dutch consumers. This consisted of in-depth interviews relating to the construct, as also the meaning of different items and phrases used in the CETSCALE (based on a literal translation of items into Dutch). This again confirmed the problems associated with using a direct translation of the CETSCALE in the Netherlands, particularly concerning the items relating to the availability of domestic products (see Appendix A).

Based on these findings, two versions of the CETSCALE were developed (Appendix A). One was a direct or literal translation of the CETSCALE based on a back translation procedure. This was intended to provide a basis for comparison. The other was a modified version, adapting the items to clarify their meaning in the Dutch context. For example, “American products, first, last and foremost,” was translated as “In general, nothing beats products from your own country.” In both questionnaires, an additional item relating to the purchase of foreign products when domestic brands were not available was included. This was designed to balance the factor analysis and examine further the existence of a second factor relating to domestic product availability. Both versions also included two attitudinal constructs, patriotism and attitudes towards travel, to assess discriminant validity. Questions relating to two criterion constructs, preferences for domestic products and brands, and reluctance to buy foreign (German) products were also included (Appendix B).

The two versions of the questionnaire were administered to samples of 126 and 127 respondents respectively, in Nijmegen, a city in the eastern part of the Netherlands. Since Nijmegen is a university town, the educational level of the sample tended to be high; approximately half of the sample had a college education. There was also a slight bias towards younger consumers; over one-third of the sample was between 18-25 (Appendix C). The socio-economic and demographic characteristics of respondents for the two questionnaire versions were approximately the same.

Data Analysis

The means and standard deviations for the two versions of the questionnaire were first examined to assess differences with regard to the skewedness of the distribution. Next, principal components analyses of both literal and modified versions of the CETSCALE based on 10 and 11 item versions were conducted to examine the underlying dimensionality of the construct. A series of confirmatory factor analyses was then performed to further test dimensionality (Fornell and Larker 1981). Next, the relationship of the control and criterion variables with both one and two factor models of the two scale versions was examined to assess their nomological, convergent and predictive validity. These findings are next discussed in more detail.

Findings

Examination of the principal components factor analyses for both scale versions suggested that the construct was not unidimensional. In the case of the literal scale version, two factors with an eigenvalue greater than 1 were identified. One represented the core items in the CETSCALE. The second included the items relating to the purchase of foreign products when domestic products or brands were not available. A similar structure was observed for the modified scale version. These analyses supported the proposition that the consumer ethnocentrism construct was more complex in the Netherlands than had originally been found in the U.S.

Confirmatory factor analyses for both scale versions further supported these findings (Table 3). For all scale versions, the two-dimensional model provided a better fit for the data than the single-factor model.[1] Using a χ2 difference test we obtained a χ2diff = 13.7, dfdiff = 1, p 0.90 criterion if some errors are allowed to correlate within a dimension. The significant difference between the single and two-factor solution remains.

-----------------------

Country A

Figure 1

Assessing Construct Equivalence in Cross-National Research

Assess criterion and predictive validity

Assess salience in Country B based on:

•Literature review of similar/related constructs/concepts

• Discussion with local researchers (local experts)

• Conduct evaluative research, i.e.: focus groups, in-depth interviews where necessary

Literal

Translation

of Scale

Develop Modified

Version of Scale/

Construct

- broaden domain

specification

- add items

Compare Contextual Similarity

Construct/Scale Developed in Country A

Examine nomological validity of literal and modified versions

Examine internal structure via principal components and/or confirmatory factor analysis

Compare

Country B

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download