The challenge: Out-of-Home is Like the Internet



Measuring Outdoor: How Media Research Uses Traffic Research to Create a Ratings Currency

James Tobolski, Arbitron, Inc.

William McDonald, Arbitron, Inc.

Joshua Chasin, Warp Speed Marketing, Inc.

September 24, 2003

Measuring Outdoor: How Media Research Uses Traffic Research to Create a Ratings Currency

Section 1. Abstract

The field of traffic research is a high-stakes field populated by a school of serious, dedicated, educated, and passionate researchers. Traffic research is high stakes because, for example, a municipality might choose to undertake a $500 million bond offering to build a new road or bridge based on forecasts of traffic flow throughout a metropolitan or other geographic area.

But traffic research is vastly different from media research, at least as practiced in the US. Perhaps the most dramatic difference is the skew in media research toward primary data collection, whereas in traffic research a large body of empirical data already exists (government traffic counts), so much of the advanced work is done in development and deployment of statistical, predictive models.

The Out-of-Home ratings service prototyped by Arbitron in Atlanta represents a new paradigm in media research. Ultimately, of course, all ratings services derive their utility from the extent to which they provide users with insight about exposure to advertising (or, in practice, potential exposure, or “opportunity to see/hear.”) But because the core consumer behavior to be measured in an Out-of-Home audience measurement service is traffic, as opposed to media consumption, the prototype methodology in Atlanta was developed by combining the knowledge bases of two wholly different disciplines—media research and traffic research.

This hybrid approach was necessary too because of the great challenges in adequately reporting on opportunity to see for Out-of-Home in the US, where the medium is so fragmented and granular that traditional approaches to audience measurement would fall short. Indeed, the paper draws a parallel to the Internet as another medium new to audience measurement, in which traditional, existing paradigms for audience measurement are currently being challenged.

The resulting system that has been developed includes components that will be quite familiar to the US media researcher—and some that will be new and different. The primary difference is the extent to which the system relies on a data expansion algorithm, or statistical model, in order to fill all the cells (by demographic and inventory unit) in the market. As this paper will demonstrate, such an approach is not without precedent in the Out-of-Home media measurement arena; and, that it is the appropriate approach when the behavior to be projected is traffic behavior.

Perhaps the major finding presented in this paper is the fact that primary respondent data collection in and of itself is insufficient as a means for creating Out-of-Home audience estimates. Primary data collection is a vital component in such a system; however for Out-of-Home it must be supplemented by statistical modeling in order to create the level of granularity necessary for the buying and selling of advertising. This is a conclusion that several international media markets have already reached; it is a new finding within the context of US media measurement.

Finally, this paper will show how, by combining best practice media research with best practice traffic research, the system described herein is able to generate actionable estimates at an extremely granular level, for in excess of 98% of the 7,500+ inventory units in the Atlanta geography covered.

Section 2. The Challenge: When Traditional Audience Measurement Paradigms Fall Short

Out-of-Home is Like the Internet

In a recent article, Media Researcher Erwin Ephron noted that “Outdoor does not compete with TV, radio or print for the bulk of discretionary advertising dollars, because, for one thing, there is no good planning data for the medium.”[i] In the same article, in discussing potential solutions, Ephron goes on to note that: “The travel data can be either site-centric or consumer-centric. (Ironically these are the same measurement issues the Internet is debating.)”[ii]

Dr. Joseph Philport, president of the Traffic Audit Bureau (TAB), has observed that the Daily Effective Circulation counts (DECs) provided by the TAB can co-exist in harmony with a US Out-of-Home ratings service. Dr. Philport observed that there was a place for both “site-centric” (the DEC) and “user-centric” (ratings) data.[iii]

To those acquainted with the issues confronting Internet measurement, as Ephron notes, this paradigm is quite familiar. And this is appropriate—because like Internet measurement, Out-of-Home exposure measurement poses a daunting challenge to the traditional model of media audience measurement. In short, both media are so granular—the number of inventory units presenting an “opportunity to see” is so vast and fragmented—that traditional media research approaches relying on data collection among a random probability sample—followed by data cleaning and editing, and then weighting and projection to the universe-- fall short.

The problem, quite simply, is that the appropriate sample size to provide measurement with sufficient statistical reliability is a function of the average audience size of the media vehicle being measured. On the Internet, there are thousands of web sites, millions of web pages, and innumerable opportunities to serve an exposure to a web visitor. Traditional media research approaches cannot possibly hope to provide sufficiently large sample sizes to measure the granularity of Internet behavior with the same robustness that a radio ratings, TV ratings, or print ratings service does. Subsequently, Internet audience reportage has bifurcated into two approaches: site-centric, involving a census of visitors taken from traffic logs for individual sites (but which cannot be combined across site to create duplication); and user-centric, or sample-based measures, which face sample size challenges but which can generate multi-site duplication patterns.

The Internet: With respect to Internet measurement, one innovative approach to user-centric measurement has been developed by comScore. ComScore recruits respondents in a non-random fashion, enabling them to offer an Internet panel of roughly 1.5 million respondents, whose online behavior is tracked and used to generate reports. While some in the media research community may have trouble accepting a methodology that is not built around random probability sampling, comScore’s approach seems a clever way of addressing the research needs of the Internet by getting around practical and economic limits on sample size that deployment of a random probability sample would pose. While the comScore approach has not been vetted by the Media Ratings Council accreditation process, it has been subject to an ARF review.[iv]

Out-of-Home: Out-of-Home is generally a local medium, requiring measurement on the local market level. Arbitron has identified over 7,500 pieces of Out-of-home inventory in Atlanta, the first US market in which the company has deployed a test ratings service.[v] In Chicago, which is under consideration as the first Arbitron expansion market, the company has identified over 13,000 pieces of inventory.

Typically, ratings services require a cell in-tab of 30 or more respondents before reporting an estimate. With 13,000 pieces of inventory and 16 age/sex cells (18-24; 25-34; 35-44; 45-49; 50-54; 55-64; and 65+ for both males and females), this implies that an 18+ sample must be large enough to generate at least 4,680,000 inventory exposures—just among the sample! (12 cells X 30 minimum in-tab X 13,000 inventory units.) And this is a simplistic estimate; it fails to take into account the fact that the inventory on the most heavily-trafficked roads will account for a disproportionate number of the exposures (exposures to inventory in Atlanta, or any US market for that matter, are by no means distributed evenly across the inventory units.)

Subsequently there would be three important variables to consider in determining the appropriate sample size to measure a major US market, using the traditional media research paradigm:

• How many exposures or impressions can the researcher collect per respondent?

• How are an individual respondent’s exposures distributed across inventory? In other words, does a given respondent generate repeated exposures to the same set of inventory? The higher the intra-clustering effect among individual respondents, the more respondents would be required to cover a broader array of inventory units.

• How deep into the 13,000 pieces of inventory will it be necessary to report usable data, in order to have a viable service?

Under the traditional ratings service methodological approach, these three variables would yield some guidance with respect to the necessary sample sizes for meeting marketplace need.

Out-of-Home Ratings the Traditional Way: How Much is Enough?

Traditionally, media researchers consider minimum counts per survey respondent group cell as the driver in determining total sample size requirements. However, this approach for Out-of-Home would result in unreasonably (and unnecessarily) large sample sizes. (If the survey instrument was an electronic technology, such as GPS, these sample sizes would make cost unfeasible.)

The Atlanta test provided some empirical data to help estimate how large a survey sample would have to be in order to report on the Out-of-Home medium at the level of granularity to which US media buyers and sellers are accustomed.

The following assumptions are made:

• As noted above: that reportage needs to be sufficiently discrete to create the demographic cell data that serves as building blocks for assembling broader target demographics; that is, 16 age/sex breaks.

• That the vast preponderance of identified inventory must be reportable with non-zero audience estimates at the demographic cell level. (There is a direct relationship between sample size and the percentage of inventory with reportable estimates, in a traditional media measurement construct.)

Key Finding: Respondent Exposures per week: Arbitron estimates that the 50 respondents in the Atlanta test who carried personal GPS units were exposed to approximately 1500 discrete inventory units per week.[vi]

Inventory exposure distributions by inventory type: How do the 1500 exposures distribute by inventory type? For this exercise we will assume four quartiles of inventory class based on the probability that an individual respondent will be exposed to any inventory in that class in a week. This step helps us to distribute the 1500 exposures by inventory type.

To illustrate the sample size calculation, we will walk through inventory class A in table 1 below:

Table 1: Calculation of Target Sample Size

7-Day GPS Instrument

Market Total Inventory Units: 13,000

A |B |C |D |E |F |G |H |I |J | |

Inventory Class |

% of Inventory |

Inventory Distribution |Probability

Of Class Exposure |

Class Distribution |Min. Cell Size |

#

Cells |Col. F X

Col. G |Col. C X

Col. H |Col. I

Col. E | |A |25% |3250 |0.90 |563 |30 |16 |480 |1560000 |2773 | |B |25% |3250 |0.70 |438 |30 |16 |480 |1560000 |3566 | |C |25% |3250 |0.50 |313 |30 |16 |480 |1560000 |4992 | |D |25% |3250 |0.30 |188 |30 |16 |480 |1560000 |8320 | | |100% |13000 |2.40 |1500 | | | | | | |

• Column B: We are assuming an equal distribution of inventory by quartile—that is, Inventory class A is the most heavily-trafficked 25% of inventory; class B the second-most-heavily trafficked, and so on. Presumably class A would comprise the largest boards on the busiest roads.

• Column C: Given a market of 13,000 inventory units, 25%, or 3,250, will fall into each quartile.

• Column D: For each quartile, an estimated probability that a given respondent will be exposed to any inventory in that class in a week. Type A is the most heavily-trafficked; here we assume that there is a 90% probability; or, that 90% of the population will be exposed to at least one class A inventory unit in a week.

• Column E: Given a total weekly exposure of 1500 discrete units, we use the exposure probabilities in column D to distribute these 1500 respondent inventory units by quartile. The calculation here is (class probability) / (sum of class probabilities) X (total respondent inventory units per week). For class A: (.9) / (2.4) * 1500, or 563. Over a third of the inventory units to which the average respondent is exposed would fall into the highest quartile.

• Column F: Minimum in-tab cell size required for reporting. We are assuming a count of 30; conceivably the ratings service might require a larger cell in-tab for reporting estimates, driving the final required sample up.

• Column G: For each class, the 16 age/sex cells as above.

• Column H: The product of columns F and G; the bare minimum sample size required would be 480—30 per cell for each of 16 cells. If every respondent was exposed to every inventory unit, 480 would be an appropriate target sample. Obviously this is not a realistic scenario.

• Column I: Total required discrete respondent/exposures. The product of the 480 required respondents multiplied by the inventory units in the class. For type A, with 3,250 inventory units, we would require a total of 1,563,000 discrete respondent/inventory exposures. In other words, 1.56 million instances of one respondent being exposed to a discrete piece of inventory.

• Column J: Final required sample size. We divide the 1.56 million required discrete respondent/inventory exposures by the number of discrete exposures per respondent per week in the class. 563 of the discrete 1500 inventory units to which the average respondent is exposed fall into class A. Therefore, in order to report on all the class A inventory at the inventory/demographic cell level, a sample size of 2,773 respondent-weeks would be required.

Market Considerations

We have seen that in order to measure the 25% of inventory which receives the most discrete exposures, a sample would need to cover 2,773 person-weeks (i.e., 2,773 respondents providing a week of data each.) However, it is an easier task to report on the largest inventory units than to report on the lower-rated units. Arbitron market diligence in the US has led the company to conclude that a service would need to go beyond the top quartile in order to be commercially viable. Indeed the very largest units are often sold based on supply and demand; the industry’s need for ratings is largely driven by a need to develop schedule-level estimates for packages that include all types of inventory, not just the types with the most traffic.

In other words: from a user perspective, depth and granularity of data reported is an important research quality driver in Out-of-Home audience measurement.

To include the second quartile of inventory units—reporting now on the 50% inventory that is most heavily trafficked—would require a sample size of 3,566 person-weeks.

Expanding the service to cover all four quartiles of inventory would require a sample size of 8,320 person-weeks. This is over ten times as large as existing and proposed services based exclusively on 7-day GPS measurement.

The above illustration assumes equal distribution of the 16 age/sex cells within the sample. If the cells are not equally distributed in the sample (e.g., if males 18-24 represent only 5% of the total sample) than sample requirements would need to be somewhat larger.

Section 3. A Global Perspective

At this point it becomes reasonable to explore ways in which researchers in other countries have grappled with the challenges of Out-of-Home measurement. It is important to note that the media landscape varies greatly by nation, and the reader should bear this in mind. Different countries have different inventory types, sizes, tolerances for clutter, and legislation restricting placement of Out-of-Home advertising.

POSTAR:

Perhaps the best-known Out-of-Home audience measurement system is Great Britain’s POSTAR (a contraction standing for “Poster Audience Research”). POSTAR is a Joint Industry Committee that was charged with developing a way to address “an almost intractable set of problems”[vii] in Out-of-Home measurement. POSTAR is comprised, in essence, of seven steps:

1. Traffic counts: As many traffic counts as possible are obtained from the appropriate municipalities, and related to individual poster sites via neural networking. This process uses poster site characteristics to attribute counts to sites even where government data is not available.

2. Pedestrian counts: Pedestrian counts from a sample of locations are taken manually. A similar neural networking approach as above is applied to this data to project pedestrian counts for all sites.

3. Coverage calculation: A sample is taken measuring person-level travel behavior in great detail. Trips are mapped and related to individual poster sites. This enables the development of daily coverage for different OTS levels, and build of OTS levels over time.

4. Dispersion factor: Also from the above survey, a factor was developed to determine the impact of geographic dispersion of campaign inventory on the exposure patterns of campaigns.

5. Visibility Adjusted Impacts: A two-stage process, using eye tracking and time exposed to an inventory unit, to develop an adjustment to audience levels at the gross and campaign (but not the inventory-specific, apparently) level.

6. Refinements: These include adjustments to accommodate specific user geographies (exposure of persons in geography A to a campaign in geography B), to account for hours of daylight based on seasonality; and, to account for illumination.

7. Data access: Assemblage of all this data, factoring, and manipulation into a usable desktop system.[viii]

Intriguingly, the POSTAR system does not rely on the traditional media research paradigm. Outside of the pedestrian count work and the national survey to develop factors, there are no “respondents”, per se. In what may be an unfair oversimplification of POSTAR, the methodology may well be characterized as the application of factors and modeling to existent municipal traffic counts.

ROAM:

ROAM (Research on Outdoor Audience Measurement) is a system developed to provide audience measurement for Out-of-Home inventory units in the five major markets in Australia.

Australia represents a unique situation, because the Australian government mandates that an annual sample of 40,000 persons complete one-day travel diaries. Participation in this survey is akin to jury duty in the US; if called, eventually you must serve. Subsequently, these 40,000 travel diaries—placed for purposes of civic planning and engineering—enable the Australian media research community to piggyback onto the government effort by taking advantage of data from 40,000 respondents annually, already collected, tabulated—and paid for.

ROAM is an elegant system that takes the raw data from these travel diaries as input, and uses data about nature and demographics of origin and destination points as inputs into a sophisticated computer model that generates inventory-specific data with demographic detail at the inventory unit level. ROAM is provided to users with an advanced mapping system, enabling the user to “see” the performance of different campaigns against different demographics and geographies (for example, how does a campaign of inventory located in the business district work in reaching persons living in a certain suburban area?)[ix]

While Australia poses a far simpler challenge to the media researcher than the US market does—only 5 local markets to measure, and the travel diaries already in place—it is clear that ROAM wasn’t built in a day.

Other Voices, Other Developments:

In a November, 2001 presentation on measurement needs in Europe, Neil Eddleston, Managing Director of JC Deceaux noted that “data capture is not a measurement system”-- meaning that the collection of respondent-level data does not in and of itself sufficiently constitute a ratings service for Out-of-Home. This presentation further observed that GPS “in its full form” was unproven, and that in order to create a robust Out-of-Home measurement and reporting system, “Data modeling is essential.”[x]

Finally, a significant development in global Out-of-Home measurement took place on June 20, 2003 when the ARF and ESOMAR included a day on “Ambient Media” in their Symposium on Worldwide Audience Measurement. Several relevant papers were presented:

• Peter Kloprogge, Marcel Van der Kooi, and Lex van Meers reported on the Dutch Out-of-Home system. Similar to the Australian system, the Dutch system is able to take advantage of government-commissioned work (an annual one-day travel study of 120,000 persons collecting time, start point, end point, travel mode, and motivation for each trip.) Route generation modeling is performed based on these data points, plus respondent demographics and infrastructure data.[xi]

• Mary Falbo and Joanne Van der Burgt presented on the use of GPS by the Canadian Outdoor Measurement Bureau (COMB) for developing reach/frequency modeling for Out-of-Home in Canada. The Canadian GPS work, involving samples of 100 in each of three cities, was used to calibrate existing reach models. Future work is intended to produce a national data base of driving patterns, and hence exposures to outdoor advertising, by geographic and demographic targets.[xii]

• Andrea Mezzasalma presented the findings of a GPS-based methodology in a collection of cities and towns in Italy. With respect to sample size issues, Mezzasalma noted that in the town of Messina, every inventory unit had at least a 3% reported net reach, based on a sample of 624 person-weeks (312 respondents keeping the GPS device for 2 weeks each.) However, this data is for total universe, not for discrete age/sex cells; additionally, the Messina work included only 120 pieces of inventory.[xiii]

In Summary:

It becomes clear that in the field of Out-of-Home audience measurement around the world, measurement and reporting systems are being developed that rely on statistical modeling heavily as a core, indispensable component. Models are being developed for audience measurement and reportage, as well as for refinement of reach and frequency calculations. Often these models use GPS data as inputs, or drivers; although in the case of ROAM, and in the Netherlands, the modeling takes advantage of survey instruments placed by the government.

GPS appears to be a technology that is gaining international prominence as a data collection instrument for measuring Out-of-Home advertising exposure. In most cases, though, limited sample sizes are restricting application of GPS to total (e.g., persons 18+) universe projections; or, to the development of broader behavioral patterns which may then be incorporated into models or existing systems.

Section 4. When Worlds Collide: US Media Researchers and Traffic Researchers Working Together

Billboards Are Not Magazines

In thinking about the development of an Out-of-Home ratings service, it quickly became apparent that Out-of-Home was uniquely different than any other medium.

It is axiomatic to state that the nature of this difference is that a billboard or poster does not have any “content”; that the ad exists in an independent state free of any medium. TV commercials are embedded within TV shows; magazine ads are embedded within magazines.

But for the most part, Out-of-Home advertising exists on a road, on a moving vehicle, in a rail or bus station, in an airport, along a city street. In developing a media ratings service, typically the researcher measures exposure to media vehicles; or, more precisely, endeavors to measure and report on the behavior of media usage. For broadcast, print, and online research, the somewhat insular world of the media researcher is quite sufficient for developing an audience measurement service. Media researchers know how to research media behavior.

But in Out-of-Home, the relevant behavior that requires measurement and reportage is not readership or viewing. It is not really media consumption at all. Rather, the relevant behavior is traffic behavior; how people move through and use the travel grid within a market. Media researchers—even the best ones—cannot profess to possess the same expertise in traffic research as traffic researchers do. (It may be worth noting that the opposite is also true.)

Subsequently, when Arbitron began exploration of the development of an Out-of-Home ratings service that would be able to provide a depth of detail on as wide a breadth of inventory as possible, the company quickly came to the conclusion that the task at hand might require a new approach to development of a media audience measurement currency service—one informed by media research best practices, as well as best practices in traffic research. A significant contributing factor to this conclusion was the survey of global best practices for measuring Out-of-Home audiences as described above.

In particular, the traditional US media research paradigm of input-throughput-output represented by random probability respondent data collection (input); data cleaning, editing, weighting, and projection to universe (throughput); and delivery to the user’s desktop (output) would prove insufficient to the task at hand.

With some inspiration from the international landscape, and in particular POSTAR and ROAM, Arbitron turned to the field of traffic research in order to develop a hybrid best-practice approach.

Exposing Traffic Researchers to Media Constructs

In undertaking an overview of traffic research, Arbitron quickly discovered two things:

• There was an abundance of public domain data available about traffic behavior. These included municipal road segment traffic counts, and government transportation studies. (The interested reader might want to visit the website for the US Bureau of Transportation Statistics: .)

• The traffic research domain was dominated by sophisticated modeling and data expansion algorithms.

The data components that Arbitron had identified as necessary for reporting on Out-of-Home advertising exposure at the demographic/inventory level were as follows:

• One-day unduplicated reach

• Seven-day unduplicated reach

• One-day gross impressions

• Seven-day gross impressions

All other “media math”—including schedule reach, frequency, Gross Rating Points, etc.--would be easily derived from these data, and from US Census-based universe estimates.[xiv] This assumes, of course, that:

• The smallest increment of reporting would be the single day. The potential exists in the future to report at the daypart level by classifying trips based on time of day.

• Schedule reach in excess of seven days (e.g., 28 days) can be calculated using a negative beta binomial model driven by the single-day and seven-day data points.

Frequency data may be derived by dividing gross impressions by unduplicated reach.

The Traffic Research Perspective

The traffic research perspective on the needs of media measurement was encouraging, enlightening, and “out-of-the-box” with respect to US media research deployment. While in general the traffic researchers with whom we consulted were fascinated by the application of traffic data within an advertising construct (as opposed to an engineering construct) and well-versed in the data collection instruments under consideration (travel logs, GPS technology), the traffic research paradigm suggested that the best course was to model the above statistics based on existing government traffic counts and apply industry models of traffic flow, modified to generate the outputs required for media math.

Additionally, the necessary sample sizes required to generate the required volume and granularity of data with sufficient statistical robustness, Arbitron was assured, would be so large as to be technically and economically unpractical for developing an Out-of-Home ratings service.

The Media Researchers Respond

Arbitron was intrigued.

Despite the fact that POSTAR has been successful, and much lauded, for creating currency ratings data for Out-of-Home with essentially no ongoing primary data collection[xv] (save of course for the otherwise unavailable pedestrian counts), Arbitron recognized that there are several very good reasons for basing an Out-of-Home ratings service on primary data collection:

• Systems based solely on traffic flow models tend to be unable to accommodate issues of duplication of exposure, which are of paramount importance in developing “media math.” In media terms, primary data collection is necessary in order to disentangle Gross Impressions into the component parts of reach and frequency. Direct linkages are required between who is traveling, where they travel, and to what inventory units they are exposed at the persons level (the introduction of consumer-centric, as opposed to location-centric, measures.)

• Regular updating of primary data collection would allow the ratings service to capture marketplace and environmental changes in behavior stemming from things like new construction (shopping malls and places of employment serve as attractors in travel behavior); changes in the business climate; marketplace demographic shifts (e.g., neighborhood gentrification would alter traffic behavior with respect to demographic and socio-economic characteristics).

• Changes in available transit options would change the patterns of travel. For example, the introduction of a new bus route or service into a market would alter the transportation patterns of persons in that market, shifting trips from automobile or pedestrian travel to bus.

On the other side of the coin, however, there are some compelling arguments in favor of a model-based approach:

• First and foremost: the ability of modeling to generate data at a discrete level for the vast preponderance of inventory units.

• The opportunity to introduce a system into the marketplace which would be economically viable (without sacrificing granularity.)

• A model-based approach could be rolled out into far more markets than one that depended solely on primary data collection and traditional projection. This is especially true when the primary data collection mode is GPS, since electronic measurement is generally the most costly approach to media research data collection. A model-driven methodology can be scaled for smaller markets by making adjustments that would require a lesser dependence on the primary data collection components (and in particular, on the more costly GPS primary data component).

Section 5. The Atlanta Test: The Approach

The Four Components

Not surprisingly, when Arbitron weighed all these factors, the company developed a hybrid, multi-phased approach for testing in the Atlanta market. The original test included four primary components:

• The deployment of 1304 travel logs;

• A GPS sample of 50 respondents (targeted for selection from among the travel log responders);

• A proprietary data expansion algorithm, or model;

• Government data. The government data included traffic counts by road segment, and demographic data from the Census Bureau.

The travel log collected three days of trip detail at the origin/destination level. Logs were completed online or on paper. The default option was online data collection; the pre-contact placement letter provided a URL, and if the respondent began and completed the task online, no additional contact was necessary. If the respondent did not proceed to the web site, he or she was offered a choice of traditional paper travel logs or online, interactive travel logs during the recruitment call.

Illustration 1: A Picture of the Process

Note that Internet data collection was deployed, but the sample frame was persons 18+ in telephone households. Respondents did not have to have Internet access to participate, and as a result there was no Internet frame bias. With the exception of Men 65+, Women 55-64, and Women 65+, every age/sex cell had more logs completed online than via paper.

Table 2: Travel Log In-Tab

Paper versus Internet by Age/Sex Cell

How Does the Model Work?

Intellectual property issues and competitive concerns dictate that we walk a fine line in describing the way the methodology works. However, within these constraints, we believe we can satisfy the reader’s curiosity.

The challenge at hand was ultimately to develop a means for taking the volumes of available data, and placing or repurposing it into a media context (traffic engineers and researchers do not look at roads or transit systems as advertising media, nor do they concern themselves with matters such as duplication.)

Additionally, the objective of this test was to measure and report on the exposure to Out-of-Home inventory specifically of persons 18+ who lived in the Atlanta “metro-plus” geography comprising the test universe. Therefore Arbitron had to be sure to exclude traffic that originated outside of the market geography. This is a core component of the difference between site-centric and respondent-centric measures. The TAB’s DEC counts are based on all traffic passing a location regardless of whether it originates in-market. Arbitron was endeavoring to report on a universe of persons 18+ in Atlanta, and the exposure of these persons to inventory within the geography. In other words, the DEC includes through traffic; the ratings data would not.

Creating the Media Math: The Atlanta test methodology integrates and projects GPS and travel log respondent data, traffic data and modeling, US Census data, and outdoor inventory locations, in order to produce estimates of reach and frequency by demographic group. This process provides audience numbers to more than 7,500 inventory locations in Atlanta. The overall approach that Arbitron uses to create Media Math numbers for outdoor inventory ratings is diagrammed below. The Media Math also contains the building blocks for creating media plans by combining locations and days, for target audience demographics.

Arbitron’s proprietary approach uses survey GPS and travel log data, to calculate reach as the net number of persons making trips by outdoor inventory locations. Frequency estimates come from traffic model total impressions divided by those reach estimates.

During the process of computing travel routes (based on trip origin and destination zones) from GPS and travel log data, Arbitron assigns demographics to those paths by applying respondent data to road segments having outdoor media. Exposure frequency is estimated as demographically weighted gross impressions divided by reach for each surveyed road segment with inventory. Values are expressed in percentages of the population for specific demographic categories for each road segment with inventory, followed by data integration and projections of those estimates to all Atlanta outdoor inventory locations.

Arbitron applies a Negative Binominal (Gamma-Poisson) Model to those estimates to arrive at one-day and seven-day outdoor audience ratings. For the Atlanta test we have not provided data beyond 7 days; however this technique will be used to go beyond one week to project 28-day ratings. This involves focusing on the Poisson exposure distribution for any one individual and the Gamma distribution of individual Poisson rates across the population. The model has two parameters: Mean exposure rate in the population, which comes from the travel log and GPS data, and the variance of individual exposure rates about the mean, which comes from the variance of those rates. The basic unit of analysis is road segments per day, coupled with generic descriptors for those units such as residential area, downtown, shopping area, major highway; weekday, weekend day, etc., sorted by traveler demographics and trip purpose characteristics. The application of the Negative Binomial Model produces reach and exposure frequency numbers for each demographic group and works for any combination of road segments and any number of days.

The result is audience data for each road segment with inventory in the Media Math database (in this case, for the entire Atlanta Area), which contains reach, frequency, and gross impressions for an average day and week for total persons, broken into demographics by age, gender, race/ethnicity, education, and income.

Illustration 2: Schematic Depiction of Arbitron’s Out-of-Home Ratings Media Math

Methodological Commentary: On the Efficacy of GPS

It is important to address the efficacy of GPS technology in the Out-of-Home ratings mix, especially with the media research industry calling for personal, portable, passive electronic measurement.

GPS stands for Global Positioning System. A GPS technology-based unit uses the network of global positioning satellites to triangulate respondent location based on longitude and latitude, by locking in on three different satellites (in much the same way that the epicenter of an earthquake is located based on readings from three different points.) Subsequently GPS can provide personal, portable, passive electronic measurement at a very discrete level of individual respondent movement.

Arbitron deployed a sample of 50 GPS respondents in the Atlanta test. Moving forward, the company’s expectation is that GPS will play an increasingly important role in the primary data collection component of the methodology. If the GPS unit is small, personal, portable, and has sufficient battery life to last for a week; and if the researcher can “recycle” the devices by having them returned by the respondents, then the technology can play an invaluable role in collecting traffic and travel data from individuals.

However, as observed by Neil Eddleston of JC Deceaux, it is important to note the difference between a data collection instrument and a methodology. GPS as a data collection instrument is but a part of the four-piece methodology described herein.[xvi] To the extent that primary data collection is an important part of the Out-of-Home ratings process—and in major US markets (perhaps the top-25 or 30) this will almost certainly be the case—GPS is probably the state-of-the art. However, a key finding of this paper is that primary respondent data collection in and of itself is insufficient to provide an actionable ratings service in the Out-of-Home space.

Moving Forward

Finally, it is important to note that Atlanta was a methodological proof of concept test, and to remember that by definition a methodological test is designed to yield learning. Moving forward, the balance between the component pieces as deployed in the test may very well be modified based on Arbitron’s experience working with the data and the modeling; by the accumulation of knowledge in managing and maximizing response from both GPS and travel log participants; and, by advancements in technology. With respect to technology, it is quite likely that future service rollout will rely more heavily on GPS for the primary data collection component as unit costs come down, the devices become smaller and more reliable, and battery life increases.

Section 5. The Atlanta Test: Findings

Reportable data on 98% of inventory at the demographic cell level

The most significant finding of the Atlanta test is that the four-part methodology succeeded in estimating audience for over 98% of over 7,500 inventory units identified in the survey area. No US-based initiative has ever come close to approximating this kind of coverage. These estimates included demographic-level data for 1-day gross impressions, 1-day unduplicated reach, 7-day gross impressions, and 7-day unduplicated reach. From these building blocks some of the more complex media statistics have been reportable, including the gross impressions of a showing at the demographic target level.

The next tier of deliverable would naturally be schedule-level audience data. Moving forward, software tools would be in place for aggregating schedule-level unduplicated reach, average frequency, effective frequency, and Cost per GRP.

As for some of the specific data findings for Out-of-Home exposure, Arbitron will have released the data set to Outdoor Operators, advertisers, and agencies by the time this paper is released. We expect a lively and informative industry dialogue on the reported data to be an important governor in how we proceed in modifying the test approach for commercialization.

Section 6. Conclusion

The US Out-of-Home advertising landscape is fragmented, under-measured, and by most accounts under-utilized in the advertising mix as a consequence of this under-measurement. In the Atlanta Out-of-Home ratings test, Arbitron successfully demonstrated the prototype for an audience measurement system that:

• Combines primary data collection with sophisticated modeling, reflecting the best-practice construct in place internationally;

• Incorporates the knowledge and wisdom of traffic research into a media research framework, creating a new paradigm (at least domestically) for approaching a difficult media measurement challenge;

• Provides estimates at the demographic level for over 98% of the 7,500+ identified pieces of inventory in the Atlanta market area;

• Can be cost-effectively be deployed in US local markets.

-----------------------

[i] Erwin Ephron, “Some Pajamas. A Few Thoughts On Measuring outdoor”; (2003)

[ii] ibid.

[iii] Remarks presented by Dr. Joseph Philport at the Advertising Research Foundation Out-of-Home Steering Committee meeting, June 2, 2003.

[iv] Cook, William, An ARF Methodological Review of comScore Networks, Inc; netScore, 2001; available at .

[v] Arbitron, Inc. tabulation of inventory in the Atlanta “metro plus” geography, based on input from TAB, Clear Channel, Lamar, and Viacom.

[vi] Arbitron Atlanta Out-of-Home ratings test.

[vii] IPA write-up. “POSTAR”, February 1997; bulletin number 709.

[viii] Ibid.

[ix] Brendan Cook, from the methodological CD-ROM distributed by ROAM

[x] Eddleston’s presentation may be viewed as a .pdf at:

[xi] Klopregge. Van der Kooi, and van Mears, The Dutch Outdoor Study, presented at ARF/ESOMAR, June 20, 2003.

[xii] Mary E. Falbo and Joanne Van der Burgt, The Canadian Experience in Developing an Outdoor Reach and Frequency Model, presented at ARF/ESOMAR, June 20, 2003.

[xiii] Andrea Mezzasalma, The Pros and Cons of Using GPS in Outdoor Research: The Italian Experience, presented at ARF/ESOMAR, June 20, 2003.

[xiv] Of course, factors such as actual inventory location, illumination, and directionality would have to be taken into account in generating actual estimates.

[xv] For example Robert McCann, Nielsen Media Research, noted in his address to the OAAA/TAB conference on June 19, 2003 that the advent of POSTAR nearly doubled Out-of-Home’s share of adspend in Great Britain within 5 years.

[xvi] Indeed there is a fifth part that should probably be mentioned: the collection and manipulation of inventory files in order to plot the data against the actual pieces of inventory.

-----------------------

[pic]

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download