FORECAST ERROR: 2020 PRESIDENTIAL ELECTION PREDICTORS

FORECAST ERROR: 2020 PRESIDENTIAL ELECTION PREDICTORS

By Timothy Martyn Hill. Edited version published online at

PART 0: TAGLINE

The Republican President Donald John Trump lost the popular vote to his Democratic rival Joseph Robinette Biden and received 232 electoral votes to Biden's 306 when the Electoral College met in December 2020. Timothy Martyn Hill reviews the predictions - and the errors - that were made

PART 1: THE ELECTION

Early in 2020 the 45th President of the United States Donald J. Trump looked forward to the coming election. A billionaire property developer who had acceded to the presidency after a surprise win in 2016, he anticipated winning his second election as most sitting first-term Presidents do. Then the pandemic happened.

In a remarkable November 3rd election in which earlier postal votes played an unprecedented role, President Trump's on-the-day lead was worn away as the postal votes were laboriously counted. Despite repeated legal challenges, the individual states certified their votes one-by-one and Joseph R. Biden won the Electoral College when it met in December 2020.

Billions of dollars had been spent, modellers had predicted, bookies had taken bets, pollsters had polled. Which of them had predicted the outcome and how far out had they done so? This article sets out to answer that question, by analysing the performance of pollsters, seat and vote modellers, and betting firms all the way up to election day 2020.

PART 2: ASSESSMENT

To assess the performances of predictors, we convert all predictions made to a two-party-forced format, meaning that the predictions for undecided voters, "don't knows", and third-party and independent candidates will be proportionally reallocated to the official Democratic and Republican candidates for president. Then, to judge the accuracy of a prediction, we use two metrics: Mean Absolute Error (MAE) and WIN (a metric that scores a prediction on whether or not it predicted the eventual winner).

For a fuller discussion of two-party forced format, our metrics and our choice of predictors, see our article on the 2016 election at this link:

PART 3: WHICH RESULTS SHALL WE MEASURE PREDICTIONS AGAINST?

The counting of votes is depicted in fiction as a simple process, but in reality it is more complicated. In normal elections, as the days progress, small errors are found or a recount changes the vote slightly. It is rare for these changes to make a difference to the final outcome, but it does introduce a slight error. Consequently even reputable media sources may disagree as to the exact number of popular votes a candidate receives, or the number of digits needed after the decimal point, so the normal process for statistical purposes is to wait until an electoral commission or a legislative body publishes the exact numbers in one document. In the US such a body would be the Federal Electoral Commission.

However, the 2020 election was not a normal election. The repeated questioning of the votes and the repeated requests by President Trump and other associated bodies for recounts meant that keeping a running total of the votes as they were counted was unusually difficult. Although the Electoral College votes were known and fixed, sources for the total popular votes disagreed and I could not find two definitive single sources that agreed on the percentage to two decimal places (see Appendix 4e in the PDF version of this

article). Consequently, I had to wait until the Federal Election Commission issued the official figures which are given in Table 1 to one decimal place.

They are presented alongside the two-party-forced version of the results, while the WIN parameter designates the winner and the post-facto probability of success.

Table 1: Estimated results at the time of writing of the 2020 US Presidential Election.

President Party

PV% 2pf WIN ECV %

2pf WIN Source

Biden

Democratic 51.3% 52.2% 1 306 56.9% 56.9% 1 [0114c]

Trump Republican 46.9% 47.8% 0 232 43.1% 43.1% 0 [0114c]

Other

Other

1.8% -

- 0

0%

-

- [0114c]

Total

100% 100%

- 538 100% 100%

- [0114c]

PART 6: HOW WELL DID OUR PREDICTORS DO?

So, how did our predictors behave? Well? Badly?

Nationwide Opinion Polls The website "538" [0105a] lists 87 distinct entities that conducted, commissioned, or published nationwide opinion polls for the 2020 Presidential election.

To match our previous article on the 2016 election, we selected the following:

? ABC News/Washington Post ? CNN ? Rasmussen ? Reuters ? NBC News

The selections were predictors of popular vote. The resulting MAEs and WINs are given in Table 2. If a predictor issues two or more predictions for a given day then we will take just one or take an average for that day.

Table 2: National opinion polls of the popular vote close to the 2020 Presidential Election

Predictor

PVB PVT Winner Source 2pfB 2pfT ResB ResT MAE WIN

ABC/WaPo 54 42 Biden [0106a] 56.3% 43.8% 52.2% 47.8% 4.1% 1

CNN

54 42 Biden [0106b] 56.3% 43.8% 52.2% 47.8% 4.1% 1

Rasmussen 48 47 Biden [0106c] 50.5% 49.5% 52.2% 47.8% 1.7% 1

Reuters/Ipsos 52 45 Biden [0106d] 53.6% 46.4% 52.2% 47.8% 1.4% 1

NBC

52 42 Biden [0106e] 55.3% 44.7% 52.2% 47.8% 3.1% 1

avg 2.9% 1

In terms of predicting a popular vote winner, our five selections were large by the standards of past POTUS elections (see PDF Appendix 11-13), although all of them predicted the eventual winner. Note, however, that four out the five overestimated Biden's lead.

Modellers And Other Predictors: Journal national predictions In October 2020 the online version of PS: Political Science and Politics (Volume 54, issue 1) listed[1018x] various predictors of the national popular vote and/or electoral vote. Combined with the author's own searches, that yielded sixteen predictors in total.

To match our previous article on the 2016 election, we selected the following predictions by:

? Abramowitz ? Erikson and Wlezien

? Enns and Lagodny (instead of Ray Fair). Ray Fair declined to predict, stating that his model "...has nothing to say about the effects of pandemics", [1018d] although he occasionally updated his outputs. Instead, we substituted Enns and Lagodny: not an exact match but their model had an economic component.

? Norpoth ? FiveThirtyEight/Silver

The selections were predictors of popular vote and of Electoral College vote. The resulting MAEs and WINs are given in Tables 3 and 4.

Table 3: Journal predictions of the Electoral College close to the 2020 Presidential Election.

Predictor

ECVB ECVT ECVO Winner Source

2pfB 2pfT ResB ResT MAE WIN

Abramowitz

319 219 0

Biden [1018a]

319 219 306 232 4.8% 1

Enns and Lagodny 290 248 0

Biden [1018p][1018x] 290 248 306 232 5.9% 1

Norpoth

176 362 0

Trump [1018m]

176 362 306 232 48.3% 0

538/Silver

347 191 0

Biden [1018e]

347 191 306 232 15.2% 1

avg 18.6% 0.75

Table 4: Journal predictions of the popular vote close to the 2020 Presidential Election.

Predictor

PVB PVT PVO Winner Source

2pfB

2pfT

Erikson&Wlezien 0.55 0.45 n/a Biden [1018c][1018x] 55.00% 45.00%

Enns&Lagodny 0.545 0.455 n/a Biden [1018p][1018x] 54.50% 45.50%

538/Silver

0.536 0.452 0.012 Biden [1018e]

54.25% 45.75%

ResB 52.2% 52.2% 52.2%

ResT 47.8% 47.8% 47.8% avg

MAE WIN 2.8% 1 2.3% 1 2.1% 1 2.4% 1

In terms of predicting a popular vote winner, our five selections were fairly reliable for popular vote, not so much for electoral college votes. All of them except Norpoth predicted a Biden win. But most overestimated the size of Biden's lead, except for Enns and Lagodny who underestimated his lead in the EC, and Norpoth who predicted a Biden loss.

In passing we note in sadness the failure of Norpoth's Primary Model. It was a simple and hitherto robust model which uses the votes cast in the presidential primaries (the process parties go through to select their candidates months before the election). If, like Fair, he had recognised the vulnerability of his model to the pandemic he might have withdrawn it.

Modellers And Other Predictors: statewide predictor aggregators Previously we had neglected statewide predictors, specifically statewide opinion polls, because their lower frequency and asynchronicity make them difficult to use. Neverthless well-resourced analysts may spend a considerable period of time investigating them and produce their own aggregated predictions based on those polls or other elements. Unlike the predictions above, which were more measured and were published in political journals, these were more ad-hoc.

There are three statewide predictor aggregators that we can use and they are ? FiveThirtyEight ? RealClearPolitics ? Sabato's Crystal Ball

The selections were predictors of Electoral College vote. The resulting MAEs and WINs are in Table 5 below.

Table 5: Predictions of EC vote made by statewide predictor aggregators close to the 2020 POTUS Election.

Predictor

ECVB ECVT ECVO Winner Source 2pfB 2pfT ResB ResT MAE WIN

FiveThirtyEight/Silver 348 190 0

Biden [0108a] 348 190 306 232 15.6% 1

RealClearPolitics

319 219 0

Biden [0108b] 319 219 306 232 4.8% 1

Sabato's Crystal Ball 321 217 0

Biden [0108c] 321 217 306 232 5.6% 1

avg 8.7% 1

The MAEs for these are not exactly great, but they are better than the more formal journal predictions and RealClearPolitics's prediction was close to the actual outcome. But, again, all overestimated Biden's lead.

Betting Odds And Spreads The website "oddschecker" [0104a] lists 20 fixed-odds and 3 exchange bookmakers that accepted bets for the 2020 Presidential Election. Combined with Ladbrokes and Coral (who had left Oddschecker earlier in the year) that yielded 25 bookmakers.

To match our previous article on the 2012 election, we selected the following:

? Betfair Sportsbook: a London-based bookie that covers fixed-odds ? Betfair Exchange: the same London-based bookie that covers exchange betting ? Ladbrokes: a London-based bookie that covers fixed-odds ? William Hill: a London-based bookie that covers fixed-odds ? PaddyPower: a Dublin-based bookie that covered fixed-odds. Merged with Betfair in 2020

The bets pay out if the selection gains a projected majority in the Electoral College (and hence the Presidency), not if the selection wins the popular vote. Consequently, all of our selections were measuring the probability of winning the Electoral College. If we compare those probabilities to the result (Biden won) then the resulting MAEs and WINs are given in Table 6.

Table 6: Odds on EC win from fixed-odds and exchange bookmakers prior to the 2020 Presidential Election.

Predictor

OddsB OddsT Winner Source DecB* DecT* 2pfB 2pfT ResB ResT MAE WIN

Betfair S'book _2/5 _2/1 Biden [1103a] 71.4% 33.3% 68.2% 31.8% 1

0

31.8% 1

Betfair Exchange _4/9 _2/1 Biden [1103a] 69.2% 33.3% 67.5% 32.5% 1

0

32.5% 1

Ladbrokes

_4/9 _13/8 Biden [1103c] 69.2% 38.1% 64.5% 35.5% 1

0

35.5% 1

William Hill

_4/9 _15/8 Biden [1103b] 69.2% 34.8% 66.6% 33.4% 1

0

33.4% 1

PaddyPower

_2/5 _2/1 Biden [1103a] 71.4% 33.3% 68.2% 31.8% 1

0

31.8% 1

* = decimal odds

avg 33.0% 1

As is the case in previous elections, the odds have a very large MAE. But please note that odds are normally measured by calibration and examining them by MAE is not the standard approach. However, that argument must be counterbalanced by the fact that if we wish to compare odds to other methods in this election, we must use a common comparator.

Again, the predictors predicted a Biden win.

PART 7: CONCLUSION

Judging from the early commentary on election night and immediately after, the 2020 US Presidential Election was to be seen as another entry in the long line of prediction failures. These "hot takes" were further complicated by the heated debate surrounding the count of postal votes, which were unusually large due to the pandemic. The AAPOR report in July 2021 reinforced this impression with respect to polls, pronouncing them the worst in 40 years for the national popular vote.[0801a]

The author notes that the vast majority of predictors predicted a Biden win see Table 7 for summary) and that the polling error was comparable to 1996. In normal circumstances this would not be enough, but given the fact that the election was held in the middle of a pandemic this must surely be considered adequate.

However, one fact is indisputable: the predictors - with only a few exceptions - overestimated Biden's lead.

Table 7: Summary of predictors close to the 2020 Presidential Election.

Predictor

# of predictors assessed Average MAE on a 2pf basis

Bookmakers

5

33.0%

Journal predictions of the EC 4

18.6%

State predictor aggregators 3

08.7%

National opinion polls

5

02.9%

Journal predictions of the PV 3

02.4%

Average WIN 1 0.75 1 1 1

This article measures how predictors performed, when compared to each other and (for the polls) when compared to previous elections. But this was an extraordinary election and how the predictors were perceived, how they were judged and the implications for modellers, pollsters, political scientists and bookmakers as the events between Election Day 2020 and Inauguration Day 2021 unfolded are outside the scope of this article.

Consequently, they will be examined in the next entry in the Forecast Error series, tentatively entitled "Forecast Error: POTUS 2020 part 3: The Aftermath"

PART 9: DISCLAIMER

The author is a statistician who works in the private sector. To pass the compliance requirements of the author's employer please note that the author's affiliation for this article is "independent" and the views expressed in this article are those of the author and in no way do they reflect the opinions of any entity or organisation with which the author is ? or has in the past, or may in the future be ? affiliated.

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download