Sunday, January 6, 2019

BTRTN: How We Did in Our BTRTN Midterm Election Predictions

Tom with the report card on our predictions.

The 116th Congress is now seated.  Well, sort of.  Our government is shut down and very little is actually happening (arguably that has been true for quite some time).  The garbage piles up, Congress’ approval rating sinks even further (down to 9% in a recent YouGov poll), and Donald Trump continues to astonish in his single-minded focus on building a wall that both conservative immigration experts and border patrol agents think is a very low priority on their immigration wish lists.

Image result for report cardSo, with the new Congress seated but otherwise at a standstill, more or less, we decided it might finally be a good time to give an assessment of how we did in forecasting the November, 2018 midterm elections.  Why did we wait so long?  Well, we had hopes that all of those straggler elections would be resolved so we could give a definitive final answer.  But, alas, we have one to go, North Carolina’s 9th district, which still has yet to be certified due to allegations of widespread fraud, in a very close race.  The machinations to fill the seat continue.

And so we go forward with our assessment.  We looked the results in two ways:

1.     The “overall" outcome – how did we do in projecting the total number of seats each party would end up with in the Senate, House and governorships?

2.     The “specific" outcomes – how did we do in projecting each of the 505 elections (434 House ex-NC9, 35 Senate and 36 Governor)

We also looked at our results in an “absolute” sense -- how our predictions did versus the actual results -- and also on a relative basis -- how we did compared to the more widely-known prognosticators. 

OVERALL OUTCOME

The bottom line?  It was a very good year for us at BTRTN.  Referencing the chart below:

·        We did get the “big picture” right, predicting correctly that the Dems would take over the House, the GOP would retain the Senate and the Dems would make extensive inroads in winning governorships

·        And we were quite close in the House, in particular, with the Dems picking up +40 net seats versus our forecast of +36.

·        We were not quite as happy with our Senate picks, as we thought the GOP would “hold serve” by maintaining their 51/49 majority, but they actually picked up two more seats.  (But it’s hard to feel too bad, since we correctly forecasted 33 out of 35 races.)

·        And we were pretty pleased with the governor races, where we had the Dems picking up +9 sate houses and they actually came in at +7.

BTRTN
House
Senate
Governors
Predicted % chance D takeover
78%
6%
n/a
Predicted Dem/GOP split
231/204
49/51
25/25
Actual Dem/GOP split
235/199
47/53
23/27
Predicted Dem net gain
D + 36
No change
D + 9
Actual Dem net gain
D + 40
R + 2
D + 7

All in all, it’s hard to do much better than this, especially given how many “toss-up” races there were (see below).

SPECIFIC OUTCOMES

How did we do, race-by-race? Overall we were accurate on 96% of the races, getting only 21 races wrong out of the total of 505 (excluding NC9).

Race
Total
Right
Wrong
% Right
Senate
35
33
2
94.3%
House
434
419
15
96.5%
Governor
36
32
4
88.9%
Total
505
484
21
95.8%

In the Senate, we got every race right except for Indiana and Florida. 

·         Indiana was very difficult, a toss-up race that had no polling in November, and the polling in October on average had incumbent Democrat Senator Joe Donnelly up by a measly +1.4 points over the eventual winner, Republican Mike Braun.  Donnelly was sort of an “accidental” Senator to begin with, having defeated Richard Mourdock in 2012 in a seat long held by moderate GOP mainstay Richard Lugar.  Mourdock defeated Lugar in the GOP primary as part of the Tea Party movement, and then proceeded to run a gaffe-filled campaign which included an incredible statement about abortion, when he said that rape was “something that God intended to happen.”  Braun won the race going away, 52/44.

·         Florida was the Dems’ big disappointment on Election Night, where they lost close races for both Senator and Governor.  Democrat incumbent Bill Nelson was up by +3.3 points on average over seven November polls, leading in six of them.  We had it as a toss-up, but that type of polling – right up to Election Day – usually translates into a win.  But Nelson lost in a ridiculously tight race, 50.1% to 49.1% to Governor Rick Scott – that’s about 10,000 votes in a race in which over 8 million were cast.

As for the Governors, we had four wrong, again all toss-ups:

·         Florida was a very similar story as on the Senate side, with Dem Tallahassee Mayor Andrew Gillum leading in the November polls by +4.2 points on average over Republican Ron DeSantis but losing another epically close one, 49.6% to 49.2%.

·         There was very sparse polling in Ohio, only three polls in October, and none in November.  Those three polls all favored Democrat Richard Cordray in his bid to replace termed-out GOP Governor John Kasich.  But the GOP’s Mike DeWine won this one by +4.

·         Iowa had only three polls in October and November, two favoring Democrat Fred Hubbell.  But Republican Kim Reynolds won by +3 points.   

·         Nevada went the other way.  The one November poll had the GOP’s Adam Laxalt up by +4 points, but it was Democrat Larry Sisolak who won by +4.

We won’t detail all the House races but this was perhaps our best showing, with only 15 wrong.

As you read this, you might appreciate the difficulty in getting any of these races right.  The chart below gives some dimension to that challenge.  Most of those races are, of course, considered “solid” – nearly 400 of them.  But even two of those went the other way, Oklahoma’s 5th district and South Carolina’s 1st.

We did well with the “likely” or “leans,” getting only three wrong for a 95% rate. But perhaps we are proudest of our “toss up” record, races deemed “too close to call.”  Normally you might expect to do little better than 50/50, but we got two-thirds of them correct.

Ratings
Total
Right
Wrong
% Right
Solid
394
392
2
99.5%
Likely/Lean
64
61
3
95.3%
Toss-up
47
31
16
66.0%
Total
505
484
21
95.8%

VERSUS OUR “COMPETITORS”

We are, of course, a small fish in a big pond, and one might ask how we did versus the better-known prognosticators.  The answer appears to be “very well” but this is much harder to do than one might think.

Here is the comparison as we see it.  BTRTN appears to have done the “best” in forecasting the House; Five Thirty Eight and Larry Sabato’s Crystal Ball did better on the Senate; and several of the Five Thirty Eight models did best on the gubernatorial races.  The chart compares how many seats Democrats were forecasted to emerge with in comparison to how many they actually did secure.

ACTUAL
BTRTN
Sabato
Five Thirty Eight
"Lite"
"Classic"
"Deluxe"
House
235
231
229
227
227
230
Senate
47
49
48
48
48
48
Governor
23
25
25
25
24
24

Having said that, there is some room for controversy on the Five Thirty Eight front.  Nate Silver does not make things easy.  He has three separate models, and for each he lists probability tables and you have to look very carefully to find the “mode” estimate (that is, his most likely predicted outcome).  He also has a fancy map with a colorful dot for each district.  To discern his “prediction” you have to look at each individual “toss up” race and then add them to the other data he provides on the “leans” and the “solids.”  Quite cumbersome, but that’s how I divined the numbers above (which don’t line up with his probabilistic mode predictions).  We welcome any help in deciphering his predictions, and here is the link to the charts we used for the table above.


Other predictors, like Charlie Cook, just leave many races as toss-ups, never forecasting which way they will go.  That’s no fun!

Regardless of all this, you can conclude two things:

1)     It was a very good year for pollsters and prognosticators:  there was no major miss as in 2016, and

2)     We at www.borntorunthenumbers.com hung with them best of them in the 2018 midterms, as we generally have done for the last 10 years



No comments:

Post a Comment

Leave a comment