New Brunswick post-mortem: PC surge falls short

The results of last night's provincial election - when they finally emerged - should not have been too much of a surprise. But in the end, thanks to a misleading final election eve poll, the race was not as close as it seemed it was going to be.

As the polls had suggested would be the case for over a year, Brian Gallant's Liberals won the New Brunswick provincial election and will form a majority government. They took 27 seats and 42.7% of the vote, with the Progressive Conservatives taking 21 seats and 34.7% of the vote. The Greens pulled off an upset by getting their leader, David Coon, elected. The party took 6.6% of the vote province-wide, a very respectable result for a Green party over and above their seat win.

The New Democrats, despite taking 13% of the vote, the party's best performance in its history, did not win a seat or even come particularly close to winning one. This is an anomaly of the first-past-the-post system, and suggests that the strategy of focusing on a small number of seats, perhaps even one, is the better strategy to follow when a party has no realistic shot at a broader breakthrough.

The seat count for the Liberals and Progressive Conservatives fell well within the ranges that were projected (17 to 31 for the Liberals, 18 to 32 for the PCs). But the model had suggested the PCs could edge out the Liberals with fewer votes - and that was indeed the case at various points during the counting. It was also pointed out how close things were, and that three seats were projected with just 50% confidence. Two of those seats did indeed go to the Liberals instead of the Tories, enough alone to have changed the identity of the projected overall winner.

The riding-level projections were not as accurate, with 39 of 49 being called correctly. But in only five ridings was the potential winner not identified by the likely ranges, which is a decent performance.

What happened in those five ridings? In Tracadie-Sheila, the numbers were skewed due to the performance of NDP leader Roger Duguay there in 2010. In Fredericton South and Fredericton-Grand Lake, it seems the Liberals did not get the votes they were expected to in the capital. And in Charlotte-Campobello and Madawaska Les Lacs-Edmundston, the sophomore PC incumbents under-performed significantly.

Missing the Greens' win in Fredericton South was unavoidable, as the party was only polling marginally better than its 2010 performance. Strong local campaigns by small parties (Kris Austin of the People's Alliance in Fredericton-Grand Lake is another example) are impossible to register in province-wide polling. Surprises like these two will always slip through the cracks.

Had the actual results been plugged into the model, the results would have been comfortably within the ranges, and a Liberal victory would have been projected (though at a few points in the evening I would have been very nervous). The riding-level projection would not have been any more accurate.

Why is this the case? For one, smaller provinces are harder to forecast because there are fewer votes in each riding. That means there is much more potential for local factors to swing a riding more easily.

But perhaps most important was the lack of polls, the lack of regional polling data, and the significant miss by Forum Research in its election eve survey.

Despite the small sample and age of the poll, the Corporate Research Associates' final survey of September 15-18 proved to be quite accurate, missing no party by more than the margin of error. Only the over-estimation of the Liberals by 3.3 points was relatively large, but it was still a very respectable showing. It would have been interesting, however, to see what CRA would have reported at the very end of the campaign and whether it would have come up with the same result with a larger sample.

Forum's poll was off the mark, but for one main reason: a significant over-estimation of PC support. They were closer for the Liberals than CRA was, and were also closer for the NDP. But in having the PCs over-estimated by over five points, Forum's result was well outside the margin of error and skewed the perception of the race to an important degree. Last night's result was no tie. Making an error in the numbers is always a problem, but painting an inaccurate portrait of a race compounds the error exponentially.

My projection would have been better off without the Forum poll (had Forum not published, my final projection would have had a cumulative error of just 4.3 points), but it is impossible to know beforehand which polls will perform well. It would have seemed a good idea to ignore Forum's final Alberta poll in 2012, despite the fact it was the only one that suggested Wildrose might not win. And it would have been a bad idea to prefer CRA's final poll in Nova Scotia in 2013 to Forum's. Sometimes, it is just the luck of the draw. Forum struck out last night, in part because the methodology they use invites the potential for a skewed sample. If they had polled throughout the final weekend and attempted (any or more) callbacks, they might have gotten closer to the mark.

Overall, however, the election was not a failure for the polls. CRA did a very good job and, aside from the final survey, Forum was lock-step with that firm along the way. Both firms captured the fall in NDP support and, with the exception of the final Forum poll, both suggested that the Liberals were the ones to beat as the campaign unfolded. They also captured the PC's modest gains as the campaign came to a close, but the surge the party experienced was nowhere near as large as expected. They did outperform the polls at the beginning of the campaign by a few points, but not nearly enough to seriously challenge for government (though they almost did thanks to their very efficient vote distribution).

Are there lessons to learn from this election? Emphasizing the limitations of a forecast with thin polling data is an important thing to do, and I tried to do that in my final projection. Perhaps in the future it would be a better idea to do away with precise projections when the data is thin in order not to give the illusion of certainty. In small provinces, that may be the way to go.