I go over the numbers and the implications, and what to draw from the results, in my CBC article here. Please read it and come back here for the nitty-gritty.
So let's get down to it.
There was a late surge for the Liberals that pushed them over the top, but some of the polls that had older data (even only a few days old) were not able to capture the wave that pushed them from between 35% and 38% to the 39.5% the party actually got. For that reason, the vote projection was also lagging a little bit. Nevertheless, it still painted an accurate picture of what was going on, particularly when considering the likely ranges.
The seat projection model did not perform up to my expectations. When the results were plugged into it, it still projected a Liberal minority - though the chances of a majority government were higher.
The Liberal result was far more comfortably within the high-to-maximum range, but was still out of the likely range of outcomes. The Conservative result falls further out of the likely range than it did initially, though the NDP's projection is improved and the Bloc's projection hardly changes.
The regional projections edge closer to the mark, but you can still see the problems the model had.
In British Columbia, the Conservatives were greatly over-estimated, with both the Liberals and NDP under-estimated as a result. Alberta would have been accurate, but the NDP would have been under-estimated in the Prairies to the benefit of the Conservatives.
In Ontario, the Conservatives would have been over-estimated at the expense of the Liberals. In Quebec, the Bloc and Conservatives would have been accurately pegged, but the New Democrats greatly over-estimated, at the expense of the Liberals again. And in Atlantic Canada, the Liberal sweep is still unimaginable - even at the maximum ranges. It is the only region of the country where the maximum ranges would have still not been enough, demonstrating how unusual it was that the Conservatives and New Democrats lost the ridings that they did.
So I think there are some things to learn from how the results were different from the projections. It is something I tackle in the CBC piece.
I have not yet gone through the riding-by-riding results, but will in the coming days.
The performance of the polls
The polls did a very good job, with Nanos Research and Forum Research taking the cake both at the national and regional levels.
The list below includes only those polls conducted and publicly released in the week before Election Day. Abacus Data and Innovative Research did not release a final poll in this period.
Nanos and Forum finished at the top of the table, the biggest mark against them being the under-estimation of the Conservative vote. Nevertheless, for all of the parties neither missed the mark by more than 1.9 points.
(This is looking at Nanos's one-day result, though the three-day result was also relatively close to the mark.)
Ipsos Reid, Mainstreet Research, and Léger were all quite close as well, putting the Liberals at 38%, the Conservatives between 30% and 33%, and the New Democrats between 21% and 22%. That slight over-estimation of the NDP vote was behind the under-estimation of the Liberals.
EKOS had less total error than Léger, but Léger told a story that was in my view closer to the reality in that it gave the Liberals an eight-point lead over the Conservatives, suggesting a majority was possible (the Liberals won by 7.6 points). EKOS's poll, which put that gap at just 3.9 points, was the biggest source of doubt in the Liberals' ability to get close to that majority mark the night before the election. The Angus Reid Institute also contributed to that doubt, and was at the bottom of the table.
Note that, as its post-election release mentions, Mainstreet Research did indeed send me their topline numbers before the votes were counted (and, as the report says, the data was also sent to Postmedia). That result would put them in the top three with Nanos and Forum. But the numbers were not published before the election results were known and I imagine they would not have been published had they been off the mark (Quito Maggi of Mainstreet says in a comment below that a commitment had been made to Postmedia to publish the results of the poll regardless of the election result). It would not have been fair of me to judge Mainstreet based on its unpublished numbers had they been wrong, so it isn't fair of the other pollsters to judge Mainstreet by its unpublished numbers because they were right.
But it should be noted that Mainstreet's final polls of the campaign (published before the date or not) were quite close to the mark.
But it should be noted that Mainstreet's final polls of the campaign (published before the date or not) were quite close to the mark.
Also note that, as Nanos did not publish any regional numbers for its one-day sample, I have graded them below according to their three-day sample. And as they combined their Alberta and Prairie results, I have left them out of the rankings for those regions.
The polls were quite good in British Columbia, though the ranges were wide. But there was no systemic over- or under-estimation of any of the parties, and in fact nearly every pollster was almost exactly on the mark for the Conservatives and NDP. Insights West had the best result by a wide margin.
There was a systemic under-estimation of the Conservatives in Alberta, which seemed to benefit the NDP more than it did the Liberals, who were well-gauged. Angus Reid, Ipsos Reid, Mainstreet, and Forum did well here.
Considering the small sample sizes, the polls did very well in the Prairies, with the results falling well within the ranges of the polls done in the region. Léger had the best results.
Ontario was very well polled. All the pollsters had the Liberals between 43% and 46%, when the party took just under 45%. They under-estimated the Conservatives across the board, however, and in most cases it was the NDP that got the extra points. Nanos and Forum did well here.
Quebec was the miss of the election. Only Forum had the Liberals anywhere near the 35.7% the party took, with all the other pollsters under-estimating the Liberals by a significant mount. The NDP was better gauged by most pollsters, though some over-estimated them. The Conservatives were mostly over-estimated, while the Bloc was generally well polled. Nanos and Forum did the best here.
And in Atlantic Canada, the polls either got close to the Liberal result or under-estimated them. With few exceptions, the polls over-estimated the NDP and Conservatives across the board.
On the whole, though, the polls did a very good job. The only real error was in Quebec, and that was enough to make the difference between the 38% the Liberals were pegged at nationally in half the polls published in the last week and the 39.5% the party actually took on election night. That is a performance the pollsters can be proud of.
I'll have more on how the seat projections performed at the riding level soon. That should be illuminating in why the projections missed the mark.
