A minority result for the PQ was always a distinct possibility, but it turned out to be a lot smaller than expected. The Liberals were best positioned to finish second, but they out-performed expectations to a significant degree. And the CAQ, despite a good haul of votes, did not manage to win as many seats as they might have and a few of their star candidates did not win in their ridings.
ThreeHundredEight's projection gave a slight edge to a PQ majority, but this overall result was envisioned. The problem with the projection was that the Liberals were under-estimated in the polls, which meant that the projected results over-estimated both the PQ and the CAQ.
The Quebec election was not a repeat of the Alberta election. Equating the two would be a silly mistake. The polls in Quebec were generally within the margin of error for the PQ, CAQ, and QS and they only under-estimated the Liberals. The polls from the Alberta election gave Wildrose a 10-point lead when the result was a 10-point win for the Progressive Conservatives. Nothing close to that occurred last Tuesday.
In terms of the Alberta election, the question of "what went wrong?" can certainly be asked. The public polls did not capture a swing in the last week that internal polls did manage to identify. The question that should be asked in Quebec is "why were the results different?"
Polls are often erroneously seen as simply a way to predict an outcome. They are, of course, a snapshot and things can change - but voting intentions rarely shift to a significant degree in a matter of days. That is why results usually match the last polls of a campaign. But polls are most useful as a way to understand and contextualize what is going on during an election campaign. After the election is over, they can be a very good tool to understand what happened. And projections based on polling data do the same.
Of course, polls and projections will always have a degree of error and I will take a look at the errors the projection made below. But working from the idea (which has been proven over time, notwithstanding the few exceptions) that in a normal election polls can track the voting intentions of the population accurately and that projections can turn those numbers into accurate seat counts, a comparison of polls/projections to results paints a very good picture, not of how polls and projections failed, but of what worked and what didn't for the parties themselves.
How would the seat projection model have worked had the polls been bang-on in each of the model's six regions? Polls for three of them (Montreal island, Montreal suburbs, and Quebec City) were available throughout the campaign while polls for the other three (Eastern, Central, and Western Quebec) were available for some of the polls.
The model would have awarded the Parti Québécois 53 seats, the Liberals 42, the CAQ 28, and two to Québec Solidaire. In other words, a strong result for the PQ and QS and an under-estimation of Liberal strength to the benefit of the CAQ.
The riding accuracy, at 81.6%, would not have changed but the potential winners would have been identified in 94.4% of ridings when using the seat projection ranges.
Those ranges (using the polling volatility in the last week of the campaign as a guide, as the model normally does) would have encompassed the final result: 41-65 for the PQ, 34-55 for the Liberals, 18-38 for the CAQ, and 1-2 for QS. The projection would have called for a PQ minority, a strong Liberal opposition, and a third place finish for the CAQ, while the odds would have been considered very unlikely for a PQ majority or a CAQ second-place finish. This would have been a good call.
This is where the results of the election and how they differ from polls and projections can tell us something about what went right for the Liberal campaign. In the Alberta election, to take one example, the projection model would have done very well with accurate polls with one exception - the extraordinary resilience of Liberal incumbents in Edmonton and (especially) Calgary. With the Liberal vote dropping so steeply in Alberta as a whole, no projection model that wasn't based on a hunch could have forecast that the Liberal incumbents in these two cities would prove so difficult to topple. While that was an example of the projection model being wrong, it said more about the Liberals in those ridings than it did about the model itself.
 |
| Region vote projections vs. results |
The performance of the Liberals in central Quebec was quite remarkable. Polling data that was available for the region in the mid-point of the campaign, when the Liberals were far from out of the race, showed that central Quebec was not a strong part of the province for the party. It put them at 27%, behind the PQ and the CAQ. As their vote tanked in the polls in the regions of Quebec and among francophones (not to mention the riding polls that showed the Liberals with little support in Nicolet-Bécancour, Saint-François, Saint-Maurice, Sherbrooke, and Trois-Rivières), the projection put the Liberals at between 23% and 29% in the region. They ended up with 32.2% of the vote, beating out the CAQ (30.9%) and the PQ (28.4%).
As a result, this was the region with the most amount of errors and the region that skewed the projection against the Liberals most importantly. Their vote in this highly francophone region turned out in great numbers, voting for their incumbent MNAs in particular. Along with a surprising degree of resilience in the Montreal suburbs, particularly Laval, this was the region of Quebec that 'saved' the Liberals from a far more significant defeat.
 |
| Seat projections vs. results |
The PQ was projected to take between 38% and 42% of the vote in this region, but ended up with 35.9%. Neither the Liberals nor the CAQ were outside of their projected ranges in the suburbs, but both were at the higher end of expectations. The 450 was considered the make-or-break region for both the PQ and the CAQ, and in the end this was correct. The PQ won 15 seats instead of the projected 19, putting them at the very bottom of their expected results. They could have won as many as 24 seats in the region, enough alone to have given them a majority government.
For the CAQ, they did as well as expected in and around Montreal and Quebec City. The projection gave them seven seats in Quebec City, which they indeed won, and seven in the suburbs. They took six.
But it was in francophone Quebec outside of the two main centres where the CAQ vote did not materialize. The party won no seats in eastern Quebec instead of the projected one, one seat in western Quebec instead of the projected three, and five seats in central Quebec instead of the projected nine. But, generally speaking, the projection was quite good in estimating the CAQ's vote - and the projection model would have still given them nine more seats than they actually won if the polls had been completely accurate.
Why? The CAQ's under-performance is one case where the error that the projection made tells us something important about the CAQ. The CAQ's vote share aligned quite closely with the polls and the projection, but their seat result did not.
The reason for this is that the CAQ's vote was far more uniform than the ADQ's. If the ADQ had gotten 27.1% of the vote in this election, they very likely would have won around 28 seats. But despite the CAQ having swallowed up the ADQ and its MNAs, the party is not the same. It does use the same populist and vaguely nationalist language of Mario Dumont, but the party and its leader take a more centrist approach. That changed the party's voter profile somewhat, enough to give the CAQ relatively even support - particularly in central and western Quebec. The model expected the CAQ's vote to be more heavily concentrated in central Quebec to the detriment of the western part of the province - instead, the CAQ's vote was more uniform and that meant a lot of second place finishes.
When the next election occurs, which considering the minority government in place could not be very long, the model will undoubtedly perform better because it will be able to use the CAQ's vote as a baseline, rather than the ADQ's.
Another factor which made the projection over-estimate the CAQ was the adjustment made for the floor-crossers in Blainville, Deux-Montagnes, and Sanguinet. While the adjustment would have worked in most elections, for whatever reason it did not work in this one. It over-estimated the support of Daniel Ratthé, Benoit Charette, and François Rebello to a significant degree. Without the adjustment in place, the model would have correctly given Blainville to Ratthé but Deux-Montagnes and Sanguinet to the PQ. The results for each party would have been quite accurate - indicating that the floor-crossers did not personally bring any large number of votes with them.
If we look closely at the results, the projection did well on some of the minor details. Québec Solidaire's results were projected very well, with the result of 12% on the island of Montreal (their most important region) being off of the projection by 0.1 point. The model also did quite well with Option Nationale, the Greens, and the other parties.
The projected ranges were correct for every party in some tough-to-call ridings as well: Jean-Lesage (correctly forecasting a close three-way race), Groulx, and Gouin, among others. Generally speaking, the model did quite well on an individual basis for each party in the ridings where the winner was called correctly. The ridings that were not called correctly often had very large errors, suggesting that fiercely local circumstances were at play in many of them.
But the turnout adjustments may not have helped matters. While it did correctly assume that QS would be over-estimated and the PLQ under-estimated, and the adjustment did not effect the projection for the CAQ significantly, it was wrong to consider the PQ under-estimated in the polls. The last surveys by CROP and Léger were excellent for the PQ, and the last surveys by Forum and EKOS over-estimated the PQ greatly.
The model made a bet that, in most elections, would have been correct. Going forward, I am not sure if this is a bet that is worthwhile to make. The adjustment seems to have caused a lot of confusion during the campaign, with readers not understanding why I had the CAQ so low and the PQ and PLQ so high, no matter how many times I explained it. Perhaps it would be best for ThreeHundredEight to limit itself to poll aggregation and what those polls would deliver in terms of seats, with the high and low ranges being used to estimate how the polls are expected to be wrong. I welcome suggestions from readers.
With the surprising performance of Jean Charest's Liberals, the Quebec election threw almost everyone for a loop. But the results might have had more to do with the Liberals' stronger organization and the mythical prime à l'urne playing a role than the errors of polls and projectors. Quebec's distinct society does seem to come with distinct elections - no one will be surprised if next time the result is, well, a surprise.
