The answer the poll provided was: maybe, but certainly not to the same extent as we've seen in other surveys. But that is a boring answer. The answer a lot of people saw in this instead was: "welp, so much for that idea."
I think that is a very simplistic way to look at the results of this poll. I'll get into that in more detail below, but first let's just take a look at the overall numbers.
The Liberals dropped four points to 33% in Quebec, while the New Democrats were unchanged at 30%.
The Bloc Québécois was also steady, at 17%. The Conservatives picked up three points to hit 16%, and the Greens were unchanged at 4%.
The Liberals have been wobbling back and forth in CROP's polling for some time, as shown in the last six polls from the company: 38%, 34%, 37%, 32%, 37%, 33%. This recent drop would seem to fit into that oscillation.
But what about the Conservatives?
The increase of three points is within the margin of error (or would be, of similarly sized probabilistic samples), so it could just be a statistical fluke. I think, however, that with the gains we have seen in other polls it stands to reason that it isn't a statistical fluke. In CROP's previous 10 polls, for instance, the Conservatives averaged just 13%.
And the Tories made gains throughout the province. They were up among francophones and non-francophones, made a significant increase on the island of Montreal, and inched upwards in Quebec City and the regions of Quebec. Only in the 'couronne', where they fell by a single point, did the party take a step backwards. And 15% of respondents picked Stephen Harper as the preferred person to be prime minister, his best result in a CROP poll since at least June 2013.
Nevertheless, that 16% is well below the current aggregate of 21%, and the 23% to 26% we've seen in the last two polls by EKOS. That would seem to settle it, then, yes? Much ado about nothing in Quebec. A large sample poll from a Quebec-based company showing a marginal increase is a very strong argument against Conservative gains in the province.
But I don't think we should stop there. A lot of people have accepted this CROP poll as the be-all-and-end-all of polling Quebec. I think that is a bit much. CROP is a good pollster. But they are far from the decisive and conclusive voice.
Consider that CROP hasn't been tested all that much in recent elections. In the 2011 federal election, the final CROP poll exited the field on April 20, almost two weeks before Election Day. In the 2014 provincial election, CROP was out of the field on March 16, more than three weeks before Election Day. In recent contests, only in 2012 did CROP put out a late campaign poll. But it was one conducted over the telephone, which tells us little about the accuracy of their online panel.
Much is also made of the large sample size. In this poll, CROP gathered the opinions of 884 decided voters. A standard national poll would have less than 250 responses from the province.
But the last three Forum and EKOS polls that have shown the Conservative gains in Quebec sampled a total of 1,037 decided voters. So sample size is not the issue. The reliability of the sample is another question entirely, however. Whether it be some 900 panelists or some 1,000 Quebecers willing to take an automated telephone survey, if the sample itself is of poor quality it doesn't matter how many people are interviewed.
And this brings up another interesting question. Is there a methodological difference in the results? Since the beginning of 2015, the Conservatives have averaged 23% support in Quebec in six IVR polls. In four online polls, they have averaged just 17%. Could this be a sort of 'shy Tory' effect in Quebec, an issue Joël-Denis Bellavance mentioned last week on CTV's PowerPlay? Is there a reason that online panels would be influenced by that while IVR polls would not be? In 2012, the closest thing to that would have been a 'shy Charest' effect, and the online polls did post lower Liberal numbers than Forum's IVR surveys.
All of this is not to say that CROP's results should be discounted - far from it. But it should make you ponder whether one poll really has such a monopoly on the truth. Instead, each poll adds to what we know, and CROP gives us a very good piece of information. It tells us Forum and EKOS have probably been too high for the Tories. But CROP might still be too low. Methodological influences and house effects can be very important.
Take, for example, the last time CROP was in the field in December. At around the same time, six other polls had been conducted. Look at the differences between the consensus of those six polls (the average) and CROP's findings:
Does this mean the exact same thing could be happening with this latest poll? Not necessarily. And does it mean that CROP is wrong while the others are right? Again, no. What it does mean is that each pollster has what is called a 'house effect', a methodological bias caused by any number of sources: mode of contact, the people sampled, the questions asked, the weightings applied, etc.
This is the benefit of using an aggregate of polls. It can iron out these differences, and get us closer to what might be the truth. CROP's polls are very valuable for their large samples, regional breakdowns, and local knowledge. But that doesn't mean other polls are clueless - in fact, the non-Quebec-based pollsters did very well in Quebec in 2011.
The regional breakdown
Let's get back to the poll itself.
The New Democrats held on to the lead among francophones with 32%, narrowly ahead of the Liberals at 29%. The Bloc was at 20%, while the Conservatives were at 15%. Note, though, that for the NDP they are polling quite a bit lower than the 35% to 39% recorded by CROP in polls done between June and November of last year.
Among non-francophones, the Liberals had a 13-point drop to 49%, with the NDP steady at 19% and the Conservatives at 18%. The Bloc made a big jump of 10 points to 10%, suggesting that in this case we may be looking at a statistical anomaly since it seems unlikely that about 1 in 6 non-francophones surveyed in December decided to de-camp from the Liberals and head to the Bloc.
The Liberals led in and around Montreal, with 34% on the island and 44% around it. The New Democrats were not far behind in Montreal with 31%, though they dropped eight points to 24% in the 'couronne'. The Conservatives placed third on the island of Montreal with a jump of 11 points to 18%, but were in fourth behind the Bloc (20%) with 9% in the suburbs.
The Conservatives held the lead in Quebec City, though, with 38% support. That was virtually unchanged from December, and the Tories have not polled so highly here in two consecutive CROP surveys since the beginning of 2012. The NDP was at 33%, while the Liberals were down to 16% support.
In the rest of the province, the NDP was narrowly ahead with 31% as the Liberals dropped to 30%. The Bloc was steady at 20%, while the Conservatives were at 14%.
On who would make the best prime minister, Thomas Mulcair was in front with 25%. Justin Trudeau experienced a big tumble, dropping five points to 23%. As mentioned, Harper was up to 15%, a three-point increase.
If we ignored what other surveys have been saying, how would we look at this poll? For the most part, we'd consider it par for the course. These are the sorts of numbers CROP has been putting out for months. A close race between the NDP and Liberals, but with the NDP ahead among francophones. The Conservatives up slightly, which we'd consider just a wobble, but good numbers in Quebec City. The Bloc continuing to flounder.
Overall, we'd probably consider it a decent poll for all three federalist parties. The Liberals still lead, and look well-positioned for gains in and around Montreal (the drop they experienced on the island was due to those odd results among non-francophones, which are sure to be reset with the next poll). The NDP still leads among francophones, and so should retain the bulk of their seats. The Conservatives could make big gains in Quebec City.
But in light of other polls, we can look at this in two ways. The first is to consider that the Conservatives may not be making the gains other polls have suggested they are, and that their hopes need to be tempered. The second is to see in this the same trends that other polls have recorded, and that Quebec will indeed be a battleground for all three national parties. As usual, time will tell.
