Wednesday, February 16, 2011

Little change in Harris-Decima poll, and a comparison of recent results

Yesterday, Harris-Decima added their voice to the polling cacophony, showing very little change in their national numbers but giving further evidence of the Conservatives' double-digit lead over the Liberals.

After this report on the Harris-Decima poll, I take a look at their numbers in conjunction with those of EKOS and Ipsos-Reid. Three polls with the same inaccuracies? It seems unlikely.

In this poll, as in others, statistically significant shifts in voting intentions in Ontario are the story.First off, compared to Harris-Decima's last poll the Conservatives have a gained only a point, and now lead with 37%. The Liberals are down one to 27%, while the New Democrats are also down one point to 14%.

The Bloc Québécois and Greens have each gained a point and stand at 10%. All of these shifts are within this telephone poll's +/- 1.8% margin of error (19 times out of 20).

As Harris-Decima has reported their regional margins of error, I will not round them as I did for Ipsos-Reid's poll yesterday. I will use them, however, to reverse-engineer and estimate the poll's sample sizes for my model.

In Ontario (MOE +/- 3.1), the Conservatives have gained four points and now lead with 43%. The Liberals are unchanged at 34%, while the New Democrats are down three to 12%. The Greens are unchanged at 10%. As in other recent polls, the Conservatives have made gains in the province over-and-above any likely statistical wobbling.

The Bloc Québécois is steady at 40% in Quebec (MOE +/- 3.6), while the Conservatives (+1) and Liberals (-1) are tied at 19%. The NDP is down two to 12% in the province. More good results for the Tories.

In British Columbia (MOE +/- 5.0), the Conservatives are up one point to 35%. The Liberals, down three, trail with 29% while the NDP is down five to 19%. The Greens have gained seven points, and are in it at 16%.

The Conservatives have gained four points in Atlantic Canada (MOE +/- 5.6) and now lead with 39%, echoing Ipsos-Reid's results. The Liberals are down 13 points to 29%, while the NDP is up nine to 24%.

In Alberta (MOE +/- 5.6), the Conservatives have dropped five points to 56%, while the Liberals are up three to 21%. The only noteworthy shift here is a six point gain for the NDP, who stand at 12%.

And in the Prairies (MOE +/- 5.6), the Conservatives have dropped only one point and lead with 48%. The Liberals are up three to 22%, while the NDP is steady at 20%.

With this poll, the Conservatives would win 20 seats in British Columbia, 27 in Alberta, 21 in the Prairies, 61 in Ontario, eight in Quebec, and 12 in Atlantic Canada for a total of 150. That's 12 more than in my last projection for a Harris-Decima poll.

The Liberals would win 12 seats in British Columbia, one in Alberta, four in the Prairies, 37 in Ontario, 13 in Quebec, and 16 in Atlantic Canada for a total of 85, down nine.

The Bloc Québécois would win 53 seats in Quebec, unchanged.

And the New Democrats would win four seats in British Columbia, none in Alberta, three in the Prairies, eight in Ontario, one in Quebec, and four in Atlantic Canada for a total of 20, down three.

Harris-Decima also looked into leadership, finding that 46% of Canadians have a favourable impression of Stephen Harper. Another 43% have an unfavourable impression, and according to the press release this is the first time Harper has had a net positive rating since November 2009.

Michael Ignatieff, on the other hand, is at his lowest level ever at 25% favourable and 51% unfavourable.

Jack Layton splits 44% to 38%, Elizabeth May splits 25% to 30%, and Gilles Duceppe splits 53% to 30% (in Quebec only).

A comparison of recent polls

When EKOS showed a 12.5-point lead for the Tories on Friday, the Conservatives objected and the poll was called an outlier. Then Ipsos-Reid came along and confirmed EKOS's numbers on Monday, and Harris-Decima did the same on Tuesday.

But what of their methodological limitations? The inability to get a truly random sample? The regional sub-samples being too small to tell us anything? Why, these polls are unreliable and we should just stop talking about them!

Well, no. Polls aren't perfect, and some are less perfect than others, and there are some methodological hurdles to overcome in the future. But polls still work, and are a reasonable measure of public opinion. The proof is in the pudding, as they say, and all three of the polling firms that have reported over the last week (and who were in the field on overlapping days) have told the same story of a shift in public opinion.

Not even the small regional sub-samples varied widely. Ipsos-Reid, EKOS, and Harris-Decima all conduct their surveys by telephone, but not in the same way. Ipsos-Reid and Harris-Decima appear to use traditional telephone methods, while EKOS uses an IVR system which reaches both landlines and cell phones. And sample sizes were very different: EKOS's poll surveyed 1,652 people, Ipsos-Reid's surveyed 1,001 people, and Harris-Decima surveyed 3,025 people. For a small region like Atlantic Canada, the MOE varied between 14% in the Ipsos-Reid poll to only 5.6% in Harris-Decima's poll. And yet...We see very little difference in their results for the Conservatives and the Liberals. What's more, there is even little difference in how the vote has shifted in each of their polls.

In British Columbia, the Conservatives in each poll have gained compared to the last polls conducted by these firms, previous polls which were all conducted at about the same time in January. The Liberals lost support in all three polls as well.

In Ontario, the Conservatives gained between 4 and 6 points in all three polls, while the Liberals either held steady, lost four points, or lost eight points. That tells a significant story of Tory gain and Liberal loss, and in Quebec the Conservatives have gained in every poll.

At the national level, the results were 37%, 39%, and 37% for the Conservatives and 25%, 25%, and 27% for the Liberals.

Clearly, these guys must know what they're doing. Or they are all wrong in the exact same way.

But does it matter? An election could be two or three months away, or more than a year away. What do these polls say about how Canadians will vote at the point? Well, not much - but that isn't what these polls are arguing. And that doesn't make them completely useless.

Oddly enough, a similar discussion is taking place in the United States. Nate Silver of FiveThirtyEight recently wrote on his blog about some presidential primary polls. He was criticized for it, as these curmudgeons argued that the polls were meaningless. I'll quote what Nate had to say on the issue today. The parallels between the American pre-primary season and this Canadian pre-writ period are obvious. The full post can be found here.

However, it is clearly incorrect to suggest that a candidates’ post-primary favorability numbers cannot be predicted at all by what they are early on. For the 18 candidates in the study who actually endured a primary (not counting George W. Bush in 2004, who did not), the correlation of the candidate’s net favorability ratings between the two periods was .63.

That is not a terribly strong correlation by any means, and the number might change some if the study covered more years and included candidates like Mr. Dole and Mr. Reagan. Nevertheless, the relationship is highly statistically significant. Even at this early stage, polls tell us something — not everything, not a lot, but something — about how the candidates are liable to be perceived next year following the primaries.

...

Mr. Nyhan
[one of Nate's critics] has written that early primary polls “don’t matter” and that they are “useless” — and several other bloggers have echoed these statements. That just isn’t true. Yes, as a first approximation, the rule of thumb “don’t pay much attention to early primary polls” is probably better than “pay a lot of attention to early primary polls,” given the way that the media tends to overrate their importance. But Mr. Nyhan’s statement is hyperbolic.

There are many aspects of politics that can be investigated, reported on, and studied. Polling is one of them. Clearly, not everyone believes polls are helpful or even necessary. They are entitled to their opinion and entitled to argue it, but the rest of us who find them interesting and worthwhile, and there are a lot of us, should and will continue with business as usual.