The results for Victoria are also relatively unsurprising. Murray Rankin of the NDP led with 47%, while Donald Galloway of the Greens was up six points to 26%. Paul Summerville of the Liberals was third with 14%, while Dale Gann of the Tories had 12% support. The gain for Galloway is outside the margin of error, and certainly an interesting aspect of this race.
But the results in Durham and Victoria don't raise any alarm bells. The results in Calgary Centre do.
Forum found that Joan Crockatt of the Conservatives had dropped 16 points to 32%, putting her only two points ahead of the Liberals' Harvey Locke. His gain of two points is within the margin of error. Chris Turner of the Greens, however, gained 12 points to reach 23%, putting him in a solid third place and in the running. The NDP's Dan Meades was in fourth with 12%.
While I'll grant Thomas Mulcair his criticism of polling in Alberta in his dismissal of Meades' low numbers (he's actually up four points, though that is within the margin of error), his statement that the polls indicated the NDP was in trouble in Quebec in the last federal election doesn't hold water. The NDP was in a very comfortable position in the final week of the campaign in the province.
Is it possible that the Conservatives are neck-and-neck in Fortress Alberta? It is not shocking to see the Liberals at 30% in Calgary. Their provincial counterparts do better in that city and it isn't outlandish to see the Liberals at this level of support considering their province-wide polling at the moment. Having the Greens so high and the Conservatives so low is unusual, and would mean that local factors are seriously at play.
This is plausible, considering some of the stories that have appeared in local media lately. There is the split between the PCs and Wildrose and the fact that Turner is a relatively good candidate for the Greens. Nevertheless, this is Alberta.
It is worth noting the difference in sample sizes between Calgary Centre and the other two. Surveying only 376 people is not huge, and the lower number of respondents suggests that the response rate was quite low in the riding.
Forum uses the IVR method, which can have mixed results. Nate Silver recently calculated that "robodial" polls performed worse than their online and live-caller counterparts in the recent American election, finding an average error of five points for IVR polling compared to 3.5 points for telephone and 2.1 points for online surveys. However, some IVR firms performed well while others were near the bottom. This is actually to be expected - robodialling is the cheapest way to do polling so it is more likely that incompetent polling firms would still take a stab at it. There is more chances that the men will be separated from the boys when it comes to IVR polling than for online and telephone polling, as those methods are much more expensive and non-serious players are less likely to use them.
So where does Forum lie on the spectrum? We actually have some recent by-elections to use as a guide. Forum was active in both Kitchener-Waterloo and Vaughan when those ridings held by-elections in early September. Forum actually did quite well, with an average error of 3.3 points per party (PC, NDP, Liberals, and Greens only) in Kitchener-Waterloo and 1.8 points in Vaughan. In both ridings, Forum made the right call, including the upset NDP victory.
Forum also did well in the three riding polls they released during the provincial Quebec election. They had polled Orford, Saint-François, and Sherbrooke for a local English-language newspaper, and had an average error of two points per party (PQ, PLQ, CAQ, and QS only) in Orford and Sherbrooke and 1.5 points in Saint-François. So far, so good.
But a general rule in polling is that if something looks like an outlier, it probably is. Does the Calgary Centre poll look like an outlier? From the outside looking in, considering the riding's history, it looks like it could be. Locals may have a better clue.
However, while some IVR firms did well in the recent presidential election and Forum has had a string of good riding level polls of late, the method they use appears to be more vulnerable to wild variations between polls.
The Quebec provincial election is a good example of that. Let's look at the variations from poll to poll for the top four parties in the polls done by CROP (using live-callers), Léger Marketing (using an online panel), and Forum (using IVR). For example, if the PQ gained two points and the Liberals lost three points in a poll, while the CAQ and Québec Solidaire held steady, that would be a total variation of five points.
Léger had the most consistent polling, with an average total variation from poll-to-poll of 4.4 points. In one of their polls, all four parties had the same level of support as they did in their previous survey. In their most volatile poll, they had a total of eight points changing between the four parties.
CROP's polling was also consistent, with an average total variation of 5.8 points. Their lowest total change between polls was three points, their highest was nine.
Forum, however, had very inconsistent polling. On average, their numbers shifted by 14 points from one poll to the next. On two occasions the total variation was over 20 points between polls, and only once was the total variation lower than Léger's and CROP's average.
This is not to impugn Forum's accuracy; it is certainly possible (though less likely) that Forum was accurately depicting a volatile race while Léger and CROP were inaccurately suggesting a more stable electorate. But it does raise alarm bells when we see that the vote in Calgary Centre has shifted so radically, considering Forum's recent experience with IVR polling.
In the end, Forum could turn out to be right on the money and Calgary Centre could indeed be a very close race. If another poll emerges we will have a better idea of whether that is the case or not. For the moment, though, it might be better to exercise caution.
