What's in a Question?

With only seven national pollsters who are infrequently active, it is difficult to compare them and their methodologies. It doesn't mean we can't try, though.

There are two general ways to ask a survey question concerning vote intentions. The first is to ask something along the lines of:

"If an election were held today, would you vote for the Conservative candidate in your area, the Liberal candidate in your area, the NDP candidate in your area, the Green candidate in your area, or the Bloc Quebecois candidate in your area?"

The pollsters always rotate the order of the parties so that none is given an advantage. This question gives people options from which to choose. It is more likely to get a response from undecided or inattentive voters.

The second question is something along the lines of:

"If an election were held today, which party would you vote for?"

This question does not name the parties and so forces the respondent to give an answer without knowing which parties are his or her options. It is more likely to not get a response from undecided or inattentive voters but is, in my estimation, a more accurate way to judge current public opinion.

Judging from their detailed reports, Strategic Counsel, Angus-Reid, Léger Marketing, and Ipsos-Reid ask the first type of question. EKOS asks the second, while Nanos - in their latest poll at least - asks respondents to give their first and second preference, but does not name the options. While it isn't exactly the same question as asked by EKOS, for the sake of argument I'm going to lump them together.

Harris-Decima does not put the question in their detailed reports. I've sent them an email and hopefully will get a response.

Taking the average result from each of the polling firms' latest poll, we get the following totals for each of the parties when they are listed in the question:

Conservatives - 38.3%
Liberals - 28.8%
New Democrats - 15.0%
Bloc Quebecois - 9.3%
Greens - 7.8%

When respondents are not given a list of the parties, the average result is:

Conservatives - 36.8%
Liberals - 31.6%
New Democrats - 14.4%
Bloc Quebecois - 9.8%
Greens - 7.6%

Looking at it this way, we see that the NDP, Bloc, and Greens don't really benefit either way. But the Liberals certainly tend to do better when respondents aren't prompted with the list of parties.

Another factor that can change the results of polling is whether "leaners" are included. Several polling firms, when given the "I don't know" or "I haven't decided" response to the first question, ask a second. That second question asks what party the respondent is leaning towards voting for.

From what I can tell, Strategic Counsel, Léger Marketing, and Ipsos-Reid include leaners in their results. Angus-Reid, EKOS, and Nanos only use decided voters.

This is the average result for the "leaners included" pool:

Conservatives - 38.7%
Liberals - 29.3%
New Democrats - 14.3%
Bloc Quebecois - 8.7%
Greens - 8.3%

And the decided results:

Conservatives - 36.8%
Liberals - 30.0%
New Democrats - 15.2%
Bloc Quebecois - 10.2%
Greens - 7.0%

This indicates that Liberal, NDP, and Bloc voters are more committed than Conservative or Green voters.

Of course, all of this should be taken with a grain of salt. There isn't a large enough sample size to really determine how much influence the question has on the response.