Turnout and the Ontario by-elections

UPDATE: New polls from Campaign Research and Forum Research were released on July 31, and they have been used to update the By-Election Barometer. In terms of the turnout model as discussed below, applying it to Forum's latest polls changes the look of them rather significantly.

A few weeks ago, I discussed how a simple turnout model was able to significantly increase the accuracy of polls in some recent elections. Depressingly so - the model requires ignoring the voting intentions of those under the age of 35. But nevertheless the model worked. Can it be applied to the Ontario by-elections that will be taking place on Thursday?

The first question to ask, however, is whether the turnout adjustment should be applied. Earlier this month, I took a detailed look at Forum Research's by-election track record and found it to be decent. If we take those same polls and apply the turnout model, we see that it turns these decent polls into very good ones. (You will need to click on the chart in order to magnify it.)

In only two of the seven by-elections Forum has recently polled would the turnout model have made the poll worse. And these were the by-election polls for Victoria (out of the field 14 days before the election) and Calgary Centre (out of the field nine days before the election). In every other case, the turnout model would have improved the polls. On average, the model reduced the amount of error by 3.3 points. If we exclude the three polls taken more than a week before the vote was held, the turnout model improved the polls by an average of 7.5 points. That is rather significant. The polls in Westside-Kelowna and Kitchener-Waterloo, for instance, would have been dramatically improved.

Now that it is quite clear that the turnout model can improve polls, let's apply it to the latest set of by-election polls Forum has recently released.

In Etobicoke-Lakeshore, the seven-point deficit that Peter Milczyn was last recorded to have against Doug Holyday is increased to 11 points. Based on the amount of error in Forum's by-election polls, a margin of seven points was just about the limit of what you would consider a toss-up race. At 11 points, the margin seems insurmountable. It makes Etobicoke-Lakeshore, already leaning towards the Progressive Conservatives, a much more solid bet to go Tory.

London West is also made more solidly PC, as it improves Ali Chahbar's 36% to 40%, giving him a 10-point lead over the New Democrats' Peggy Sattler. At five points, the race could easily go either way. At 10 points, the Tories look much more likely to pick up the seat. The Liberals, still mired at 17%, do not seem to be a factor whatsoever.

Of all the polls Forum put out, the one in Ottawa South seems the most likely to be at fault. A poll by Campaign Research showed a much closer race just a week before, and the riding has a strong Liberal streak. But the turnout model suggests Forum may even still be under-estimating Matt Young's lead, inflating his support from 48% to 51%, while John Fraser's shrinks to 32% from 34%.

The news continues to be grim for the OLP in Scarborough-Guildwood. There, Forum (and Campaign) had given the Liberals their only lead. It was worth six points for Mitzie Hunter against Ken Kirupa, while Adam Giambrone remained well behind at 21%. The turnout model transforms the race into a toss-up, giving Hunter 37% of the vote to 36% for Kirupa. Giambrone stays at 21%. If the turnout model has it here, Scarborough-Guildwood could be a surprise.

Windsor-Tecumseh seems to be the riding least likely to surprise anyone, as Percy Hatfield has a wide lead over the Tories and Liberals in both the adjusted and unadjusted numbers. His lead is reduced from 35 points to 34 points, but that hardly seems to be of concern. The model suggests that the OLP could do even worse than expected here.

In fact, across the board it appears that turnout will work against the Liberals. The adjustment is nowhere worth more than two points, but with the PCs gaining between two and four points due to turnout it puts the OLP in a much more precarious position. Without the adjustment, the Liberals could expect to win Scarborough-Guildwood and have a shot at Etobicoke-Lakeshore and, maybe, Ottawa South. With the adjustment, the Liberals are out of the running in every riding but Scarborough-Guildwood. If these sorts of trends hold, the Progressive Conservatives could pick up three seats (maybe even four) and the NDP one. That would be quite a blow against the government.

Final by-election polls are to be expected this week. I will be interested to see how the NDP is doing in London West and whether the Liberals will remain competitive in Scarborough-Guildwood, Etobicoke-Lakeshore, and Ottawa South. But as it stands, Thursday is setting up to be a difficult day for Kathleen Wynne.

Interview with Don Mills of Corporate Research Associates

Today's column for The Globe and Mail is the first of three that will be running over the next few weeks, looking at polling in Canada. Each piece will be focusing on the modes of contact used by polling firms today: live-caller telephone, IVR, and online. 

For this series, I interviewed a number of Canada's leading pollsters via email. I'll be posting the transcripts of these email interviews over the next few weeks here to accompany the Globe articles. Today, we talk to Don Mills, Chairman and CEO of the Corporate Research Associates. This Halifax-based firm conducts its political polling over the telephone with live-callers - though in a growing trend, they have just recently announced the launch of a new online panel. Firms that stick to only one mode to do their polling are becoming rarer and rarer.

308: While other firms have moved to online panels and IVR polling, CRA continues to use live-callers. Why have you stuck with this methodology?

DM: The methodology has stood the test of time. Our industry standards do not allow the use of margin of error for online research which are considered samples of convenience. Many companies violate industry standards in this regard. We agree with that standard.

308: What do you consider the strengths of telephone polling with live-callers compared to other methodologies?

DM: Its ability to produce random representative samples is still the biggest strength in my opinion.

308: What are its limitations?

DM: Very costly relative to other methodologies in terms of data collection and also hard to find those in the 18-24 year old age group to interview,

308: Generally speaking, how does telephone polling compare to other methodologies in terms of costs and effort?

DM: Generally it depends on the population to be surveyed and the length of the survey. Hard to reach segments are particularly expensive for telephone research. For general population type research, telephone research is likely 25%  or more expensive than online on average.

308: What challenges do you face in building a representative sample, considering falling response rates and increased use of cell phones over landlines?

DM: Response rates for public opinion type research have not actually changed dramatically in the last number of years. It has always been difficult, but not impossible, to find those aged 18-24 years old. Interestingly enough, although those 18-24 are more likely to use cell phones personally, there is also a high proportion who still live at home with their parents who have landlines and who can still be reached on landlines as a consequence.

308: What role does weighting play in producing accurate results?

DM: Weighting is still important in random telephone research to properly reflect the known distribution of the population. It is nearly impossible to have proportional representation in random telephone research.

308: CRA has been in the business for a very long time. How has political polling changed over the years? 

DM: Surprisingly little in my opinion. Young people continue to be disengaged in the political process for the most part as in the past. Older people continue to believe in the process and to be the most likely to vote. I think the challenge today is mainly among those in the child rearing years who have very busy lives with their children with little time to think about anything but their jobs and their families. This is the group that is likely now less engaged in the political process as a result.

308: How has the business of polling in general changed?

DM: There was a time when media invested in polling that was not just about the horse race, delving more into the issues to better understand the basis for supporting parties. This is no longer the case. Political polling is much more superficial than it was in the past.

308: Do you face any particular challenges as a pollster based in Atlantic Canada?

DM: Not particularly. Our company has a well established reputation which helps us when collecting public opinion. We have always been a non-partisan company and have never conducted polling for any political party. I think that enhances our creditability with both the political parties and the media. I believe our track record has been outstanding as well.

308: Your polls tend to be conducted over several weeks, instead of over a few days. Why is that, and what are the benefits or problems associated with this?

DM: Most of our political polling is done on a predictable quarterly basis, the same period each year. The consistency of our process provides a larger time frame than we would if we were working with media. In that case, doing survey over a few weeks rather than a few days is not really an issue.  Our media work is normally over shorter time periods, although the rolling polling we did for the Telegraph Journal in the last New Brunswick election was pretty much a gold standard for tracking change over the course of the election.

308: CRA also tends to have some of the largest proportions of undecideds of any polls produced in Canada. Why?

DM: Most of the time we do not try to force an opinion. We also report those undecided, don’t plan to vote and prefer not to say together in the our numbers while other companies do not. This looks like the percentage of undecided is higher than is actually the case. Near elections, we typically ask a leaning towards question which reduces the undecided percentage considerably. I have never personally worried about those undecided because my experience is that a lot of those people are not actively engaged and probably don’t vote and those that do vote along the line of the decided group.

308: What changes, if any, need to be made to ensure that telephone polling produces good results in the future?

DM: We have strict standards for telephone research such as assuring that interviewers are well trained and respectful, that the proper number of callbacks are done properly. Adherence to the standards necessary to conduct valid telephone research remains the key to achieving reliable results. There needs to be some industry effort to promote the value of participating in survey research as a benefit for citizens as well. In a free society, how much value is associated with the ability to express your opinions in the hope of changing life for the better?

Ontario PCs have advantage in latest Forum poll

Forum Research put out its latest Ontario provincial numbers via the Toronto Star this morning, finding the Progressive Conservatives to have a five-point edge over the Ontario Liberals. But the change in support since Forum's last poll is nothing significant. The potential effect of turnout in such a hypothetical election, however, could be.

Forum was last in the field on June 26, another of their snap, one-day polls. The PCs gained one point since then, to lead with 36% support. The Liberals were down two points to 31%, while the New Democrats were up three points to 27%. While that is the biggest change of the main parties, it is still within the theoretical margin of error. However, if we look at where the parties stood at the end of May, we see that the NDP has picked up six points over that time while the Liberals have dropped seven. That could be a worrying trend for the OLP if it doesn't stop.

The Greens were down three points to 5%, while 1% of respondents said they would vote for another party. From the sample of decided and leaning voters, it appears that about 6% of respondents were undecided, no different from Forum's last poll.

This poll does have the same sampling issues that we usually see with Forum's surveys: 60% of the sample is over the age of 55, rather than the roughly 35% of adult Ontarians who are actually over that age. Only 8% of the sample was under the age of 34. This proportion of the population takes up closer to 27% of Ontario adults. This means that Forum has to apply some pretty important weights.

In that context, it is remarkable how steady the Tories have been in Forum's polling. They have been at 34%, 35%, or 36% in the firm's last six polls going back to March. The Liberals have wobbled more significantly, with between 31% and 38% over that time. The NDP has also seen a good deal of fluctuations, with between 21% and 27% support. This suggests that voters are swinging between the OLP and the NDP.

Regionally, only two shifts in support appear worth highlighting: an 11-point gain for the Liberals in eastern Ontario (good news for John Fraser in Ottawa South) and a 10-point gain for the NDP in southwestern Ontario (good news for Percy Hatfield in Windsor-Tecumseh and Peggy Sattler in London West). The PCs have the edge in eastern Ontario, northern Ontario (which stretches down to around Barrie in Forum's estimation), and the 905 area code, while the NDP is ahead in the southwest and the Liberals in Toronto.

With these regional numbers, the Progressive Conservatives would win 46 seats and probably form a minority government. The Liberals would narrowly edge out the NDP for the Official Opposition role, with 31 seats to 30 for the New Democrats.

Outside of Toronto and eastern Ontario, the Liberals are at a severe disadvantage, especially considering their weakness in the southwest and northern parts of the province. The NDP is able to win a whole swathe of seats in those areas, while the PCs dominate the GTA and take a good number of seats in the rest of the province as well.

But what if we take ThreeHundredEight's simple turnout model into account? With that adjustment, the Progressive Conservatives would be expected to take 38% of the vote to 33% for the Liberals and only 23% for the New Democrats. If we apply that adjustment to the regional numbers, we get a seat haul of 49 for the Progressive Conservatives, 37 for the Liberals, and only 21 for the New Democrats. Still a PC minority, but the Liberals would be in a far stronger position. The NDP is penalized most significantly in the southwest, where they drop to eight seats (four more go to the PCs and two more to the OLP).

This gives a very clear demonstration of how turnout can impact outcomes. The PCs get to within only five seats of a majority government, close enough for just a slight error in the polling or seat modelling to put them over the top. The Liberals go from being in danger of falling to third place to being clearly in second, while the NDP goes from hopeful Official Opposition to third place again.

But their leader, Andrea Horwath, remains first in approval ratings. She scored 43% in this poll, compared to 35% disapproval. Her numbers have been very steady for some time. Among NDP voters her approval rating soars to 73%, while it is still at a very solid 45% among Liberals.

Kathleen Wynne's approval rating stands at 35%, while her disapproval has increased by five points to 48%. Among Liberals, her approval rating is 72%. Interestingly, she scored 41% among New Democrats - an increase of 10 points since last month.

Tim Hudak remains at the bottom of the pack, with an approval rating of 27% (he has been at either 26% or 27% since March). His disapproval rating is at 51%, and his approval rating is at only 58% among PC voters. The silver lining for him, though, is that Wynne is approaching his level of unpopularity.

The situation in Ontario, despite a little back and forth in the numbers, has been pretty static since Kathleen Wynne became premier. The PCs and Liberals trade the lead and the NDP remains in third, but well above where they were in 2011. It makes calling an election a gamble for every party: the PCs could return to opposition, the Liberals could lose power, and the NDP could lose the influence they have in a minority legislature. If the plug is pulled this fall or if budget negotiations fail in the spring, no leader is likely to have any great confidence in victory. Better to just roll the dice and end the uncertainty.

EKOS poll shows flagging Liberal support

Late last week, EKOS Research released their most recent federal polling results via iPolitics, showing that the Liberals have been losing support since shortly after Justin Trudeau's leadership victory. However, neither the Conservatives nor the New Democrats have really been able to take advantage.

EKOS was last in the field on May 22-26 (though that poll was not released until last week as well), and since then the Liberals dropped 4.3 points to 30.4%. That drop is outside the margin of error, and part of a larger trend as the party had been at 38.6% in EKOS's Apr. 30-May 2 poll.

But the Conservatives continue to tread water well below where they need to be. They were up 2.1 points to 28.4%, but that is not a statistically significant amount of change and is only 2.2 points up from where EKOS had them in their earlier poll. It is a similar story for the NDP, who were up 2.5 points to 23.4% support, but that is after having dropped in EKOS's poll from the end of May. The party is still below where they were at the end of April.

The number of undecideds and non-responses was 17%, mostly unchanged from EKOS's previous poll. The Greens had 8.5% support, while the Bloc Québécois was at 6.3% and support for other parties at 3.1%.

A note about EKOS: they have a good disclosure policy, with their reports being very complete and full of numbers. It would be best if they could also include their unweighted sample sizes in their reports, but the firm is not shy to share this information when asked and Frank Graves is always willing to talk about methodology in great detail.

These numbers from EKOS are interesting, as the firm has had a tendency in the past to low-ball the three major parties. But that has not been the case in their last two polls, with the Liberals at 39% and 35%. Their two previous polls had the leading party at less than 30%, and you have to go back to March 2012 to find an EKOS poll where the leading party had more than 33% support. So, this suggests that EKOS is recording a real dip in Liberal support, something that other firms have been hinting at. It is too early to definitively say that the honeymoon is over for Trudeau, but it is certainly not too early to ask the question.

Regionally, support for the parties was mostly wobbling within the margin of error and similar to what other polls have been showing. But the Liberals did lose significantly in Ontario (down 5.5 points to 32.7%) and Quebec (down 6.3 points to 29%). Elsewhere, the New Democrats were narrowly ahead in British Columbia (which has become a real three-way race in the last few months) while the Tories were in front in Alberta and the Prairie provinces. The Liberals were still ahead in Atlantic Canada, but have moved away from the 50% they were enjoying in April-May.

Of interest in this poll is the support among immigrant Canadians. Jason Kenney is credited with having done a great deal of good work for the Conservatives among these voters, and he has retained some responsibility in this domain despite his change of ministry. The poll suggests the Conservatives still have the advantage among this demographic, with 35% to 33% support for the Liberals. But they would prefer to have a wider edge than that.

On turnout, the Liberals would likely do better than the Conservatives as they were ahead among voters 45 and older.

In terms of seats, the Conservatives would still narrowly beat the Liberals due to their advantage in the West and in rural Ontario, taking 126 to 112 for the Liberals. The New Democrats would hold 60 seats, buoyed in large part by their strong B.C. numbers, while the Bloc Québécois would take advantage of the weakness of the NDP in Quebec and win 38 seats. The Greens would take two.

The numbers in Quebec are particularly interesting. Though the Liberals double their support, they aren't in a strong position to win a large number of seats due to their relative weakness outside of Montreal and a few select areas of the province. With the NDP dropping so steeply, the small gain the Bloc has made in this poll (worth only three points from their 2011 showing) pays big dividends. In fact, a huge proportion of Quebec's seats are won by a tiny proportion of the vote (35% or less), with only some five to 10 points separating the Bloc, NDP, and Liberals. It means that it would not take much to radically transform the numbers in the province.

In the B.C. election, the forecast did not actually get a huge number of seats wrong (in fact, of all the prognosticators ThreeHundredEight got the most correct), but the ones that were wrong went all in the same direction, magnifying the error. But on average, of those that were wrong the probability forecast averaged about 68% confidence that the right call was being made (not exactly high). That translates to a margin of about nine to ten points. So if we apply that range to the federal forecast, we see that Quebec is the real battleground.

That is because with a range of just under 10 points, the number of seats each of the parties could win in Quebec is wide enough to drive a truck through: 11-50 seats for the Bloc Québécois, 20-38 seats for the Liberals, 0-37 seats for the New Democrats, and 5-12 seats for the Conservatives. No other province showed that wide a variety of potential outcomes, suggesting that the next election could be won or lost in Quebec. The Conservatives are powerless in that regard, and need to hope that Quebec is split three ways instead of plumping for one party as the province did in 2011.

Ontario also did show some wide variations, at between 32-60 seats for the Tories and 37-69 seats for the Liberals, but that was primarily due to all of the close Conservative-Liberal seats in the province (the NDP ranged only between 17 and 28). British Columbia had a high proportion of close races (12-24 seats for the NDP, 9-17 for the Liberals, and 4-17 for the Conservatives), while Alberta, the Prairies, and Atlantic Canada had only a handful of seats that could go either way.

These ranges show that, while the Liberals have a less efficient vote than the Conservatives in the precise projection, they have greater potential. The Liberals could win between 90 and 163 seats with these numbers from EKOS (taking into account the potential for EKOS's numbers to be wrong) while the Conservatives could win between 87 and 152 seats. That likely means either official opposition or minority government for either party, but neither is at the magic 169 needed to form a majority government in the new House.

The New Democrats could win between 34 and 106 seats, which means that if the three parties end up very close in the popular vote the NDP does have the potential (tiny as it is) to come ahead in the seat count. They also have the potential to return to fourth-party status behind the Bloc Québécois, who could win as many as 50 seats and return to prominence or fall just short of official party status in the House of Commons.

The Greens could win between two and four seats, the extra two being very unlikely wins in Ontario and the North.

This gives a good indication of where the parties stand: the race is very much between the Liberals and Conservatives but the NDP is still in the running, while depending on how things go for the other parties the Bloc can remain on the fringes or return to influence. Quebec is the biggest battleground, while the Conservatives' fate lies in Ontario. Federal politics in Canada remain at a crossroads - and likely will continue to be straight through to 2015.

Forum's by-election track record

Five by-elections are scheduled for August 1 in Ontario, and all five of them are setting up to be interesting races. At least four of them could be considered toss-ups, while the fifth could mean defeat for the incumbent. As by-elections go, these are not bad for a hot summer.

Because of these by-elections, we have heard (and will continue to hear) much from Forum Research. The firm is - by far - the most publicly active political polling firm in the country, and it never misses an opportunity to poll for a by-election (it even put out three for the foregone conclusion by-election in Westside-Kelowna). It has already put out some new numbers for these Ontario races, so it might be a good idea to take a look at the firm's track record in by-elections.

Forum releases a good amount of data with their polls, showing unweighted sample sizes. As I've discussed before, this gives us a good idea of how representative the raw sample is, but tells us nothing about what Forum does with that sample as weighted numbers are not provided. In some cases, it seems that a lot of tweaking needs to be done in order to get the sample sizes right. Unfortunately, the firm is reluctant to provide details about how they go about building its samples and what it does with them. As this is proprietary information, that is within their right - but hopefully they are telling their media partners about it before the numbers are published.

Instead, when asked Forum relies on its own track record which, admittedly, is pretty good. Here is how Forum has performed in the last year of by-elections (the 'Days' column refers to the number of days between the poll and the by-election):

CLICK TO MAGNIFY

As you can see, in these seven by-elections Forum has yet to pick the wrong winner or even get the order of the parties mixed up, even in difficult by-elections like Kitchener-Waterloo or Calgary Centre. But if we take a very critical eye to the firm's performance, we see that only their call in Vaughan for the top four parties had every number within the margin of error.

For those by-elections where the final poll was taken a week or two before the vote, that is more than forgivable. And even then Forum did rather well, showing the Greens to have real strength in Victoria, calling the top three parties within 2.7 points in Calgary Centre, and pegging the NDP as second and the Liberals as third in Durham.

However, the record is not completely clean. Even with a small sample of 177 decided voters, Forum was outside of the margin of error in Westside-Kelowna for the B.C. Liberals, under-estimating them by 10.7 points. The 6.3-point miss on the NDP's vote was within the margin of error, but is still rather substantial. In Labrador, the NDP was over-estimated by almost five points and in Kitchener-Waterloo the Progressive Conservatives were under-estimated by almost six.

Overall, however, the record is rather good for the difficult task of calling by-elections. With the exception of Westside-Kelowna, Forum's polls taken within a few days of the vote have done well. On average, Forum's by-election polls have had a margin of error of +/- 3.5 points for each party (and +/- 3.2 points in polls taken a few days before the election).

What does this tell us about the five by-elections in Ontario?

It certainly suggests that four of them are complete toss-ups. If we apply the +/- 3.5-point margin of error to their latest set of polls, we see that the Tories could potentially win four of the five (or none), the Liberals could retain three (or none), and the NDP is in the running for two (and at least one).

Windsor-Tecumseh is the only one that does not seem to be in question, as the New Democrats range between 48.5% and 55.5% of the vote, compared to 18.5% to 25.5% for the Progressive Conservatives. 

The next 'easiest' call would have to be London West, where the Tories would have between 32.5% and 39.5% of the vote, compared to a band of 25.5% to 32.5% for the NDP and 20.5% to 27.5% for the Liberals.

Despite Doug Holyday's candidacy, Etobicoke-Lakeshore actually appears to be the best shot the Liberals have at retaining a seat, with a range of between 41.5% to 48.5% of the vote to 35.5% to 42.5% for Holyday. In Scarborough-Guildwood, the +/- 3.5-point margin gives the Liberals 35.5% to 42.5%, compared to 30.5% to 37.5% for the Tories.

By this measure, Ottawa South is the closest riding, with the Liberals at 38.5% to 45.5% of the vote against 34.5% to 41.5% for the Progressive Conservatives. However, the Forum poll suggests its sample might be skewed towards the Tories, as 39% of respondents said they voted for the party in 2011, while only 40% said they voted for the Liberals. In fact, the Liberals took 49% of the vote to 33% for the PCs.

Undoubtedly, we will hear more from Forum before the voting begins on August 1. In Windsor-Tecumseh, the question is whether the New Democrats can hold their lead. In the rest of the by-elections, the results will be closely watched as none of them have an obvious winner.

A deeper look at polls from Environics and Ipsos-Reid

The two most recent federal polls unleashed on the world came from Ipsos-Reid and Environics Research Group. Both polls were taken at the end of June, which means that they are now a little dated. So instead of looking at the overall numbers, let's take a look at some of the numbers that weren't included in their original releases.

First, the Environics poll. In addition to the usual regional breakdown that Environics includes in its release, the polling firm also records 'sub-regional' support and was kind enough to pass those numbers along.

The last time I had these numbers was in June 2012, at the height of Thomas Mulcair's honeymoon with Canadians. Compared to those numbers, the New Democrats have fallen 11 points in Environics's polling, while the Conservatives have been down five and the Liberals gained 15.

At the sub-regional level, however, we can see where some of these gains and losses have disproportionately taken place. Compared to the shift in national support, the Liberals have surged disproportionately in Toronto (Environics defines Vancouver, Toronto, and Montreal as their CMAs) by 22 points to 50% support. They were up 28 points, also to 50%, in Newfoundland and Labrador. Both of those shifts in support are well outside the margin of error, even for the small samples used for these regions.

It should be noted, however, that Environics also did a poll of 1,000 Newfoundlanders and Labradorians at around the same time as their national June 2012 poll, and compared to those numbers the Liberals were up only 16 points.

Liberal support grew by about the same amount as it did nationally in Saskatchewan (14 points), Montreal (13 points), and Vancouver (13 points), but only by 11 points in Manitoba.

For the Conservatives, their support fell disproportionately in Vancouver (10 points) and Saskatchewan (8 points), while it maintained itself better in Newfoundland and Labrador (down two points), Toronto (three points), and Montreal (two points). But in these areas, the Conservatives did not have much further to drop.

And the New Democrats suffered most in Newfoundland and Labrador (down 25 points and losing the lead), Montreal (down 15 points), and Toronto (14 points), while they fell only four points in Saskatchewan. They were also down 12 points in Vancouver and eight points in Manitoba.

With these levels of support, the Liberals would win five seats in Newfoundland and Labrador to two for the NDP, while the Tories would be shut out. In Manitoba, the Tories would take eight, the Liberals four, and the NDP two, and in Saskatchewan the Conservatives would win 11, the NDP two, and the Liberals one.

The extra bit of precision for the Prairie provinces does have an effect, as using the overall Prairie numbers to project seats short-changes the NDP by two (giving one extra seat to the Liberals and Conservatives).

For the Ipsos-Reid poll, the firm did something different in trying to assess likelihood of voting. They found that, among the general population, the Liberals led with 33% to 30% for the Conservatives and 28% for the New Democrats. The regional numbers would deliver seats in the following proportions:

The three parties would be almost even, with 121 for the Conservatives, 112 for the Liberals, and 102 for the New Democrats, with the Bloc and Greens taking three seats together.

The Conservatives pull more seats from fewer votes because of their advantage in the western provinces and in Ontario. A tie there (34% to 34%) is more advantageous for the Tories. But the party would only win five seats east of the Ottawa River.

When Ipsos-Reid looked at only those who were most committed to vote (about 60% of the sample, interestingly enough considering that is a very plausible turnout number), the Liberals were boosted to 35%, with the Tories dropping to 29% and the NDP to 26%. For those wondering, that is counter to what my simple turnout model would have given for this poll (34% CPC to 33% LPC and 24% NDP). With Ipsos's regional numbers for likely voters (which the firm was also nice enough to pass along), the Liberals move ahead in seats:

The Liberals win 126 seats to 114 for the Conservatives and 95 for the NDP. The big difference is in Ontario, as the turnout numbers give the Liberals 38% to 34% for the Conservatives. But turnout benefits the Tories in B.C. and the NDP in the Prairies and Quebec.

Despite the difference between Ipsos-Reid's total sample and likely voters being relatively marginal (no party moved by more than two points nationally), the consequences are far more important. In the first scenario, the Conservatives might actually try their luck at a minority government. They would likely be defeated by the Liberals and NDP, but if those two parties did decide to work together they would each need to be given almost equal heft in a coalition.

In the second scenario, the Conservatives likely would not try to form a minority government and instead the Liberals would try their hand at it (yes, I know that the Tories could try to continue to govern but I'm assuming they wouldn't, as Jean Charest did in Quebec). If they did work with the NDP, they would be able to call more of the shots holding 57% of the coalition's seats instead of 52%, and finishing much further ahead in the popular vote (nine points instead of five).

This emphasizes just how important likely voter numbers will be in the next election. I am happy to see that more firms are starting to think about how they will go about this in 2015.

June 2013 federal polling averages

Up until the end of the month, June was looking to be a polling desert. But in the end, six national firms and one Quebec firm reported by the end of the month, and in all over 9,000 Canadians were polled on their federal voting intentions. The result: Justin Trudeau's honeymoon may be on the wane.

Nevertheless, the Liberals have maintained a sizable lead over the Conservatives. They averaged 34.4% support in June, a drop of 5.6 points from where they stood in May. The Conservatives were up 1.3 points to 28.9%, while the New Democrats were up 0.5 points to 23.8%.

The Greens picked up 2.1 points to hit 6.2% and the Bloc Québécois increased by 1.2 points to 5.3% support. 1.4% of Canadians (+0.4) said they would vote for another party. On average, about 15% said they were undecided.

That drop of almost six points for the Liberals is certainly significant - the last time a party has dropped that much in one month was between April and May 2011, when the Liberals plummeted to 19%  in the election. The Liberal decline is positive news for both the Tories and NDP as they have arrested their four months of consecutive decrease.

But it is difficult to compare apples to apples, as these six national firms have not been in the field in such a small window since the spring of 2012. If we compare to the last time they were all in the field, which goes back to September-December 2012, we see that the Liberal rise that the leadership campaign and Trudeau's win caused took from both the NDP and Conservatives.

Since September-December, and looking only at these polling firms, the Conservatives and NDP have each dropped around five and six points, respectively. The Liberals have picked up almost 11.

But it has to be recognized that June was not a particularly good month for the Liberals, at least on the face of it. The party was down in every region of the country, but in several places they are coming down from at least four year highs. With the exception of the May averages, the Liberals are still polling at some of their highest levels since before 2009.

Starting on the West Coast, the Liberals averaged 30.1% support, a drop of 2.8 points, in B.C. The NDP was up 0.3 points to 28.5%, while the Conservatives hit their lowest point since before 2009 with 28.1% support. The Greens gained 3.4 points to hit 12.3%.

In Alberta, the Conservatives picked up 8.3 points to reach 55.5%, putting them where they usually are in the province. The Liberals dropped 9.7 points to 23.3%, but that was largely due to the anomalous Forum poll from May that put the party at an implausible 42% in Alberta. Ignoring last month, the Liberals are doing better than they have since before 2009. The NDP was up 0.5 points to 13.6%, while the Greens were down 0.5 points to 4.9%.

In the Prairies, the Conservatives were up 7.9 points to 45.5%, while the Liberals were down 8.3 points to 25.1%. The NDP continued at their post-election low, falling 0.8 points to 21.6%. The Greens were down 1.4 points to 6.4%.

The Liberals were down 3.7 points to 37% in Ontario, though with the exception of last month that is still their best since September 2010. The Conservatives were up 0.3 points to 33.6% and the NDP was up 2.5 points to 22.3%. The Greens gained two points, averaging 6.2% support.

In Quebec, the Liberals dropped 3.3 points to 37.7%, again their recent best with the exception of last month. The NDP gained for the second consecutive month (the first time that has happened since February-March 2012, when Thomas Mulcair won the leadership), picking up 0.8 points and averaging 28.5%. The Bloc Québécois was up 2.2 points to 19.5%, while the Conservatives dropped to single digits with a 0.2 point fall to 9.9%. The Greens were up 0.9 points to 4%.

And in Atlantic Canada, the Liberals fell 3.8 points to 47.2% while the NDP was up 5.4 points to 27.7%, approaching their haul from the 2011 election. The Conservatives were down 3.1 points to only 19.2%, while the Greens were up one to 4.7%.

With these levels of support, the Liberals would likely win 134 seats, with the Conservatives taking 120, the New Democrats winning 80, and the Bloc and Greens each winning two.

This represents a drop of 18 seats for the Liberals from the May projection, with the Conservatives picking up 11, the NDP six, and the Greens one.

That extra Green seat is Victoria, as I have added the by-election's results to the projection model. With the NDP down and the Greens up since that close contest, it follows that Victoria would fall into the Green camp - particularly since the area just elected its first Green MLA.

The Liberals dropped five seats in Ontario and four apiece in Atlantic Canada and Alberta, while the Conservatives were up five in Ontario, four in Alberta, and three in the Prairies. The NDP made four of their gains in Atlantic Canada.

In the end, the 40% support the Liberals were registering in May was probably never going to be sustainable. New leaders almost always experience a burst in support, and that does seem to be what was going on. If the party settles in at the mid-30s, it would be hard to call that anything but a spectacular success in the context of the last election. The Conservatives are still struggling mightily, but they may have hit their floor. How the NDP's support plays out in the coming months will be the thing to watch.

The quick and easy (and effective) turnout model

After the polling debacle in the British Columbia election earlier this year, many pollsters identified turnout as a major issue in the error, and not in terms of the number of people who turned out. Rather, the problem was with who turned out. I looked at this issue in an article for The Globe and Mail shortly after the B.C. election, and a new article by Angus Reid for Maclean's identifies the same problem. In that article, he mentions that if Angus-Reid had used a turnout model based on who went to the polls in 2009, they would have given the NDP a three-point lead instead of the nine-point lead they actually awarded the party.

While that still would have been off, it would have been a lot closer and would have suggested that the race was getting too close to call. The egg on the faces of the pollsters (and those who base their work on polls, such as myself) would have been a little thinner.

Ever since the federal and provincial elections in 2011, I have played around with trying to guess at how the polls were going to miss turnout. This was employed in the manner of adjustments to the projection based on how the polls had been wrong in similar situations before. This helped me get very close to the results for the B.C. Greens and Conservatives in the election, and got me a little closer to the truth in Alberta (I probably would have had a Wildrose majority instead of a minority without it).

But the results haven't been satisfactory, and the B.C. election showed how necessary it was (Alberta had some other dynamics going on, not related to turnout). The 2011 federal election was also influenced by turnout, as it turned a Conservative minority government into a majority one.

I could try to design some complicated model for estimating turnout, but in most cases the simpler a forecasting model can be the better it is. One of the criticisms of Nate Silver's model is that it is overly complicated for only a slightly better performance than simpler alternatives. He has alluded to that himself, and discusses the benefits of simplicity in The Signal and the Noise (required reading).

I've determined that a very simple turnout model can, in fact, be quite effective. That model has to do entirely with age. Younger Canadians do not go to the polls in as large numbers as middle-aged Canadians, who do not vote in as large proportions as seniors. And, it seems, those who do vote tend to vote in similar ways to those who are older than them.

In order to reflect this, the simple turnout model I will be using in future elections (when the data is available, and as a separate calculation from the polling aggregation) employs the following formula: discard the results of those under the age of 35 and double the results from respondents 55 and older. That's it.

This is easy to do as most pollsters tend to report these age groups. When they use other definitions, a little tweaking needs to be employed. When they do not report these results, I will have to estimate them based on what other polls are showing or ignore them entirely. The results of this method are surprisingly good.

The chart below shows the difference between the final projection and what that final projection would have been if using this simple turnout model (ignoring polls without age breakdowns) in four recent elections. A few notes: in Alberta, I have compared the turnout projection to the unadjusted final projection - I was employing turnout adjustments for every party in that election already. And in Quebec, since Léger and CROP do not report age breakdowns, I have only applied the turnout model to Forum Research's final poll.

As you can see, in every case the turnout model performs better. In some cases, the difference is dramatic: total error would have been reduced by about a third in British Columbia and almost entirely erased in Ontario. The results in that province are particularly striking - no party would have been missed by more than 0.3 percentage points!

For British Columbia, the difference is the most consequential since it would have changed the narrative from an easy NDP victory to a lead of only 2.3 points. In the case of Angus-Reid, if the firm would have used my simple turnout model instead of a more complicated one based on 2009's turnout figures, they would have had the NDP's lead at only two points (42% to 40%) instead of the three reported in the Maclean's article. Ipsos-Reid would have also had a two-point lead (44% to 42%) instead of the seven-point gap they actually had, while Forum Research would have given the Liberals a two-point edge (43% to 41%) instead of the reverse.

In Quebec, Forum's adjusted numbers would have tied Léger's for the best performance, while in Alberta their final poll would have given Wildrose a one-point advantage instead of two points. The final polls of Angus-Reid and Abacus Data would have been made worse, however. Turnout was not the issue in Alberta, it seems.

UPDATE: I had forgotten that Léger did indeed release age breakdowns with their final Quebec poll (they normally do not release this information). As you can see in the discussion in the comments, applying the turnout model to the Léger and Forum polls and averaging them gives a very good result: 33% PQ, 30.5% PLQ, 27% CAQ, and 6% QS for a total error of only 1.5 points!

For Ontario, the results would have been simply astounding. The polls by EKOS Research, Angus-Reid, Abacus Data, and Ipsos-Reid would have all been closer with this very simple turnout model.

This demonstrates how Canadian election polling needs to rely on more dramatic turnout models in order to get closer to election results. This is problematic, however, since it implies that the pollster with the best model, and not the best methodology, would get the plaudits. In order to be acclaimed for having the best methods, pollsters should use a turnout model based on the questions in the poll itself, like Ipsos-Reid did in their most recent federal survey. If a pollster can estimate turnout correctly, as well as the results of the vote, they will have proven themselves exceedingly competent. This requires full reporting, however, as otherwise the public will not be able to determine if the model or the methodology won the day.

For ThreeHundredEight, this simple turnout model should provide readers with some decent expectations of results. At the very least, it should help prevent some surprises. But because the model is so simple, and verges more on a gimmick than anything based on voting behaviour research, I will only be including it as an extra piece of information. The site will abandon turnout modelling for the projection itself, and instead focus on polling averages and ranges based on how polls have been wrong before. Hopefully, this will give readers the best possible understanding of the dynamics of an on-going campaign.

Updates to federal averages and by-elections

On Friday, I had to say good-bye to my eight-year-old desktop PC. It had suddenly, and tragically, decided to kick the bucket. Luckily, I back-up regularly and the hard-drive was not shot so nothing of importance was lost. But it did throw a monkey wrench into my plans for updating the site on Friday.

This morning, I have updated the federal averages with the latest Ipsos-Reid poll and have put all of the by-elections in the By-Elections Barometer up to date, taking into account all the polling that I missed while I was away the week before last. I have also added the two new races in Etobicoke-Lakeshore and Scarborough-Guildwood.

As there were a lot of updates to do to the barometer, and since this new computer and Windows 8 has slowed me down considerably while I get the hang of it, I won't have any more detailed post to put up until tomorrow. But check out the new aggregates and by-election forecasts in the meantime.

The Environics Institute

The site has a new sponsor this month. It is the Environics Institute, a non-profit organization that conducts public opinion research. Considering that its work might be of particular interest to readers of ThreeHundredEight, I asked Dr. Keith Neuman, Executive Director of the Environics Institute, to provide us with an introduction to his organization.

**********

308: Could you tell us a bit about the Environics Institute and its mission?

KN - The Environics Institute is the only organization in Canada to focus on public opinion and social research in the public interest (as a non-profit entity).  It was founded in 2006 by Michael Adams (also co-founder of the commercial company Environics Research Group in 1970), when he realized there was no entity in Canada like the respected Pew Research Center in the US doing independent public interest research. 

Our initial project was the first-ever national survey of Muslims in Canada, based closely on a similar survey Pew conducted in five European countries and the USA. September 11th made Muslims the new “other” group in the western world, and this community was poorly understood and treated with suspicion.  The survey results provided a more accurate and constructive picture of our country’s Muslim population, in part by demonstrating how much they share in common with other Canadians.  It was well covered in the media (thanks to a partnership with the CBC) and much appreciated by the Muslim community.

The Institute’s mission is to promote relevant and original public opinion and social research on important issues of public policy and social change in Canada.  The underlying premise is that through such research that organizations in all sectors, as well as individuals, can better our country today, how it has been changing, and where it may be heading.  A major focus of the Institute’s mandate is to survey individuals and groups not usually heard from, asking questions not normally asked.  All of our research is in the public domain (available through our website at www.environicsinstitute.org), and in most cases we publicize our studies in partnership with major media organizations such as the CBC and the Globe and Mail.

In addition to conducting its own research projects, the Institute promotes the importance and use of public opinion research as important input into public policy and decision-making.  We also aim to serve as a centre of excellence for responsible public opinion research methods and application, through education, training and consulting.

308: Does the Institute do any political polling?

KN - The short answer is no. The mandate of the Institute is to conduct important research that is not being done by other organizations, to address important gaps in our knowledge and understanding.  Political polling is being well covered by others in Canada, and so is not an area where we need to direct our efforts.  We do have an interest in the broader topics of democracy and civic engagement, and this is reflected in some of our studies.

308: What kind of research has the Institute done in the past?

KN - Our research falls into three broad streams:

1.  Our most substantial projects focus on conducting meaningful research with poorly-understood and often marginalized parts of our country’s diverse population.  In addition to our survey of Muslims (which we hope to update this fall), the Institute conducted the landmark Urban Aboriginal Peoples Study (www.uaps.ca), which was the first to focus on Aboriginal communities in our cities (more than half of our Aboriginal population now lives in cities).  We are currently in the early stages of launching a major study of the diverse Black community in the Greater Toronto Area. For these projects we use a community-based research model, in which we actively engage the community being studied in all phases of the project, including an initial scoping of the issues to be addressed.

2.  Our second stream addresses the Canadian population as a whole, and focuses on building understanding of Canadian public opinion on public policy and social trends as they are today and how they are changing over time.  Our signature project is Focus Canada, which is the longest-running public opinion research program in Canada (launched in 1976) that provides a credible, independent and sustained source of Canadian public opinion on important issues facing the country. 

Focus Canada was a syndicated study conducted on a quarterly basis by Environics Research until 2009 when the federal government instituted a moratorium prohibiting its own use of any form of syndicated public opinion research. In 2010, the Environics Institute took over Focus Canada as a non-profit project and is now conducting it as an annual survey. The emphasis is on continuing the ongoing tracking at the national and regional levels to identify how Canadian public opinion is changing (or not) over time on a wide range of issues.

3.  Finally, the Institute also conducts research that places Canada in an international context, to help us better understand how current and shifting public opinion and social values in this country compare with what is happening in other parts of the world.  For instance, the Environics Institute was the Canadian partner in the 2012 AmericasBarometer (www.AmericasBarometer.org), a multi-country public opinion survey on democracy and governance across the Americas that is conducted every two years by a consortium of academic and think tank partners in the hemisphere.

308: What are some of the most interesting findings you’ve had in your research?

KN - There are many interesting and often counter-intuitive findings from our research, including the following:

·         Most Muslims in this country want to fit in and be part of broader society, but most other Canadians think they do not. This is an example of where research can uncover a positive story that can lead to broader public acceptance (2006 Survey of Muslims in Canada).

·         Canadians feel strongly connected to the world outside the country’s borders, and this is reflected in a variety of ways. This includes a substantial financial contribution to international organizations and family members (i.e. through remittances), which in 2007 was estimated to total $27 billion, dwarfing the federal government’s official development assistance total of $4 billion (Canada’s World Poll, 2008).

·         Contrary to popular belief, most Aboriginal Peoples living in Canadian cities are not transient, but have made the city their home and is where they want to be. For many, the city is proving to be a venue for creative development of Aboriginal culture, rather than remaining bound to rural traditions.  The overall picture of Aboriginal life in the city is very different from what one sees on most of the country’s reserves (Urban Aboriginal Peoples Study, 2008-09).

·         Canadians believe being a good citizen is more than having a passport and obeying the law.  Just as important are having an active commitment to ones community, and being accepting of others who are different.  Moreover, Canadian-born and foreign-born citizens share a remarkably similar vision of what it means to be a good citizen (Canadians on Citizenship, 2012).

308: My readers might be more familiar with the Environics Research Group, which does political polling. Can you explain how the two organizations are different?

The Environics Research Group is one of Canada’s leading public opinion and market research companies, and as such is a commercial, for-profit business that conducts research on behalf of its clients. The Environics Institute is a non-profit entity that conducts its own research or collaborates with partners in launching joint projects that would not otherwise be done.  The Institute has no formal business or financial relationship with Environics Research (or with other businesses that are part of the Environics Group of Companies).  The Institute maintains its own financial and administrative systems, its own website and social media presence, and has an external Board of Directors. 

**********

Thanks to Dr. Neuman for taking the time to answer these questions, and I hope some of you will take a look at the work the Institute is doing.

Latest averages for Canada, Ontario, and Quebec

After a much needed break, it is time to get back to work and take a look at the polls that were missed while I was away. There were some interesting results, but for now let's look at what the current aggregates are showing.

The Liberals continue to lead nationwide, with 35.8% support to 29.1% for the Conservatives and 22.9% for the New Democrats. But that is down from the recent highs that the Liberals have been experiencing, where the aggregate put them at 39% or even 40% support. Nevertheless, the lead they hold over the Conservatives is significant and  the New Democrats still appear to be treading water.

The Liberals are ahead by sizable margins in Ontario, Quebec, and Atlantic Canada, and are in front by a few tenths of a percentage point in British Columbia. That is the tightest race at the moment, with the Liberals at 29.9%, the Conservatives at 29.4%, and the NDP at 27.8%. The Greens are strongest there at 11.8%.

In Ontario, the Liberals are ahead by 5.5 points in the aggregate with 38.7% to 33.2% for the Tories. In Quebec, the Liberals have slipped just below 40%, but that puts them well ahead of the NDP (26.1%) and the Bloc Québécois (17.9%). And in Atlantic Canada, the Liberals dominate with 47.6% to 28% for the NDP and only 18.8% for the Conservatives.

But the government is still well in front in Alberta (though they are no longer closer to 60% than 50% support) and in the Prairies, where their vote has probably been the sturdiest.

At the provincial level in Ontario, Kathleen Wynne's Liberals have fallen behind the Progressive Conservatives. However, the margin is minuscule: 34.5% for the PCs and 33.3% for the OLP. The NDP is still polling above their election haul of 2011, with 23.9% support.

It is difficult to determine whether there is any real cause for concern here for the Liberals. They have fallen 2.2 points since the last update at the end of May, but the PCs have hardly budged (up 0.3 points) and the NDP actually dropped 0.4 points. Instead, the provincial Greens made the gain (from 5.1% to 7.3%), one that is almost certainly a statistical anomaly. The best that can be said right now is that the OLP and PCs are in a very close race.

The race in Quebec is much more lop-sided, however. The opposition Liberals under Philippe Couillard have moved ahead dramatically, averaging 37.7% support. The Parti Québécois is down to 26.2%, though that is slightly better than where they stood a few weeks ago (primarily due to their lower results in CROP's polling).

The Coalition Avenir Québec continues to be unable to make up any ground, and has actually dropped to only 20% support. With Couillard at the helm of the PLQ, the usefulness of François Legault's party seems to have diminished in the eyes of many voters who were tired of both the PQ and Jean Charest's version of the Liberal Party.

Among the minnows, Québec Solidaire is best placed with 9.1% support, while Option Nationale was at 3.4% before Jean-Martin Aussant's resignation as leader. Where that small party will go from here remains to be seen.

During the summer months, political polling tends to stagnate. So, we should probably expect the parties in Ontario, Quebec, and at the federal level to be roughly where they are today in September. But two months is still a long time, even if most Canadians have checked out. Considering the stakes in Ontario and Quebec, where Wynne and Marois have flimsy minorities, and the historical lows for Stephen Harper's government, these two months could be important.