Weighing the polls (again)

In the wake of the 2011 federal election, I took a look at what had been the Achilles' heel of my projection model at the time: the daily reduction in weight of a given poll. For the federal election, I reduced the weight of a poll by 7% per day. This was not aggressive enough to be able to adequately record the increase in New Democratic support in the final week of the campaign, throwing everything off.

Accordingly, after some analysis I decided to instead decrease the amount of weight given to a poll by 23% per day. This worked pretty well during the last set of provincial campaigns, enough so that it was not the cause of any major errors.

That does not mean that the daily weight reduction cannot be dialed in more accurately. However, polls in the five campaigns acted very differently from one province to the next. On the one hand, in Ontario, Manitoba, and (especially) Prince Edward Island, increasing the rate of reduction increased the degree of accuracy of the vote projection model. On the other hand, in Saskatchewan and Newfoundland and Labrador, decreasing the rate of reduction upped the accuracy.

The chart above records the total cumulative error of the projection compared to the vote results for major parties. This means that if the model is under-estimating the support of one party by two points and over-estimating the support of another by two points, that equates to a total error of four points. A daily rate of reduction of 0.75, for example, means a rate of reduction of 25% per day.

For the federal election, a rate of reduction of about 25% would have worked well. For Ontario, a reduction of around 40% or more would have been best, while 35%-40% for Manitoba would have yielded better results. For P.E.I., the accuracy increased with every increase of the rate of reduction, but for Newfoundland and Labrador and Saskatchewan it was the opposite.

What this chart shows is that over the six elections, a daily rate of reduction of between 30% and 40% would have yielded the best cumulative result of 43.7 points of total error. This indicates that, in the future, a daily reduction in this range would be best.

But how to decide? A reduction of 30% would have worked better in the federal, Newfoundland, and Saskatchewan campaigns, a reduction of 40% would have been the better option in Ontario, Manitoba, and Prince Edward Island.
What I've done in the chart above is rank each rate of reduction from one to six for each election. The lowest total when adding up these rankings, then, should tell us which rate of reduction would have had the better results across the board.

That turns out to be a reduction of 35% per day, or by a factor of 0.65. This is the top rate for Manitoba, and it ranks in the middle for every other election. Upping the rate of reduction to 40% would have been too much for the federal, Newfoundland, and Saskatchewan elections, while putting it to 30% would not have been the best result for anyone.

Moving forward, the vote projection model will be using this 35% daily reduction during a campaign. Outside of a campaign, I will be applying that 35% reduction on a weekly basis.

But that is only one part of the story. The next task is to figure out the difference between poll results and election results, so that the vote projection model can bridge the gap between the two. Stay tuned.