Saskatchewan: Projection vs. Results

The election in Saskatchewan was held a little more than two months ago, so I apologize for the lateness of this analysis. It is a bit of necessary housekeeping, however. 

If this sort of post-mortem is not of interest, I invite you to read my article today on The Globe and Mail website here. It takes a somewhat quirky approach to analyzing the NDP leadership race.

Saskatchewan was always going to be the easiest to project of the five provincial elections held in 2011. A massive win by Brad Wall was a foregone conclusion, while the New Democrats were the only ones capable of forming the Official Opposition. The real challenge was nailing down the number of seats, which in the end proved somewhat more difficult than I had expected. But the projection for Saskatchewan was, in fact, one of the best for the projection model.

Of course, Brad Wall's Saskatchewan Party did win its majority government, taking more seats and more of the vote than expected. Dwain Lingenfelter's NDP was handed a major defeat as Lingenfelter himself was defeated in his riding. The NDP leader stepped down shortly afterwards.

ThreeHundredEight projected 43 seats for the Saskatchewan Party and 15 for the New Democrats. Instead, the SP won 49 and the NDP only nine. The biggest problem in the model was, as will be shown, the difficulty in forecasting the SP's support in Saskatoon and especially Regina. In the rural parts of the province, the model did exceptionally well.

But with the actual provincial vote the projection would have been no different: 43 SP and 15 NDP.

The model was off by six seats per party, which is a relatively large amount. But the model only made six incorrect calls, all going the same way. This gave the projection a riding accuracy of 89.7%, which is one of its better results.

The seat ranges suggested that the NDP could do better than projected but only slightly worse. The reality was quite different, however.

The seat ranges with the actual provincial vote, on the other hand, were much better: 43-48 for the Saskatchewan Party and 10-15 for the New Democrats. That was much closer to the end result.

The projection model under-estimated Saskatchewan Party support by 1.8 points, forecasting 62.4% to the actual 64.2%. It over-estimated NDP support by 1.6 points and Green support by 0.2 points, for an overall error of 1.2 points per party. That is quite acceptable, and one of the model's best results.

The chart above summarizes some of the results of the individual riding projections for each party.

Overall, the average victory margin in the six wrong calls was 8.4%, a relatively large amount. But aside from these six ridings, the model performed very well.

Projected Results

Every riding projection for the Greens was within 5% of the actual result, while almost four out of every five NDP projections was within that margin. Another three-quarters of SP projections was within 5%. In all, 67.2% of ridings were projected to within 5% for every party. That is the model's second best performance, ranking only behind Manitoba 2011.

On average, the Saskatchewan Party was under-estimated by three points in each riding, while the NDP was over-estimated by 2.4 points. The Greens were slightly over-estimated by 0.3 points per riding.

The margin of error per party per riding was +/- 2.8%, again the model's second best performance after Manitoba.

The real problem was in the cities. Whereas the model was off by an average of +/- 2.0% per party per riding projection outside of Regina and Saskatoon, within those cities the model was off by an average of +/- 3.9%. If we just concentrate on the SP and the NDP, that margin of error in the cities was +/- 5.6%. The need for a regional model becomes all the more apparent - the polls with regional breakdowns had indicated that the SP's support was higher in the cities than should have been expected by province-wide trends.

The model did make many close calls in individual ridings. In three of them, the projections for the Saskatchewan Party, the NDP, and the Greens were all bang-on. Two of them were rural (Batoche and Wood River), while one of them was urban (Saskatoon Massey Place). In six other ridings, the average error per party was less than 1%.

The projection for the 2011 Saskatchewan election was, in the end, a good one, with the exception of the model's performance in Regina. A few tweaks to incorporate regional data, which are being made in models for other provinces going forward, would have resulted in a very close projection. This makes the Saskatchewan projection a qualified success - the problems are easily recognizable and avoidable for the future.