Tuesday, April 24, 2012

Most Pollsters Did Not Pick Up Alberta Swing Voters

In this article by Ira Basen, the pollsters involved in the Alberta election rely on the Alberta election results to confirm their polls. As most Albertans know, most pollsters were far off the mark with about a 22 percent error rate.

Six polls were conducted between April 16 to April 21. The Wildrose Alliance Party scored between 40% to 41%, and the PC Party scored 31% to 34%. The actual election results were almost the reverse: PC Party 43.98% and the Wildrose Alliance Party 34.51%. The pollsters failed to record accurately the voting intentions of Alberta. Although these intentions may change overtime.

Interestingly, Forum Research was the only pollster to conduct polls in the last days of the election:
Saturday April 21: Wildrose Alliance Party 41% and PC Party 32%
Sunday April 22: Wildrose Alliance Party 38% and PC Party 36%
Monday April 23: Wildrose Alliance Party 34% and PC Party 44% (almost identical to the actual election results.)

The constant stream of poll results which favored the Wildrose Alliance Party may have worked against this party, by creating fear of a government further to the right. The federal Conservative Party of Canada, an offshoot of the Alberta Reform Party and with similar ideology to the Wildrose Alliance Party, have given Canadians a taste of the far right. In an Alberta election with no polls, the Wildrose Alliance Party may have squeaked in with a majority.

Polling, accuracy, and the contest for bragging rights

Everyone expects this race to be close, perhaps decided by just a fraction of a percentage point. There are several strongcandidates. But none belongs to a political party.

This is a race between pollsters, not politicians.

For Canada's highly competitive, billion-dollar market research industry, election night is the equivalent of Grey Cup Sunday.

When the votes are all counted, which polling firm will have come closest to getting the numbers right, and how close will that be?

Public opinion polling has been around since the 1930s and has been shown to yield remarkably accurate results, 19 times out of 20.

But for many people, polling still carries the whiff of snake oil.

How can 1,000 people, chosen at random, represent the views of 33 million? How can a pollster know what I'm thinking when none of them have ever asked me?

That's why the polling industry's credibility is on the line every election campaign.

Many people wouldn't mind seeing pollsters get knocked down a few notches. They can be irritating and arrogant on TV, pontificating about what "Canadians" are thinking.

If nothing else, we want to show that we're not as predictable as we might appear.

Indeed, that almost happened in 2004.

That was not a good election for Canada's pollsters. All the big firms seriously over-stated the Conservative and NDP vote, while under-estimating the Liberals by amounts that lay outside their margins of error.

The NDP won 15.7 per cent of the vote in 2004, far less that the 19 per cent predicted in the final Ekos poll, and the 20 in the SES (now Nanos) one.

The Liberals ended up with 36.7 per cent of the vote, not the 32 per cent predicted in the final Ipsos poll.

Despite these results, the polling industry was unapologetic.

Polls, they reminded us, are "snapshots in time." They are designed to tell you what people are thinking when they are asked a question, and are supposed to have no predictive value.

This can be quite handy when doing political polling. Pollsters stop polling a day or two before the voting starts. So if the actual results don't match the latest polling data, don't blame the pollster.

Maybe they're right. Maybe large numbers of people did change their minds at the last minute. We don't know. But it must be nice to be in a business where you can never be proven wrong.

If you are keeping score, the past two campaigns have been better for the pollsters.

2006 was a particularly good year. Nanos won the crown for most accurate polling. Its last poll, taken the day before the vote, came within 0.01 per cent of the final totals for all four major parties.

You can't get much closer than that. Other major polling firms were not far behind.

2008 was not as good. Bragging rights that time went to Angus Reid. Its final, online poll proved to be the most accurate. But it was still about a percentage point off the actual vote total for each of the parties.

Most of the other companies floundered when it came to the Conservative vote, under-estimating it by around three percentage points.

This year's election campaign comes at a time when the industry is under scrutiny as never before.

A few months ago, two of its leading lights, Allan Gregg of Harris Decima and Frank Graves of Ekos, complained to the Canadian Press that the industry needed to address serious methodological challenges if it hoped to remain credible.

In response, the industry's trade association, the Market Research and Intelligence Association, took out a full-page ad in Ottawa's Hill Times newspaper declaring its "confidence in the results of our polling and in the value that we provide to Canadians."

But the reality is that the polling business is changing dramatically.

Randomized telephone polling, which has been the backbone of the industry for decades, is rapidly giving way to online polling. See my earlier post.

This may be the last campaign where the telephone is the primary instrument of choice for pollsters. Everyone in the industry is scrambling to figure out where things are heading next.

For proof, you just have to look at the wide range of approaches that three of Canada's leading pollsters are employing.

Nanos continues to use randomized telephone surveys.

Ekos also uses the telephone, but it has replaced humans with machines. People who receive an Ekos robo-call are asked to punch in their answers to machine-generated questions on the keypads of their telephones.

Angus Reid uses an online survey. It randomly selects 2,000 people who have been non-randomly selected to participate in the company's marketing surveys, and asks them to answer questions about their political preferences.

All of the companies ask some variation of the "If a federal election were held today who would you vote for" question, but again, there are important differences.

Nanos asks people to name their favourite party. Unlike the others, it doesn't prompt them by providing the names.

All of them follow up the voter preference question with other questions designed to test, among other things, how strong the respondent's commitment is to their chosen party, and how likely they are to vote.

After all, with voter turnout now hovering below 60 per cent, roughly four out of every 10 people surveyed by pollsters probably won't wind up voting at all.

Figuring out who those people are, and properly factoring them into the equation, is critical to arriving at an accurate number.

All of these questions and methodologies reflect the latest in social science research and all are focused on one objective: to come closest to predicting how Canadians will vote on May 2.

Right now, there are still some fairly wide gaps among the polls, up to five percentage points difference in assessments of the NDP as well as the Conservative vote.

Over the next few days, you can expect to see those gaps narrow, as voters firm up their decisions, and pollsters increase the number of people they survey in order to reduce their margin of error.

It's a high-stakes, high-profile game that no one wants to lose.

Which is why politicians will not be the only people watching with sweaty palms as the votes are counted on Monday night.

Final Poll Records Swing to the PC Party

No comments:

Leave a Comment

Thank you for sharing your perspective.