Mainstreet admits 'big polling failures' after predicting Nenshi would lose Calgary election

The polling firm that consistently predicted Bill Smith would be elected mayor of Calgary and challenged third-party observers who raised questions about its results now admits to "big, big polling failures."

"We were the worst by far," Mainstreet Research president Quito Maggi said the day after incumbent Naheed Nenshi defeated Smith by eight percentage points.

For weeks prior to Monday's vote, Mainstreet released poll after poll suggesting Smith was leading Nenshi by between nine and 17 points.

"These numbers point to the near certain election of Bill Smith on Oct. 16," Maggi said in an Oct. 7 news release.

"It's likely Nenshi will not be the only incumbent to be the victim of this change election," he said.

In the actual vote, not only did Nenshi win — so, too, did all 10 incumbent councillors who ran for re-election.

'Wonky sample'

Maggi believes his company's methodology isn't fundamentally flawed, but said it's investigating whether it failed to connect with some younger voters and those who don't speak English as their first language.

The company is also looking into the possibility that some people were simply hanging up on the automated calls that Mainstreet relies on to reach voters and gather their opinions.

In particular, Maggi admitted the second poll his company did, which had Smith ahead by 17 points, was based on a "wonky sample."

"We knew that it wasn't a great sample, but it's the sample we get," Maggi said.

"Our code says we have to publish polls, even when we think it might be an outlier. And I certainly think that second one was, for sure, an outlier, absolutely."

The demographic breakdown of that poll put Smith ahead by 19 points among female voters and gave Smith a staggering 36-point lead among voters aged 18 to 34.

Both of those demographics are traditionally stronger for Nenshi, which prompted several political scientists to raise questions about Mainstreet's results.

Other polls put Nenshi ahead

Mainstreet did the bulk of the public polling during the election campaign, releasing three polls over a period of three weeks that were paid for by Postmedia, the company that owns the Calgary Sun and Calgary Herald.

But late in the race, a poll commissioned by the LRT on the Green Foundation — a non-profit citizens group dedicated to light-rail transit expansion in Calgary — was released, which had Nenshi in the lead.

Then, in an unusual move, a group of academics opted to release partial results from a larger study they were conducting that included public polling of voter preferences in Calgary.

That polling, conducted by Forum Research, also suggested that Nenshi was leading Smith.

Both of the non-Mainstreet polls included demographic breakdowns more in line with what political observers expected when it came to young voters and female voters.

"Part of the reason we felt compelled to release some of these results is we understand the power of polls," said Michael McGregor, a Ryerson University professor and lead researcher in the academic group that commissioned the Forum poll.

"I'm not going to go out there and say, perhaps like some other people have said, that this is how the election is going to be. I will say that this is, I think, a good indication of how the electorate feels about the candidates."

Mainstreet vs. political scientists

As pundits and political scientists publicly questioned Mainstreet's results during the campaign, Mainstreet didn't sit back and accept the criticism lightly.

Maggi personally engaged on Twitter, in particular, with Duane Bratt, a professor at Mount Royal University.

Bratt was especially critical of the demographic breakdown in Mainstreet's numbers, while Maggi questioned Bratt's credibility.

Meanwhile, in an interview with 660 News the day before the election, Mainstreet executive vice-president David Valentin suggested political commentators in Calgary were biased.

"Certainly I've seen a lot of behaviour from political scientists that I would say is quite shocking in this election campaign, and some of it, quite frankly, is quite appalling," he said.

In that same interview, Valentin said pundits who were critical of Mainstreet could expect a response after the vote, as the company planned on "singling people out" for "what exactly it is they said and did" about the poll results.

"I think anyone who comments to the media should expect that their comments are going to receive scrutiny after the fact," he said.

"I think that's fair."

Pre-election bravado

Mainstreet also released a "polling scorecard" just before the election, encouraging Calgarians to compare its results and those of the other polling firms to the actual vote totals on Monday.

(Neither Forum Research nor Asking Canadians, the firm that conducted the poll for LRT on the Green, asserted that their polling results were predictive of actual voting outcomes.)

The scorecard was the latest in a series of moves from Mainstreet asserting confidence in the firm's results.

"If polling were poker, this is the part where I would go all in," Maggi tweeted on Oct. 11, the day the LRT on the Green poll was released.

"I would bet $10 million we're closer than that pseudo poll today," he said.

On the Friday before the election, the day the Forum poll was released, Valentin also reiterated his faith in Mainstreet's numbers.

"Some people are going to have a very bad Monday, but not me," he tweeted.

Post-election humility

As the election results rolled in and Nenshi's victory became clear, Maggi said he was incredulous, at first.

"It was just unbelievable to us what the result was," he said.

"I had actually said as much as it would take a miracle for Naheed Nenshi to get re-elected, and then he won by seven per cent."

But as the night wore on, Maggi struck a more conciliatory tone, even agreeing with one online commenter that the result made Mainstreet look "silly."

"Silly is an understatement," Maggi tweeted in response. "I take full responsibility for this error and launching a full investigation on the polling error."

He spent the next couple of days defending Mainstreet from legions of online critics.

"Admittedly, the industry and myself could use a touch of humility," he tweeted on Wednesday.

"Will do my part."

Edmonton mayor pans automated polls

Meanwhile, Edmonton Mayor Don Iveson said automated phone polls, in general, are unreliable and harmful to the public discourse, in his view.

"We should stop reporting things that are off by 10 or 15 per cent and distort or oversample or undersample certain groups. But that's just me," he told reporters.

"And I know some of your outlets have decided to do that, but I'm just going to say as a free-time, political commentary that those polls are just not useful. They distort the public discourse," Iveson said.

"They do a disservice to citizens, quite frankly, because it's not what a credible poll with good methodology shows."