Are online polls any good?

·4 min read

Polling and market research firms are increasingly relying on online polling, but Canadian firms are split on whether they think this is a better way to gauge public opinion.

Results from online-only polling cannot be considered a probability sample, which uses some form of random selection. However, polling companies who use this method, such as Léger, have been including notes with their results, indicating what the margin of error would have been for comparison's sake, if a probability sample had been used.

Joanna Everitt, a political science professor at the University of New Brunswick, said there are issues with online polls, but they do tell us something.

She noted that, as a general rule, the more people who are surveyed, the smaller the margin of error, though if it’s not a random sample you can’t make that sort of analysis.

Phone surveying also has issues, she said, noting that a low response rate from people now filtering their calls also skew results. Online panels are often responded to by individuals interested in current events, Everitt said. If a survey is buried in your email box, you might complete a survey only if you are interested in a topic, she said.

Access to online resources is also a factor, Everitt said, adding that according to a recent report from the Canadian Radio-television and Telecommunications Commission, New Brunswick had more households that only had landlines and no internet than most other regions in the country.

It is also more likely that online surveying skews younger, Everitt said.

But Margaret Brigley, CEO of Narrative Research, a firm that draws from an online panel to conduct each survey, meaning no margin of error can be assigned to results, said even compared to 10 years ago, many more people are online, even older demographics. The cost effectiveness is also driving the trend, she said, and it’s also useful that people can complete an online survey at any hour of the day.

Nik Nanos, chair of Nanos Research, said his firm uses a mix of telephone and online methods, so that a margin of error can still be attributed, he said.

Unlike surveys which draw respondents from a panel people have opted to join, “nobody can volunteer. Everyone is randomly selected via the phone,” he said, so it is a proper probability sample and a margin of error in the traditional sense can be applied. Even if a survey may be completed online, respondents have spoken to a live person first, he said.

“It’s much more expensive, but we have greater confidence in the results,” he said.

Still, Nanos said, question design and timing are probably more important.

Mainstreet Research firmly believes random samples obtained through the phone are best and clients prefer this, said Joseph Angolano, the firm's vice-president.

Mainstreet uses interactive voice recordings, live phone calls and online tools to conduct its work. However, Angolano noted that Europe and the United States are moving to online panel designs that are considered randomized and thinks Canada may be moving in this direction. But , he noted bots and click farms have been issues for some U.S. firms.

Christian Bourque, executive vice-president of Léger, which uses online panels, said the question of whether something can be considered random is part of a 15-year academic debate at the Canadian Research and Insights Council.

Bourque questioned how long we can go on considering phone sampling as random and unproblematic since response rates are so low and social desirability bias has been found to be more prevalent when speaking to a real person instead of answering an online survey, he said, noting that this seems to have been at play in some polling around Brexit.

On 338Canada, a website of electoral projections created by Philippe Fournier who teaches at Cégep de Saint-Laurent in Montreal, Léger and Nanos Research each received an A+ ranking based on accuracy, consistency and transparency, though they use different methods.

With businesses, governments and non-profits looking to understand Canadians to know how to respond to the pandemic, some polling companies said they have been busier than usual.

Nanos said not only has his firm been busier, during the lockdown, early in the first wave, respondents were participating at higher levels than usual.

Clara Pasieka, Local Journalism Initiative Reporter, Telegraph-Journal