Canada’s troubled National Household Survey: We get to do it all again in 2016

The results of the 2011 National Household survey – the controversial replacement to Canada’s mandatory census – has been rolling out over the past several months, collecting corrections, criticisms and delays along the way.

The latest issue came this week, with the announcement that the next batch of intel would be delayed for a month due to data problems. It’s been a rocky road, to be sure. But the good news is, changes could be made before we have to do it again three years from now.

The already-dubious National Household Survey took a further shot to its credibility this week when it was announced that "issues in data processing" would delay the release of the next batch of information.

Two batches of information set for release on Wednesday would now be held a month and released on Sept. 11. For what it is worth, the information will focus on income, earnings, housing and shelter costs.

Previously released data, on aboriginal communities and immigration, as well as education and migration, have led to doubts about the compilation methods as well as the results.

[ Related: Stats Canada delays final release of National Household Survey ]

So the announcement that the next batch has been delayed due to issues hasn't exactly curried any favour.

“We were in the final stages and some of the results seemed odd, a bit,” census manager Marc Hamel told the Globe and Mail. “When we went back to the data-processing steps, we discovered that one of the steps was not applied correctly.”

Officially, Stats Canada has only called the problem an "issue with data processing" and says previous data was not affected. (On July 25, it released a correction to its migration statistics.)

[ More Brew: Toronto Star to offset advertising losses with online paywall ]

The data dump, whenever it comes, will be the third and final batch of information released from the 2011 National Household Survey, the voluntary questionnaire that replaced Canada's mandatory census with questionable results.

Ahead of the last information dump, chief statistician Wayne Smith said that we would "do a disservice to Canadians by trying to persuade them that this data is somehow flawed, when clearly it is not, and clearly it is more robust than anything else out there."

Robert Gerst, statistical methodologist at Calgary's Converge Consulting Group Inc., considers that bar fairly low, calling the NHS results a "worthless" replacement for the mandatory census.

"In effect, this replaced a random sample with a non-random sample. Non-random samples have their place, but making conclusions about the population isn’t one of them," he wrote.

"As a result, no conclusions about the Canadian population can be drawn from the NHS. Since making these types of conclusions is the whole point of a census, the NHS data is worthless."

It is not entirely clear what data was hit by the recent processing issues, nor what the processing issues were. A request for clarification has been made to Statistics Canada.
Also not clear is how the latest snafu will affect the next census taken in 2016.

The good news is that Statistics Canada is currently considering what shape the 2016 census will take, and asking data users to determine how best to address their needs.

There is currently no plan, or chance, to return to a mandatory census. But perhaps the next National Household Survey can learn from the hiccups it faced this time around.

Or at least, perhaps Stats Canada can catch all of its data processing issues earlier than the eve of the information’s release.

Want to know what news is brewing in Canada?
Follow @MRCoutts on Twitter