Pioneering AI researcher to advise RBC's machine learning lab

A pioneer in machine learning from the University of Alberta is teaming up with the Royal Bank of Canada on artificial intelligence research.

Richard Sutton, a professor at the school's department of computer science and a graduate of the University of Massachusetts, will advise the bank's machine learning research division and collaborate with RBC's second AI research lab, to be located in Edmonton.

Sutton specializes in the same branch of machine learning that Google's AlphaGo computer program used, in part, to beat one of the highest-ranking professional players of the board game Go — until recently, a notoriously difficult game for computers to play.

The announcement is the latest in a string of AI-related partnerships, acquisitions and investments that have been struck in Canada in recent months — the most high-profile of which have involved Facebook and Google, which have been in a fierce competition for access to talent.

'The father of reinforcement learning'

For over three decades, Sutton has specialized in reinforcement learning. In this branch of machine learning, an algorithm is designed to receive either a reward or penalty based on its behaviour, and learns to make choices that will result in the most reward — and, hopefully, most desired behaviour — over time.

That behaviour could be anything from a self-driving car successfully navigating a busy highway without incident, to a computer that most sends elevators most efficiently from floor to floor.

Yann LeCun, Facebook's Director of AI, called Sutton "the father of reinforcement learning" in a Facebook post last March.

In fact, Sutton wrote the book, publishing the first edition of the textbook Reinforcement Learning: An Introduction in 1998 with his co-author and former academic advisor Andrew Barto.

Sebastian Thrun — himself an early pioneer in AI and self-driving cars, who later co-founded Google's X lab for research and development efforts in 2007 — reviewed the book shortly after its publication, calling it a worthy read for AI novices and veterans alike.

But despite the technique's long history, Sutton said in a statement that, in the world of finance, researchers have "only scratched the surface of what reinforcement learning can do."

Banking on disruption

"All of these efforts really began, not only because AI is critical for us overall — and it's part of our future, really — but also because we've seen a lot of developments come out of Canadian universities that have shaken up the industry," said Foteini Agrafioti, head of RBC Research, in an interview.

At an investor conference in Toronto earlier this month, executives from TD, RBC, National Bank and Scotiabank said that artificial intelligence may be the most disruptive technology to come for banks.

Scotiabank, for example, gave the University of Toronto a gift of $1.75 million last September for the creation of a "Scotiabank Disruptive Technologies Venture" — going in part towards "research and independent study" in artificial intelligence and the sponsorship of the school's artificial intelligence conference.

Similarly, RBC announced the creation of its machine learning research division in October, as part of an initial partnership with the University of Toronto. It's located within the school's Banting Institute.

A spokesperson declined to say how much the company has invested in AI research to date, saying only that "these investments will be in the tens of millions of dollars over the coming years."

The 20-person research lab in Toronto is run by Agrafioti, the co-founder and co-inventor of Nymi, a wearable heart rate monitor that its creators hoped could replace passwords and other forms of authentication with the unique rhythm of the wearer's heart.

Machine learning — which includes branches such as deep learning and reinforcement learning — is "one of those types of technology that you would really find underpinning many different applications across our businesses," said Agrafioti, adding "it's typically used to manage fraud, and risk, as well as client services."

RBC plans to submit its first research paper for peer-review to this year's International Conference in Machine Learning, which will be held in Sydney, Australia in August.