Martha White: Getting reinforcement learning ready for the real world

Martha White wants to know how things work.

It could be the "elegant complexity" of mathematical functions which grabbed her attention as a young student. Or the way she'll sometimes let a weed grow in her garden at home, just to see what will happen. White is not content with just knowing something works -- she wants to know how it works. And how to make it work better.

"People say I have a 'high ping rate.' I often pepper people with questions. I just generally like understanding the things around me," she says.

As an Associate Professor of Computing Science at the University of Alberta, a Canada CIFAR AI Chair and a Fellow at Amii, the Alberta Machine Intelligence Institute, White focuses on building artificial intelligence that can handle the complexities of the real world.

It's a question that first caught her attention as an undergrad studying math and computer science at the University of Alberta. While there, she became interested in the work of professor (and Amii fellow) Michael Bowling. After speaking with Bowling and discovering machine learning, she was drawn to the discipline – particularly the promise AI shows in solving real-world problems.

White continued with a PhD in Computing Science at the U of A. When applying for jobs, she found herself drawn to an offer from Indiana University Bloomington, an institution with a growing department and excitement about the use of machine learning. After a few years there, she made the return to her alma mater, joining a growing group of researchers and businesses turning to Alberta for AI opportunities.

The University of Alberta and Amii have become global leaders in reinforcement learning (RL), a form of artificial intelligence that allows machines to tackle complex problems using trial-and-error learning. White says that much of the work on RL so far has been theoretical or done in laboratory settings. However, she says that is about to change.

"Reinforcement learning in the real world is coming. It's just like, a few years ago, there used to be just a little bit of machine learning, and then it was everywhere. That's going to happen with RL."

As with any scientific advancement, there are still hurdles in applying reinforcement learning in the real world. Much of White's research is on improving RL techniques to address these concerns, such as sample efficiency. RL algorithms are often very sample inefficient -- essentially, they have to obtain a lot of experience to properly learn how to do a task. That can limit applicability to real-world applications. For these systems to work well in the real world, White also aims to make reinforcement learning agents more stable, capable of learning and incorporating new information over long periods. The challenges are complex, she says, but the potential benefits of real-world RL are worth it.

"I want the things I do to help people. I would like my work to make the world a better place today."

White will speak more about applying reinforcement learning to real-world problems in a keynote speech during Amii's AI Week, held May 24 - 27 in Edmonton.