Dr Richard Sutton
Reinforcement Learning and Artificial Intelligence

Dr Rich Sutton has been appointed as the iCORE Chair in Reinforcement Learning in the Department of Computing Science at the University of Alberta. His research program will be a cornerstone of the new Alberta Ingenuity Centre for Machine Learning (AICML), which was recently established in Edmonton. To develop the research program, Dr Sutton has received an iCORE Chair and Professor Establishment (CPE) grant of $600,000 per year for five years, for a total of $3 million. This represents roughly 50 percent of the total $6.75 million budget. The University of Alberta is contributing approximately $540,000 per year for a total of $2.7 million.

Biographical Information
Dr Richard S. Sutton was born in Toledo, Ohio, and grew up in Oak Brook, Illinois, a suburb of Chicago. He received a BA degree in psychology from Stanford University in 1978, and MS and PhD degrees in computer science from the University of Massachusetts in 1980 and 1984. He worked for nine years at GTE Laboratories in Waltham as principal investigator of their connectionist machine-learning project, and for three years at the University of Massachusetts in Amherst as a research scientist in the computer science department. In 1998-2002 Dr Sutton worked at AT&T Labs in Florham Park, New Jersey. In August of 2003 he took up his post as professor of computing science at the University of Alberta.

He is the author of the original paper on temporal-difference learning and, with Andrew Barto, of the textbook Reinforcement Learning: An Introduction. He is a fellow of the American Association for Artificial Intelligence.


Research Program Overview
The research program will be making advances in reinforcement learning in artificial intelligence. Reinforcement learning applies to any task that involves taking a sequence of actions where the effect of one action influences the progression of subsequent actions - for example, flying a helicopter, playing backgammon, scheduling elevators, scheduling constrained resources. Another part of this kind of learning is a defined long-term goal - for example, staying airborne until the destination is reached, winning a game, making most efficient drops and pick-ups to twelve stories with three elevators, or optimizing the allocation of scarce resources. The research explores ways to learn the best options in decision-making situations, and takes into account long-term benefits as well as immediate rewards.

Reinforcement learning methods are generating increasing attention because they can be applied as part of a computer system's normal operation, without requiring special supervision or training information. The computer system does the learning in the course of operation. So, for example, rather than having to reprogram an elevator system when a new busy client moves onto the sixth floor, the elevator control system would learn to optimize its pattern based on new usage.

Reinforcement learning has had a significant impact on several disciplines, including computer engineering, psychology, and neuroscience. It helps to develop optimized computerized control systems, but it also asks helps to understand the way humans make decisions, and how this might be implicated in the wiring of the brain.

The most common application of reinforcement learning can be found in robotics, but it is also important in many applications of process control, communications networks, and finance.

Research team collaborations
The team will be built up to include the director (Dr Richard Sutton), a CRC chair (Dale Schuurmans), two associated faculty members, ten graduate students, two postdoctoral fellows, and two programmer/technicians. The iCORE Chair builds on the substantial strengths in machine learning and artificial intelligence with current University of Alberta faculty, such as Professors Dale Schuurmans, Robert Holte, Russ Greiner, Jonathan Schaeffer, Martin Muller, Michael Buro, Michael Bowling, Hong Zhang, and Vadim Bulitko. In addition, the research team will benefit from University of Alberta's broader strengths in artificial intelligence, from Professor Randy Goebel in knowledge representation, Professor Walter Bischof in computer vision, Professor Francis Jeffery Pelletier in logical methods, and Professor Renee Elio in cognitive science.


Related Links :
Dr Sutton's Homepage
Alberta Ingenuity Centre for Machine Learning (AICML)

About the Alberta Ingenuity Center for Machine Learning (AICML)
The Alberta Ingenuity Centre for Machine Learning (AICML) represents an exciting, nationally and internationally recognized node of researchers working in the field of Machine Learning. The Centre supports and promotes curiosity-driven Machine Learning research, and leading edge scientific and commercial applications in the bioinformatics and interactive entertainment industries. Dr Sutton and his team will work closely with the AICML, sharing interests and administrative resources.