AAAI 2018 New Orleans, Louisiana, USA

A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress.

2pm – 6pm, February 2, 2018

Inverse reinforcement learning (IRL) seeks to find the preferences of another agent using its observed behavior, thereby avoiding a manual specification of its reward function. IRL is appealing because of its ambitious potential to use data recorded in everyday tasks (e.g., driving data) to build autonomous agents capable of modeling human behavior and socially collaborating with others in our society. Research related to IRL has grown tremendously in recent years because the reward function is inherently more transferable compared to the observed agent’s control policy, and IRL has potentially ground-breaking applications such as autonomous vehicle control, predicting the future behavior of the demonstrator to facilitate multi-agent decision making. This tutorial will provide a comprehensive and unified review from early research to current methods as well as open questions in IRL. The tutorial requires no prior knowledge of IRL but assumes a familiarity with basic probability and statistics.

This idea has been studied extensively in economics by game theorists and investigated in preliminary ways by sub-groups in AI for game playing and planning. Preliminary outcomes emphatically suggest that reasoning about types is an important tool for solving problems involving interactions with high uncertainty about agent behaviors and in which extensive online learning based on trial-and-error is undesirable or infeasible. This half-day tutorial will provide a comprehensive and unified introduction to the theory and practice of type-based methods spanning early research in game theory to the latest work in AI, as well as outlining open problems for future research. The tutorial requires no prior knowledge of multiagent theory but assumes familiarity with basic probability and statistics.

Tutorial Outlines.

Tutorial Slides.

Saurabh Arora is a Ph.D student in Computer Science Department at The University of Georgia. His research-work focuses on ‘on-line inverse reinforcement learning under occlusion.’ Prashant and Saurabh are co-writing a comprehensive survey article on IRL that will be submitted to AI Journal shortly.

Dr. Prashant Doshi is a tenured Professor of Computer Science and faculty fellow of the AI Institute at The University of Georgia, USA. His research interests lie broadly in artificial intelligence and robotics. Specifically, he studies automated decision-making under uncertainty in multiagent settings, non-cooperative game theory, and robot learning, specifically inverse reinforcement learning. He was a visiting professor at the University of Waterloo in 2015, and he has also had short stints at the IBM T. J. Watson Research Center. He has published 125+ articles and papers in journals, conferences, and other forums in the fields of agents and AI. His research has led to publications in the Journal of AI Research, Journal of AAMAS, AAAI, IJCAI and AAMAS conferences. In 2011, Prof. Doshi was awarded UGA’s Creative Research Medal for his contributions to automated decision making. Prof. Doshi teaches introductory courses on AI and Robotics to undergraduate and graduate students, and a course on decision making under uncertainty to graduate students, all of which are well received among the students.