Robotics: Science and Systems XVI
Learning from Interventions: Human-robot interaction as both explicit and implicit feedback
Jonathan Spencer, Sanjiban Choudhury, Matt Barnes, Matthew Schmittle, Mung Chiang, Peter Ramadge, Siddhartha SrinivasaAbstract:
Scalable robot learning from seamless human-robot interaction is critical if robots are to solve a multitude of tasks in the real world. Current approaches to imitation learning suffer from one of two drawbacks. On the one hand, they rely solely on off-policy human demonstration, which in some cases leads to a mismatch in train-test distribution. On the other, they burden the human to label every state the learner visits, rendering it impractical in many applications. We argue that learning interactively from \textit{expert interventions} enjoys the best of both worlds. Our key insight is that any amount of expert feedback, whether by intervention or non-intervention, provides information about the quality of the current state, the optimality of the action, or both. We formalize this as a constraint on the learner's value function, which we can efficiently learn using no regret, online learning techniques. We call our approach Expert Intervention Learning (EIL), and evaluate it on a real and simulated driving task with a human expert, where it learns collision avoidance from scratch with just a few hundred samples (about one minute) of expert control.
Bibtex:
@INPROCEEDINGS{Spencer-RSS-20, AUTHOR = {Jonathan Spencer AND Sanjiban Choudhury AND Matt Barnes AND Matthew Schmittle AND Mung Chiang AND Peter Ramadge AND Siddhartha Srinivasa}, TITLE = {{Learning from Interventions: Human-robot interaction as both explicit and implicit feedback}}, BOOKTITLE = {Proceedings of Robotics: Science and Systems}, YEAR = {2020}, ADDRESS = {Corvalis, Oregon, USA}, MONTH = {July}, DOI = {10.15607/RSS.2020.XVI.055} }