Robotics: Science and Systems XVII

Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Study

Jianlan Luo^*, Oleg Sushkov^*, Rugile Pevceviciute^*, Wenzhao Lian, Chang Su, Mel Vecerik, Ning Ye, Stefan Schaal, Jonathan Scholz
^* These authors contributed equally

Abstract:

Over the past several years there has been a considerable research investment into learning-based approaches to industrial assembly; but despite significant progress these techniques have yet to be adopted by industry. We argue that it is the prohibitively large design space for Deep Reinforcement Learning (DRL); rather than algorithmic limitations per se; that are truly responsible for this lack of adoption. Pushing these techniques into the industrial mainstream requires an industry-oriented paradigm which differs significantly from the academic mindset. In this paper we define criteria for industry-oriented DRL; and perform a thorough comparison according to these criteria of one family of learning approaches; DRL from demonstration; against a professional industrial integrator on the recently established NIST assembly benchmark. We explain the design choices; representing several years of investigation; which enabled our DRL system to consistently outperform the integrator baseline in terms of both speed and reliability. Finally; we conclude with a competition between our DRL system and a human on a challenge task of insertion into a randomly moving target. This study suggests that DRL is capable of outperforming not only established engineered approaches; but the human motor system as well; and that there remains significant room for improvement. Videos can be found on our project website: https://sites.google.com/view/shield-nist.

Download:

Bibtex:

  
@INPROCEEDINGS{Luo-RSS-21, 
    AUTHOR    = {Jianlan Luo AND Oleg Sushkov AND Rugile Pevceviciute AND Wenzhao Lian AND Chang Su AND Mel Vecerik AND Ning Ye AND Stefan Schaal AND Jonathan Scholz}, 
    TITLE     = {{Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Study}}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2021}, 
    ADDRESS   = {Virtual}, 
    MONTH     = {July}, 
    DOI       = {10.15607/RSS.2021.XVII.088} 
}