Robotics: Science and Systems XIX

Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation

Gagan Khandate, Siqi Shang, Eric T Chang, Tristan L Saidi, Johnson Adams, Matei Ciocarlie

Abstract:

In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment’s transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots.

Download:

Bibtex:

  
@INPROCEEDINGS{Khandate-RSS-23, 
    AUTHOR    = {Gagan Khandate AND Siqi Shang AND Eric T Chang AND Tristan L Saidi AND Johnson Adams AND Matei Ciocarlie}, 
    TITLE     = {{Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation}}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2023}, 
    ADDRESS   = {Daegu, Republic of Korea}, 
    MONTH     = {July}, 
    DOI       = {10.15607/RSS.2023.XIX.020} 
}