Robotics: Science and Systems XIX
Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation
Gagan Khandate, Siqi Shang, Eric T Chang, Tristan L Saidi, Johnson Adams, Matei CiocarlieAbstract:
In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment’s transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots.
Bibtex:
@INPROCEEDINGS{Khandate-RSS-23, AUTHOR = {Gagan Khandate AND Siqi Shang AND Eric T Chang AND Tristan L Saidi AND Johnson Adams AND Matei Ciocarlie}, TITLE = {{Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation}}, BOOKTITLE = {Proceedings of Robotics: Science and Systems}, YEAR = {2023}, ADDRESS = {Daegu, Republic of Korea}, MONTH = {July}, DOI = {10.15607/RSS.2023.XIX.020} }