Robotics: Science and Systems XVII

On complementing end-to-end human behavior predictors with planning

Liting Sun, Xiaogang Jia, Anca Dragan


High capacity end-to-end approaches for human motion (behavior) prediction have the ability to represent subtle nuances in human behavior; but struggle with robustness to out of distribution inputs and tail events. Planning-based prediction; on the other hand; can reliably output decent-but-not-great predictions: it is much more stable in the face of distribution shift (as we verify in this work); but it has high inductive bias; missing important aspects that drive human decisions; and ignoring cognitive biases that make human behavior suboptimal. In this work; we analyze one family of approaches that strive to get the best of both worlds: use the end-to-end predictor on common cases; but do not rely on it for tail events / out-of-distribution inputs — switch to the planning-based predictor there. We contribute an analysis of different approaches for detecting when to make this switch; using an autonomous driving domain. We find that promising approaches based on ensembling or generative modeling of the training distribution might not be reliable; but that there very simple methods which can perform surprisingly well — including training a classifier to pick up on tell-tale issues in predicted trajectories.



    AUTHOR    = {Liting Sun AND Xiaogang Jia AND Anca Dragan}, 
    TITLE     = {{On complementing end-to-end human behavior predictors with planning}}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2021}, 
    ADDRESS   = {Virtual}, 
    MONTH     = {July}, 
    DOI       = {10.15607/RSS.2021.XVII.037}