Robotics: Science and Systems XVII

Optimal Pose and Shape Estimation for Category-level 3D Object Perception

Jingnan Shi, Heng Yang, Luca Carlone


We consider a category-level perception problem; where one is given 3D sensor data picturing an object of a given category (e.g.; a car); and has to reconstruct the pose and shape of the object despite intra-class variability (i.e.; different car models have different shapes). We consider an active shape model; where—for an object category— we are given a library of potential CAD models describing objects in that category; and we adopt a standard formulation where pose and shape estimation are formulated as a non-convex optimization. Our first contribution is to provide the first certifiably optimal solver for pose and shape estimation. In particular; we show that rotation estimation can be decoupled from the estimation of the object translation and shape; and we demonstrate that (i) the optimal object rotation can be computed via a tight (small-size) semidefinite relaxation; and (ii) the translation and shape parameters can be computed in closed-form given the rotation. Our second contribution is to add an outlier rejection layer to our solver; hence making it robust to a large number of misdetections. Towards this goal; we wrap our optimal solver in a robust estimation scheme based on graduated non-convexity. To further enhance robustness to outliers; we also develop the first graph-theoretic formulation to prune outliers in category-level perception; which removes outliers via convex hull and maximum clique computations; the resulting approach is robust to 70 − 90% outliers. Our third contribution is an extensive experimental evaluation. Besides providing an ablation study on a simulated dataset and on the PASCAL3D+ dataset; we combine our solver with a deep-learned keypoint detector; and show that the resulting approach improves over the state of the art in vehicle pose estimation in the ApolloScape driving datasets.



    AUTHOR    = {Jingnan Shi AND Heng Yang AND Luca Carlone}, 
    TITLE     = {{Optimal Pose and Shape Estimation for Category-level 3D Object Perception}}, 
    BOOKTITLE = {Proceedings of Robotics: Science and Systems}, 
    YEAR      = {2021}, 
    ADDRESS   = {Virtual}, 
    MONTH     = {July}, 
    DOI       = {10.15607/RSS.2021.XVII.025}