A recent paper by members of the DCIST alliance develops a method for tightly coupled object shape and pose optimization. Inspired by DeepSDF, which uses neural networks to regress a Signed Distance Function (SDF) description of object shape, they propose a bi-level object shape model named ELLIPSDF, to support joint object pose and shape optimization. At the coarse level, ELLIPSDF uses an ellipsoid model as it provides simple geometric constraints for object pose and scale initialization. A latent shape vector and an SDF neural network decoder are used at the fine level. The bi-level object model of ELLIPSDF allows initialization of object pose and scale from multi-view bounding-box measurements, followed by joint pose and ellipsoid-SDF shape optimization. ELLIPSDF was validated via large-scale experiments on the ScanNet dataset with multiple object categories.

Figure: Visualization of intermediate ELLIPSDF stages. First column: RGB image, depth image, instance segmentation (yellow), fitted ellipse (red) for a chair in ScanNet scene 0461. Second column: mean shape and ellipsoid with initialized pose. Third column: optimized fine-level and coarse-level shapes with optimized pose.

 

Capability: T1C1A – Multi-Modal Representations of Knowledge

Points of Contact: Nikolay Atanasov (PI) and Mo Shan 

Video: https://youtu.be/qVqr8j5E0ho

Paper: https://arxiv.org/pdf/2108.00355.pdf

Citation: M. Shan, Q. Feng, Y. Jau and N. Atanasov, “ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description,” IEEE/CVF International Conference on Computer Vision (ICCV), 2021.