Hierarchical Planning for Heterogeneous Multi-Robot Routing Problems via Learned Subteam Performance

A recent paper by members of the DCIST alliance proposes a new hierarchical planner for task allocation problems where tasks correspond to heterogeneous multi-robot routing problems defined on different areas of a given environment. The researchers tackled this complex planning problem with a novel planner which breaks down the complexity of the original problem into two subproblems: the high-level problem of allocating robots to routing tasks, and the low-level problem of computing the actual routing paths for each subteam. The planner uses a Graph Neural Network (GNN) as a heuristic to estimate subteam performance for specific coalitions on specific routing tasks. It then iteratively refines the estimates to the real subteam performances as solutions of the low-level problems become available. On a testbed problem having an area inspection problem as the base routing task, the experiments show that the new hierarchical planner is able to compute optimal or near-optimal (within 7%) solutions approximately 16 times faster (on average) than an optimal baseline that computes plans for all the possible allocations in advance to obtain precise routing times. The experiments also show that a GNN-based estimator can provide an excellent trade-off between solution quality and computation time compared to other baseline (non-learned) estimators.

Capabilities: T2C1 E+G (Simultaneous Task Assignment and Planning)

Points of Contact: Nicholas Roy (PI) and Jacopo Banfi

Video: N/A

Paper: https://ieeexplore.ieee.org/document/9705621

Citation: J. Banfi, A. Messing, C. Kroninger, E. Stump, S. Hutchinson, N. Roy. “Hierarchical Planning for Heterogeneous Multi-Robot Routing Problems via Learned Subteam Performance.” IEEE Robotics and Automation Letters, to appear (2022).

CORSAIR: Convolutional Object Retrieval and Symmetry-AIded Registration

This paper considers online object-level mapping using partial point-cloud observations obtained online in an unknown environment. We develop an approach for fully Convolutional Object Retrieval and Symmetry-AIded Registration (CORSAIR). Our model extends the Fully Convolutional Geometric Features (FCGF) model to learn a global object-shape embedding in addition to local point-wise features from the point-cloud observations. The global feature is used to retrieve a similar object from a category database, and the local features are used for robust pose registration between the observed and the retrieved object. Our formulation also leverages symmetries, present in the object shapes, to obtain promising local-feature pairs from different symmetry classes for matching. We present results from synthetic and real-world datasets with different object categories to verify the robustness of our method.

Capability: T1C1A: Multi-Modal Representations of Knowledge

Points of Contact: Nikolay Atanasov (PI) and Qiaojun Feng

Video: https://youtu.be/uIr4Q08BKqo 

Paper: https://arxiv.org/pdf/2103.06911

Citation: T. Zhao, Q. Feng, S. Jadhav and N. Atanasov, “CORSAIR: Convolutional Object Retrieval and Symmetry-AIded Registration,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021

Robust multimodal data association

A fundamental problem in robotic perception is matching identical objects or data, with applications such as loop closure detection, place recognition, object tracking, and map fusion. The problem becomes more challenging when matching is done jointly across multiple, multimodal sets of data, however, the robustness and accuracy of matching in the presence of noise and outliers is greatly improved in this setting. At present, multimodal techniques do not leverage multiway information, and multiway techniques do not incorporate different modalities, leading to inferior results. To address this issue, members of the DCIST alliance developed a principled mixed-integer quadratic framework to formulate the multimodal, multiway data association, and a novel algorithm, called Multimodality association matrIX fusER (MIXER), to find solutions. MIXER uses a continuous relaxation in a projected gradient descent scheme that guarantees feasible solutions of the integer program are obtained efficiently. Experiments demonstrated that correspondences obtained from MIXER are more stable to noise and errors than state-of-the-art techniques. Tested on a robotics dataset, MIXER resulted in a 35% increase in the accuracy of data association (measured as the F1 score) when compared to the best alternative.

Capability: T1C1C

Points of Contact: Jonathan How (PI), Kaveh Fathian

Video: NA

Paper: https://arxiv.org/pdf/2111.14990.pdf

Citation: P. C. Lusk, R. Roy, K. Fathian, J. P. How, “MIXER: A Principled Framework for Multimodal, Multiway Data Association,” in IEEE ICRA workshop on robust perception, 2021.

ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description

A recent paper by members of the DCIST alliance develops a method for tightly coupled object shape and pose optimization. Inspired by DeepSDF, which uses neural networks to regress a Signed Distance Function (SDF) description of object shape, they propose a bi-level object shape model named ELLIPSDF, to support joint object pose and shape optimization. At the coarse level, ELLIPSDF uses an ellipsoid model as it provides simple geometric constraints for object pose and scale initialization. A latent shape vector and an SDF neural network decoder are used at the fine level. The bi-level object model of ELLIPSDF allows initialization of object pose and scale from multi-view bounding-box measurements, followed by joint pose and ellipsoid-SDF shape optimization. ELLIPSDF was validated via large-scale experiments on the ScanNet dataset with multiple object categories.

Figure: Visualization of intermediate ELLIPSDF stages. First column: RGB image, depth image, instance segmentation (yellow), fitted ellipse (red) for a chair in ScanNet scene 0461. Second column: mean shape and ellipsoid with initialized pose. Third column: optimized fine-level and coarse-level shapes with optimized pose.


Capability: T1C1A – Multi-Modal Representations of Knowledge

Points of Contact: Nikolay Atanasov (PI) and Mo Shan 

Video: https://youtu.be/qVqr8j5E0ho

Paper: https://arxiv.org/pdf/2108.00355.pdf

Citation: M. Shan, Q. Feng, Y. Jau and N. Atanasov, “ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description,” IEEE/CVF International Conference on Computer Vision (ICCV), 2021.