Learning to Regrasp by Learning to Place

Our system creates and leverages a regrasp graph to solve the regrasping problem. Each node $\mathcal{N}$ in the regrasp graph includes stable poses $\{\mathcal{T}^{i}_{g}\}$ for all objects $\{\mathcal{O}_{i}\}$. An edge $\mathcal{E}_{uv}$ connects two nodes $\mathcal{N}_{u}$ and $\mathcal{N}_{v}$, if there is one and only one object $\mathcal{O}_{j}$ whose stable pose is changed between the two nodes, and there exists a valid pair of grasping operation $\mathscr{G}_{k}^{j}$ and placement operation $\mathscr{P}_{k}^{j}$ that transform the object’s stable pose. The edge indicates that the robot can first grasp the object $\mathcal{O}_{j}$ from its stable pose under one node and then place it at the stable pose in the other node.

Initially, all objects $\{\mathcal{O}_{i}\}$ are stably placed in the scene. All objects’ initial poses define the first node in the regrasp graph. For each object $\mathcal{O}_{j}$, we use a deep-learned stable object placement pose prediction model (Sec. 4.2) to estimate the object’s alternative stable poses under the current surrounding environment. Each alternative stable pose of the object $\mathcal{O}_{j}$, along with other objects’ stable poses, leads to a new node in the regrasp graph. Whenever a new node is created, our system checks whether there are edges connecting it to the rest nodes and add these edges to the graph.

Leveraging the constructed regrasp graph, the solution of a regrasp problem is a path from a starting node $\mathcal{N}_{0}$ to a target node $\mathcal{N}_{t}$ that contains a stable object placement pose $\mathcal{T}_{g}^{t}$ allowing a valid grasping operation $\mathscr{G}_{l}^{t}$ to achieve the target grasp pose $\mathcal{G}_{*}^{t}$ for the target object $\mathcal{O}_{t}$. We use depth-first-search algorithm to search for a valid path.

Given an object $\mathcal{O}_{i}$ and a static surrounding environment $\mathcal{S}=\mathfrak{O}-\{\mathcal{O}_{i}\}$, we first train a generative model that learns to propose a diverse set of poses $\{\mathcal{T}_{k}^{i}\}$ for the object $\mathcal{O}_{i}$ to be stably placed in the environment $\mathcal{S}$.

Fig. 3 (left) illustrates the inputs, outputs, and design overview for this network. In our implementation, we represent both the partial observations of the object $X_{i}\in\mathbb{R}^{m\times 3}$ and surrounding environment $X_{\mathcal{S}}\in\mathbb{R}^{m\times 3}$, captured by the robot’s RGB-D camera, as 3D point clouds consisting of $m=1,024$ points. We first employ two PointNet++ encoders [26] to extract global point cloud features $f_{i}\in\mathbb{R}^{128}$ and $f_{\mathcal{S}}\in\mathbb{R}^{128}$. To propose multiple solutions to the stable poses of the object $\mathcal{O}_{i}$, we sample a random Gaussian noise $z\in\mathbb{R}^{3}$ as an additional input and use an Multilayer perceptron (MLP) to extract a feature $f_{z}\in\mathcal{R}^{64}$. We concatenate $f_{i}$, $f_{\mathcal{S}}$, and $f_{z}$ to produce a single joint feature $f_{i,\mathcal{S},z}$ and finally employ another MLP to decode a final 6-DoF pose ${\hat{\mathcal{T}}}_{g,z,\mathcal{S}}^{i}$ for the object $\mathcal{O}_{i}$. For brevity, we omit the subscripts $g$ denoting the global world coordinate, $\mathcal{S}$ indicating the environment, and the superscript $i$ representing the shape $\mathcal{O}_{i}$ in the following paragraph. The 6-DoF pose is composed of a 3D translation and a 3D orientation. For the orientation part, we adopt the axis-angle representation, which has been shown to be effective for pose prediction task [27].

Sampling different Gaussian noises $\{z_{l}\}_{l=1}^{r}$, we obtain a diverse set of pose predictions $\{\hat{\mathcal{T}}_{z_{l}}\}_{l=1}^{r}$. Provided with a ground-truth list of stable poses $\{\mathcal{T}_{z_{l}}\}_{l=1}^{r_{*}}$ generated by running several random interaction trials in physical simulation, we define a loss $\mathcal{L}_{M}$ to train our pose prediction outputs.

$$\mathcal{L}_{M}(\{\hat{\mathcal{T}}_{z_{l}}\}_{l=1}^{r},\{\mathcal{T}_{z_{l^{\prime}}}\}_{l^{\prime}=1}^{r_{*}})=\sum_{l=1}^{r}\min_{l^{\prime}}L(\hat{\mathcal{T}}_{z_{l}},\,\mathcal{T}_{z_{l^{\prime}}})+\sum_{l^{\prime}=1}^{r_{*}}\min_{l}L(\hat{\mathcal{T}}_{z_{l}},\,\mathcal{T}_{z_{l^{\prime}}})$$

(1)

where $L$ measures the distance between two 6-DoF poses. We implement $L$ as the average $L_{2}$ distance between the corresponding points of the object point clouds after applying the two poses.

We train a model to classify the stability of the sampled poses. For each predicted pose $\hat{\mathcal{T}}_{z_{l}}$, we first concatenate the transformed object point cloud $\hat{\mathcal{T}}_{z_{l}}(X_{i})$ with the surrounding environment point cloud $X_{\mathcal{S}}$. Then, we augment this combined point cloud with one extra 1D arrays of ones and zeros, to indicate if a point belongs to $X_{i}$ or $X_{\mathcal{S}}$, along the XYZ dimension to form a tensor of shape $(M+M,3+1)$. With this augmented input, a straightforward method for stable classification would be using a PointNet++ [26]. However, we find that this naive approach does not provide satisfying results due to the intrinsic difficulty of analyzing stability among two separate geometry components.

To facilitate learning, we propose to solve this problem via a multi-task supervised learning approach. After obtaining the feature from the PointNet++ backbone, we have three task branches that perform stable classification, contact classification, and contact point regression simultaneously. For contact point regression, we use a PointNet++ decoder to regress the 3D offset $v_{i}$ from each point $p_{i}$ of the input point cloud to its nearest ground-truth contact point with a loss defined as:

$$\mathcal{L}_{offset}=\sum_{p_{i}\in X}L(p_{i}+v_{i},\operatorname*{arg\,min}_{c\in C}\left\|c-p_{i}\right\|_{2})$$

(2)

Here $X$ denotes the input point clouds including the transformed object point cloud $\hat{\mathcal{T}}_{z_{l}}(X_{i})$ and the environment point cloud $X_{\mathcal{S}}$. Based on the predicted $\hat{\mathcal{T}}_{z_{l}}$ pose, we calculate the nearest ground truth stable pose from Eqn. 1 and collect the corresponding contact point sets between the object and the environment in the simulation denoted as $C$. We implement $L$ as smooth $L1$ loss. We also adopt a variance loss $L_{variance}$ similar to [27] to reduce the variance of the group {$p_{i}+v_{i}$} that correspond to the same contact point. We train two MLP classifiers for stable pose and contact classification with losses denoted as $\mathcal{L}_{stable}$ and $\mathcal{L}_{contact}$ respectively. Our total loss is defined as:

$$\mathcal{L}_{C}=\mathcal{L}_{stable}+\lambda_{1}\mathcal{L}_{offset}+\lambda_{2}\mathcal{L}_{contact}+\lambda_{3}\mathcal{L}_{variance}$$

(3)

where $\mathcal{L}_{stable}$ and $\mathcal{L}_{contact}$ are standard binary cross entropy losses.

Leveraging the above stable pose classifier function denoted as $\mathcal{V}$, we apply CEM [25] to search for a locally optimal goal pose starting from the initial predicted poses. We first apply the predicted transformation $\hat{\mathcal{T}}$ to the object point cloud $X_{i}$ to get a point cloud $\tilde{X}_{i}$. A point $a$ in the CEM searching action space is a 6D transformation $\mathcal{T}_{a}$ which transforms the object point cloud $\tilde{X}_{i}$ into $\mathcal{T}_{a}(\tilde{X}_{i})$. A point $s$ in the CEM state space is the transformed object point cloud along with the surrounding environment point cloud $(\mathcal{T}_{a}(\tilde{X}_{i}),X_{\mathcal{S}})$. When selecting the action $a$, we run a derivative-free optimization method CEM [25] to search within the 6D pose space to find a 6D transformation $\mathcal{T}_{a}$ associated with the highest score in the value model $\mathcal{V}(s)$.

$$a^{*}=\operatorname*{arg\,max}_{a}{\mathcal{V}(\mathcal{T}_{a}(\tilde{X}_{i}),X_{\mathcal{S}})}$$

(4)

The 6D pose associated with $a^{*}$ is the final stable placement pose produced by our model.

Our pipeline samples and refines 128 poses for each object and supporting item pair. We use the pose classifier to estimate the scores for these 128 poses and use a threshold of 0.8 to filter out the poses with low scores. To verify the remaining poses, we run a forward simulation in PyBullet [28] to check whether the objects at these poses are stable. The results of the stable placement accuracy for each category are reported in Table 1.

We first adopt a baseline (Random) which is based on a random pose sampling strategy. The rotation value is sampled from SO(3) uniformly, and translation value is uniformly sampled within this region of [$\pm 0.05$, $\pm 0.05$, $z$+0.02] in the supporting item’s local frame, where $z$ is determined by the maximal height of each supporting item. We use the meter as the unit of translation value throughout the experiment section.

We slightly modify the approach in  [12] to work in our setting and use it as another baseline ( Jiang et al. [12] ), in which a stable pose classifier is learned based on hand crafted features. We randomly sample 128 poses in the local frame of the supporting environment, and classify these poses using the stable pose classifier. We evaluate the accuracy of these stable poses in PyBullet [28] and report the results in Table 1. The results suggest that even with random sampled poses, a simple classifier with hand-craft feature can improve the stable placement accuracy. We also design another baseline that adopts heuristic rules for pose sampling but with the same classifier from [12]. For this baseline ( Jiang et al. [12] + heuristic), the object translation in the horizontal dimensions (XY dimensions) is sampled in the radius of $0.9*r$ and the translation in the gravity dimension (Z dimension) is determined by $z+0.01$, where $r$ and $z$ are the radius and maximal height of supporting item respectively. We additionally rotate the object to make the longest axis of the object horizontally. While this baseline can achieve some reasonable results in easy situations when the supporting item has a nearly flat horizontal surface (i.e., box-wrench, box-spatula), it perform worse with supporting items with complicated geometry (i.e., bowl, mug). The results demonstrate that a good pose proposal algorithm is also of great importance to the success of stable object placement. With carefully designed stable pose proposal network and stable pose classifier network, our framework achieves the best results over all the approaches.

We evaluate whether the pose proposal network can generate a reasonable distribution of the stable poses for placing the object over the supporting items. A diversifying stable pose distribution is crucial for the success of finding desirable regrasping solutions. For each input object and supporting item pair, we draw 128 poses from the proposal network, and transform the candidate object to the proposed stable poses. The qualitative results are presented in Fig. 4. Our visualization demonstrates that our proposed stable poses yield diverse object placements.

To quantitatively measure the diversity of the stable object placements, we define two poses as two different poses if their translation difference is larger than $\tau_{1}$ and the rotation angle between them is larger than $\tau_{2}$. In our experiment, $\tau_{1}$ and $\tau_{2}$ are set to be $0.03m$ and $30^{\circ}$ respectively. We report the average number of different stable object placements in Table 2. The results showing that although random uniform sampling combined with heuristic rules can generate diverse stable poses for some easier environment (i.e., boxes with flattened surface), it can not handle complicated geometry. Our proposed method outperforms the baseline by a large margin.

To verify the usefulness of the multi-task joint learning framework, we adopt the same network architecture but with different combinations of task losses. The quantitative results on the test set are shown in Table 3. We use the accuracy and precision metrics to measure the performance of each model. While the model trained with only stable loss reaches a reasonable performance, adding auxiliary losses improves the stable precision from 0.906 to 0.910. Our experiments demonstrate that the contact prediction and contact offset regression are beneficial to the stable pose learning task. This is reasonable because a strong correlation between each subtask eases the joint learning process: 1) for all stable poses, the placed object must have contact points with the supporting environment, and 2) the offset from the support and the object should be consistent with each other.

Our model takes raw point clouds of the object and surrounding environment along with random variables as inputs, and outputs a list of stable poses. However, these poses can not be 100% accuracy, the failure cases are shown in Fig. 9. We analyze that some predicted poses might be sensitive to tiny perturbation, or missing parts of the depth scans, these predicted poses might result in slight collisions between the object and the surrounding environments. These failure cases could be mitigated if force/tactile sensors are leveraged to produce closed-loop manipulation policies.

The regrapsing task is quite challenging. A typical regrasping process includes the object placement stage and the object regrasping stage.

During the object placement stage, when the robot is placing the object (for example, a spoon) into the supporting item (for example, a bowl), the final object pose might be stable but different from the predicted desired pose. The 6D pose of the supporting item might also change when receiving external forces in the process. These changes in the object and supporting item’s 6D poses might ruin the following regrasping stage. For example, the spoon might fall into the bottom of the bowl, which makes the spoon’s regrasping process impossible.

In the object regrasping stage, the robot might fail to find a feasible collision-free approach plan. Given that the robot finds a collision-free approach, the grasping process might still fail due to sensor noise, friction. In the regrasping stage, although the object is already at a stable pose, it might still be sensitive to external perturbations resulting in failure grasps. The most failure cases are caused by the false stable object pose predictions.