Learning Efficient Policies for Picking
Entangled Wire Harnesses: An Approach to
Industrial Bin Picking

Industrial bin picking has been developed for decades. Prior works have primarily focused on model-based approaches such as 3D object localization or 6D pose identification [2, 13, 14] and grasp planning [7, 8, 15]. Alternatively, model-free methods do not require known object information and can produce grasp poses for novel objects. Domae et al. [1] proposed to plan grasps considering collisions between the gripper and the objects from a single depth image. Several works leverage deep learning to mitigate the challenges of complex physical interactions and environment uncertainties. Mahler et al. [16] trained a model from synthetic data to produce collision-free grasps for daily objects. Matsumura et al. [5] proposed a learning-based method to plan robust grasps for industrial parts. However, there remain challenges in handling difficult objects. Recently, several works tackled the challenges of those objects which are (1) difficult to be recognized, e.g., transparent or shiny objects [6, 17], (2) difficult to perform grasping, e.g., thin and elliptical objects [18, 19], and (3) involved with rich physical interactions, e.g., tangled-prone objects [9, 10]. So far, these approaches have focused on rigid objects. Leao et al. [20] proposed a method to pick up soft tubes by fitting shape primitives to clutter, but it does not work on dense clutter or objects with irregular shapes. Objects with non-rigid nature and complex geometries like wire harnesses are relatively unexplored and remain challenging in the industrial bin picking domain.

Deformable object manipulation has primarily focused on two object classes: 1D (cable, rope) and 2D (fabric, cloth). Several studies adopt specially designed motion primitives to accomplish various manipulation tasks such as knot tying/untying [21, 22, 23], spreading cloth [24] or whipping ropes [25]. Using deformable and long objects in industrial bin picking poses new challenges. The cluttered scenes are more complex due to the entanglement issues caused by their deformable nature. Ray et al. [26] proposed to untangle herbs from a pile using a two-finger gripper. Takahashi et al. [27] proposed a learning-based separation strategy for grasping a specified mass of small food pieces. Although some works have addressed the factory automation problems for wire harnesses [11, 28, 29], robotic wire harnesses picking is less studied. In this paper, we propose a novel and efficient bin picking strategy to deal with wire harnesses.

We select Fast Graspability Evaluation (FGE) [1] - a model-free approach to detect collision-free grasps. FGE calculates pixel-wise graspability scores by convoluting a gripper’s template of contact and collision areas with the input depth map. A grasp composes a pixel location $g=(u,v)$ on the depth map and a rotation angle $\phi$ indicating the gripper’s orientation. We transform $(u,v,\phi)$ to the grasp with four degrees of freedom $(g_{x},g_{y},g_{z},g_{\phi})$ denoting the grasp point and the gripper’s orientation at the robot coordinate frame. This module outputs a set of grasps ordered by their FGE scores.

1) Action Formulation: We formulate each disentangling action $a$ with a motion scheme $\psi$ and two parameters as follows:

where the robot only performs the helix motion $\psi_{H}$ or performs the spinning motion $\psi_{S}$ after $\psi_{H}$. Note that directly executing $\psi_{S}$ after grasping may not be effective since the extracting displacement of the target object is small. We propose six separation actions $a_{h},a_{hs},a_{f},a_{fs},a_{tf},a_{tfs}$ using two motion primitives and a direct lifting action $a_{dl}$. Table I shows their notations and illustrations. We use $M$ to represent the collection of these seven actions.

2) Action Parameter Determination: To determine the parameters of each action and search for the best action, we define a numerical metric action complexity for exploring the trade-off between success rates and execution time. Let $\mathcal{A}(a)$ denote the action complexity of the action $a\in\{a_{dl},a_{h},a_{hs},a_{f},a_{fs},a_{tf},a_{tfs}\}$. It is defined by assuming that actions with larger $\theta_{H}$ or $\theta_{S}$ involve higher complexity. To reduce the search cost during exploration, we assume that the action complexity linearly scales with the success rate of each action. We find this linear relationship by executing 80 physical attempts for each action as Table I presents. Then, we use this hypothesis to determine the action parameters experimentally. Specifically, we predefine a set of possible values of $\theta_{H},\theta_{S}$ experimentally for our policy to select. $\theta_{H}$ can be selected from $\{0,\pi,2\pi,4\pi\}$ and $\theta_{S}$ can be selected between $\{0,\pi/2\}$. Note that the other parameters of the motion primitives $H=(c_{H},r_{x},r_{y},h_{0},h)$ and $c_{S}$ are fixed in our policy. Finally, we assign integers 0 to 6 as the action complexity for the discrete actions from $a_{dl}$ to $a_{tfs}$. The action parameters and execution details are included in Table I.

Our policy can explore the optimal action by minimizing the action complexity as much as possible. Let us consider a case when the robot performs $a_{tf}$ to extract an entangled object. Suppose the target object is entirely disentangled after a full circle ($a_{f}$) while the robot still needs to perform the second circle. Thus, the current observation only requires $a_{f}$ as the optimal action to ensure a successful separation with less execution time, while $a_{tf}$ is a redundant action which can also solve the entanglement but costs more time. We can observe that an optimal action has lower action complexity than a redundant action. Thus, the optimal action is required to untangle the target with minimal action complexity.

3) Prediction Model: The inference of the optimal action without object models should be conditioned on the grasp locations. We propose Action Success Prediction (ASP) to predict if the action-grasp pair can successfully separate the target. ASP learns a function parameterized by $\tau$:

where the input is a depth image $o$ $\in\mathbb{R}^{224\times 224\times 3}$ with triplicated depth values across three channels to match with the default input size of the image encoder’s backbone, a pixel-wise grasp pose $g=(u,v)$ $\in\mathbb{R}^{2}$, a categorical action $a$ $\in\mathbb{R}^{7}$ and the output is a success possibility in the range of $[0,1]$. We encode the image using a ResNet-50 backbone [30], the grasp point using a single fully-connected layer with 256 units, and the categorical action using a fully-connected layer with 14 units. Then we concatenate the output from all three branches and feed it to a fully-connected layer with 256 units and produce an action success possibility.

4) Training via Active Learning: The dataset for training ASP is entirely collected from real-world experiments. Each sample has a depth image $o$, a grasp location $g$, a labeled action $a$ and a binary success metric $S=\{0,1\}$. We execute each action for the clusters with 6, 10, 12 and 18 objects. We label each attempt with success ($S=1$) or failure ($S=0$) depending on if the robot picks a single wire harness. Due to this data collection manner, some samples in the dataset are labeled with redundant actions instead of optimal actions. To deal with this problem, we leverage active learning to train the ASP model, making it possible to predict optimal actions using this dataset. Generally, we first select several samples manually as training data to train the model, use the trained model to predict the remaining samples, query and transfer samples for training and fine-tune the model repeatedly. Specifically, we manually select the initial training data with approximately optimal actions. Note the number of samples for each action is roughly equal. Let data pool denote the left samples except for the training data. After training, we query the samples in the data pool and transfer the logical samples to the training data. Here, a sample $(o,g,a,S)$ can be determined as success-logical or failure-logical using the trained model $\tau$ and our proposed Action-Grasp Inference module (Section IV.C). Let $a_{p}=$ ActionGraspInference($o,g,M,\tau$) denote the predicted action:

• •

  Success-logical: $\mathcal{A}(a_{p})\leqslant\mathcal{A}(a)$. For samples labeled with $S=1$, the labeled action $a$ is a redundant action compared with the predicted action $a_{p}$ (Fig. 5(a)).

• •

  Failure-logical: $\mathcal{A}(a_{p})>\mathcal{A}(a)$. For samples labeled with action $a$ and failure $S=0$, the predicted action $a_{p}$ has higher action complexity (Fig. 5(b)).

During each iteration, as the number of logical samples increases, the model performance of predicting the optimal actions also improves. We define a transfer ratio $r$ representing the ratio of the number of samples that would be transferred in each iteration to the number of samples in the current data pool. The iteration stops when the data pool is empty or early stops before overfitting. Algorithm 1 shows the detail of training ASP via active learning.

At this point, we’ve obtained a set of grasp candidates, action candidates and the scores of each action-grasp pair. Our policy then needs to determine which action-grasp pair can be executed. This module infers all possible action-grasp pairs to guarantee a successful picking with minimal action complexity:

where the inputs are a depth image $o$, a collection of actions $M$, grasp candidates $G$ with FGE scores from the Model-Free Grasp Detection module and ASP model $\tau$. This module first predicts the action success possibilities of all action-grasp pairs $P=f_{\tau}(o,G,M)$. If all possibilities in $P$ are lower than the threshold $p_{thld}$, which means all action-grasp pairs cannot solve the entanglement, we select the grasp with the highest FGE score and the most complex action $a_{tfs}$. Otherwise, the best solution is determined by the action-grasp pair with the lowest action complexity. If multiple grasps share the same action complexity, we select the pair with the highest FGE score.

Our dataset contains 722 samples. We set the ratio of active learning $r=0.4$ and use a simple decision threshold of $p_{thld}=0.5$ over the softmax of each action’s success possibility to classify success (1) or failure (0). We train the network using binary cross-entropy loss function and the Adam optimizer. We stop training after three times of fine-tuning as it achieves the best performance. Fig. 6 shows the accuracy and loss during active learning. The gray curve refers to the Initial Model (IM) trained using manually determined samples, which would be potentially accurate but lack robustness due to fewer data. The green line indicates the Final Model (FM), which performs the best as the fine-tuning goes on since it converges to IM but with higher data-driven accuracy.

Moreover, Table III shows the details of each iteration in active learning. Row 1-2 shows the number of samples used as the training data and left in the data pool. Particularly, 92 samples left in the data pool after the final fine-tuning are used to validate all models by checking the number of logical samples. Row 3-4 shows the ratios of logical samples increase with the fine-tuning process. Finally, Table III validates our hypothesis that more complex actions correspond to higher success rates. We respectively present the average scores predicted by FM for each action. FM can correctly predict an ascending order of possibilities as the action complexity increases. We can observe that $a_{fs},a_{tf},a_{tfs}$ share similar scores since the validation samples contain 18 objects at most. $a_{tfs}$ does not show a significantly high score due to the accumulated low scores when all predictions fail and $a_{tfs}$ is forced to be selected.

1) Physical Experiment Setup: We use a NEXTAGE robot from Kawada Industries Inc. The robot is required to grasp objects from the parts bin lying in front of it and transport them to another bin located on its left side. The robot’s left arm operates over a workspace captured as a top-down depth image by a Photoneo PhoXi 3D scanner M. A two-fingered parallel gripper is attached at the arm tip. The setup is shown in Fig. 7(a). The length of the wire harness used in this work is 74cm. After performing the analysis and physical experiments, we fix the parameters of the proposed trajectory as $c_{H}=$ (0.525,0.065)[m], $r_{x}=$ 0.1m, $r_{y}=$ 0.225m, $h_{0}=$ 0.32m, $h=$ 0.14m as well as the speed of the action since they yield high task effectiveness. We sample several waypoints on the trajectory and plan motions with a uniform velocity. We use a PC with an Intel Core i7-CPU and 16GB memory without GPU for real-world experiments and a PC with an Intel Core i5-6400 CPU, 16GB memory and an Nvidia GeForce 1080 GPU for learning.

We present three baselines. DL (directly lifting) uses FGE to detect the grasp point and executes by directly lifting ($a_{dl}$). RAND executes a random action and the grasp with the highest FGE score. TFS only executes the most complex action $a_{tfs}$ and the grasp of the highest FGE score. We also present three versions of our policy. Ours-IM is our policy using the initial model in active learning while Ours-FM uses the final model. Ours-FM-R denotes Ours-FM with a recovery module using force feedback. After performing the predicted action, we record the force from an F/T sensor mounted on the robot’s wrist to determine if the grasped wire harness is still entangled. If there exists a sudden increase of force, the target is not disentangled and the robot places it back to the parts bin.

We leverage two metrics to evaluate the bin picking performance. Success rate refers to the number of successful attempts of picking up a single object divided by the number of attempts of placing. PPH (Pickings Per Hour) is the number of successful attempts the robot can execute in one hour. Additionally, we present Avg. $\mathcal{A}$ (Average action complexity) to evaluate how the action complexity predicted by our policy varies under different entanglement scenarios.

2) Task Design: We prepare two real-world bin picking tasks. Consecutive picking aims to empty the bin filled with respectively 5, 10, or 15 objects. The robot picks up objects one by one until the bin is empty. Randomized picking refers to picking up objects from the bin filled with respectively 18-20, 20-22 and 22-25 objects. After each picking, we reload the bin and shuffle the wire harnesses to provide randomness during the task. It can encourage the robot to confront different patterns of entanglement as much as possible. Fig. 7(b) shows the bins filled with different numbers of wire harnesses.

3) Comparisons with Baselines: Table IV compares the performance of the three versions of our policy and three baselines in success rate and PPH. For consecutive picking where the goal is to empty the bin, Ours-FM and Ours-FM-R significantly increase the average success rate from 56.7% to 87.3% and 88.1% compared to DL. TFS achieves higher success rates than Ours-FM but has lower PPH since TFS only executes the time-consuming action $a_{tfs}$. Especially in the latter half of a continuous picking task when fewer objects remain in the bin, our policy can shorten the execution time by inferring adequate actions. Ours-FM-R also has lower PPH since this policy needs extra actions to place the entangled objects back in the parts bin. Furthermore, the average action complexities for the predicted actions using RAND, Ours-IM, Ours-FM and Ours-FM-R are also presented in Table IV. The average action complexity for 5 objects is significantly lower than that for 10 and 15 objects. As the number of objects in the bin increases, the action complexity of the predicted action increases. It demonstrates that entanglement frequently occurs when the bin contains more objects and requires more complex actions. We also observe that the failed attempts by baselines always drag objects outside the workspace, requiring human workers to rearrange after each attempt. Our policy helps maintain a relatively clean workspace during the consecutive picking thanks to the horizontal separation and our action-grasp inference algorithm.

For randomized picking, we compare the performance of Ours-FM and Ours-FM-R with a DL baseline as Table IV shows. More objects are involved in this task than consecutive picking. Thus, the possibilities of encountering complex entanglement patterns become higher. Ours-FM completes the task with 80% accuracy and 109 PPH, almost twice higher than DL. The results suggest that our policy can grasp the tightly intertwined objects in dense clutter. All three proposed modules collaboratively contribute to efficient bin picking from perception to manipulation planning. However, as the number of objects increases, both metrics of Ours-FM decrease. Due to heavier occlusions and visual noise, the detected grasp candidates become fewer and some entanglement patterns can hardly be recognized from the depth image. Despite this, the most complex action $a_{tfs}$ can still strive for success. Additionally, Ours-FM-R outperforms Our-FM in success rate especially when the number of objects increases thanks to the recovery module but has lower PPH. When the bin contains more than 22 objects, Ours-FM-R shows a higher success rate and PPH than Ours-FM, indicating the feedback module can help further improve the bin picking performance.

1) Visualized Results: We present visualized results of picking attempts with grasps, actions and input depth images. First, Fig. 8(a) presents the predicted action-grasp pairs of each action. It demonstrates that our policy infers the actions not only by analyzing the object number in the scene but also by reasoning about the occlusions around the input grasp point. Additionally, if the robot grasps close to the wire harness’s end, our policy tends to predict more complex actions since this case may require the gripper to handle the length by moving a larger distance. Then, Fig. 8(b) shows a set of successful pickings with the reasoned action-grasp candidates ranked by descending prediction scores. The optimal action-grasp pairs inferred by our policy are marked as red. Our policy can recognize the objects barely entangled with others that only require $a_{dl}$. As for the scenes that do not contain such objects, our policy can reason the entanglement situation and predict the proper actions. When the predicted scores of all action-grasp pairs are lower than $p_{thld}$, our policy executes $a_{tfs}$ and grasp with the highest FGE score, where the target is likely on the top of the pile.

2) Novel Wire Harnesses: To demonstrate the breadth of our method, we utilize Ours-FM for two unseen wire harnesses. They differ from those used for training in lengths and structures but have similar components (e.g., deformable cables and rigid connectors). Fig. 9 shows two novel wire harnesses and the corresponding action-grasp pairs predicted by our policy. Table V shows their length and the average action complexity of prediction with different object numbers. In the case of shorter objects (see Fig. 9(a)), our model does not predict actions with too higher complexity. The robot tends to select $a_{dl}$ and $a_{h}$ to pick up objects. Since this type of wire harness is less tangle-prone, the accuracy of picking them primarily relies on the grasp detection module while our policy can handle the potential entanglement. On the other hand, for long wire harnesses (Fig. 9(b)) whose length exceeds our bin picking working cell, Table V suggests that our policy tends to output more complex actions. However, even $a_{tfs}$ is still insufficient to separate each. More complex manipulation strategies are needed for such objects.

We observe four failure modes in the physical experiments.

• •

  Objects outside of the bin: The input image of the ASP model does not include the complete objects.

• •

  Grasp failure: The grasp failure rate is 2.1% (24/1170). A grasp fails when the robot grasps multiple objects in hand or grasps nothing. It mainly comes from vision sensor’s noise and heavy occlusion.

• •

  Tightly wedged objects: The target tightly inserts another one’s cable bundles or rigid components, making it extremely difficult to be disentangled.

• •

  Action prediction failure: Our policy sometimes predicts the wrong actions for separation due to visual noise or heavily occluded objects.

Our policy also has limitations. First, for long wire harnesses, the robot fails to extract them from the entanglement since their length exceeds the robot’s reachable areas. Second, the training phase is unique and conditioned on the structure of the objects in the dataset. It would be difficult to adopt our current policy to wire harnesses with completely different geometries.

We divide the reasons causing failure modes and limitations into two categories and provide future extensions. (1) Poor visual prediction for heavily occluded clutter: We will extend our policy by using multi-sensory inputs other than vision-only predetermined policy and force-only feedback control. We will also consider online closed-loop learning and more effective recovery methods to further improve the robustness of our policy. (2) Insufficient motion primitives: the proposed motion primitives cannot solve some complex cases and the reach range of a single robot manipulator is limited. We will consider more effective motion primitives using dual-arm or involving dynamics. It would also be interesting to design more general motion primitives to utilize our policy on various wire harnesses with different geometries.