Robots of the Lost Arc:
Self-Supervised Learning to Dynamically Manipulate
Fixed-Endpoint Cables

Dynamic manipulation and management of linear deformable objects such as ropes, chains, vacuum cords, charger cables, power cables, tethers, and dog leashes are common in daily life. For example, a person vacuuming may find that the vacuum power cable is stuck on a chair, and use dynamic manipulation to “vault” the cable over the chair. If the first motion does not succeed, the human can try again, adapting their motion to the failure. In this paper, we explore how a robot can teach itself to perform three tasks:

Task 1: Vaulting

The robot dynamically manipulates a cable to move from one side of an obstacle to another.

Task 2: Knocking

The robot dynamically manipulates a cable to knock a target object off an obstacle.

Task 3: Weaving

The robot dynamically manipulates a cable to weave it between three obstacles.

To accomplish these tasks, we propose to generate trajectories using a parameterized function. To reduce the complexity of parameterizing actions, we compute minimum-jerk trajectories using DJ-GOMP [1, 2] that take as input the apex point in the trajectory. This point, combined with task-specific starting and ending arm configurations, are used to generate a single high-speed trajectory for a physical UR5 robot.

We present a self-supervised deep imitation learning pipeline that learns to predict the apex point given target positions. Using a simulator we develop for vaulting, with a simulated UR5, we obtain an example action that completes the vaulting task. For any of the three tasks in the real world (see Fig. 1), we then have the physical UR5 continually execute actions based on the action computed in simulation, with noise added to increase dataset diversity. While the robot performs actions, one end of the cable is attached to its gripper, and the other is fixed to a wall. (Hereafter, we use cable to refer to any 1D deformable object with low stiffness, such as ropes, cords, and strings.) This process results in a self-supervised procedure for collecting a large dataset. We use the resulting data to train a deep neural network policy to predict an action (i.e., the apex point) given a target position.

This paper contributes:

1. 1.

   Three novel dynamic cable manipulation tasks: vaulting, knocking, and weaving.

2. 2.

   INDy, a self-supervised framework for collecting data and learning a dynamic manipulation policy using parameterized minimum-jerk trajectories.

3. 3.

   A simulation platform to experiment with novel tasks and policy learning.

4. 4.

   Physical experiments evaluating the policies with a UR5 robot on 5 different cables.

Manipulation of 1D deformable objects has applications in surgical suturing [3, 4], knot-tying [5, 6, 7, 8], untangling ropes [9, 10, 11], deforming wires [12, 13], and inserting cables into obstacles [14]; Sanchez et al. [15] provide a survey.

Various approaches have been proposed for quasistatic manipulation of cables, such as using demonstrations [16, 17, 18], self-supervised learning [19, 20, 21], model-free reinforcement learning [22, 23, 24], contrastive learning [25, 26], dense object descriptors [27, 28, 29], generative modeling [30, 31, 32], and state estimation [33, 34, 35, 36, 37]. These prior works generally assume quasistatic dynamics to enable a series of small deformations, often parameterized with pick-and-place actions. Alternative approaches involve sliding cables [38, 39] or assuming non-quasistatic systems that enable tactile feedback [11, 21]. This paper focuses on dynamic actions to manipulate cables from one configuration to another, traversing over obstacles and potentially knocking over objects.

More closely related is the work of Yamakawa et al. [40, 41, 42, 38, 43, 44, 45, 46] that focuses on dynamic manipulation of cables using robot arms moving at high speeds, enabling them to model rope deformation algebraically under the assumption that the effect of gravity is negligible. They show impressive results with a single multi-fingered robot arm that ties knots by initially whipping cables upwards. Because we use fixed-endpoint cables moving at slower speeds, the forces exerted on the rope by gravity and the fixed endpoint prevent us from performing knot ties. Kim and Shell [47, 48] use a small-scale mobile robot with a cable attached as a tail to manipulate small objects by leveraging inter-object friction to drag or strike the object of interest. While their work aims to tackle dynamic cable control, it relies heavily on physics models for motion primitives that are only applicable to dragging objects via cables, and uses a RRT-based graph search algorithm to address the stochasticity in the system. Other studies have developed physics models to analyze dynamic cable motion. For example, Zimmermann et al. [49] use the finite-element method to model elastic objects for dynamic manipulation of free-end deformable objects, and Gatti-Bonoa and Perkins [50] and Wang and Wereley [51] analyze fly fishing rod casting dynamics. We focus on fixed-end cable manipulation and do not attempt to model cable physics, and we aim to control cables dynamically via vaulting, knocking, and weaving, and cannot rely on the same dragging motion primitives as in Kim and Shell [47].

We propose an approach that is related to TossingBot [52] where a robot learns to toss objects to target bins. TossingBot uses RGBD images and employs fully convolutional neural networks [53] to learn dense, per-pixel values of tossing actions, and learns a residual velocity of tossing on top of a physics model in order to jointly train a grasping and tossing policy through trial and error. We take a similar approach in dynamically manipulating by reducing the learning process to learn only the joint angles of the apex of the motion. We do not rely on a physics model of the dynamic manipulation motion, and instead, leverage a quadratic programming-based motion-planning primitive to generate a fast ballistic motion that minimizes the jerk while moving through the learned apex point.

Given a manipulator arm holding one end of a cable with the other end fixed (e.g., plugged into a wall), the objective is to compute and execute a single high-speed arm motion from a fixed far-left starting location, to a fixed far-right ending location that moves the cable from an initial configuration to a configuration that matches a task-dependent goal constraint.

Let $s\in\mathcal{S}$ be the configuration of the cable and task-specific objects, including obstacles or targets, where $\mathcal{S}$ is the configuration space. Since $\mathcal{S}$ is infinite-dimensional, we consider observations of the state, and focus on tasks for which the observations are sufficient to capture progress. Let $o\in\mathcal{O}$ be the observation, where $\mathcal{O}$ is the observation space and $f_{c}:\mathcal{S}\rightarrow\mathcal{O}$ map a state to an observation (e.g., via a camera view). Let $\tau_{a}\in\mathcal{A}$ be the action the manipulator arm takes, where $\tau_{a}$ is a complete open-loop trajectory and $\mathcal{A}$ is the action space. Let $f_{\mathcal{S}}:\mathcal{S}\times\mathcal{A}\rightarrow\mathcal{S}$ be the stochastic state transition function of the cable.

Formally the objective is: for each task $i\in\{\mathrm{vaulting},\allowbreak\mathrm{knocking},\allowbreak\mathrm{weaving}\}$, to learn a task-specific policy $\pi_{i}:\mathcal{O}\rightarrow\mathcal{A}$, that when given the observation $o_{0}=f_{c}(s_{0})$ of an initial configuration $s_{0}\in\mathcal{S}$ and a goal set of observations $\mathcal{O}_{\mathrm{goal}}\subset\mathcal{O}$, computes an action $\tau_{a}\in\mathcal{A}$ such that $f_{c}(f_{\mathcal{S}}(s_{0},\tau_{a}))\in\mathcal{O}_{\mathrm{goal}}$. For each task, we define the goal set in terms of constraints, such as requiring an observation with the cable on the right side of an obstacle (Fig. 2 (a)).

Since the action space $\mathcal{A}$ is also infinite-dimensional, we define a parametric trajectory function $f_{\mathcal{A}_{i}}:\mathbb{R}^{3}\rightarrow\mathcal{A}$, that, when given an apex point, computes complete arm trajectory. While the exact function is a design choice, we propose that the function should have the following features: the arm motions are fast enough to induce a motion over the length of the cable, the apex point of the motion can be varied based on an $\mathbb{R}^{3}$ input, the other points in the trajectory vary smoothly through the apex, and the motions are kinematically and dynamically feasible. This converts the problem to learning $\hat{\pi}_{i}:\mathcal{O}\rightarrow\mathbb{R}^{3}$, where $\mathcal{O}$ contains position and dimension information, such that for the action $a\in\mathbb{R}^{3}$ computed by the policy, $f_{c}(f_{\mathcal{S}}(s_{0},f_{A_{i}}(a)))\in\mathcal{O}_{\mathrm{goal}}$.

We propose a deep-learning system that takes as input an image, and produces a sequence of timed joint configurations; when the robot to follows the motion, it results in the dynamic manipulation of a cable that accomplishes the task.

We use simulation to experiment with training algorithms, and to find a feasible base action $a_{\beta}$ to bootstrap the data collection process in the real world. Using simulation enables fast prototyping to efficiently assess qualitative and quantitative behavior of multiple base actions, but can suffer from a sim-to-real gap, motivating research teams to collect data from running robots in the real world [58, 59]. Similarly, we collect data by running a physical UR5 robot, but use simulation as a means to obtain a functional action that can be improved upon when collecting data.

To model cables in simulation, we use a Featherstone [60] rigid-body simulator from BulletPhysics via Blender [61]. Cables consist of a series of small capsule links connected by spring and torsional forces. We tune the spring and torsional coefficients by decreasing bending resistance and stopping before the cable noticeably behaves like individual capsules rather than a cable, and increasing twist resistance until excessive torsion causes the capsules to separate.

When finding a feasible $a_{\beta}$ to transfer to real, we only simulate vaulting. The obstacle height, radius, and location are randomized for each trial, along with the initial cable state. We use 10 different obstacle and initial cable settings in simulation, and select the input apex point with highest success rate out of 60 random points. Then, we take this apex point directly to physical experiments as the base action $a_{\beta}$ to collect data, as the physical UR5 exhibits similar behavior to that in simulation after applying the predefined configurations from simulation, as seen in Fig. 4.

We use a physical UR5 robot grasping to dynamically manipulate a cable attached to a wall 4.5 m away, on the three tasks, with 5 different cable types. The system obtains observations from a Logitech C270 720p webcam placed 0.75 m above the robot arm base. It scales and crops the image to a 512x512x3 segmented RGB image that feeds into a ResNet-34 [62] deep neural network implemented with PyTorch [63]. We initialize weights using He initialization [64].

In physical experiments of the 3 tasks (see Fig. 5 for layout), we use large, white plastic bricks as the obstacle(s), which can vary in size and location. The obstacles we use for experiments are 0.15 m to 0.75 m in width and 0.3 m to 1.5 m in height. To generate training data, we randomize the location and size of the obstacles in each trial. We test the system with 5 cables listed in Table I.

We define difficulty tiers based on the distance of the obstacle to the base of the robot.

We evaluate INDy along with two baselines:

Baseline with Fixed Apex.

In the three tasks, for each configuration, a human annotator sets one set of apex joint angles that can accomplish the task. The apex joint angles are fixed with respect to the obstacle itself, and they remain the same across different locations of the same obstacle.

Baseline with Varying Apex.

In vaulting and weaving, for each configuration (including obstacle size and location), a human annotator sets the apex joint angles such that the robot arm’s end effector is 15 cm above the center of mass (COM) of the obstacle, and aligned horizontally with the COM of the obstacle. For knocking, a human annotator makes the robot arm end 10 cm above the COM of the target object, and aligned horizontally with the COM.

We find the baselines from simulation, where we observe that the fixed and varying apex motions could induce enough velocity to make the cable vault the obstacle.

We measure the correlation between simulated and real setups, benchmark the three dynamic cable tasks, and investigate generalization to different cables. The project website supplements these results with videos.

This paper explores learning three dynamic cable manipulation tasks: vaulting, knocking, and weaving. Experimental results suggest that the proposed procedure outperforms two baseline methods.

In future work, we will generalize the learned policy to different cable lengths and masses, and more complex obstacles and target objects. We will apply the parameterized trajectory method to other dynamic cable manipulation tasks without fixed-endpoints of the cable, inspired by the adventures of Dr. Indiana Jones.

This research was performed at the AUTOLAB at UC Berkeley in affiliation with the Berkeley AI Research (BAIR) Lab, the CITRIS “People and Robots” (CPAR) Initiative, and the Real-Time Intelligent Secure Execution (RISE) Lab. The authors were supported in part by donations from Toyota Research Institute and by equipment grants from PhotoNeo, NVidia, and Intuitive Surgical. Daniel Seita is supported by a Graduate Fellowship for STEM Diversity (GFSD). We thank our colleagues who provided helpful feedback, code, and suggestions, in particular Francesco Borrelli, Adam Lau, Priya Sundaresan, and Jennifer Grannen.