Learning to Localize, Grasp, and Hand Over
Unmodified Surgical Needles

Automation of surgical subtasks is an active area of research with a rich history. Prior literature has studied automation of tasks related to surgical cutting [34, 23, 19], debridement [23, 16], hemostasis [26], peg transfer [12, 13, 24], and suturing [31, 33, 7, 32, 9, 28]. While automated suturing has been studied in prior work [28, 31], suturing without modifications such as painted fiducial markers is an open research problem. Recent work studies robust and general approaches to specific subproblems within suturing, including the precise manipulation of surgical needles during suturing from needle extraction [32] to bimanual regrasping [7], which is the focus of this work.

Needle manipulation is also studied by [32], where the approach studies the extraction of needles from tissue phantoms to compute robust grasps of the needle even in self-occluded configurations. Bimanual needle regrasping was studied in detail by [7], with impressive results on simulation-trained policies that take needle end effector poses as input. We extend their problem definition to consider multiple handoffs of unmodified needles and end effectors, which requires perception of the needle and robot pose from images without color segmentation. Needle grasping has also been studied using visual servoing policies in [9], where the needle is painted with green markers to track its position during closed loop visual servoing. [35] study tabular RL policies for needle regrasping in a discretized space in a fixed setup with known needle pose and experiments without the needle on the dVRK. The experiments in [35] suggest that value iteration-trained policies can mimic expert trajectories used for inverse reinforcement learning. In contrast, we present an algorithm compatible with significantly varying initial needle and gripper poses using only image observations. We additionally present many physical experiments with a needle, evaluating the success rate and speed of the algorithm.

Visual servoing (VS) is a popular technique in robotics [11, 18], and has recently been applied to compensate for surgical robot imprecision in the surgical peg transfer task [24]. While classical VS approaches typically make use of hand-tuned visual features and known system dynamics [6, 5], recent work proposes learning end-to-end visual servoing policies from examples [20, 14]. To reduce the need for tuned features and dynamics models and also reduce the number of training samples required to create a robust VS policy for bimanual needle regrasping, we present a hybrid approach that combines coarse motion planning with fine control, where a learned VS policy is only used in parts of the task where high precision is required. This framework, called intermittent visual servoing (IVS), was studied in detail by [24], where the system switches between a classical trajectory optimizer and imitation learning VS policy based on the precision required at the time. Inspired by this technique, we present an IVS approach to bimanual needle regrasping, that combines coarse perception and planning with fine VS control. Because this task requires reasoning about depth across several directions, we present a multi-view VS policy that learns to precisely hand over the needle based on several camera views.

Active perception is a popular technique with many variations to localize objects prior to manipulation by maximizing information gain about their poses [22, 29, 36, 2, 1]. This has been studied in the context of robot-assisted surgery, where the endoscope position is automatically adjusted via a policy learned from demonstrations to center the camera focus on inclusions during surgeon teleoperation. In this work, we actively servo the needle to highly visible poses to maximize the accuracy of its pose estimate. This is most similar to [1], where the authors propose an algorithm to actively select views of a grasping workspace to uncover enough information about unknown objects to plan grasps.

In the HOUSTON problem, the surgical robot starts with a curved surgical needle with known curvature and radius grasped by one gripper and must accurately pass it to the other gripper and back. This is challenging due to the needle’s reflective surface, thin profile, and pathological configurations [32, 7] as depicted in Figure 2. Once the needle is successfully passed to the other end effector, it is passed back to the first end effector. This process is repeated $N_{\mathrm{max}}$ times, or until the needle is dropped. We also consider a special case of this problem, the single-handover version, in which $N_{\mathrm{max}}=1$.

Let $p_{L}(t)$ and $p_{R}(t)$ denote the poses of the left and right grippers, respectively, at discrete timestep $t$ with respect to a world coordinate frame. The needle has pose $p_{N}(t)$ with respect to the world frame. Observations of the workspace are available via RGB images from a stereo camera, $I_{L}(t)$ and $I_{R}(t)$, or overhead monocular RGB images $I_{O}(t)$ from an RGB camera. The left and right cameras in the stereo pair have world poses $p_{\rm st,left}$ and $p_{\rm st,right}$, respectively. The overhead camera has pose in world frame $p_{\rm O}$. Each trial starts with the needle in the left gripper and ends when the needle is dropped. Additionally, the trial terminates if no successful handoff occurs in $\tau_{\rm max}$ timesteps.

At timestep $t$, the algorithm is provided observation $y(t)$ which contains images from the sensors: $y(t)=\left(I_{L}(t),I_{R}(t),I_{O}(t)\right)$. The algorithm outputs a target pose and jaw state for each end effector $u(t)=((p_{L}(t+1),\varphi_{L}(t+1)),(p_{R}(t+1),\varphi_{R}(t+1)))$, where $\varphi_{L}(t+1)\in\{0,1\}$ indicates the whether the left jaw is closed at timestep $t+1$.

In order to deterministically evaluate HOUSTON policies in a wide variety of needle configurations, we discretize the needle-in-gripper pose possibilities by choosing a number of categories across three degrees of freedom:

1. 1.

   The needle’s curve can face either towards or away from the camera, providing 2 possibilities

2. 2.

   The gripper may hold the needle either at the tip, or 30° inwards following the curvature of the needle. This degree of freedom has 2 possibilities.

3. 3.

   The rotation of the needle about the tangent line $\omega$ to the point the gripper intersects ranges from $0\degree$ to $180\degree$, and is discretized into 7 bins as in Figure 3.

This gives a total of 28 possible needle configurations and we perform a grid search over these possibilities. We chose these configurations to be representative of those seen in suturing tasks post-needle extraction.

While the robot encoders provide an estimate of the gripper poses $p_{L}(0)$ and $p_{R}(0)$, the precise needle pose is unknown due to cabling effects of the arms. We assume access to a stereo RGB pair in the workspace, an overhead RGB camera, and the transforms between the coordinate frames of these cameras and the robot arms.

We evaluate HOUSTON by recording: i) the number of successful handoffs in a multi-handoff trial, ii) the success rate per arm of single handoffs beginning from each configuration in III-C, and iii) the average time for each handoff.

In the first phase, the algorithm repositions the needle to a pose where the other arm can easily grasp it without collisions and the cameras can clearly view it. Throughout execution of the coarse policy, we parameterize the needle as a circle of known radius, and measure its state in world frame as a center point $c_{p}$, normal vector $c_{n}$, and a needle tip point $c_{t}$ as shown in Figure 6. Active Needle Presentation consists of two stages: needle acquisition and handover positioning. The needle acquisition stage moves the needle to maximize visibility, and the positioning stage uses visual servoing to move the needle into a graspable state.

After the active presentation phase described in Section IV-A and the pose of the needle is relatively accurately known and accessible to the grasping arm, we execute a grasping policy $\pi_{\rm grasp}$ to grasp the needle. However, the needle pose estimate after the first phase may not be perfect, so we must visually servo to compensate for these errors when grasping. Even if the needle pose was perfectly known, reliable grasping of a small needle is still challenging due to the positioning errors of the robot, which are a result of its cable-driven arms [24, 13, 12, 30, 25, 21]. The policy splits corrective actions between the $x$- and $y$-axes with two sub-policies, $\pi_{\mathrm{grasp},x}$ and $\pi_{\mathrm{grasp},y}$. Each policy uses RGB inputs ego-centrically cropped around the grasping arm, with the $x$- and $y$-axis policies using $140\times 200$ pixel crops from the inclined camera and $70\times 200$ pixel crops from the overhead camera respectively. The cropping forces the policy to condition based on the relative position of the gripper and needle without the ability to overfit to texture cues from other parts of the scene. The fine-grained grasping policy and image crops are displayed in Figure 5. We ablate different design choices for the grasping correction policy and also present open-loop grasping results in Section V.

The grasping subpolicy $\pi_{\mathrm{grasp},y}$ is a neural network classifier that outputs whether the grasping arm should move in the $+y$ (down in the crop) or $-y$ (up in the crop) direction. $\pi_{\mathrm{grasp},x}$ is trained similarly to output whether the grasping arm should move in the $+x$ or $-x$ directions. The policies are trained by collecting offline human demonstrations through two methods: 1) we sample poses for arms in the workspace such that needle orientation is perturbed by $10\degree$ about each axis, then move the robot to a good grasping position via a keyboard teleoperation interface. 2) we execute the pre-handover positioning routine and position the robot in the desired grasp location by hand, after which the robot autonomously iterates through offsets in the $\pm x$ and $\pm y$ directions, labeling actions according to the offset from goal position. We experimentally find that separating the policy across two axes significantly improves grasp accuracy (Section V). A separate grasping policy is trained for each arm on 100 demonstrations each. Each demonstration takes 5-10 actions and each dataset takes about an hour to collect. The policies $\pi_{\mathrm{grasp},x}$ and $\pi_{\mathrm{grasp},y}$ are each represented by voting ensembles of 5 classifiers, each of which have three convolutional layers and two fully connected layers. Details about model architectures are located in the project website.

During policy execution, we iteratively sample actions first from $\pi_{\mathrm{grasp},x}$, then $\pi_{\mathrm{grasp},y}$, waiting for each to converge before continuing. We multiply the action magnitude by a scalar $\beta_{decay}=0.5$ every time the network outputs the opposite action of the previous timestep. Servoing terminates when action magnitude decays to under $0.2$mm. This enables implicit convergence to the goal without explicitly training the policy to stop. After $x$ and $y$ convergence, we execute a simple downward motion of 1cm to grasp the needle.

To evaluate the method for the task of active needle presentation, we compare to the following baselines:

• •

  Depth-based presentation: Instead of using the stereo RGB network to detect needle pose, we use a depth image-based detection algorithm to detect the needle pose and servo it to the flat grasping pose. This method takes the depth image from the built in depth calculation from the stereo camera, $I_{D}(t)$, as input, masks the gripper out of the depth image using the dVRK’s forward kinematics, then performs a volume crop around the end effector and fits a circle of known radius to the points using RANSAC to extract the state.

• •

  No-Sim-Data: This is an ablation of the RGB stereo segmentation network that is only trained on the small dataset of real data.

To evaluate the design choices used in the fine-grained grasping policy, we compare to the following baselines:

• •

  Open Loop: Executes an open loop grasping motion to grasp the needle based only on needle geometry and inverse kinematics.

• •

  Shared $(x,y)$ Grasp Policy: Trains a single policy to output $x$ and $y$ displacements, and takes both $I_{O}(t)$ and $I_{L}(t)$ as input.

To evaluate whether the system can transfer to needles unseen in training, we evaluate HOUSTON on three additional needles as in Figure 7.

We perform experiments using the daVinci Research Kit (dVRK) surgical robot [15], a cable-driven surgical robot with two needle drivers, a gripper which can open 1cm.

For perception, the setup includes a Zed Mini stereo camera angled slightly downwards to face the arms, and an overhead Zivid One Plus M camera facing directly down. Stereo images are captured at 2K resolution, and overhead images are captured at 1080p. Locations of the arms relative to each of the cameras is statically calibrated.

We evaluate HOUSTON and baselines on the single handover task in Table I, and perform multiple systematic passes over the $28$ starting configurations described in Section III-C. We find that HOUSTON is able to more reliably perform the task than comparisons, which either experience many presentation errors or many grasp positioning errors.

We evaluate HOUSTON on the multiple handover task with $N_{\rm max}=50$ with two different starting configurations (Table II). We observe that in the first configuration, the algorithm completes $26.20$ successful handovers on average and $38.60$ in the second. In three trials, no errors occur, and we manually terminate them after $50$ successful handovers.

HOUSTON encounters three failure modes:

• •

  P: Presentation error: the robot fails to present the needle in an orientation that is in the plane of the table with the needle tip pointing toward the grasping arm. This may lead to grasping angles that are unreachable or out of the training distribution for the grasping arm.

• •

  X: Grasping positioning error (X): the $x$-axis grasping policy fails to line up with the needle prior to executing the$y$-axis grasping policy.

• •

  Y: Grasping positioning error (Y): the $y$-axis grasping policy fails line up with the needle prior to grasping.

We categorize all of the failure modes encountered in Table I. We find that the open loop grasping policies are not able to consistently position well for grasping. HOUSTON has failures that are evenly distributed across the failure modes. Grasp policy servoing errors stem mainly from needle configurations that are far outside the distribution seen in training. Presentation phase failures stem primarily from mis-detection of the needle true tip, either because of incomplete segmentation masks or from drift in robot kinematics causing the most distal needle point to not be the tip. This causes the servoing policy to rotate the needle away from the camera, after which sometimes it loses visibility and fails to bring the needle to the pre-handover pose. Multi-handoff failures most frequently arise because of subtle imperfections in grasp execution where the needle rotates to a difficult angle. During the subsequent handover the needle can become obstructed by the holding gripper, inhibiting the grasping policy.