Automatic Tissue Traction Using Miniature Force-Sensing Forceps for Minimally Invasive Surgery

In our forceps, the current $F^{e}_{g}$ is estimated and observed in each running cycle via (3). Originally, we plan to apply a group of parameters $[K_{P,\>1},\>K_{I,\>1},\>K_{D,\>1}]$ with controller (13) to control the grasping force $F_{g}$. However, from (3), we find that $F_{g}$ is positively related to $\theta$. Therefore, we set the corresponding parameters $[K^{\theta}_{P,\>1},\>K^{\theta}_{I,\>1},\>K^{\theta}_{D,\>1}]$ of different $\theta$ configurations as below (14) to minimize the difference in response.

where $f(\theta)=\frac{\sin{\alpha_{0}}}{\sin(\frac{\theta}{2}+\alpha_{0})}$ is a function related to $\theta$.

In addition, considering the deformation of the spring and the press deformation of the tissue, we set the speed of the lower driver ${}^{R}\dot{d}_{l}$ as

This can compensate for the elastic motion and minimize the forceps tip movement, i.e., keeping ${}^{T}{}d_{p}=0$ by setting ${{}^{R}{}d_{l}(\tau)=-{}^{T}t_{s}(\tau)}$ with $\int^{t}_{0}\,{{}^{R}{}\dot{d}_{l}(\tau)}\,d\tau=-\int^{t}_{0}\frac{{}^{T}{}t_{s}(\tau)}{{}^{L}{}d_{u}(\tau)}\,{}^{L}\dot{d}_{u}(\tau)\,d\tau$.

Consequently, the input for the upper driver in the tissue grasping process with controlled $F_{g}$ can be set as

where $e_{1}(t)=F^{\ast}_{g}(t)-F^{e}_{g}(t)$, $F^{\ast}_{g}(t)$ is the target grasping force, and $F^{e}_{g}(t)$ is the current estimated grasping force.

Fig. 4 shows the experimental setup, where a commercial pressure sensor (RFP603 200g, customized by RFP^®, China) was adopted as the reference. Associating with a linear voltage converter (K-CUT, China), the pressure sensor had a resolution of 0.01N in its linear sensing range [0, 0.6]N. The forceps grasped the commercial pressure sensor, which directly measured the reference grasping force. Because the pressure sensor’s sensing area was a 10mm-diameter circle, 3D-printed jaws were rigidly fixed to the forceps’ jaws to ensure sufficient contact. These extension jaws were configured with $\theta=10^{\circ}$, $30^{\circ}$, and $50^{\circ}$ to provide different grasping angles. Consequently, $l_{3}$ in (3) was set as $l^{\prime}_{3}$ in the following experiments. To avoid the slide between the forceps and the pressure force sensor, we fixed the sensor to the lower extension jaw with a thin glue pad. We tuned the controller parameters empirically by referring to the Ziegler–Nichols method. We observed acceptable performance with $[K_{P,\>1},\>K_{I,\>1},\>K_{D,\>1}]=[20,\,1,\,1]$ and applied them to the following evaluation experiments.

Because the grasping force is positively related to the grasping angle $\theta$, as shown in (3), the maximum grasping force that the forceps can produce varies with different $\theta$. Moreover, to cover the range of $F_{g}$ estimation as much as possible, we set the amplitude of the target grasping force profile $F^{\ast}_{g}$ to 0.2N, 0.25N, and 0.3N for experiments with $\theta=10^{\circ}$, $30^{\circ}$, and $50^{\circ}$, respectively, and note them by $i\in(1,2,3)$. Referring to [16, 33, 39, 40], we set the frequency of $F^{\ast}_{g}$ to 1/30Hz, 1/15Hz, and 1/10Hz and repeatedly applied them to experiments with different $\theta$ configurations.

Fig. 5(a), (b), and (c) present the grasping force results of experiments conducted with the targeted grasping force frequency equalling 1/30Hz, 1/15Hz, and 1/10Hz, respectively. For clarification, we plotted results with the same $F^{\ast}_{g}$ frequency in the same subplot. $F_{g,\>i}^{\ast}$, $F^{e}_{g,\>i}$, and $F^{r}_{g,\>i}$ denote the input target grasping force, the forceps-estimated grasping force, and the measured reference grasping force, respectively. Fig. 5(d) shows the errors between $F^{\ast}_{g}$, $F^{e}_{g}$, and $F^{r}_{g}$ with boxplots, and the root mean square (RMS) errors are marked with pentagrams. We also noticed that the error between $F^{\ast}_{g}$ and $F^{e}_{g}$ tended to increase when the frequency of the targeted force $F^{\ast}_{g}$ increased, and the maximum error (0.095N) was observed in the experiment with 1/10Hz and $\theta=50^{\circ}$. On the contrary, errors between $F^{\ast}_{g}$ and $F^{e}_{g}$ showed slight variations, yet all were below 0.06N. This indicates the grasping force $F_{g}$ can be estimated and controlled effectively during the grasping process.

In our forceps, the current $F^{e}_{p}$ is also estimated and observed in each running cycle via (2). We assume the tissue has been grasped stably for the pulling stage, and the upper driver is locked by setting

Then, the pulling force control can be achieved by directly relying on the lower driver’s motion. Consequently, the input for the lower driver in the pulling process with controlled $F_{p}$ can be set as

where $e_{2}(t)=F^{\ast}_{p}(t)-F^{e}_{p}(t)$, $F^{\ast}_{p}(t)$ is the target pulling force, and $F^{e}_{p}(t)$ is the current estimated pulling force.

Similarly, we evaluate the pulling force control by actuating the forceps to follow a series of inputted force profiles $F^{\ast}_{p}$ with different amplitudes and frequencies. The experimental setup is shown in Fig. 4, where the reference pulling force $F^{r}_{p}$ is measured by a commercial gauge force sensor (SBT630B 1kg, SIMBATOUCH, China) that is connected to the pressure sensor by an elastic. The forceps initially grasped the commercial pressure sensor with a grasping force $F_{g}=0.2N$. Then, the forceps pulled the pressure sensor with an input target pulling force profile $F^{\ast}_{p}$, having an amplitude of 2.0N. $[K_{P,\>2},\>K_{I,\>2},\>K_{D,\>2}]=[10,\,2,\,5]$ were chosen and applied to the following evaluation experiments. Similar to the tissue grasping evaluation, three frequencies, 1/30Hz, 1/15Hz, and 1/10Hz, were chosen and repeatedly applied to experiments with three grasping angles $\theta=10^{\circ}$, $30^{\circ}$, and $50^{\circ}$ noted by $i\in(1,2,3)$.

Fig. 6(a), (b), and (c) present the comparisons of pulling force $F_{p}$ in experiments conducted with $F^{\ast}_{p}$ frequency equal to 1/30Hz, 1/15Hz, and 1/10Hz, respectively. For clarification, we plotted results with the same $F^{\ast}_{p}$ frequency in the same subplot. $F_{p,\>i}^{\ast}$, $F^{e}_{p,\>i}$, and $F^{r}_{p,\>i}$ denote the input target pulling force, the forceps’ estimated pulling force, and the measured reference pulling force, respectively Fig. 6(d) shows the errors between $F^{\ast}_{p}$, $F^{e}_{p}$, and $F^{r}_{p}$ with boxplots, and RMS errors are marked with pentagrams. The error between $F^{\ast}_{p}$ and $F^{e}_{p}$ tended to increase when the frequency of $F^{\ast}_{p}$ increased, and the maximum error 0.4N was observed in the experiment with 1/10Hz and $\theta=50^{\circ}$. On the contrary, errors between $F^{e}_{p}$ and $F^{r}_{p}$ showed slight variations, yet all were below 0.16N. This indicates that $F_{p}$ can be estimated and controlled efficiently during the pulling process.

The results above have shown the potential of the proposed controllers in automatic tissue traction, which consists of a grasping stage followed by a pulling stage. In particular, the grasping stage can directly employ the controller developed in Sec. III-A, i.e., pure grasping force control using (15) and (16).

The pulling stage may also just use the pure pulling force controller proposed in Sec. III-B; however, a notable coupling between $F_{p}$ and $F_{g}$ was observed in the experiments in Sec. III-B. For example, Fig. 7 presents the $F_{g}$ variation resulting from the changed $F_{p}$ corresponding to experiments in Fig. 6(a), and the maximum values of these variations were 0.073N, 0.096N, and 0.116N for $\theta=10^{\circ},\,30^{\circ},$ and $50^{\circ}$, respectively. This force coupling phenomenon was also observed in [20, 21, 22] when these studies evaluated their forceps’ force-sensing capabilities.

The force coupling phenomenon can be explained by (2) and (3). Referring to them, we can formulate grasping force as

where $F_{d}=F_{p}+F_{s}$. This indicates that the grasping force $F_{g}$ is positively related to the pulling force $F_{p}$.

When dealing with delicate and critical tissues, it is often desirable to maintain a controlled grasping force while pulling the tissue. Therefore, we propose to simultaneously control the grasping and pulling forces during the tissue pulling stage. This can be conveniently achieved by setting the inputs for lower and upper drivers in the pulling process with (18) and (16), respectively.

Here, we adopted the same experimental setup shown in Fig. 4. The experimental procedures simulated the tissue traction process with the following considerations. First, the forceps grasp the pressure sensor following a target grasping force profile $F_{g}^{\ast}$. Then, during the pulling process, the forceps pull the pressure sensor following a target pulling force profile $F^{\ast}_{p}$ while keeping the grasping force at a constant value $F^{\ast,\,t}_{g}$. $F^{\ast,\,t}_{g}$ simulated a threshold to limit the grasping force and avoid unexpected tissue breaking.

Specifically, the targeted grasping force $F^{\ast}_{g}$ first gradually increased from zero to the threshold $F^{\ast,\,t}_{g}=0.2$N during the grasping phase ($0-5$s). Then, it kept constant during the pulling phase ($8-18$s) when the target pulling force $F^{\ast}_{p}$ increased from 0 to 2.0N. Finally, both $F^{\ast}_{g}$ and $F^{\ast}_{p}$ kept constant during ($18-25$s). During ($5-8$s), $F^{\ast}_{g}$ and $F^{\ast}_{p}$ were constant to visually separate the grasping and pulling phases.

Each experimental procedure was repeated five times. The standard deviation plots Fig. 8(a) and (b) show the comparison of estimated, targeted, and reference values of $F_{g}$ and $F_{p}$, respectively. Since all the experiments followed the same $F^{\ast}_{g}$ and $F^{\ast}_{p}$ profiles, their results almost overlaid. In these traction experiments, we are more concerned with the force variations during the pulling process, and Table I lists the worst-case errors (see Appendix-A) of $F_{g}$ and $F_{p}$ during this period.

We can see the mean, RMS, and maximum errors between $F^{\ast}_{g}$ and $F^{e}_{g}$ are all under (0.0106, 0.0128, 0.0377)N, which were comparable with that of experiments shown in Fig. 5. The mean, RMS, and maximum errors between $F^{\ast}_{p}$ and $F^{e}_{p}$ are all under (0.2480, 0.2890, 0.5529)N, which were also comparable to that in experiments shown in Fig. 6. Moreover, compared to that without controlled grasping force (see Fig. 7), the maximum deviation of $F_{g}$ in the pulling phase reduced by around 300% for $\theta=50^{\circ}$ (from 0.116N to 0.0377N), 470% for $\theta=30^{\circ}$ (from 0.096N to 0.0202N), 460% for $\theta=10^{\circ}$ (from 0.073N to 0.0151N). These indicate that simultaneous $F_{p}$ and $F_{g}$ control can be effectively achieved during the pulling process.

In addition, Fig. 8(c) shows the corresponding driving force $F_{d}$ and spring elastic force $F_{s}$ of these experiments. We can see that less $F_{d}$ and $F_{s}$ were required for forceps with larger $\theta$ to achieve the same $F^{\ast,\,t}_{g}$. Moreover, the driving force $F_{d}$ remained nearly constant during the pulling phase while the spring elastic force $F_{s}$ (equal to the supporting force) gradually decreased. This was because the constant grasping force required constant driving force, and the system had to reduce the elastic force $F_{s}$ to ensure the increased pulling force $F_{p}$. This phenomenon confirmed the analysis in (19) and revealed that the decoupling control of $F_{g}$ and $F_{p}$ was achieved by indirectly controlling $F_{d}$ and $F_{s}$.

Fig. 9(a) shows the experimental setup, where two UR robotic arms were adopted as the macro-level actuators. The ex vivo chicken tissue was placed in a human body phantom to simulate the lesion, and instruments were implemented through simulated minimally invasive ports on the phantom. Both universal robotic arms, UR-1 and UR-2, have been pre-docked to the initial state, and the micro-level actuators were responsible for performing the automatic traction-assisted tissue resection task locally. During the tissue traction, no manual manipulation was applied to the forceps. Specifically, UR-1, carrying the forceps for tissue traction, was locked during the operation, while UR-2 was teleoperated to adjust the pose of the scissors for tissue cutting.

To integrate the force controller and the robotic controller, we further designed a software architecture, as depicted in Fig. 9(b), which contains a forceps controller and a scissors controller. The forceps controller was responsible for the tissue traction. Dell Latitude 5420 laptop equipped with an i5-11${th}$ @ 2.6GHz CPU received the task state and sent the target forces $\mathbf{F}^{\ast}$ to the force controller. Subsequently, the force controller generated the inputs $\mathbf{u}=[^{R}\dot{d}_{l},\,^{L}\dot{d}_{u}]^{\top}$ and sent them to the forceps’ micro-level actuator of the forceps. The operator could supervise the procedure and input orders through a designed graphic user interface (GUI), shown in Fig. 9(a). This GUI also displayed real-time data, including sensed multiple forces, pulling distance, endoscope image, and camera frame of the sensing module.

The scissors controller was responsible for the tissue cutting. A touch device (3D Systems, USA), shown in Fig. 9(a), enabled the teleoperation of the UR-2 robotic arm and facilitated the transmission of cutting commands to the micro-level actuator of the scissors. User movements captured by the touch device were abstracted by the touch ROS driver provided by [41]. These movements were then transformed and relayed to the robot’s end-effector frame, located at the tip of the scissors.

The force controller, state estimator, and signal generator for micro-level actuators were implemented on an STM32H7 microcontroller, which communicated with the laptop and ROS via serial ports. The UR-2 robot was connected to another computer running ROS via an Ethernet cable, and the touch device was linked to the computer through a USB port. The publishing rate for the UR-2 robot was 500 Hz, while the touch device’s was 1000 Hz. Each pose trajectory command sent to the UR-2 robot had a duration of 1 ms.

For typical tissue resection, we noticed that the required pulling force at the first cut is more significant than that of the following cuts. Therefore, the targeted pulling force $F_{p}^{\ast}$ should be reduced after each cut. Moreover, because of the elasticity, the tissue body tends to collapse after the connection part has been cut out. This indicates that the cutting surface may passively be revealed by the tissue collapse when the tissue has been pulled up to a certain displacement.

Considering the above phenomena and patterning after the clinical dissection procedure [18], we designed an operation flow with automatic traction. The automatic traction was built upon predefined grasping and pulling forces profiles, complemented by the pulling distance information. The forceps first reached the tissue with touch detection by comparing the estimated $F_{p}$ with a set threshold $F_{p}^{*,\>t}=-0.05$N (the negative sign indicates the operation is opposite to pulling tissues). Next, the tissue was grasped to the target grasping force $F_{g}^{\ast}=0.3$N, followed by pulling up and holding with an initial target pulling force $F_{p}^{\ast,\>0}=0.25$N. After the pulling, the scissors were teleoperated to the target tissue, and cutting was performed. Then, the operation alternated between pulling and cutting several times, and the target pulling force $F_{p}^{\ast,\>i}$ reduced to $\rho F_{p}^{\ast,\>i-1}$ after the $i^{th}$ cut. Here, $i\in[1,\>m]$, $m$ is the number of cuts performed, and $\rho=0.8$ is the reduction ratio. Once reaching the target force $F_{p}^{\ast,\>i}$ in each pulling, the forceps kept static with no pulling distance $d_{p}$ change to provide a constant cutting surface. The incremental pulling distance $\Delta d_{p}$ between cuts was limited by a threshold $\Delta d_{p}^{t}=20$mm, and the total pulling distance $d_{p}$ was limited by $d_{p}^{t}=30$mm. A threshold for cutoff detection $F^{\ast,\,c}_{p}=0.05$N was also included. When the estimated pulling force $F^{e}_{p}$ after each cut was less than $F^{\ast,\,c}_{p}$, the forceps would not pull up the tissue further. Then, the operator was required to check whether the tissue had been cut off and confirm whether another cut was needed or could be moved out directly. The above operation flow is also summarized in Appendix-B.

Although through the distributed software framework design and direct memory access (DMA) communication, the local controller could achieve high-frequency operation with minimized signal interference, the updating frequency of estimated elastic supporting force $F_{s}$ was limited to 30Hz by the camera (OVM6946, Omnivision Inc., USA) update rate. The frequency could be improved by adopting a camera with a higher rate. Consequently, the performance of force estimation and control could be further improved. Replacing the spring with flexures that have different stiffness, the sensitivity and resolution of the sensing module can also be configured. Additionally, the grasping force $F_{g}$ estimation is related to the supporting force $F_{s}$, and the sensible range of $F_{s}$ (limited by the spring’s deformation) is $[0-5]$N in our prototype. Therefore, the sensible range of $F_{g}$ is also limited and positively related to grasping angle $\theta$.

For the PID controller, the proportional term produces an output value proportional to the current error value and generally operates with a steady-state error. Then, the integral term is required to eliminate the steady-state error. However, a large proportional or integral term could result in a notable overshoot [42]. This overshoot can introduce excessive forces and bring risks to tissue manipulation. Consequently, we tended to select smaller $K_{P}$ and $K_{I}$ values during the controller tuning process. In addition, a large derivative term may impact the controller’s stability, but a small one can improve it [42]. Therefore, smaller $K_{D}$ values were also preferred. These parameters can be fine-tuned to enhance the performance in specific aspects for different preferences.

We demonstrated the system has the ability to track force profiles with different frequencies and amplitude, as shown in Fig. 5, Fig. 6, and Fig. 8. Because the actuators’ displacements and speeds can not change rapidly, the response speed of force control is limited, and there are rise times for the system to track the target forces. These rise times approximately equal the execution times of the grasping and pulling tasks. For example, using the experimental setup shown in Fig. 4, we verified the system’s step response, and the results are shown in Fig. 11. For target grasping force $F_{g}^{\ast}$ with 0.35N step signal, a 1.5s rise time with 1.6mm driving cable displacement $d_{c}$ was observed. For target pulling force $F_{p}^{\ast}$ with a 1.0N step signal, a 0.5s rise time with 4mm pulling distance $d_{p}$ was observed. However, the force variation in tissue traction often involves considerable tissue deformation, and no rapidly changed force is required [18]. In addition, ex vivo experiments show that the forceps can effectively complete the adjustment of forces between cuts. Therefore, these rise times will not compromise the effectiveness of tissue traction.

To further identify the potential benefits of tissue traction with controlled pulling force $F_{p}$, we conducted a group of resections on ex vivo tissue relying on pulling distance $d_{p}$. During the operation, the incremental pulling distance $\Delta d_{p}$ was set to 10mm, while the target pulling force $F^{\ast}_{p}$ and its reduction ratio $\rho$ were disabled. Fig. 12(a) and (b) show that the tissue slid and split due to excessive pulling forces $F^{\ast}_{p}$, respectively. For each experiment, $F_{g}$, $F_{p}$, and $d_{p}$ were recorded, and the top endoscopy snapshots show the critical events of the operation.

In Fig. 12(a), the target grasping force $F^{\ast}_{g}$ was set to 0.3N as that in Fig. 10. Because the tissue was pulled up with $\Delta d_{p}$ after each cut without intentional $F_{p}$ control, the excessive $F_{p}$ was observed between 50$-$53s with $F_{p}$ = 0.38N (which was limited by a threshold of 0.25N in Fig. 10) and the tissue slide occurred at 66s as the tissue was excessively pulled with $d_{p}$ = 25 mm and $F_{g}$ = 0.37N.

We then increased the target grasping force $F^{\ast}_{g}$ to 0.35N in the experiment presented in Fig. 12(b). Although the tissue slide was avoided, we observed tissue split. The excessive $F_{p}$ was observed between 45$-$55s with $F_{p}$ = 0.4N, and the tissue slide occurred at 63s as the tissue was excessively pulled with $d_{p}$ = 22mm and $F_{p}$ = 0.24N. This tissue split could be avoided with controlled $F_{p}$, as the estimated $F_{p}$ after $C_{2}$ was 0.03N at 61s, which is less than the cutoff detection threshold $F^{\ast,\,c}_{p}=0.05$N in Fig. 10. This means the remaining tissue was too fine and should not be pulled up further after $C_{2}$.

We also conducted tissue traction with the grasping process achieved by (15) and (16), and the pulling process relied on (18) and (17), i.e., the grasping force $F_{g}$ is not controlled during the pulling process. The tissue also has been successfully cut off, and the result is shown in Fig. 13. Since the required $F_{p}$ in the tissue-pulling process is small (0.25N in these experiments), the force coupling was not obviously observed. However, another interesting phenomenon caught our attention: the grasping force $F_{g}$ in Fig. 13 gradually decreased (0.04N reduction at the end, 12% of 0.3N) after each cut. This could result from the interaction between the scissors and tissue, which caused rapid force variation and impaired the grasping condition. This reduced grasping force could increase the risk of tissue slides in the following operations.

Nevertheless, with decoupling force control, the grasping force $F_{g}$ in the experiments presented in Fig. 10 recovered to the target value $F_{g}^{\ast}$ (with 0.01N errors at the end, 3% of 0.3N). Compared to the experiment in Fig. 13 that has a 12% reduction of $F_{g}$, enabling grasping force control in the pulling process could reduce tissue slide risk. This phenomenon further supports the criticality of simultaneous multi-force control in tissue manipulations.

The used parameters in tissue resections, including touch detection threshold $F^{\ast,\,t}_{p}$, targeted grasping force $F_{g}^{\ast}$, initial targeted pulling force $F_{p}^{\ast}$, reduction ratio $\rho$, pulling distance incremental threshold $\Delta d_{p}^{t}$, total pulling distance threshold $d_{p}^{t}$, and cutoff detection threshold $F^{\ast,\,c}_{p}$, were empirically chosen. For more precise operation, these parameters should be identified according to clinical requirements. To help understand this, we also conducted automatic traction-assisted resections on ex vivo beef tissue.

We first conducted a resection with the same parameters in Fig. 10, i.e., initial target pulling force $F^{\ast,0}_{p}$ = 0.25N, reduction ratio $\rho$ = 0.8, and target grasping force $F^{\ast}_{g}$ = 0.3N. Fig. 14(a) presents the result, and the top snapshots show the critical stages. Although the tissue was successfully cut off with five cuts, the operation was more challenging as the cutting surface overlapped due to insufficient pulling force $F_{p}$, which made it more difficult for the scissor to reach, as shown in $C_{3}$ of the snapshots. This may be because beef tissue is tougher than chicken tissue [44].

Consequently, we slightly increased $F^{\ast,0}_{p}$ to 0.3N, $\rho$ to 0.9, and $F^{\ast}_{g}$ to 0.35N to provide a better cutting surface. Then, two resections were repeatedly conducted, and the results are shown in Fig. 14(b) and (c). The top snapshots are the critical stages corresponding to the trial of Fig. 14(b). The snapshots show that the cutting surfaces in Fig. 14(b) have been improved compared to Fig. 14(a), and no tissue slide, break, or split occurred.

In practical surgery, external sensing modules, such as a vision system, could also provide more information for decision-making. With additional information and considerations of clinical constraints, the tissue resection strategy has the potential to be extended to more general operations.