\corrauth

Amir Hossein Hadi Hosseinabadi, Robotics and Controls Laboratory (RCL), Electrical and Computer Engineering Department, University of British Columbia, Vancouver, British Columbia (BC), Canada,
V6T 1Z4, Canada

Force Sensing in Robot-assisted Keyhole Endoscopy: A Systematic Survey

Amir Hossein Hadi Hosseinabadi and Septimiu E. Salcudean¹¹affiliationmark: ¹¹affiliationmark: Robotics and Controls Laboratory (RCL), Electrical and Computer Engineering Department, University of British Columbia, Vancouver, British Columbia (BC), Canada [email protected]

Abstract

Instrument-tissue interaction forces in Minimally Invasive Surgery (MIS) provide valuable information that can be used to provide haptic perception, monitor tissue trauma, develop training guidelines, and evaluate the skill level of novice and expert surgeons.Force and tactile sensing is lost in many Robot-Assisted Surgery (RAS) systems. Therefore, many researchers have focused on recovering this information through sensing systems and estimation algorithms.

This article provides a comprehensive systematic review of the current force sensing research aimed at RAS and, more generally, keyhole endoscopy, in which instruments enter the body through small incisions. Articles published between January 2011 and May 2020 are considered, following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. The literature search resulted in 110 papers on different force estimation algorithms and sensing technologies, sensor design specifications, and fabrication techniques.

keywords:

Force and Tactile Sensing, Medical Robots and Systems, Force Control, Haptics and Haptic Interfaces, Telerobotics

1 Introduction

In Minimally Invasive Surgery (MIS), surgical access is provided through small incisions or natural orifices in the body. A surgical instrument is operated by the surgeon for tissue manipulation. Compared to open surgery, MIS provides less tissue trauma, postoperative pain, patient discomfort, wound complications and immunological response stress Wottawa et al. (2016), lower risk of infection Soltani-Zarrin et al. (2018) and blood loss Dai et al. (2017), shorter hospital stay Bandari et al. (2020), faster recovery Lee et al. (2015), and improved cosmetics Aviles et al. (2016) all of which lead to improved therapeutic outcome and efficiency Otte et al. (2016) and lower morbidity and morality Aviles et al. (2017) making MIS cost-effective Faragasso et al. (2014). Nonetheless, the ergonomically cumbersome posture increases surgeon fatigue. The limited instrument dexterity and visual perception of the scene Haghighipanah et al. (2017); Haouchine et al. (2018), and the non-intuitive hand-eye coordination due to fulcrum motion reversal decrease accuracy and contribute to surgeon fatigue Hadi Hosseinabadi et al. (2019). The high level of psychomotor skills needed increases the operation time and require a longer learning curve Shahzada et al. (2016). The sense of touch is reduced by friction in the access port and instrument mechanism.

In Robotic MIS (RMIS), the surgical instrument is controlled by a robotic manipulator and operated by a remote surgeon. The robotic operation restores hand–eye coordination Aviles et al. (2015a) and innovations in tool design improve dexterity leading to improved ergonomics that reduce surgeon fatigue Stephens et al. (2019); Bandari et al. (2020). The enhanced 3D surgical vision, automatic movement transformations, fine motions, filtering of physiological hand tremor and motion scaling lead to improved surgery precision Sang et al. (2017). However, the surgeon is isolated from the surgical site by robotic manipulators that do not provide the haptic perception Juo et al. (2020). This deprives the surgeon of a rich source of information. Thus, many studies are targeted towards the reconstruction and evaluation of haptic feedback.

Haptics can be either tactile or kinesthetic Okamura (2009). The tactile perception is through the cutaneous receptors in the skin which can sense, for example, texture or temperature Mack et al. (2012). The kinesthetic force feedback is perceived by mechanoreceptors in the muscle tendons to detect force, position and velocity information about objects Juo et al. (2020). Traditionally surgeons use palpation to characterize tissue properties, detect nerves and arteries Bandari et al. (2017), and identify abnormalities such as lumps and tumors Lv et al. (2020); Puangmali et al. (2012). Moreover, the surgeons rely on the sense of touch to regulate the applied forces. Excessive forces can lead to tissue trauma, internal bleeding, and broken sutures. Insufficient forces however can lead to loose knots and poor sutures. Sang et al. (2017); Li et al. (2016).

Direct Force Feedback (FF) and Sensory Substitution (SS) are the most common approaches of presenting surgeons with force information. While the direct method provides the most intuitive interaction Li and Hannaford (2017), it is the most challenging one to implement, as it requires a method of force sensing and a safe and robust teleoperation interface for force reflection. A compromise between transparency and stability of different teleoperation frameworks is reported by Hashtrudi-Zaad and Salcudean (2001). In sensory substitution, visual, auditory, or vibro-tactile signals provide haptic perception to the surgeon. While safety can be easily guaranteed, this method can cause discomfort, distraction, and cognitive overload. In general, visual methods are shown to be the most effective feedback modality. Abdi et al. (2020)

In summary, the introduction of haptic perception is proven to decrease operation time Abiri et al. (2017), facilitate training, improve accuracy, and enhance patient safety for novice surgeons in complex tasks Juo et al. (2020). More experienced surgeons learn to infer force information from visual cues such as the tissue and instrument deformations and the stretch in sutures Aviles et al. (2017). Additionally, force information can be used to automate surgical robot tasks in dynamic and unstructured environments Kuang et al. (2020), to identify tissues in real time, to create tissue-realistic models and simulators for training Stephens et al. (2019), and to perform surgical skills assessment Soltani-Zarrin et al. (2018).

1.1 Comparison to the Existing Reviews

An extensive review of haptic perception and its efficacy in RMIS is presented by Amirabdollahian et al. (2018). This review concluded that while there is a consensus on the need for haptic and tactile feedback, no commercial system is yet available that addresses this need. More recently, El Rassi and El Rassi (2020) presented a brief overview of haptic feedback in teleoperated robotic surgery. Overtoom et al. (2019) and Rangarajan et al. (2020) surveyed virtual haptics in surgical simulation and training. The latter followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines to identify the relevant literature. The authors similarly affirm the efficacy of haptic feedback in surgical education. None of the publications above review the developments in the field of force sensing and estimation.

Abdi et al. (2020) reviewed research since 2000 on the efficacy of haptic feedback in teleoperated medical interventions. The authors present a concise overview of the force-sensing literature with 44 references cited over a wide range of medical applications. Although the review provides a general understanding of the challenges and complexities in instrument-tissue force measurement, it is not a comprehensive presentation of the prominent developments and the articles were subjectively selected with no evaluation criteria. Additionally, the records were only classified based on the sensing technology and the sensor location; However, the instrument’s dexterity level, the sensing Degrees of Freedom (DoFs), and the performance measures were not compared. A comparison of its references with the records cited in our review shows an overlap of only 20 out of 110 papers. Bandari et al. (2020) reviewed tactile sensing literature over the past twenty years. It also includes some literature on force-sensing in neurosurgery and microsurgery procedures. Although the authors presented a comprehensive review with 121 references, a comparison of the included articles with the records in this paper shows an overlap of only 8 out of 110 papers which are mostly on developments related to the gripping force sensing.

This article is a systematic review based on the PRISMA guidelines that expands on the sensor design requirements and presents the most recent developments in force sensing and estimation in keyhole endoscopy. We discuss how research has evolved over the past decade and provide suggestions for future research directions. The closest publications to our review are the surveys by Puangmali et al. Puangmali et al. (2008) and Trejos et al. Trejos et al. (2010) which were published about a decade ago, and therefore there are no overlapping papers with those reviews.

Refer to caption — Figure 1: PRISMA flow diagram

2 Methodology

A systematic survey was conducted by following the PRISMA guidelines (see Figure 1) and it was based on Google Scholar, Web-of-Science, PubMed, and IEEE Xplore Digital Library repositories. The period for the review is over the past decade, from January 2011 until May 2020. The following keywords were used for identification: Force sensing, Kinesthetic, Tactile, Haptics, Minimally Invasive Surgery, MIS, MIRS, Robot-Assisted, RMIS, RAS, RAMIS, Laparoscopy, and Endoscopy. For every year, the first 20 pages of search results in Google Scholar were surveyed (total of 2000 records). The same approach was used for the identification of records through the other repositories (PubMed: 213, Web-of-Science: 42, and IEEEXplore: 40). For screening, the duplicates were removed and the identified records were skimmed through to mark the ones that are relevant to keyhole endoscopy. The articles that refer to force sensing in microsurgery, neurosurgery, and needle insertion were excluded because they involve a different set of requirements and challenges. Specifically, microsurgical instruments such as those used in neurosurgery and retinal surgery Gonenc et al. (2017) have a much smaller diameter (less than 2 mm) and do not require an articulated wrist, which complicates the actuation system and sensors’ power and signal routing. Moreover, Bandari et al. (2020) briefly discussed the force-sensing literature in microsurgery, neurosurgery, and needle insertion. 114 articles were found eligible for a complete review. Throughout the review, the references of the selected papers were surveyed and the relevant articles that were not initially identified were added, thus increasing the total number of eligible records to 129. The work progressions and duplicate publications were removed to lead to the 110 articles included in this survey.

The included articles are tabulated for an easier comparison of the method, the sensor location, the sensing DoFs, the dexterity of the instrument under study, and the results. The Dexterity Index (DI) for different instruments is defined according to the Table 1 and Figure 2. Depending on the sensor location, the sensing DoFs are defined as instrument or wrist tri-axial forces (F_X, F_Y, F_Z) and moments (M_X, M_Y, F_Z), and the gripper normal (F_N), shear (F_S), and pull (F_P) forces as depicted in Figure 3. In summarizing the results, the following acronyms were used: ACC: Accuracy, ERR: Maximum absolute error, MAE: Mean-Absolute-Error, NRMSE: Normalized-Root-Mean-Square-Error, RES: Resolution, RMSE: Root-Mean-Square-Error, RNG: Range, and SENS: Sensitivity.

Table 1: Dexterity index definition for MIS instruments

Dexterity Index (DI)	Instrument Functionality and DoFs
-	Standalone Testing
0	Palpation
1	Grasping
2	Grasping + Axial rotation
3	Grasping + Flexion
4	Grasping + Flexion + Axial rotation
5	Grasping + Flexion + Abduction
6	Grasping + Flexion + Abduction + Axial rotation

3 Design Requirements

3.1 DoF, Range, Resolution, Accuracy, Bandwidth and Sampling Rate

The grasping force, the instrument lateral and axial forces, and the axial torque are the most relevant DoFs to improve accuracy and provide an effective haptic experience in MIS applications Wee et al. (2016); Soltani-Zarrin et al. (2018); Bandari et al. (2020). Deformations in the sensor structure or displacements in its components are the physical surrogates that are monitored for force estimation. Thus, there are always trade-offs between the sensor’s structural rigidity, resolution and sensitivity, and range Puangmali et al. (2012). The grip force can reach up to 20 N in da Vinci instruments during needle handling or knot-tying O’Neill et al. (2018); Bandari et al. (2020); however, pinch forces as large as 4 N can cause damage to delicate tissue Hong and Jo (2012); Abiri et al. (2019). The maximum allowable suture pull force is 4-6 N Spiers et al. (2015); Dai et al. (2017). The optimal kinesthetic force range suggested for MIS applications is $\pm$ 10 N in all directions and $\pm$ 20 N for grasping Khadem et al. (2016); Wee et al. (2017). No requirement on bending moments and axial torque is specified in the literature Hadi Hosseinabadi et al. (2019). Resolutions of 0.06 N Wang et al. (2014); Choi et al. (2017) and 0.2 N Soltani-Zarrin et al. (2018) are suggested for FF and SS schemes, respectively. The human just-noticeable difference (JND) is 10% Wang et al. (2014); Kim et al. (2017) in the range of 0.5 to 200 N increasing to 15-27% below 0.5 N Hwang and Lim (2017) which can be considered as a requirement on the sensor accuracy. The human’s temporal resolution is 320 Hz for force discrimination and up to 700 Hz for vibration detection Puangmali et al. (2012). However, the desired bandwidth of the force sensor is usually dictated by the application (FF, SS, vibration detection, etc.) and desired noise and resolution performance. A sample rate of 500 Hz is considered appropriate for direct force feedback applications Jones et al. (2017). Sample rates as low as 30 Hz can be effective in visual SS modality.

3.2 Size, Mass, and Packaging

Surgical instruments are inserted into the body through a cylindrical port of 12-15 mm in diameter Spiers et al. (2015); Li et al. (2015b). The outside diameter of the instrument is desired to be less than 10 mm Shi et al. (2019). The sensor should be lightweight to not significantly increase the instrument inertia. The operation rooms are filled with equipment that can cause electromagnetic interference, and the electrocautery tools operate at high voltages Lim et al. (2014); Seok et al. (2019). Thus, the sensors require insulation for electrostatic protection and shielding against electromagnetic interference Peña et al. (2018). The sensors that enter the body also require sealing against humidity and debris ingression Trejos et al. (2014).

3.3 Sterilizability

Surgical instruments are cleaned and sterilized for reuse; the former refers to removing debris from the device and the latter is the elimination of microorganisms that can cause disease Trejos et al. (2014). The common sterilization methods are plasma and gamma radiation, the use of chemicals (alcohol, ethylene oxide or formaldehyde), and steam sterilizations Bandari et al. (2020). Steam sterilization is the fastest and the most preferred method Soltani-Zarrin et al. (2018) which is performed in an autoclave at 120-135°C, 207 kPa and 100% humidity for 15-30 minutes Spiers et al. (2015); Zhao and Nelson (2015). This harsh environment can be destructive to many transducers, signal conditioning electronics, wire insulations, bondings, and coatings.

3.4 Biocompatibility

The sensors for use in MIS must abide by ISO 10993 which entails a series of standards for evaluating the biocompatibility of medical devices Trejos et al. (2014). For biocompatibility electrical components often require coatings that interfere with sterilizability Spiers et al. (2015).

3.5 Adaptability and Cost

Instruments are disposed after 10 to 15 uses due to accelerated cable fatigue Anooshahpour et al. (2014); Kim et al. (2018a); Xue et al. (2018). The EndoWrist instruments retail at $2k-$5k Spiers et al. (2015). If the sensor is integrated into the instrument and is to be disposed, it should not increase the instrument price significantly. An adaptable solution that can be easily used on different instruments is desirable.

4 Location

The sensors can be placed in the instrument mounting interface, the instrument base, proximal (outside the body) and distal (inside the body) shafts, the actuation mechanism (cables/rod), the trocar mount and its distal end, the articulated wrist, and the gripper jaws (Figure 4).

While the size, sterilizability, biocompatibility, and insulation requirements are more relaxed for the sensors placed outside the body, these locations are more prone to the factors causing sensor inaccuracy. The sensors at the instrument interface, base, and proximal shaft can have the electronics isolated from the patient Van Den Dobbelsteen et al. (2012). The sensors in the instrument shaft can gain high precision in the lateral direction, but experiments Maeda et al. (2016); Lv et al. (2020) showed that they do not provide high resolution in the axial direction unless the structure is modified to amplify axial strains. The sensors in the instrument shaft and trocar cannot independently measure the gripping force He et al. (2014) and measuring cable tensions cannot provide information on the axial force. The sensors integrated into the trocar and instrument interface are usually adaptable Wang et al. (2014).

Sensors placed at the gripper jaw provide the most accurate readings and have the most stringent design constraints. They are difficult to fabricate, package, mount Kim et al. (2018b); He et al. (2014), and shield Seok et al. (2019); Wang et al. (2014) and have limited adaptability which makes them cost-prohibitive for disposable instruments Xue et al. (2019). Additionally, the electronics are usually placed away from the transducer which deteriorates the Signal to Noise Ratio (SNR) Suzuki et al. (2018). Placing the force sensor at the grasper may also conflict with functional requirements for monopolar or bipolar cautery instruments He et al. (2014).

Figure 5 summarizes the severity level of different sources that contribute to the sensing inaccuracy as a function of the sensor location (scale of 1 to 3; 1 is minimum, 3 is maximum and ✘ is no effect). It also compares how stringent the listed design requirements are for each sensor location (scale of 1 to 3; 1 is the least, 3 is the most and ✘ refers to not a requirement). The distribution of the records included in this survey as a function of the sensor locations and the sensing technologies are shown in the same figure. It is evident that the sensorless techniques have the majority of publications over the past ten years. Additionally, the Micro-Electro-Mechanical (MEM) and Fiber Brag Grating (FBG) technologies have been widely adopted in the fabrication of miniature transducers that can be integrated into the gripper jaws. An overview of different transduction technologies is presented in the next section.

Table 2: Sensorless force estimation: model-based

Author	Method	Sensing DoFs	Instrument / DI	Results
1- Li et al. (2013)	Pneumatic Actuation Muscles (PAM) were used in design of a custom forceps. Disturbance observers were used on the pneumatic actuation system and the robot joints.	Instrument tri-axial forces	Custom Developed (CD) RMIS forceps / 5	ERR $<$ 0.4N RNG: 0-3.5N
2- Tsukamoto and Ishii (2014)	Proposed a three step Robust Reaction Torque Observer (RRTO): 1) Cancel the error in estimated torque (overshoot correction) 2) Identify and compensate the inertial torque component in the drive train (inertia compensation) 3) Estimate the gripping torque.	Grip torque	CD RMIS instrument / 4	Plot comparison, not quantified.
3- Anooshahpour et al. (2014)	Proposed two quasi-static models on cable dynamics (Pull & Pull-Push) that take tendons friction & compliance into account. A linear combination of the two models provided a close estimation of the output gripping torque.	Grip torque	EndoWrist Needle Driver / 6	Plot comparison, not quantified. RNG: $\pm$ 40Nmm
4- Lee et al. (2014)	The gripper was actuated by a pneumatic catheter balloon to provide a uniform gripping force. The pneumatic pressure was monitored to estimate the grip force.	Grip force	CD RMIS instrument / 5	ERR $<$ 0.3N RNG: 0-10N
5- Zhao and Nelson (2015)	A wrist actuation design using planetary gears was proposed to decouple the motions in different DoFs. The motor currents were used to estimate the forces in static and dynamic scenarios.	Instrument lateral forces, grip force	CD RMIS instrument / 5	ERR $<$ 0.4N RNG: 0-2N
6- Lee et al. (2015)	Proposed the compensation of the gripping torque which was experimentally identified as a function of the instrument posture. The experiments were on: EndoWrist 1) ProGrasp 2) Large Needle Driver 3) Dissecting forceps.	Grip force	EndoWrist instruments / 6	1) ERR $<$ 10.69% 2) ERR $<$ 13.03% 3) ERR $<$ 16.25%
7- Haraguchi et al. (2015)	The articulated wrist was replaced by a machined spring. The instrument was pneumatically driven. A 3-DoF continuum model of the spring distal joint and the pneumatic pressure were used for force estimation.	Forceps tri-axial	CD RMIS instrument / 6	ERR $<$ 0.37N RNG: $\pm$ 5N
8- Yoon et al. (2015)	Proposed the use of Sliding Perturbation Observer (SPO) to estimate the reaction force. The presented method compensated for the Coulomb friction.	Grip force, pitch torque, instrument axial torque	EndoWrist ProGrasp / 6	RNG: F_G: 0-10N Pitch torq.: $\pm$ 150Nmm Axial torq.: $\pm$ 1Nm
9- Rahman and Lee (2015)	Cascaded fuzzy logic in Sliding Mode Control with SPO (SMCSPO) to separate different types of disturbances. A rectified position information was defined and used to estimate the perturbation (grip force).	Grip force	EndoWrist ProGrasp / 6	RNG: 0-15N
10- Li et al. (2016)	Proposed the use of an Unscented Kalman Filter (UKF) based on a dynamic model that considered cable properties, cable friction, cable-pulley friction, and a bounding filter. The proposed approach used motor currents and motor encoder readings.	Grip force	Raven-II 10 mm gripper / 6	ERR $<$ 50% RNG: 0-1N
11- Anooshahpour et al. (2016)	Proposed the use of Preisach approach to model the input-output hysteretic behavior in a da Vinci instrument.	Grip force	EndoWrist Needle Driver / 6	ERR $<$ 0.6N RNG: 0-6.5N
12- Sang et al. (2017)	Developed and identified a dynamic model for the Patient Side Manipulator (PSM) of the daVinci Standard system and the surgical instrument. The identified model was used for external force estimation.	Instrument tri-axial forces	EndoWrist Needle Driver / 6	ERR $<$ 0.1N RNG: $\pm$ 1.5N
13- Haghighipanah et al. (2017)	Evaluated two approaches for force estimation on the 3^rd link of the Raven-II system: 1) Further expanded on the approach in Li et al. (2016) by adding cable tension estimation. 2) The force was estimated by measuring the cable stretch using a linear encoder.	Instrument axial force	Raven-II 10 mm gripper / 6	ERR 1) $<$ 4N, 2) $<$ 3N RNG: 0-10N #2 Provided better estimation at lower forces
14- Li and Hannaford (2017)	Used the Gaussian Progress Regression (GPR) supervised learning approach because of its ability to deal with uncertainties and nonlinearity. The model inputs were motors encoder, velocity, and current.	Grip force	Raven-II 10 mm gripper / 6	ERR $<$ 0.07N RNG: 0-1N

Table 3: Sensorless force estimation: model-based - continue.

Author	Method	Sensing DoFs	Instrument / DI	Results
15- Xin et al. (2017)	Developed the dynamic model of one jaw by using the Benson model to describe the dry friction. The parameters were experimentally identified for an instrument designed based on the concept in Zhao and Nelson (2015).	Grip force	CD RMIS instrument / 5	ERR $<$ 0.25N RNG: 0-2.5N
16- O’Neill et al. (2018)	Evaluated motor current command and measurement, and differential gearbox as proximal torque surrogates and used Neural Networks (NNs) to estimate the distal gripping torque considering all three surrogates as inputs.	Grip force	daVinci Si Maryland grasper / 6	ERR $<$ 0.37N RNG: 0-11N
17- Huang et al. (2018)	Proposed the use of NNs optimized by a Genetic Algorithm (GA) for force estimation. The model inputs were the motors’ positions, velocities, and currents.	Grip force	CD RMIS instrument / 5	ERR $<$ 0.06N RNG: 0-1.6N
18- Takeishi et al. (2019)	Suggested the use of pneumatic actuators and NN for force estimation. Low accuracy in abrupt forces was reported. All the analysis was model-based in MATLAB.	-	Simulation	RNG: 0-10 N
19- Abeywardena et al. (2019)	A NN architecture with LSTM was proposed that used motors currents as the inputs. The model was trained for different stages of no grasp, closing, and opening.	Grip force	EndoWrist ProGrasp / 6	ERR $<$ 0.4N RNG: 0-20N
20- Stephens et al. (2019)	The performance of NNs, decision tree, random forest, and support vector machine models were compared in the angle and gripping torque estimation of each jaw. It concluded that the NN estimations were reliable when trained and tested on each jaw, on the same tool, and withing the frequency of the training data.	Grip force	EndoWrist ProGrasp / 6	ERR $<$ 0.07 N RNG: 0-5.5N
21- Wang et al. (2019b)	Proposed an external force estimation method based on cable-tension disturbance observer and the motion control strategy.	Grip force	CD RMIS instrument / 5	ACC $>$ 85% RNG: 0.1-2N

5 Sensing Technologies

Sensing technologies and the corresponding number of articles in this survey is shown in Figure 6.

5.1 Sensorless

Table 4: Vision-based force estimation

Author	Method / Task	Sensing DoFs	Stereo / Mono	Results
1- Martell et al. (2011)	Image processing algorithms were utilized for suture strain estimation by identifying the suture line and tracking the displacement of markers. The achieved resolution in strain estimation was two orders of magnitudes smaller than the known strain to failure of most suture materials (20+%). / Suture pull	Force magnitude	Mono	Strain resolution of 0.2% and 0.5% was achieved in one-marker tracking on stationary suture and two-marker tracking on moving suture, respectively.
2- Kim et al. (2012)	The soft-tissue deformation was obtained by processing the stereoscopic depth image as a surface mesh. It was compared against the original organ shape from pre-operative images. A spring damper model was used for force estimation. / Tissue push and pull	Gripper tri-axial forces	Stereo	No results were presented
3- Noohi et al. (2014)	A virtual template, based on assuming soft tissue local deformation to be a smooth function, was used to estimate the tissue deformation without a-priori knowledge of its original shape. The force magnitude was estimated by using a biomechanical model. / Tissue push	Gripper tri-axial forces	Mono	In force magnitude: ERR $<$ 0.12N RMSE = 0.07N RNG: 0-2.5N
4- Faragasso et al. (2014)	A force sensing device composed of a linear retractable mechanism and a spherical visual feature was installed on the endoscope. The force was estimated as a function of the size of the spherical feature in the image. / Palpation	Instrument axial force	Mono	RES: 0.08N RMSE = 0.13N RNG: 0-1.96N
5- Aviles et al. (2014)	The method used a 3D lattice to model the deformation of soft tissue. An RNN estimated the force by processing the information provided by the 3D lattice and the surgical tool motion. / Tissue push	Tissue normal force	Stereo	MAE = 0.05N RMSE = 0.062N RNG: 0-3N
6- Aviles et al. (2015a)	The RNN’s full feedback architecture in Aviles et al. (2014) was replaced by local and global feedback. The RMSE and computation time were improved. / Tissue push	Tissue normal force	Stereo	RMSE = 0.059N RNG: 0-3N
7- Aviles et al. (2015b)	The network in Aviles et al. (2015a) was upgraded to a recursive neural network LSTM based architecture which improves the force estimation accuracy. / Tissue push	Tissue normal force	Stereo	RMSE = 0.029N RNG: 0-3N
8- Otte et al. (2016)	The tissue deformations from Optical Coherence Tomography (OCT) and the instrument trajectories were used as inputs to a Generalized Regression Neural Network (GRNN) to estimate the instrument-tissue forces. / Tissue push	Force magnitude	OCT Scanner	RMSE = 3mN RNG: 0-20mN
9- Aviles et al. (2016)	Evaluated the effect of dimensionality reduction on the performance of the RNN+LSTM architecture proposed in Aviles et al. (2015b). It showed that implementation of a Probabilistic Principal Component Analysis (PPCA) significantly reduced dimension (75% reduction) and improved accuracy. / Tissue push	Tissue normal force	Stereo	ERR $<$ 2% RMSE = 0.02N RNG: 0-3N
10- Giannarou et al. (2016)	The tissue deformations were estimated by finding stereo-correspondences based on tissues salient features and the use of probabilistic soft tissue tracking and thin-plate splines (TPS). The displacements were used to estimate forces based on a biomechanical model. / Tissue push	Force magnitude	Stereo	MAE = 0.07N RNG: 0-0.8N
11- Aviles et al. (2017)	This was an extension to Aviles et al. (2016) where the proposed RNN+LSTM architecture was extended to three-axis force components. Z-axis was normal to the tissue, X and Y axes were planar with the tissue surface. / Tissue push	Tissue tri-axial forces	Stereo	RMSE: All DoF $<$ 0.02N RNG: F_x: $\pm$ 0.6N, F_y: $\pm$ 2N, F_Z: $\pm$ 6N
12- Hwang and Lim (2017)	Same as Aviles et al. (2015b) with a deeper network, fully connected layers, and a sequence of mono 2D images as inputs. The results were on a sponge, a PET bottle, and a human arm with changes of light and pose. / Push	Tissue (object) normal force	Mono	RMSE: Sponge:0.05N, PET bottle:0.17N, Arm:0.1N RNG: Sponge:0-3N, PET bottle:0-7N, Arm:0-2N
13- Haouchine et al. (2018)	A biomechanical map of the organ shape was built on-the-fly from stereoscopic images. It used 3D reconstruction and meshing techniques. / Tissue push and pull	Force magnitude	Stereo	Plot comparison, not quantified.

Table 5: Vision-based force estimation - continue

Author	Method / Task	Sensing DoFs	Stereo / Mono	Results
14- Gessert et al. (2018)	Took an undeformed reference volume and a deformed sample volume from OCT as inputs into a Siamese 3D-CNN architecture and output a 3D force vector. The results were compared with CNNs that take the difference or the sum of the undeformed and deformed volumes, and a CNN that takes 2D projected surface images as inputs. / Tissue push	Tissue tri-axial forces	OCT Scanner	In force magnitude: MAE: 7.7 mN RMSE: 4.3 mN RNG: 0-1 N
15- Marban et al. (2018)	A semi-supervised learning model consisting of an encoder+LSTM network was suggested. The encoder learned a compact representation of the RGB frames from video sequences. The LSTM network used the tool trajectory information and the output of the encoder for force estimation. / Tissue push	Instrument tri-axial forces and moments	Mono	RMSE: F_Z:0.89 N RNG: F_Z:0-8 N
16- Marban et al. (2019)	It used a CNN+LSTM architecture that processed the spatiotemporal information in video sequences and the temporal structure of tool data (the surgical tool-tip trajectory and its grasping status). / Tissue push and pull	Instrument tri-axial forces and moments	Mono	Plot comparison, not quantified. Concluded that both video sequences and tool data provide important cues for the force estimation.

Table 6: Strain-gauge force sensing.

Author	Method / Location	Sensing DoFs	Instrument / DI	Results
1- Jones et al. (2011)	Custom torque sensors were placed at the instrument interface between the driver and the driven knobs. / Instrument interface	Grip force, Pitch torque, Instrument axial torque	EndoWrist instruments / 6	No results presented
2- Van Den Dobbelsteen et al. (2012)	A tension/compression load cell was installed in line with the actuating rod of the grasper. / Actuating rod	Grip force	Karl-Storz laparoscopic grasper / 1	ERR $<$ 0.09N RNG: 0-2N
3- Hong and Jo (2012)	Custom grasper jaw with flexure hinges was designed to make a compliant structure. / Gripper	Gripper normal (F_N) and pull (F_P) forces	Standalone testing / 1	RES: F_P:43mN, F_N:7.4mN RMSE: F_P=95mN, F_N=37mN RNG: $\pm$ 5N
4- Baki et al. (2012)	Strain gauges were installed onto a custom-designed flexure out of Titanium fabricated by EDM. / Distal shaft	Instrument tri-axial forces	Standalone testing / 0	ERR $<$ 4% RES: 5mN RNG: $\pm$ 2N
5- He et al. (2014)	Custom designed sensors for measuring cable tension were installed at the instrument base. / Instrument base (cable tension)	Gripper normal and 3-DoF forces	MicroHand robot instrument / 6	ERR $<$ 0.4N RNG: F_x,F_y: $\pm$ 3.5, F_Z: $\pm$ 2N, F_G: 0-11N (CS at an external sensor)

Table 7: Strain-gauge force sensing - continue.

Author	Method / Location	Sensing DoFs	Instrument / DI	Results
6- Moradi Dalvand et al. (2014)	Strain gauges were installed on the lead-screw actuation mechanism and a sleeve. / Actuating rod & Distal end of a sleeve	Instrument lateral (F_L) and Grip force (F_g)	CD RMIS instrument for 5mm fenestrated inserts / 2	MAE: F_L $<$ 0.05N, Dir. $<$ 3° RMSE: F_L: $<$ 0.05N, Dir. $<$ 5.7° RNG: F_L: $\pm$ 1N, F_g: 0-5N
7- Wang et al. (2014)	An instrumented cover plate at the instrument interface and sensorized docking clamps at the trocar mount measured the z-axis and lateral forces, respectively. / Instrument interface & trocar mount	Instrument tri-axial forces	EndoWrist instruments / 6	RMSE $<$ 8% RNG: F_x,F_y: $\pm$ 8N, F_Z: $\pm$ 12N
8- Talasaz et al. (2014)	Strain gauges were installed on the actuating cables and the RMIS instrument was attached to the robot flange through a 6 axis ATI Gamma F/T sensor. / Actuating Cables and instrument interface	Instrument tri-axial forces, axial and pinch torques, grip force	EndoWrist Needle Driver / 6	ERR: F_X,F_Y,F_Z $<$ 0.12N
9- Yu et al. (2014)	Small-size six-dimensional force/torque sensor with the structure of double cross beams. / Articulated wrist	Wrist 6DoF forces & moments	CD RMIS instrument / 5	ERR $<$ 4.5% RNG: F_X,F_Y,F_Z: 10N, M_X,M_Y: 150Nmm, M_Z: 50Nmm
10- Trejos et al. (2014)	Strain gauges were installed on the rod that actuated the grasper and on the distal end of the instrument shaft. / Actuation rod and distal shaft	Instrument lateral and grasping forces	Manual Laparoscopic grasper / 1	ERR: F_X,F_Y,F_G $<$ 0.2N RNG: F_X,F_Y: $\pm$ 5N, F_G: 0-17N
11- Spiers et al. (2015)	Custom torque sensors were placed at the instrument interface between the driver and the driven knobs. / Instrument interface	Grip force, Pitch torque, Instrument axial torque,	EndoWrist Needle Driver / 6	RNG: $\pm$ 6N
12- Li et al. (2015b)	Strain gauges were installed on a custom-designed tripod flexure. / Distal shaft	Instrument tri-axial	Standalone testing / 0	ERR: F_X, F_Y $<$ 1%, F_Z $<$ 5% RNG: F_X, F_Y: $\pm$ 1.5N, F_Z: $\pm$ 3N
13- Li et al. (2015a)	Strain gauges were integrated into a custom-designed Flexural-hinged Stewart platform. / -	6 DoF forces and torques	Standalone Testing / 0	RES: F_X, F_Y: 0.08N, F_Z: 0.25N M_X,M_Y,M_Z: 2.4Nmm RNG: F_X, F_Y, F_Z: $\pm$ 30N, M_X,M_Y,M_Z: $\pm$ 300Nmm
14- Ranzani et al. (2015)	Two custom holders with integrated ATI F/T sensors were designed for the instrument and the fulcrum point. / Instrument interface and fulcrum point	Instrument tri-axial forces	MIS laparoscopic grasper / 1	ERR $<$ 2.7% RNG: $\pm$ 4N
15- Maeda et al. (2016)	An ATI Mini40 force sensor was mounted to the shaft of the instrument. / Proximal shaft	Instrument lateral and axial forces, axial torque	CD RMIS laparoscopic forceps /6	Sensor performance not quantified.
16- Khadem et al. (2016)	Integrated a tension/compression load cell inline with the lead-screw actuation and a 6-axis ATI Mini45 at the instrument base. / Actuating rod and instrument interface	Gripper pull force, grip force	CD RMIS laparoscopic grasper	ERR: F_G $<$ 0.5N RNG: F_G: 0-5N
17- Wee et al. (2016, 2017)	Presented a force-sensing sleeve with 4 strain gauges adaptable to standard MIS instruments. / Distal shaft	Instrument tri-axial forces, axial torque	MIS Laparoscopic Grasper / 1	RES: 0.2N RMSE: F_X, F_Y $<$ 0.088N RNG: F_X, F_Y: $\pm$ 5N
18- Barrie et al. (2016)	A tension/compression load cell was installed in line with the actuating rod of the grasper. / Actuating rod	Grip force	Johan fenestrated grasper / 1	Sensor performance not presented
19- Seneci et al. (2017)	Proposed a disposable sensor clip for the gripper. The gripper was fabricated by Selective Laser Melting (SLM) and the sensor clip was 3D printed. / Gripper	Gripper normal force	Standalone testing / -	ERR $<$ 0.2N RNG: $\pm$ 5N
20- Trejos et al. (2017)	Strain gauges were installed onto the proximal and the distal shafts. / Distal and proximal shafts	Instrument tri-axial forces	MIS laparoscopic grasper / 1	ERR: F_X, F_Y $<$ 0.2N, F_z $<$ 1.7N RNG: F_X, F_Y: $\pm$ 5N, F_Z: $\pm$ 12N

Table 8: Strain-gauge force sensing - continue.

Author	Method / Location	Sensing DoFs	Instrument / DI	Results
21- Li et al. (2017a)	Extension on Li et al. (2015a) in which the sensor was integrated into the surgical instrument. / Articulated wrist	Wrist 6DoF forces and moments	CD RMIS instrument / 6	RES: F_X, F_Y: 0.12N , F_Z: 0.5N, M_X,M_Y,M_Z: 7Nmm RNG: F_X, F_Y, F_Z: $\pm$ 10N, M_X, M_Y, M_Z: $\pm$ 160Nmm
22- Kim et al. (2017)	A 3 axis I-Beam force sensor using strain gauges were designed to replace the trocar support. / Trocar mount	Instrument lateral, trocar axial friction	EndoWrist instruments / 6	RMSE: F_X $<$ 0.39N, F_Y $<$ 0.20N, F_Z $<$ 0.35N RNG: F_X, F_Y: $\pm$ 15N, F_Z: $\pm$ 10N
23- Schwalb et al. (2017)	It is similar to the overcoat method by Shimachi et al. (2003). The instrument was mounted to an inner tube that was attached to a 6 axis F/T sensor. / Instrument interface	Instrument tri-axial forces	CD RMIS instrument / 6	RES: 0.09N RNG: $\pm$ 9N
24- Yu et al. (2018a)	Axial load cells measured the cables tensions and a NN was used for friction compensation. / Actuating cable	Gripper normal (F_N) and shear (F_S) forces	CD RMIS instrument / 5	ERR: F_N $<$ 10%, F_S $<$ 8% RNG: F_N: 0-2N, F_S: $\pm$ 2.5N
25- Kong et al. (2018)	Characterized the grip force over 50k grasps of one instrument using torque sensors at the instrument interface. Trained different NNs with an error threshold of 2 Nmm. The NN inputs were the proximal position, velocity, and torque measurements. / Instrument interface	Grip torque	EndoWrist Maryland grasper / 6	ERR $<$ 2Nmm
26- Karthikeyan and Nithya (2018)	A custom flexure was designed and populated with strain gauges. / Articulated wrist	Wrist tri-axial forces	CD RMIS instrument / 5	RNG: 0-1.5N
27- Novoseltseva (2018)	The axial force was measured by a thin plate between the proximal shaft and the sterile adapter. The lateral forces were measured by a flexure at the trocar. / Proximal shaft and trocar distal end	Instrument tri-axial forces	EndoWrist Needle Driver / 6	ERR: F_X $<$ 0.4N, F_Y $<$ 0.65N, F_Z $<$ 0.63N RES: F_X: 0.03N, F_Y: 0.02N, F_Z: 0.2N RNG: F_X, F_Y: $\pm$ 19N, F_Z: $\pm$ 12N
28- Peña et al. (2018)	Vapor-deposition fabrication techniques were used to directly print strain gauges on the instrument shaft. The material cost was $0.09 per transducer. / Distal shaft	Instrument lateral forces	EndoWrist Needle Driver & Fenestrated grasper / 6	ERR $<$ 0.8N RNG: $\pm$ 5N
29- Takizawa et al. (2018)	A disposable pneumatic cylinder with a strain gauge on its inner wall actuated the grasper. The transducer and the pneumatic pressure were used for force estimation. / Actuation system	Grip force	CD MIS laparoscopic grasper	RNG: 0.1-0.25N
30- Yu et al. (2018b)	A custom gripper with double E-type beams flexure and populated with strain gauges was designed. / Gripper	Gripper normal (F_N), shear (F_S), pull (F_P)	Standalone testing / 1	RMSE: F_N=23mN, F_S=2.2mN, F_P= 93mN RES: 0.01N RNG: $\pm$ 2.5N
31- Wang et al. (2019a)	Combined the cable-drive dynamics, the cable tension measurement, and a Particle Swarm Optimization Back Propagation Neural Network (PSO-BPNN) to develop a joint torque disturbance observer. / Cable tension	Grip force	CD RMIS grasper / 5	ERR $<$ 0.25N RNG: 0-2N
32- Xue et al. (2019)	Four micro force sensors were used for cables tension measurement. The cable tension and a model of the cable drive system (with coupling and friction effects) were used to estimate the grasping forces. / Cable tension	Grip force	EndoWrist needle driver / 6	ERR $<$ 0.4N RNG: 0-12N After stability is reached (Hysteresis effect)

Sensorless refers to the case where the sensors used for force estimation are already inherent in the surgical robot Stephens et al. (2019). In model-based approaches, the sensors are the encoders and the motor current measurements. In the vision-based techniques, the sensor is the visual feedback of the surgical site through mono or stereo cameras.

5.1.1 Model-Based

Model-based techniques can be categorized into 1) analytical models developed based on first principles, 2) disturbance observers and Kalman filters that utilize a dynamic model and the control loop commands and feedback signals, and 3) data-driven models which consider the instrument as a black-box and fit a quantitative model to a customized set of input and outputs. The model-based literature is summarized in Tables 2 and 3. They include 15 analytical models, eight of which are physics-based and 7 studies use observers or Kalman filters. There are 5 articles on the use of data-driven models.

The accurate dynamic model of the surgical instruments is challenging to obtain due to the many sources of nonlinearities e.g. friction, backlash Sang et al. (2017), tendons compliance Anooshahpour et al. (2014) and creep Haghighipanah et al. (2017), elastic deformations, actuators performance variations (the motors’ brush conductivity and change in the armature winding resistance) Li et al. (2016), hysteresis Anooshahpour et al. (2016), inertia, and gravity Wang et al. (2014). Additionally, any model relies on a set of measurements (calibration or training set) that are usually taken at the beginning and used throughout the estimation. It is experimentally shown that the tool behavior changes with time which deteriorates the estimation accuracy Kong et al. (2018); Hadi Hosseinabadi et al. (2019). The environmental parameters such as temperature and humidity can also affect the instrument characteristics Li and Hannaford (2017). An alternative approach is the implementation of online adaptation and identification methods that are highly nonlinear, complex, and computationally demanding. This limits their effectiveness in real-time applications Anooshahpour et al. (2014); Li et al. (2016). Dynamic modeling is particularly difficult in instruments with coupled degrees of freedom Zhao and Nelson (2015); Xin et al. (2017); O’Neill et al. (2018). Lee et al. Lee et al. (2015) showed that for the same input force by the surgeon, the grip force of the daVinci EndoWrist grasper can vary up to 3.4 times depending on its posture. As a result, despite the extensive research work, force estimations that rely on dynamic models do not provide highly reliable results yet, especially in the instrument’s lateral direction Fontanelli et al. (2017). In comparison, the data-driven techniques based on supervised learning Li and Hannaford (2017); Stephens et al. (2019) provide more accurate force estimations.

Table 9: Optical force sensing: LIM

Author	Method / Location	Sensing DoFs	Instrument / DI	Results
1- Puangmali et al. (2012)	Presented a 3-axis force sensor with a flexible tripod structure, a stationary reflecting surface, and a pair of transmitting and receiving fibers per axis. The light source and photodetectors are remote or at the instrument base. / Distal shaft	Instrument tri-axial forces	Standalone testing / 0	ERR: $<$ 5%FS RES: 0.02N RNG: F_X, F_Y: $\pm$ 1.5N, F_Z: $\pm$ 3N
2- Ehrampoosh et al. (2013)	Proposed an optical sensor design comprised of three Gradient-Index lenses (GRIN-lens) transmitting-receiving fiber-optic collimators, a flexible structure, and a reflective plate. / Distal shaft	Instrument tri-axial forces	Standalone testing / 0	RNG: $\pm$ 6N.
3- Fontanelli et al. (2017)	Used four optical proximity sensors to measure the deflection of the instrument shaft w.r.t the fixed trocar. The sensor was 3D-printed for proof of concept. / Trocar distal end	Instrument lateral forces	Adaptable to any EndoWrist Instrument / 6	ERR $<$ 12% RNG: $\pm$ 4N
4- Hadi Hosseinabadi et al. (2019)	Optical force sensor comprising of an IR LED, a bicell photodiode, and a slit installed on the proximal shaft of the instrument. The proposed concept provided sub-nanometer resolution in deflection measurement. / Proximal shaft	Instrument lateral forces	EndoWrist instrument / 6	RMSE= 0.03N RNG: $\pm$ 1N
5- Bandari et al. (2020)	A moving cylinder bends a fiber sitting on two fixed cylinders. Rate-dependent learning-based support-vector-regression was used for calibration. / Gripper	Grip force	CD MIS laparoscopic grasper	ERR $<$ 0.2N RES: 0.002N RNG: 0-2N

Table 10: Optical force sensing: FBG

Author	Method / Location	Sensing DoFs	Instrument / DI	Results
1- Haslinger et al. (2013)	Similar to the DLR’s miniature 6 axis force-torque sensor Seibold et al. (2005) with the strain gauges replaced by FBGs. The sensor structure was a Stewart platform to provide enhanced stiffness. / Articulated wrist	Wrist 6-DoF forces and moments	DLR MICA instruments / 6	ERR $<$ 18.6% RNG: F_X,F_Y,F_Z: $\pm$ 6.9N, M_X, M_X: $\pm$ 59.34Nmm, M_Z = $\pm$ 49.53Nmm
2- Lim et al. (2014)	Two optical FBGs were integrated into the forceps. Each fiber had two gratings for measuring the mechanical strain (on the surface) and for temperature compensation (at the center of the bending neutral axis). / Gripper	Gripper normal force (F_N)	CD MIS laparoscopic grasper / 1	RES: 1mN RNG: 0-5N
3- Song et al. (2014)	3-axis force sensor with 4 longitudinal bendable beams populated with FBGs. Four other FBGs were integrated as references for temperature compensation. / Articulated wrist	Wrist tri-axial forces	CD RMIS instrument / 6	ERR: F_X,F_Y $<$ 0.1N, F_Z $<$ 0.5N RNG: $\pm$ 10N
4- Yurkewich et al. (2014)	Integrated 3 FBGs on the distal shaft and another FBG into the moving jaw of the grasper. / Distal shaft and gripper	Instrument lateral force (F_L), grip force (F_G)	MIS arthroscopic grasper / 1	RMSE: F_L= 0.213N, Dir = 4.37°, F_G = 0.747N RNG: F_L: $\pm$ 10N, F_g: 0-20N
5- Shahzada et al. (2016)	Four FBG sensors were attached to the instrument distal shaft in a two cross-section layout which is insensitive to the error caused by combined force and torque loads. / Distal shaft	Instrument lateral forces	EndoWrist Needle Driver / 6	ERR $<$ $\pm$ 0.05N (95% confidence interval) RES: 0.05N RNG: $\pm$ 2N
6- Choi et al. (2017)	Custom flexure with three FBGs and an overload protection mechanism. The calibration algorithm was based on a two-layer NN. / Articulated wrist	Wrist tri-axial forces	Standalone testing / -	ERR $<$ 0.06N RNG: $\pm$ 12N
7- Suzuki et al. (2018)	Four FBGs were integrated into the articulated wrist. The differential wavelength shift was used to achieve robustness to temperature and gripping force. / Articulated wrist	Wrist bending forces and moments	CD RMIS instrument / 6	RNG: F_X,F_Y: $\pm$ 0.5N, T_X,T_Y: $\pm$ 50Nmm
8- Soltani-Zarrin et al. (2018)	Two grasper designs with sliding stretchable T-shaped parts for enhanced axial strain. Axial FBGs were at the grasper’s bending neutral axes and its surface. / Gripper	Gripper normal (F_N) and pull (F_P) forces	Standalone testing / -	ERR: 1: F_N $<$ 0.57N, F_P $<$ 0.78N 2: F_N $<$ 0.81N, F_P $<$ 0.9N RNG: F_N: 0-10N, F_P: 0-6N
9- Xue et al. (2018)	The cable tensions were measured by FBGs pasted in the grooves on inclined cantilevers integrated into the Instrument base. / Instrument base (Cable tension)	Grip force	CD MIS laparoscopic instrument with local actuation / 5	ERR $<$ 0.5N RES: 0.14N RNG: 0-15N
10- Shi et al. (2019)	A force sensing flexure combining a Stewart base and a cantilever beam. The FBG was integrated along the central line of the flexure with its two ends fixed in grooves. / Distal shaft	Instrument axial force	Standalone testing / -	No radial constraint: MAE: F_Z $<$ 0.26N, RES: 21mN, RNG: F_Z: 0-12N With radial constraint: MAE: F_Z $<$ 0.12N, RES: 9.3mN, RNG: F_Z: 0-7N
11- Lv et al. (2020)	The force sensor had a miniature flexure based on a Sarrus mechanism to achieve high axial sensitivity and a large measurement range. An FBG was tightly suspended along the central axis of the flexure. / Distal shaft	Instrument axial force	Standalone testing / -	ERR $<$ 0.06N RES: 2.55mN RNG: 0–5N

Table 11: Capacitive force sensing

Author	Method / Location	Sensing DoFs	Instrument / DI	Results
1- Lee et al. (2014)	Proposed a tendon drive pulley at the instrument base with an integrated torque sensor. A 3-axis force sensor was placed into the instrument shaft. Both sensors were based on the changes in the distance between the electrode and the ground. / Distal shaft and instrument base	Instrument tri-axial forces, grip force	Custom Prototype of an MIS grasper / 1	RNG: 0-0.5N
2- Kim et al. (2015)	Two sensors consisting of a triangular prism shape and two capacitive-type transducers with an elastomeric polymer dielectric were integrated into the grasper. Molding was used to fabricate a prototype. / Gripper (tip)	Gripper normal (F_N), shear (F_S), pull (F_P), grip (F_G) forces	CD RMIS instrument for RAVEN-II / 6	RES: F_P=42mN, F_S=72mN, F_N=58mN, F_G=46mN RMSE: F_P $<$ 84mN, F_S $<$ 0.114N, F_N $<$ 73mN, F_G $<$ 95mN RNG: F_P: $\pm$ 2.5N, F_S $\pm$ 2.5N, F_N $\pm$ 5N, F_G: 0-5N
3- Kim et al. (2016)	Two sensors with 3 electrodes and common grounds were integrated into the Gripper. The dielectric was air and the signal processing electronics were local. / Gripper (base)	Gripper normal (F_N), shear (F_S), pull (F_P), grip (F_G) forces	Custom prototype of an MIS Grasper / 1	ERR: F_P $<$ 0.42N, F_S $<$ 0.15N, F_N $<$ 0.92N RNG: 0-8N
4- Lee et al. (2016)	An extension on Lee et al. (2014) with the 3-axis force sensor moved into the articulated wrist and two capacitive torque sensors in the tendon drive pulleys of the gripper jaws. / Articulated wrist and instrument base	Wrist tri-axial forces, grip Force	CD RMIS instrument for RAVEN-II / 6	NRMSE: F_X=0.039 , F_Y=0.056 , F_Z=0.026 RNG: F_X: $\pm$ 1N, F_Y: $\pm$ 1N, F_Z: $\pm$ 1.6N
5- Kim et al. (2018a)	Proposed a capacitance sensing PCB with 8 electrodes and a CDC chip and a conductive deformable structure as the common ground. A spherical cap was added to the sensor for testing it in a palpation task. / Articulated Wrist	6-DoF forces and moments	CD RMIS instrument for S-surge robot / 6	MAE: F_X, F_Y, F_Z: 5.5%FSO, M_X, M_Y, M_Z: 2.7%FSO RES: F_X: 0.22mN, F_Y: 0.31mN, F_Z: 0.11mN, M_X: 0.47mNmm, M_Y: 0.41mNmm, M_Z: 0.17mNmm RNG: F_X, F_Y, F_Z: 1N, M_X, M_Y: $\pm$ 20Nmm, M_Z: $\pm$ 10Nmm
6- Kim et al. (2018b)	Extension of Kim et al. (2016) with a slight modification in the proximal gripper jaw and calibration scheme so that the combination of the capacitive transducers also resolved the axial torque about the gripper. / Gripper (base)	Gripper normal (F_N), shear (F_S), pull (F_P), grip (F_G) forces, axial torque (T_P)	CD RMIS instrument for S-surge robot / 6	Integrated sensor: RES: F_N=1.8mN, F_S=2.0mN, F_P=3.8mN MAE: F_N:6.4%, F_S:3.4%, F_P:8.6%, T_P: 5.7% RNG: F_P, F_N, F_S: $\pm$ 5N, T_P: $\pm$ 3Nmm
7- Seok et al. (2019)	Extension of Kim et al. (2018b) with humidity compensation. An AC shield was added to minimize the temperature effects on parasitic capacitance. An anodizing process was applied for electric insulation. The sensors range was extended to 20 N. / Gripper (base)	Gripper normal (F_N), shear (F_S), pull (F_P), grip (F_G) forces, axial torque (T_P)	CD RMIS instrument for S-surge robot / 6	RNG: $\pm$ 20 N

Table 12: MEMS force sensing

Author	Method / Location	Sensing DoFs	Instrument / DI	Results
1- Lee et al. (2013)	A thin-film capacitive sensor was fabricated using MEMS silk-screening technique on a PET film. / Gripper	Gripper normal (F_N), shear (F_S), pull (F_P) forces	Standalone testing / -	SENS: F_N= 6.1% , F_S=10.3%, F_P= 10.1% RNG: 0-12N
2- Gafford et al. (2013)	The Pop-Up Book MEMS method was used to fabricate a grasper with a custom thin-foil strain gauge in a single manufacturing step. / Gripper	Gripper normal force	Standalone testing / -	RES: 30mN RNG: 0-1.5N
3- Kuwana et al. (2013)	A piezo-resistive sensor chip was manufactured by burying a substrate of several bent beams in different directions in resin. / Gripper	Gripper normal (F_N), shear (F_S), pull (F_P), and grip (F_G) forces	MIS laparosc-opic grasper (Covidien; ENDOLUNG) / 1	No results presented
4- Gafford et al. (2014)	Used Printed-Circuit MEMS (PCMEMS) technique to develop a monolithic, fully-integrated tri-axial sensor with printed strain gauges. / -	Instrument tri-axial forces	Standalone testing / -	RES $<$ 2mN RNG: F_X, F_Y: $\pm$ 500mN, F_Z: $\pm$ 2.5N
5- Nakai et al. (2017)	A 6-axis force-torque sensor chip composed of 16 piezo-resistive beams was fabricated by using ion beam etching and surface doping. The sensor is 2x2 mm and installed onto the grasper. / Gripper	Gripper 6DoF forces and moments	MIS laparoscopic grasper / 1	RNG: F_N: 0-40N, F_S: $\pm$ 12.5N, F_P: $\pm$ 12.5N M_N: $\pm$ 15Nmm, M_S: $\pm$ 100Nmm, M_P: $\pm$ 100Nmm
6- Dai et al. (2017)	Proposed a 3-axis capacitive force sensor with differential electrodes. The compressive load reduced the dielectric thickness, and shear forces changed the overlap area. The sensor was fabricated using MEMS lithography. / Gripper	Gripper normal (F_N), shear (F_S), pull (F_P) forces	EndoWrist ProGrasp / 6	RES: F_N=55mN, F_S=1.45N, F_P=0.25N RNG: F_N: 0-7N, F_S: $\pm$ 11N, F_P: $\pm$ 2N
7- Radó et al. (2018)	Used deep reactive ion etching (DRIE) to fabricate a monolithic silicon-based 3-axis force piezoresistive sensor. The sensor was covered with a semi-sphere PDMS polymer. / Gripper	Gripper normal (F_N), shear (F_S), and pull (F_P) forces, palpation	MIS laparoscopic grasper for Robin Heart robot / 1	ERR $<$ 10% RNG: 0-4N
8- Tahir et al. (2018)	Presented a piezoelectric sensor fabricated using reduced Graphene oxide (rGO)-filled PDMS elastomer composite to measure the dynamic force. / Gripper	Gripper normal	MIS laparoscopic grasper / 1	RNG: 0.5-20N

Table 13: Other force sensing technologies

Author	Method / Technology / Location	Sensing DoFs	Instrument / DI	Results
1- Vakili et al. (2011)	A Tekscan FlexiForce piezoresistive pressure sensor was integrated into one of the grasper jaws. / Piezoresistive / Gripper	Gripper normal force	CD MIS laparoscopic grasper / 1	RNG: 0-4.4N
2- Mack et al. (2012)	QTC Pills were integrated into a custom-designed support structure. / QTC / Instrument base	Instrument tri-axial forces, grip force, axial torque	CD RMIS instrument / 6	No results presented
3- McKinley et al. (2015)	Palpation probe that could be added onto the instruments. It measured the axial compression of the sliding tip using a Hall Effect sensor. / Magnetic–Hall Effect Sensor / Distal shaft	Instrument axial force	EndoWrist instruments / 6	RES: 4mN RNG: 0-1.6N
4- Srivastava et al. (2016)	Superelastic Nitinol wires were used, instead of strain gauges, in two cross-sections arrangements for strain measurement. / SMA / Distal shaft	Instrument lateral forces	EndoWrist Needle Driver / 6	RES: 55mN RMSE $<$ 32 mN RNG: $\pm$ 4N
5- Li et al. (2017b)	Proposed a compact 3-axis force sensor design with integrated signal conditioning, power regulation, and ADC. The sensor used an array of force-sensitive resistors (FSR) with a mechanically pre-loaded structure. / FSR / Distal shaft	Instrument tri-axial forces	Standalone testing / -	RES: 0.1N RNG: $\pm$ 8N
6- Jones et al. (2017)	A 3D-printed grasper face with an embedded neodymium permanent magnet was attached to a soft silicone base that was mounted on top of a 3-axis hall effect sensor. Genetic Programming algorithm was used for sensor calibration. / Magnetic–Hall Effect Sensor / Gripper	Gripper normal (F_N), shear (F_S), and pull (F_P) forces	Standalone testing / -	Hysteresis ERR: F_N $<$ 1.58N, F_S, F_P $<$ 0.31N RNG: F_N: 0-35N, F_S, F_P: $\pm$ 7N
7- Bandari et al. (2017)	Proposed a hybrid sensor that used a piezoresistive transducer to measure normal force and LIM in optical fibers to estimate the tissue deformation. The sensor was out of silicon for biocompatibility. / Piezoresistive+LIM / Gripper	Gripper normal force	Standalone testing / 1	RNG: 0-2.5N
8- Gaudeni et al. (2018)	Proposed the placement of a pneumatic balloon in a cavity on the surgical instrument or endoscopic camera. When needed, the membrane is inflated to contact the tissue. The pneumatic pressure and volume are monitored to estimate the force. / Pneumatic / Distal shaft	Palpation force	Standalone testing / -	ERR $<$ 0.24 N RMSE: 0.11 N RNG: 0-1.7 N
9- Abdi et al. (2020)	Tekscan FlexiForce and A101 piezoresistive sensors were installed onto the forceps via a custom 3D-printed mounting component. / Piezoresistive / Gripper	Gripper normal (F_N), shear (F_S), pull (F_P) forces	EndoWrist Cadiere forceps / 6	RNG: F_N: 0-15N, F_S: $\pm$ 44N, F_P: 0-44N
10- Kuang et al. (2020)	A slender shaft was excited by using a vibration motor. The structure’s tri-axial acceleration signals in time-domain showed discernible ellipse-shaped profiles when a force was applied. The acceleration profiles were characterized via regression to estimate the direction and magnitude of the applied force. / Vibration monitor / Proximal & distal shaft	Instrument tri-axial forces	Standalone testing / -	MAE: F_L = 18%, F_Z = 6% RES: F_L = 0.098N, Dir= 10°. RNG: F_L: 0-0.98N, F_Z: 0-0.95N

5.1.2 Vision-Based

The existing literature affirms that experienced surgeons use visual cues (tissue and instrument deformations and the stretch in the suture) as sensory feedback surrogates Martell et al. (2011); Noohi et al. (2014). With the 3D stereoscopic view in robotic surgery providing depth information, and the developments in the available computational power (high-performance Graphic Processing Units-GPUs, cluster computers, and cloud platforms), a noticeable shift towards adoption of vision-based techniques was observed. While mechanical models of the tissue are presented, they are mostly complex and computationally expensive Aviles et al. (2016). Most of the literature (Tables 4 and 5) implement supervised learning architectures (Recurrent Neural Network (RNN) and Long-Short-Term-Memory (LSTM) Aviles et al. (2016); Marban et al. (2019)) with the video stream as inputs to estimate the instrument-tissue interaction forces. The vision-based techniques are robust to many sources of inaccuracy listed in Figure 5. However, they can be affected by the instrument occlusion, smoke and changes in the tissue properties, lighting conditions and camera orientation. The estimation update rate cannot be faster than the video frame rate which is usually 30 Hz. This limitation makes the vision-based approached not suitable for FF applications in which the control loop is desired to execute faster than 500 Hz Jones et al. (2017). The current literature highlights that force estimation through video processing is easier in pushing tasks (characterized by smooth deformations) than those produced by pulling tasks that are characterized by irregular tissue deformations due to grasping Marban et al. (2019).

Force estimation based on using Optical Coherence Tomography (OCT) as the reference sensor is proposed by Otte et al. (2016) and Gessert et al. (2018). OCT images provide volumetric data with a resolution of a few micrometers in which the tissue compression and subsurface deformations can be reflected. Thus, they contain a richer signal space compared to the mono and stereo visions that provide only the surface information.

5.2 Strain Gauge

Strain gauges are the most commonly used transducers for force sensing Yu et al. (2018a). They are accurate and small and can be designed in different configurations for multi-axis force sensing. Although the transducers are low cost with a price of $10-$25 per unit Srivastava et al. (2016), they require special surface preparation, adhesives, and coatings for optimal performance that increases the assembly and integration cost Li et al. (2017b). When used in Wheatstone bridge arrangements, they require multiple wires for connection that makes packaging difficult for quick and seamless integration with surgical instruments Shi et al. (2019); Lv et al. (2020). Strain gauges are highly influenced by electromagnetic noise and are not suitable for use close to other tools with strong magnetic fields (e.g. electrocautery) Choi et al. (2017); Xue et al. (2018). They have low sensitivity and often require custom flexures or modifications in the load-carrying structure to amplify local strains Hadi Hosseinabadi et al. (2019). Strain gauges are fragile and require mechanical overload protection Kuang et al. (2020). They typically do not survive multiple sterilization cycles Shi et al. (2019) and lose repeatability. Trejos et al. Trejos et al. (2014, 2017) conducted an extensive study on biocompatible adhesives and coatings that can withstand the harsh environment during steam sterilization. However, none of the combinations showed reliable measurements after seven cycles. Tables 6 to 8 summarizes the articles which utilize strain gauges or commercial strain-gauge based force sensors for MIS force sensing.

5.3 Optical

Optical methods use light intensity (e.g. photodiodes, phototransistors), frequency (e.g. Fiber Brag Gratings), or phase (e.g. interferometry) modulation for force measurement. The optical signal can be locally converted to electric signals, or be transferred with fibers for distal processing. Placing the electronics away from the instrument tip makes sterilization easier. The optical fibers are flexible, scalable, biocompatible, electrically passive, insensitive to electromagnetic noise and thus MRI compatible Peña et al. (2018), durable against high radiation Lim et al. (2014), immune to water Song et al. (2014), corrosion-resistant Shi et al. (2019), and low cost Bandari et al. (2020). However, optical fibers cannot be routed into small bending radii Trejos et al. (2014). Additionally, The presence of small and intricate parts can make fabrication and assembly of fiber-based sensors costly Li et al. (2017b).

The Light Intensity Modulation (LIM) based sensors are vulnerable to light intensity variations due to the temperature or fiber bending Lv et al. (2020). This can be improved by normalizing the optical signal against the emitted power Hadi Hosseinabadi et al. (2019). Alternatively, a redundant strain-free fiber can be used to compensate for the effect of temperature or other sources of uncertainty Puangmali et al. (2012); Song et al. (2014). Table 9 summarizes the articles that address MIS force estimation based on LIM.

The FBG sensors are wave-length coded and insensitive to the changes in the light intensity. FBGs are very sensitive, have calibration consistency, and exhibit high SNR which provide repeatable and high-resolution strain measurements Shahzada et al. (2016). Multiple gratings can be accommodated into one fiber Soltani-Zarrin et al. (2018) simplifying the design and signals routing. Thus, they are also used in shape sensing Lv et al. (2020). Nonetheless, FBGs require interrogators for signal processing which the commercial systems cost between $10k to $100k Yurkewich et al. (2014). The articles which used FBGs for MIS force estimation are summarized in Table 10.

5.4 Capacitive

Capacitive methods are attractive solutions for high resolution and compact force sensor designs. Compared to strain gauges, they provide limited hysteresis in microscale and increased sensitivity Sang et al. (2017). However, they have a limited range Li et al. (2017b) and are prone to thermal and humidity drift Bandari et al. (2020). The change in capacitance can be due to the change of the overlapping area or the distance between the two electrodes; the latter provides higher sensitivity and a more linear response Kim et al. (2017). The commercially available Capacitance to Digital Converter (CDC) chips such as the AD7147 from Analog Devices significantly simplify the signal processing, which was believed to be challenging for capacitive transducers Trejos et al. (2014). However, they provide a low sampling rate. Table 11 lists the articles that are based on the capacitive transduction principle. The sterilizability and biocompatibility of the existing literature are not evaluated.

5.5 MEMS

MEM sensors (Table 12) operate based on the same physical principles discussed so far. However, MEM fabrication techniques such as deposition, etching, and lithography allow for the cost-effective production of small, fully-integrated, monolithic sensors Radó et al. (2018) with reduced lead time in prototypes and high throughput batch volumes Gafford et al. (2014). Typically, MEMS do not require manual assembly, bonding, and alignment, and provide functional devices after the fabrication process Gafford et al. (2013). By utilizing MEMS technology, it is possible to develop smart parts (e.g. grippers) with integrated sensing capability for micromanipulation Pandya et al. (2014). Biocompatible coatings can be added to MEM sensors for biomedical applications.

5.6 Other Technologies

Piezoelectric transducers do not require an external power supply and have high stiffness Li et al. (2017b). However, they are subject to charge leakage and are not suitable for low frequency and static loads Sang et al. (2017). They are also sensitive to temperature. Piezoresistive transducers used in force-sensitive resistors are scalable with low hysteresis and noise Bandari et al. (2017). Nonetheless, their linear response is limited to a small range and they drift under constant load Juo et al. (2020). They do not have the challenges associated with the integration of strain gauges, are relatively insensitive to humidity, and can be used in high temperatures above 170°C Li et al. (2017b). Shape Memory Alloys (SMAs) like Nitinol have a higher gauge factor compared to common metallic strain gauges and provide a larger range due to their stretchability. SMAs require an insulating coating for use on conductive surfaces. They can be clamped at two ends and do not require a backing material with special surface preparation. They are low cost and available at diameters as small as few microns Srivastava et al. (2016). Quantum tunneling composite (QTC) pills are flexible polymers that act as insulators in resting-state but increase conductivity when compressed. They are very sensitive, provide a wide dynamic range, and are low cost (less than 1$/pill). However, they are temperature sensitive and inaccurate in dynamic loading applications Novoseltseva (2018). Recently, vibration frequency and phase shifting due to an applied force have been measured for force estimation by the use of accelerometers. This approach is slow as it needs a few vibration cycles to generate stable and repeatable signals Kuang et al. (2020).

6 Discussion and Conclusion

In keyhole endoscopy, the surgeon’s interaction with the surgical site is via slender instruments that are inserted into the body through small incisions. Despite the many benefits to the patient, the operation is more challenging for the surgeon due to the instruments’ limited dexterity, fulcrum motion reversal, uncomfortable posture, and limited visual presentation. Additionally, the surgeon’s force perception is affected by the forces between the instrument and the skin and the instrument’s dynamics. The adoption of robotic and computer vision technologies resolves the limitations above and significantly improves the accuracy and efficiency in RMIS. However, most telesurgical systems completely isolate the surgeon from the tissue through the local/remote architecture of robotic telemanipulation. This deprives the surgeon of the rich information in palpation and direct interaction with the tissue. Without force feedback, the interaction of the surgeons with the environment is not as intuitive as direct manipulation and therefore extensive training is required. Moreover, the lack of haptic feedback leads to a higher risk of errors and longer task completion time, up to 2 orders of magnitude in complex tasks Hannaford et al. (1991), which may lead to higher surgery costs.

One active research stream in the field of robotic surgery is improving the sense of telepresence for the surgeons, also known as “transparency”. Direct force feedback is the most intuitive approach to improve transparency. For a fully transparent haptic experience, reliable interaction force sensing at the surgeon console and the instrument-tissue interface is required. This is in addition to a safe bilateral teleoperation architecture, and a local manipulator that is capable of reflecting the force commands, known as a haptic display. The extensive literature on haptic control indicates a trade-off between transparency and stability Hashtrudi-Zaad and Salcudean (2001). Alternatively, sensory substitution was proposed instead of haptic feedback, in the form of visual, auditory, or vibrotactile cues of force information. Although the safety can be easily guaranteed in systems with SS, it is not intuitive and can cause cognitive overload for the surgeon. The SS methods can also be used in MIS systems because no robotic manipulator is required for force reflection. The efficacy of different haptic feedback modalities in improving the surgical training and surgeon performance metrics has been studied extensively Amirabdollahian et al. (2018); Rangarajan et al. (2020); Overtoom et al. (2019); Abdi et al. (2020). It is shown that a transparent haptic experience and visual feedback of force information improve the performance metrics and shorten the training time for novice surgeons in complex tasks. Apart from haptic feedback, the instrument-tissue interaction forces can be used for tissue damage monitoring, surgical skills assessment, development of surgical training guidelines, and to automate tasks.

Extensive research has been conducted to estimate or sense the instrument-tissue interaction forces. The functional requirements depend on the application. While it is not necessary to estimate the tissue forces precisely to provide an appropriate haptic experience Jones et al. (2011), the bandwidth and sampling rate are important requirements to ensure low latency and smooth interaction with the remote environment. The sampling rate and bandwidth are less critical in SS.

Sensorless approaches utilize the information that is already available in the robotic manipulator; the axes positions and velocities, motors currents, and visual display of the surgical theatre. With the exponential growth, over the past decade, in the available computational power to researchers, data-driven approaches based on supervised learning Li and Hannaford (2017); Aviles et al. (2017) have been widely adopted. Among them, neural networks have shown promising results when trained and used on one particular instrument. However, they require a long and computationally-expensive training phase that is yet clinically-prohibitive. The training is based on a set of measurements at the beginning of the surgery that is used afterward for force estimation throughout the entire surgery. Proposed approaches that have an instrument’s operational parameters as inputs, do not consider the variations between instruments and the change of instruments behavior throughout its use Kong et al. (2018). Considering how the research direction has evolved over the past decade, experimentation with different model architectures, development of efficient training, and identification methods that can be automatically performed at the system start-up Spiers et al. (2015), improving the computation time and incorporation of online adaptation techniques are attractive research areas to be further investigated. Moreover, all the existing literature uses the information at the patient manipulator for force estimation, but the inclusion of the operating parameters at the surgeon console may also improve the quality of force estimation.

The sensor design is another avenue towards collecting force data at the instrument tip. The sensor can be located inside or outside the body. The closer the sensor is to the instrument tip the more accurate the measurements are. However, the size, biocompatibility, sterilizability, insulation, and sealing requirements are more stringent when such an approach is followed. Design proposals for sensor integration into the instrument tip have limited adaptability because the instruments for different types of surgery have different shapes at the tip (e.g. EndoWrist cautery forceps, graspers, dissectors, needle drivers, etc.). Therefore, a custom sensor needs to be designed for every instrument which increases the development, fabrication, and maintenance costs.

A variety of transduction principles, including resistive, capacitive, optical, piezoelectric, and magnetic have been used in the development of sensing solutions for minimally invasive procedures. While strain gauges are still the most commonly used transducers, the study by Trejos et al. Trejos et al. (2017) showed that biocompatible adhesives and coatings can only survive a maximum of 6 steam sterilization cycles. Considering that the instruments are typically used 10 times before disposed, this would lead to a 40% increase in the cost of the instruments with integrated strain gauges. Additionally, the installation of strain gauges is labor-intensive that contributes to increased cost.

A comparison of the publications summarized in this article with the surveys by Puangmali et al. (2008) and Trejos et al. (2010) indicates a noticeable shift towards utilizing FBG and MEMS technologies for the development of gripper integrated miniature sensors (Figure 5). FBGs are compact, sterilizable, biocompatible, electrically passive, and immune to electromagnetic noise. They provide high sensitivity with sub-micron resolution and can have multiple gratings embedded in one fiber which simplifies fiver and therefore optical signal management. While the commercial interrogators are expensive, there are signal conditioning solutions proposed to decrease the electronics cost Yurkewich et al. (2014). The developments in MEMS technology have overcome the barrier of scale and cost in the fabrication of delicate miniature sensors. Additionally, MEMS sensors typically do not need manual assembly and can be integrated into the desired application after production.

Another observable trend is the utilization of data-driven regression approaches for sensor calibration. Models based on neural networks and other supervised learning methods such as Gaussian Process Regression have shown unprecedented performance in handling nonlinearities and uncertainties in sensor calibration. Compared to the surgical instruments, the transducers show a more consistent response and do not need regular calibration unless removed and reintegrated. Efficient calibration approaches that can be quickly and automatically performed without operator intervention (e.g based on payload estimation) would benefit the RMIS systems.

{acks}

Amir Hossein Hadi Hosseinabadi would like to NSERC for his Canada Graduate Scholarship-Doctoral scholarship. Professor Salcudean gratefully acknowledges infrastructure support from CFI and funding support from NSERC and the Charles Laszlo Chair in Biomedical Engineering.

References

Abdi et al. (2020) Abdi E, Kulic D and Croft E (2020) Haptics in Teleoperated Medical Interventions: Force Measurement, Haptic Interfaces and Their Influence on Users Performance. IEEE Transactions on Biomedical Engineering 10.1109/tbme.2020.2987603.
Abeywardena et al. (2019) Abeywardena S, Yuan Q, Tzemanaki A, Psomopoulou E, Droukas L, Melhuish C and Dogramadzi S (2019) Estimation of Tool-Tissue Forces in Robot-Assisted Minimally Invasive Surgery Using Neural Networks. Frontiers in Robotics and AI 6: 56. 10.3389/frobt.2019.00056.
Abiri et al. (2019) Abiri A, Pensa J, Tao A, Ma J, Juo YY, Askari SJ, Bisley J, Rosen J, Dutson EP and Grundfest WS (2019) Multi-Modal Haptic Feedback for Grip Force Reduction in Robotic Surgery. Scientific Reports 9(1): 5016:1–10. 10.1038/s41598-019-40821-1.
Abiri et al. (2017) Abiri A, Tao A, LaRocca M, Guan X, Askari SJ, Bisley JW, Dutson EP and Grundfest WS (2017) Visual–Perceptual Mismatch in Robotic Surgery. Surgical Endoscopy 31(8): 3271–3278. 10.1007/s00464-016-5358-z.
Amirabdollahian et al. (2018) Amirabdollahian F, Livatino S, Vahedi B, Gudipati R, Sheen P, Gawrie-Mohan S and Vasdev N (2018) Prevalence of Haptic Feedback in Robot-Mediated Surgery: A Systematic Review of Literature. Journal of Robotic Surgery 12(1): 11–25. 10.1007/s11701-017-0763-4.
Anooshahpour et al. (2014) Anooshahpour F, Polushin IG and Patel RV (2014) Quasi-Static Modeling of the da Vinci Instrument. In: 2014 IEEE International Conference on Intelligent Robots and Systems. pp. 1308–1313. 10.1109/IROS.2014.6942726.
Anooshahpour et al. (2016) Anooshahpour F, Polushin IG and Patel RV (2016) Classical Preisach Model of Hysteretic Behavior in a da Vinci Instrument. In: 2016 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM. pp. 1392–1397. 10.1109/AIM.2016.7576965.
Aviles et al. (2015a) Aviles AI, Alsaleh S, Sobrevilla P and Casals A (2015a) Sensorless Force Estimation Using a Neuro-Vision-Based Approach for Robotic-Assisted Surgery. In: 2015 International IEEE/EMBS Conference on Neural Engineering, NER. pp. 86–89. 10.1109/NER.2015.7146566.
Aviles et al. (2016) Aviles AI, Alsaleh S, Sobrevilla P and Casals A (2016) Exploring the Effects of Dimensionality Reduction in Deep Networks for Force Estimation in Robotic-Assisted Surgery. In: 2016 Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling, volume 9786. pp. 585–592. 10.1117/12.2217000.
Aviles et al. (2017) Aviles AI, Alsaleh SM, Hahn JK and Casals A (2017) Towards Retrieving Force Feedback in Robotic-Assisted Surgery: A Supervised Neuro-Recurrent-Vision Approach. IEEE Transactions on Haptics 10(3): 431–443. 10.1109/TOH.2016.2640289.
Aviles et al. (2015b) Aviles AI, Alsaleh SM, Sobrevilla P and Casals A (2015b) Force-Feedback Sensory Substitution Using Supervised Recurrent Learning for Robotic-Assisted Surgery. In: 2015 Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. pp. 1–4. 10.1109/EMBC.2015.7318246.
Aviles et al. (2014) Aviles AI, Marban A, Sobrevilla P, Fernandez J and Casals A (2014) A Recurrent Neural Network Approach for 3D Vision-Based Force Estimation. In: 2014 4th International Conference on Image Processing Theory, Tools and Applications, IPTA 2014. pp. 1–6. 10.1109/IPTA.2014.7001941.
Baki et al. (2012) Baki P, Szekely G and Kosa G (2012) Miniature Tri-Axial Force Sensor for Feedback in Minimally Invasive Surgery. In: 2012 Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics. pp. 805–810. 10.1109/BioRob.2012.6290770.
Bandari et al. (2020) Bandari N, Dargahi J and Packirisamy M (2020) Miniaturized Optical Force Sensor for Minimally Invasive Surgery with Learning-Based Nonlinear Calibration. IEEE Sensors Journal 20(7): 3579–3592. 10.1109/JSEN.2019.2959269.
Bandari et al. (2020) Bandari N, Dargahi J and Packirisamy M (2020) Tactile Sensors for Minimally Invasive Surgery: A Review of the State-of-the-Art, Applications, and Perspectives. IEEE Access 8: 7682–7708. 10.1109/ACCESS.2019.2962636.
Bandari et al. (2017) Bandari NM, Ahmadi R, Hooshiar A, Dargahi J and Packirisamy M (2017) Hybrid Piezoresistive-Optical Tactile Sensor for Simultaneous Measurement of Tissue Stiffness and Detection of Tissue Discontinuity in Robot-Assisted Minimally Invasive Surgery. Journal of Biomedical Optics 22(7): 077002:1–10. 10.1117/1.jbo.22.7.077002.
Barrie et al. (2016) Barrie J, Jayne DG, Neville A, Hunter L, Hood AJ and Culmer PR (2016) Real-Time Measurement of the Tool-Tissue Interaction in Minimally Invasive Abdominal Surgery: The First Step to Developing the Next Generation of Smart Laparoscopic Instruments. Surgical innovation 23(5): 463–468. 10.1177/1553350616646475.
Choi et al. (2017) Choi H, Lim Y and Kim J (2017) Three-axis Force Sensor With Fiber Bragg Grating. In: 2017 Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. pp. 3940–3943. 10.1109/EMBC.2017.8037718.
Dai et al. (2017) Dai Y, Abiri A, Liu S, Paydar O, Sohn H, Dutson EP, Grundfest WS and Candler RN (2017) Grasper Integrated Tri-axial Force sensor System for Robotic Minimally Invasive Surgery. In: 2017 Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. pp. 3936–3939. 10.1109/EMBC.2017.8037717.
Ehrampoosh et al. (2013) Ehrampoosh S, Dave M, Kia MA, Rablau C and Zadeh MH (2013) Providing Haptic Feedback in Robot-Assisted Minimally Invasive Surgery: A Direct Optical Force-Sensing Solution for Haptic Rendering of Deformable Bodies. Computer Aided Surgery 18(5-6): 129–141. 10.3109/10929088.2013.839744.
El Rassi and El Rassi (2020) El Rassi I and El Rassi JM (2020) A review of Haptic Feedback in Tele-Operated Robotic Surgery. Journal of medical engineering & technology : 1–810.1080/03091902.2020.1772391.
Faragasso et al. (2014) Faragasso A, Bimbo J, Noh Y, Jiang A, Sareh S, Liu H, Nanayakkara T, Wurdemann HA and Althoefer K (2014) Novel Uniaxial Force Sensor Based on Visual Information for Minimally Invasive Surgery. In: 2014 Proceedings - IEEE International Conference on Robotics and Automation. pp. 1405–1410. 10.1109/ICRA.2014.6907036.
Fontanelli et al. (2017) Fontanelli GA, Buonocore L, Ficuciello F, Villani L and Siciliano B (2017) A novel force sensing integrated into the trocar for minimally invasive robotic surgery. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 131–136.
Gafford et al. (2014) Gafford JB, Kesner SB, Degirmenci A, Wood RJ, Howe RD and Walsh CJ (2014) A monolithic Approach to Fabricating Low-Cost, Millimeter-Scale Multi-Axis Force Sensors for Minimally-Invasive Surgery. In: 2014 IEEE International Conference on Robotics and Automation. pp. 1419–1425. 10.1109/ICRA.2014.6907038.
Gafford et al. (2013) Gafford JB, Kesner SB, Wood RJ and Walsh CJ (2013) Force-Sensing Surgical Grasper Enabled by Pop-up Book MEMS. In: 2013 IEEE International Conference on Intelligent Robots and Systems. pp. 2552–2558. 10.1109/IROS.2013.6696716.
Gaudeni et al. (2018) Gaudeni C, Meli L and Prattichizzo D (2018) A Novel Pneumatic Force Sensor for Robot-Assisted Surgery. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 10894 LNCS. pp. 587–599. 10.1007/978-3-319-93399-3_50.
Gessert et al. (2018) Gessert N, Beringhoff J, Otte C and Schlaefer A (2018) Force estimation from OCT volumes using 3D CNNs. International Journal of Computer Assisted Radiology and Surgery 13(7): 1073–1082. 10.1007/s11548-018-1777-8.
Giannarou et al. (2016) Giannarou S, Ye M, Gras G, Leibrandt K, Marcus HJ and Yang GZ (2016) Vision-based Deformation Recovery for Intraoperative Force Estimation of Tool–Tissue Interaction for Neurosurgery. International Journal of Computer Assisted Radiology and Surgery 11(6): 929–936. 10.1007/s11548-016-1361-z.
Gonenc et al. (2017) Gonenc B, Chamani A, Handa J, Gehlbach P, Taylor RH and Iordachita I (2017) 3-DOF Force-Sensing Motorized Micro-Forceps for Robot-Assisted Vitreoretinal Surgery. IEEE Sensors Journal 17(11): 3526–3541.
Hadi Hosseinabadi et al. (2019) Hadi Hosseinabadi AH, Honarvar M and Salcudean SE (2019) Optical Force Sensing In Minimally Invasive Robotic Surgery. In: 2019 International Conference on Robotics and Automation (ICRA). pp. 4033–4039. 10.1109/ICRA.2019.8793589.
Haghighipanah et al. (2017) Haghighipanah M, Miyasaka M and Hannaford B (2017) Utilizing Elasticity of Cable-Driven Surgical Robot to Estimate Cable Tension and External Force. IEEE Robotics and Automation Letters 2(3): 1593–1600. 10.1109/LRA.2017.2676347.
Hannaford et al. (1991) Hannaford B, Wood L, McAffee DA and Zak H (1991) Performance Evaluation of a Six-Axis Generalized Force-Reflecting Teleoperator. IEEE Transactions on Systems, Man, and Cybernetics 21(3): 620–633.
Haouchine et al. (2018) Haouchine N, Kuang W, Cotin S and Yip M (2018) Vision-Based Force Feedback Estimation for Robot-Assisted Surgery Using Instrument-Constrained Biomechanical Three-Dimensional Maps. IEEE Robotics and Automation Letters 3(3): 2160–2165. 10.1109/LRA.2018.2810948.
Haraguchi et al. (2015) Haraguchi D, Kanno T, Tadano K and Kawashima K (2015) A Pneumatically Driven Surgical Manipulator with a Flexible Distal Joint Capable of Force Sensing. IEEE/ASME Transactions on Mechatronics 20(6): 2950–2961. 10.1109/TMECH.2015.2415838.
Hashtrudi-Zaad and Salcudean (2001) Hashtrudi-Zaad K and Salcudean SE (2001) Analysis of Control Architectures for Teleoperation Systems with Impedance/Admittance Master and Slave Manipulators. International Journal of Robotics Research 20(6): 419–445. 10.1177/02783640122067471.
Haslinger et al. (2013) Haslinger R, Leyendecker P and Seibold U (2013) A Fiberoptic Force-Torque Sensor for Minimally Invasive Robotic Surgery. In: 2013 IEEE International Conference on Robotics and Automation. pp. 4390–4395. 10.1109/ICRA.2013.6631199.
He et al. (2014) He C, Wang S, Sang H, Li J and Zhang L (2014) Force Sensing of Multiple-DOF Cable-Driven Instruments for Minimally Invasive Robotic Surgery. The International Journal of Medical Robotics and Computer Assisted Surgery 10(3): 314–324. 10.1002/rcs.1532.
Hong and Jo (2012) Hong MB and Jo YH (2012) Design and evaluation of 2-DOF compliant forceps with force-sensing capability for minimally invasive robot surgery. IEEE Transactions on Robotics 28(4): 932–941. 10.1109/TRO.2012.2194889.
Huang et al. (2018) Huang J, Yan Z and Xue R (2018) Grip Force Estimation of Laparoscope Surgical Robot based on Neural Network Optimized by Genetic Algorithm. In: 2018 ACM International Conference Proceeding Series. pp. 95–100. 10.1145/3265639.3265660.
Hwang and Lim (2017) Hwang W and Lim SC (2017) Inferring Interaction Force from Visual Information without Using Physical Force Sensors. Sensors (Switzerland) 17(11). 10.3390/s17112455.
Jones et al. (2011) Jones D, Lewis A and Fischer GS (2011) Development of a Standalone Surgical Haptic Arm. In: 2011 Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. pp. 2136–2139. 10.1109/IEMBS.2011.6090399.
Jones et al. (2017) Jones D, Wang H, Alazmani A and Culmer PR (2017) A Soft Multi-Axial Force Sensor to Assess Tissue Properties in Real-Time. In: 2017 IEEE International Conference on Intelligent Robots and Systems. pp. 5738–5743. 10.1109/IROS.2017.8206464.
Juo et al. (2020) Juo YY, Abiri A, Pensa J, Sun S, Tao A, Bisley J, Grundfest W and Dutson E (2020) Center for Advanced Surgical and Interventional Technology Multimodal Haptic Feedback for Robotic Surgery. In: Handbook of Robotic and Image-Guided Surgery. Elsevier, pp. 285–301. 10.1016/B978-0-12-814245-5.00017-7.
Karthikeyan and Nithya (2018) Karthikeyan KB and Nithya V (2018) Implementation of Surgical Robot Instrument with a Force Feedback. In: 2017 5th IEEE Region 10 Humanitarian Technology Conference 2017, R10-HTC 2017. pp. 101–104. 10.1109/R10-HTC.2017.8288916.
Khadem et al. (2016) Khadem SM, Behzadipour S, Mirbagheri A and Farahmand F (2016) A Modular Force-Controlled Robotic Instrument for Minimally Invasive Surgery - Efficacy for Being Used in Autonomous Grasping Against a Variable Pull Force. The International Journal of Medical Robotics and Computer Assisted Surgery 12(4): 620–633. 10.1002/rcs.1727.
Kim et al. (2017) Kim S, Kim C, Park S and Lee DY (2017) A 3-DOF Sensor to Estimate the Force Applied to the Tip of a Surgical Instrument. In: 2017 18th International Conference on Advanced Robotics, ICAR 2017. pp. 143–148. 10.1109/ICAR.2017.8023509.
Kim et al. (2016) Kim U, Kim YB, Seok DY, So J and Choi HR (2016) Development of Surgical Forceps Integrated with a Multi-axial Force Sensor for Minimally Invasive Robotic Surgery. In: IEEE International Conference on Intelligent Robots and Systems. pp. 3684–3689. 10.1109/IROS.2016.7759543.
Kim et al. (2018a) Kim U, Kim YB, Seok DY, So J and Choi HR (2018a) A Surgical Palpation Probe with 6-Axis Force/Torque Sensing Capability for Minimally Invasive Surgery. IEEE Transactions on Industrial Electronics 65(3): 2755–2765. 10.1109/TIE.2017.2739681.
Kim et al. (2018b) Kim U, Kim YB, So J, Seok DY and Choi HR (2018b) Sensorized Surgical Forceps for Robotic-Assisted Minimally Invasive Surgery. IEEE Transactions on Industrial Electronics 65(12): 9604–9613. 10.1109/TIE.2018.2821626.
Kim et al. (2015) Kim U, Lee DH, Yoon WJ, Hannaford B and Choi HR (2015) Force Sensor Integrated Surgical Forceps for Minimally Invasive Robotic Surgery. IEEE Transactions on Robotics 31(5): 1214–1224. 10.1109/TRO.2015.2473515.
Kim et al. (2012) Kim W, Seung S, Choi H, Park S, Ko SY and Park JO (2012) Image-Based Force Estimation of Deformable Tissue Using Depth Map for Single-Port Surgical Robot. In: 2012 International Conference on Control, Automation and Systems. pp. 1716–1719.
Kong et al. (2018) Kong NJ, Stephens TK and Kowalewski TM (2018) Da Vinci tool torque mapping over 50,000 grasps and its implications on grip force estimation accuracy. In: 2018 International Symposium on Medical Robotics (ISMR). pp. 1–6. 10.1109/ISMR.2018.8333292.
Kuang et al. (2020) Kuang W, Yip M and Zhang J (2020) Vibration-Based Multi-Axis Force Sensing: Design, Characterization, and Modeling. IEEE Robotics and Automation Letters 5(2): 3082–3089. 10.1109/LRA.2020.2975726.
Kuwana et al. (2013) Kuwana K, Nakai A, Masamune K and Dohi T (2013) A Grasping Forceps with a Triaxial MEMS Tactile Sensor for Quantification of Stresses on Organs. 2013 International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS : 4490–449310.1109/EMBC.2013.6610544.
Lee et al. (2014) Lee C, Park WJ, Kim M, Noh S, Yoon C, Lee C, Kim Y, Kim HH, Kim HC and Kim S (2014) Pneumatic-Type Surgical Robot End-Effector for Laparoscopic Surgical-Operation-by-Wire. BioMedical Engineering Online 13(1). 10.1186/1475-925X-13-130.
Lee et al. (2015) Lee C, Park YH, Yoon C, Noh S, Lee C, Kim Y, Kim HC, Kim HH and Kim S (2015) A Grip Force Model for the da Vinci End-effector to Predict a Compensation Force. Medical and Biological Engineering and Computing 53(3): 253–261. 10.1007/s11517-014-1230-2.
Lee et al. (2016) Lee DH, Kim U, Gulrez T, Yoon WJ, Hannaford B and Choi HR (2016) A Laparoscopic Grasping Tool with Force Sensing Capability. IEEE/ASME Transactions on Mechatronics 21(1): 130–141. 10.1109/TMECH.2015.2442591.
Lee et al. (2013) Lee DH, Kim U, Moon H, Koo JC, Yoon WJ and Choi HR (2013) Preliminary Design of Multi-Axial Contact Force Sensor for Minimally Invasive Robotic Surgery Grasper. In: 2013 IEEE International Conference on Robotics and Automation. pp. 1019–1024. 10.1109/ICRA.2013.6630698.
Li et al. (2013) Li H, Kawashima K, Tadano K, Ganguly S and Nakano S (2013) Achieving Haptic Perception in Forceps Manipulator Using Pneumatic Artificial Muscle. IEEE/ASME Transactions on Mechatronics 18(1): 74–85. 10.1109/TMECH.2011.2163415.
Li et al. (2015a) Li K, An B, peng Ao W, bo Eng H, li U Y and guo Ang S (2015a) Miniature 6-axis Force/Torque Sensor for Force Feedback in Robot-Assisted Minimally Invasive Surgery. Journal of Central South University 22(12): 4566–4577. 10.1007/s11771-015-3007-7.
Li et al. (2015b) Li K, Pan B, Zhan J, Gao W, Fu Y and Wang S (2015b) Design and Performance Evaluation of a 3-Axis Force Sensor for MIS Palpation. Sensor Review 35(2): 219–228. 10.1108/SR-04-2014-632.
Li et al. (2017a) Li K, Pan B, Zhang F, Gao W, Fu Y and Wang S (2017a) A novel 4-DOF Surgical Instrument with Modular Joints and 6-Axis Force Sensing Capability. The International Journal of Medical Robotics and Computer Assisted Surgery 13(1): e1751:1–14. 10.1002/rcs.1751.
Li et al. (2017b) Li L, Yu B, Yang C, Vagdargi P, Srivatsan RA and Choset H (2017b) Development of an Inexpensive Tri-Axial Force Sensor for Minimally Invasive Surgery. In: 2017 IEEE International Conference on Intelligent Robots and Systems. pp. 906–913. 10.1109/IROS.2017.8202253.
Li and Hannaford (2017) Li Y and Hannaford B (2017) Gaussian Process Regression for Sensorless Grip Force Estimation of Cable-Driven Elongated Surgical Instruments. IEEE Robotics and Automation Letters 2(3): 1312–1319. 10.1109/LRA.2017.2666420.
Li et al. (2016) Li Y, Miyasaka M, Haghighipanah M, Cheng L and Hannaford B (2016) Dynamic Modeling of Cable Driven Elongated Surgical Instruments for Sensorless Grip Force Estimation. In: 2016 Proceedings - IEEE International Conference on Robotics and Automation. pp. 4128–4134. 10.1109/ICRA.2016.7487605.
Lim et al. (2014) Lim SC, Lee HK and Park J (2014) Grip Force Measurement of Forceps with Fibre Bragg Grating Sensors. Electronics Letters 50(10): 733–735. 10.1049/el.2013.4182.
Lv et al. (2020) Lv C, Wang S and Shi C (2020) A High-Precision and Miniature Fiber Bragg Grating-Based Force Sensor for Tissue Palpation During Minimally Invasive Surgery. Annals of Biomedical Engineering 48(2): 669–681. 10.1007/s10439-019-02388-w.
Mack et al. (2012) Mack I, Ferguson S, Rafferty K, Potts S and Dick A (2012) Interactive Force-Sensing Feedback System for Remote Robotic Laparoscopic Surgery. Transactions of the Institute of Measurement and Control 34(4): 376–387. 10.1177/0142331210382943.
Maeda et al. (2016) Maeda S, Oiwa K and Ishii C (2016) Scaling Method for Force Feedback of Forceps Manipulator Based on Beam Theory. International Journal of Pharma Medicine and Biological Sciences 5(1). 10.18178/ijpmbs.5.1.23-30.
Marban et al. (2018) Marban A, Srinivasan V, Samek W, Fernandez J and Casals A (2018) Estimation of Interaction Forces in Robotic Surgery using a Semi-Supervised Deep Neural Network Model. In: IEEE International Conference on Intelligent Robots and Systems. pp. 761–768. 10.1109/IROS.2018.8593701.
Marban et al. (2019) Marban A, Srinivasan V, Samek W, Fernández J and Casals A (2019) A Recurrent Convolutional Neural Network Approach for Sensorless Force Estimation in Robotic Surgery. Biomedical Signal Processing and Control 50: 134–150. 10.1016/j.bspc.2019.01.011.
Martell et al. (2011) Martell J, Elmer T, Gopalsami N and Park YS (2011) Visual Measurement of Suture Strain for Robotic Surgery. Computational and Mathematical Methods in Medicine 2011. 10.1155/2011/879086.
McKinley et al. (2015) McKinley S, Garg A, Sen S, Kapadia R, Murali A, Nichols K, Lim S, Patil S, Abbeel P, Okamura AM and Goldberg K (2015) A Single-Use Haptic Palpation Probe for Locating Subcutaneous Blood Vessels in Robot-Assisted Minimally Invasive Surgery. In: 2015 IEEE International Conference on Automation Science and Engineering. pp. 1151–1158. 10.1109/CoASE.2015.7294253.
Moradi Dalvand et al. (2014) Moradi Dalvand M, Shirinzadeh B, Shamdani AH, Smith J and Zhong Y (2014) An Actuated Force Feedback-Enabled Laparoscopic Instrument for Robotic-Assisted Surgery. The International Journal of Medical Robotics and Computer Assisted Surgery 10(1): 11–21. 10.1002/rcs.1503.
Nakai et al. (2017) Nakai A, Kuwana K, Saito K, Dohi T, Kumagai A and Shimoyama I (2017) MEMS 6-axis Force-Torque Sensor Attached to the Tip of Grasping Forceps for Identification of Tumor in Thoracoscopic Surgery. In: 2017 IEEE International Conference on Micro Electro Mechanical Systems (MEMS). Institute of Electrical and Electronics Engineers Inc., pp. 546–548. 10.1109/MEMSYS.2017.7863464.
Noohi et al. (2014) Noohi E, Parastegari S and Žefran M (2014) Using Monocular Images to Estimate Interaction Forces During Minimally Invasive Surgery. In: 2014 IEEE International Conference on Intelligent Robots and Systems. pp. 4297–4302. 10.1109/IROS.2014.6943169.
Novoseltseva (2018) Novoseltseva A (2018) Force Feedback for the Patient Side Manipulator of the daVinci Research Kit. Master’s Thesis, Worcester Polytechnic Institute.
Okamura (2009) Okamura AM (2009) Haptic Feedback in Robot-Assisted Minimally Invasive Surgery. Current Opinion in Urology 19(1): 102–107. 10.1097/MOU.0b013e32831a478c.
O’Neill et al. (2018) O’Neill JJ, Stephens TK and Kowalewski TM (2018) Evaluation of Torque Measurement Surrogates as Applied to Grip Torque and Jaw Angle Estimation of Robotic Surgical Tools. IEEE Robotics and Automation Letters 3(4): 3027–3034. 10.1109/LRA.2018.2849862.
Otte et al. (2016) Otte C, Beringhoff J, Latus S, Antoni ST, Rajput O and Schlaefer A (2016) Towards Force Sensing Based on Instrument-Tissue Interaction. In: 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems. pp. 180–185. 10.1109/MFI.2016.7849486.
Overtoom et al. (2019) Overtoom EM, Horeman T, Jansen FW, Dankelman J and Schreuder HW (2019) Haptic Feedback, Force Feedback, and Force-Sensing in Simulation Training for Laparoscopy: A Systematic Overview. Journal of Surgical Education 76(1): 242–261. 10.1016/j.jsurg.2018.06.008.
Pandya et al. (2014) Pandya HJ, Kim HT, Roy R and Desai JP (2014) MEMS Based Low Cost Piezoresistive Microcantilever Force Sensor and Sensor Module. Materials Science in Semiconductor Processing 19(1): 163–173. 10.1016/j.mssp.2013.12.016.
Peña et al. (2018) Peña R, Smith MJ, Ontiveros NP, Hammond FL and Wood RJ (2018) Printing Strain Gauges on Intuitive Surgical da Vinci Robot End Effectors. In: 2018 IEEE International Conference on Intelligent Robots and Systems. pp. 806–812. 10.1109/IROS.2018.8594517.
Puangmali et al. (2008) Puangmali P, Althoefer K, Seneviratne LD, Murphy D and Dasgupta P (2008) State-of-the-Art in Force and Tactile Sensing for Minimally Invasive Surgery. IEEE Sensors Journal 8(4): 371–380. 10.1109/JSEN.2008.917481.
Puangmali et al. (2012) Puangmali P, Liu H, Seneviratne LD, Dasgupta P and Althoefer K (2012) Miniature 3-axis Distal Force Sensor for Minimally Invasive Surgical Palpation. IEEE/ASME Transactions on Mechatronics 17(4): 646–656. 10.1109/TMECH.2011.2116033.
Radó et al. (2018) Radó J, Dücső C, Földesy P, Szebényi G, Nawrat Z, Rohr K and Fürjes P (2018) 3D force sensors for laparoscopic surgery tool. Microsystem Technologies 24(1): 519–525. 10.1007/s00542-017-3443-4.
Rahman and Lee (2015) Rahman N and Lee MC (2015) Actual Reaction Force Separation Method of Surgical Tool by Fuzzy Logic Based SMCSPO. International Journal of Control, Automation and Systems 13(2): 379–389. 10.1007/s12555-014-0100-x.
Rangarajan et al. (2020) Rangarajan K, Davis H and Pucher PH (2020) Systematic Review of Virtual Haptics in Surgical Simulation: A Valid Educational Tool? Journal of Surgical Education 77(2): 337–347. 10.1016/j.jsurg.2019.09.006.
Ranzani et al. (2015) Ranzani T, Ciuti G, Tortora G, Arezzo A, Arolfo S, Morino M and Menciassi A (2015) A Novel Device for Measuring Forces in Endoluminal Procedures. International Journal of Advanced Robotic Systems 12(8): 116. 10.5772/60832.
Sang et al. (2017) Sang H, Yun J, Monfaredi R, Wilson E, Fooladi H and Cleary K (2017) External force estimation and implementation in robotically assisted minimally invasive surgery. The International Journal of Medical Robotics and Computer Assisted Surgery 13(2): e1824:1–15. 10.1002/rcs.1824.
Schwalb et al. (2017) Schwalb W, Shirinzadeh B and Smith J (2017) A Force-Sensing Surgical Tool with a Proximally Located Force/Torque Sensor. International Journal of Medical Robotics and Computer Assisted Surgery 13(1). 10.1002/rcs.1737.
Seibold et al. (2005) Seibold U, Kubler B and Hirzinger G (2005) Prototype of Instrument for Minimally Invasive Surgery with 6-Axis Force Sensing Capability. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation. pp. 496–501. 10.1109/ROBOT.2005.1570167.
Seneci et al. (2017) Seneci CA, Anastasova S and Yang GZ (2017) Disposable Force Sensing Clip for Robotic Surgical Instruments. In: 2017 Hamlyn Symposium on Medical Robotics. pp. 31–32. 10.31256/hsmr2017.16.
Seok et al. (2019) Seok DY, Kim YB, Kim U, Lee SY and Choi HR (2019) Compensation of Environmental Influences on Sensorized-Forceps for Practical Surgical Tasks. IEEE Robotics and Automation Letters 4(2): 2031–2037. 10.1109/LRA.2019.2899217.
Shahzada et al. (2016) Shahzada KS, Yurkewich A, Xu R and Patel RV (2016) Sensorization of a Surgical Robotic Instrument for Force Sensing. In: 2016 Optical Fibers and Sensors for Medical Diagnostics and Treatment Applications XVI, volume 9702. pp. 153–162. 10.1117/12.2213385.
Shi et al. (2019) Shi C, Li M, Lv C, Li J and Wang S (2019) High-Sensitivity Fiber Bragg Grating-Based Distal Force Sensor for Laparoscopic Surgery. IEEE Sensors Journal : 2467–247510.1109/jsen.2019.2951782.
Shimachi et al. (2003) Shimachi S, Hakozaki Y, Tada T and Fujiwara Y (2003) Measurement of Force Acting on Surgical Instrument for Force-feedback to Master Robot Console. International Congress Series 1256(C): 538–546. 10.1016/S0531-5131(03)00356-X.
Soltani-Zarrin et al. (2018) Soltani-Zarrin P, Escoto A, Xu R, Patel RV, Naish MD and Trejos AL (2018) Development of a 2-DOF Sensorized Surgical Grasper for Grasping and Axial Force Measurements. IEEE Sensors Journal 18(7): 2816–2826. 10.1109/JSEN.2018.2805327.
Song et al. (2014) Song HS, Jeong JW and Lee JJ (2014) Optical Fiber Bragg Grating (FBG) Force Reflection Sensing System of Surgical Tool for Minimally Invasive Surgery. In: 2014 Proceedings of the 9th IEEE Conference on Industrial Electronics and Applications, ICIEA 2014. pp. 478–482. 10.1109/ICIEA.2014.6931211.
Spiers et al. (2015) Spiers AJ, Thompson HJ and Pipe AG (2015) Investigating Remote Sensor Placement for Practical Haptic Sensing with EndoWrist Surgical Tools. In: 2015 IEEE World Haptics Conference. pp. 152–157. 10.1109/WHC.2015.7177706.
Srivastava et al. (2016) Srivastava A, Xu R, Escoto A, Ward C and Patel RV (2016) Design of an Ultra Thin Strain Sensor Using Superelastic Nitinol for Applications in Minimally Invasive Surgery. In: 2016 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM. pp. 794–799. 10.1109/AIM.2016.7576865.
Stephens et al. (2019) Stephens TK, O’neill JJ, Kong NJ, Mazzeo MV, Norfleet JE, Sweet RM and Kowalewski TM (2019) Conditions for Reliable Grip Force and Jaw Angle Estimation of da Vinci Surgical Tools. International Journal of Computer Assisted Radiology and Surgery 14: 117–127. 10.1007/s11548-018-1866-8.
Suzuki et al. (2018) Suzuki H, Masuda H, Hongo K, Horie R, Yajima S, Itotani Y, Fujita M and Nagasaka K (2018) Development and Testing of Force-Sensing Forceps Using FBG for Bilateral Micro-Operation System. IEEE Robotics and Automation Letters 3(4): 4281–4288. 10.1109/LRA.2018.2864771.
Tahir et al. (2018) Tahir A, Abinahed J, Navkar N, Sadasivuni KK, Al-Ansari A and Cabibihan JJ (2018) Experimental Characterization of a Tactile Sensor for Surgical Applications. In: 2018 IEEE International Conference on Innovative Research and Development, ICIRD 2018. pp. 1–4. 10.1109/ICIRD.2018.8376322.
Takeishi et al. (2019) Takeishi H, Baldovino RG, Bugtai NT and Dadios EP (2019) A Force Sensing-Based Pneumatics for Robotic Surgery Using Neural Network. In: 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management, HNICEM 2018. pp. 1–6. 10.1109/HNICEM.2018.8666235.
Takizawa et al. (2018) Takizawa T, Kanno T, Miyazaki R, Tadano K and Kawashima K (2018) Grasping Force Estimation in Robotic Forceps Using a Soft Pneumatic Actuator with a Built-in Sensor. Sensors and Actuators, A: Physical 271: 124–130. 10.1016/j.sna.2018.01.007.
Talasaz et al. (2014) Talasaz A, Luisa Trejos A, Perreault S, Bassan H and Patel RV (2014) A Dual-Arm 7-Degrees-of-Freedom Haptics-Enabled Teleoperation Test Bed for Minimally Invasive Surgery. Journal of Medical Devices 8(4): 041004–1:15. 10.1115/1.4026984.
Trejos et al. (2014) Trejos A, Escoto A, Hughes D, Naish M and Patel R (2014) A Sterilizable Force-Sensing Instrument for Laparoscopic Surgery. In: 2014 Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics. pp. 157–162. 10.1109/BIOROB.2014.6913769.
Trejos et al. (2017) Trejos AL, Escoto A, Naish MD and Patel RV (2017) Design and Evaluation of a Sterilizable Force Sensing Instrument for Minimally Invasive Surgery. IEEE Sensors Journal 17(13): 3983–3993. 10.1109/JSEN.2017.2703883.
Trejos et al. (2010) Trejos AL, Patel RV and Naish MD (2010) Force Sensing and Its Application in Minimally Invasive Surgery and Therapy: A Survey. 2010 Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 224(7): 1435–1454. 10.1243/09544062JMES1917.
Tsukamoto and Ishii (2014) Tsukamoto Y and Ishii C (2014) Estimation of the Grasping Torque of Robotic Forceps Using the Robust Reaction Torque Observer. In: 2014 IEEE International Conference on Robotics and Biomimetics, IEEE ROBIO 2014. pp. 1650–1655. 10.1109/ROBIO.2014.7090571.
Vakili et al. (2011) Vakili K, Flander MS, Sepp TR, Corral M, Diaz JD, Slocum A and Teo GS (2011) Design and Testing of a Pressure Sensing Laparoscopic Grasper. In: Design of Medical Devices. pp. 1–8.
Van Den Dobbelsteen et al. (2012) Van Den Dobbelsteen JJ, Lee RA, Van Noorden M and Dankelman J (2012) Indirect Measurement of Pinch and Pull Forces at the Shaft of Laparoscopic Graspers. Medical and Biological Engineering and Computing 50(3): 215–221. 10.1007/s11517-012-0862-3.
Wang et al. (2014) Wang H, Kang B and Lee DY (2014) Design of a Slave Arm of a Surgical Robot System to Estimate the Contact Force at the Tip of the Employed Instruments. Advanced Robotics 28(19): 1305–1320. 10.1080/01691864.2014.933128.
Wang et al. (2019a) Wang Z, Wang D, Chen B, Yu L, Qian J and Zi B (2019a) A Clamping Force Estimation Method Based on a Joint Torque Disturbance Observer Using PSO-BPNN for Cable-Driven Surgical Robot End-Effectors. Sensors (Switzerland) 19(23). 10.3390/s19235291.
Wang et al. (2019b) Wang Z, Zi B, Wang D, Qian J, You W and Yu L (2019b) External Force Self-Sensing based on Cable-Tension Disturbance Observer for Surgical Robot End-Effector. IEEE Sensors Journal 19(13): 5274–5284. 10.1109/JSEN.2019.2903776.
Wee et al. (2016) Wee J, Brooks RJ, Looi T, Azzie G, Drake J and Ted Gerstle J (2016) Force-Sensing Sleeve for Laparoscopic Surgery. Journal of Medical Devices 10(3): 030946:1–2. 10.1115/1.4033810.
Wee et al. (2017) Wee J, Kang M, Francis P, Brooks R, Masotti L, Villavicencio D, Looi T, Azzie G, Drake J and Gerstle JT (2017) Novel Force-Sensing System for Minimally Invasive Surgical Instruments. In: 2017 Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, c. pp. 4447–4450. 10.1109/EMBC.2017.8037843.
Wottawa et al. (2016) Wottawa C, Genovese B, Nowroozi B, Bisley J, Grundfest W and Dutson E (2016) Evaluating Tactile Feedback in Robotic Surgery for Potential Clinical Application Using an Animal Model. Surgical Endoscopy 30: 3198–3209. https://doi.org/10.1007/s00464-015-4602-2.
Xin et al. (2017) Xin X, Chen X, Liu S, Zhao B, Gao P, Hu Y and Lv G (2017) Model-Based Grasp Force Estimation for Minimally Invasive Surgery. In: 2017 IEEE International Conference on Information and Automation, ICIA 2017. pp. 489–493. 10.1109/ICInfA.2017.8078957.
Xue et al. (2019) Xue R, Du Z, Yan Z and Ren B (2019) An Estimation Method of Grasping Force for Laparoscope Surgical Robot Based on the Model of a Cable-Pulley System. Mechanism and Machine Theory 134: 440–454. 10.1016/j.mechmachtheory.2018.12.032.
Xue et al. (2018) Xue R, Ren B, Huang J, Yan Z and Du Z (2018) Design and Evaluation of FBG-Based Tension Sensor in Laparoscope Surgical Robots. Sensors 18(7): 2067:1–18. 10.3390/s18072067.
Yoon et al. (2015) Yoon SM, Lee MC and Kim CY (2015) Sliding Perturbation Observer Based Reaction Force Estimation Method of Surgical Robot Instrument for Haptic Realization. International Journal of Humanoid Robotics 12(2). 10.1142/S0219843615500139.
Yu et al. (2014) Yu H, Jiang J, Xie L, Liu L, Shi Y and Cai P (2014) Design and Static Calibration of a Six-Dimensional Force/Torque Sensor for Minimally Invasive Surgery. Minimally Invasive Therapy & Allied Technologies 23(3): 136–143. 10.3109/13645706.2013.873469.
Yu et al. (2018a) Yu L, Wang W and Zhang F (2018a) External Force Sensing Based on Cable Tension Changes in Minimally Invasive Surgical Micromanipulators. IEEE Access 6: 5362–5373. 10.1109/ACCESS.2017.2788498.
Yu et al. (2018b) Yu L, Yan Y, Yu X and Xia Y (2018b) Design and Realization of Forceps with 3-D Force Sensing Capability for Robot-Assisted Surgical System. IEEE Sensors Journal 18(21): 8924–8932. 10.1109/JSEN.2018.2867838.
Yurkewich et al. (2014) Yurkewich DS, Escoto A, Luisa Trejos A, Le Bel ME, Patel RV and Naish MD (2014) Low-Cost Force-Sensing Arthroscopic Tool Using Threaded Fiber Bragg Grating Sensors. Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics : 28–3310.1109/biorob.2014.6913747.
Zhao and Nelson (2015) Zhao B and Nelson CA (2015) Sensorless Force Sensing for Minimally Invasive Surgery. Journal of Medical Devices 9(4). 10.1115/1.4031282.