\DOI

2024.0357

\historydate

Published in Healthcare Technology Letters; Received on xxx; Revised on xxx

StraightTrack: Towards Mixed Reality Navigation System for Percutaneous K-wire Insertion

Han Zhang^∗,1 Benjamin D. Killeen^∗,1 Yu-Chun Ku¹ Lalithkumar Seenivasan¹ Yuxuan Zhao¹ Mingxu Liu¹ Yue Yang¹ Suxi Gu¹ Alejandro Martin-Gomez¹ Russell H. Taylor¹ Greg Osgood² and Mathias Unberath¹ ¹Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
²Department of Orthopaedic Surgery, Johns Hopkins Medicine, Baltimore, MD, USA
E-mail: [email protected]
^∗These authors contributed equally to this work.

Abstract

In percutaneous pelvic trauma surgery, accurate placement of Kirschner wires (K-wires) is crucial to ensure effective fracture fixation and avoid complications due to breaching the cortical bone along an unsuitable trajectory. Surgical navigation via mixed reality (MR) can help achieve precise wire placement in a low-profile form factor. Current approaches in this domain are as yet unsuitable for real-world deployment because they fall short of guaranteeing accurate visual feedback due to uncontrolled bending of the wire. To ensure accurate feedback, we introduce StraightTrack, an MR navigation system designed for percutaneous wire placement in complex anatomy. StraightTrack features a marker body equipped with a rigid access cannula that mitigates wire bending due to interactions with soft tissue and a covered bony surface. Integrated with an Optical See-Through Head-Mounted Display (OST HMD) capable of tracking the cannula body, StraightTrack offers real-time 3D visualization and guidance without external trackers, which are prone to losing line-of-sight. In phantom experiments with two experienced orthopedic surgeons, StraightTrack improves wire placement accuracy, achieving the ideal trajectory within $5.26\pm 2.29$ mm and $2.88\pm 1.49$ °, compared to over 12.08 mm and 4.07°for comparable methods. As MR navigation systems continue to mature, StraightTrack realizes their potential for internal fracture fixation and other percutaneous orthopedic procedures.

1 Introduction

Refer to caption — Figure 1: (a) A real-world failure mode of a conventional mixed reality (MR) navigation system for K-wire insertion, in which the K-wire bends in an uncontrolled manner. (b) Our StraightTrack navigation system visualizes the real-time K-wire trajectory error based on head-mounted tracking of the cannula body. The user aligns the cannula body by minimizing the radius of two circles, which visualize the entry- and endpoint error of the current trajectory. The hatch mark indicates the direction of the error. (c) The cannula body features a rigid steel sheath that ensures accurate tracking of the actual K-wire trajectory up to the cortical bone. This is necessary to overcome two real-world causes of K-wire bending, namely interaction with tough soft tissue and “skating” of the K-wire tip on hard bone.

The prevalence of pelvic ring fractures in the United States was 37 per 100,000 in 2023, with particularly high mortality rates observed among patients over 65 [1, 2]. Effective treatment of these fractures often involves orthopedic surgery, such as pelvic fracture fixation, where the precise placement of screws is crucial for bone fixation and successful internal fracture management. To ensure this precision and avoid damage to vulnerable structures, accurate intra-operative guidance is essential. The cannulated screw system is widely used for this purpose, allowing surgeons to position a Kirschner wire (K-wire) along the desired trajectory before final screw placement, avoiding vulnerable structures and ensuring contact with the hard bone [3]. This system facilitates iterative adjustment of the K-wire trajectory based on X-ray image guidance, making it particularly valuable for minimally invasive internal fixation procedures in complex anatomical regions not only the pelvis, but also femoral neck, odontoid, and extremities [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]. These procedures are associated with smaller incisions, reduced pain, shorter recovery times, and overall better patient outcomes [5]. However, inserting K-wires in complex regions like the pelvis often necessitates multiple X-ray acquisitions and repeated attempts, even for experienced trauma surgeons [15, 16]. The difficulty arises from the inability of surgeons to directly visualize the bone, which increases the potential for human error during the procedure [17, 15]. Furthermore, this process can expose patients and clinicians to significant radiation, with each screw placement requiring between 20 to 100 fluoroscopic images [16, 13, 18].

Together, these factors have motivated the development of computer-assisted approaches for cannulated screw placement that minimizes X-ray image acquisition while still providing effective intra-operative guidance. Recent methodologies [19, 20, 21, 22, 23] employ 2 to 4 initial X-ray acquisitions to establish a desired trajectory, which is then visualized in 3D using Mixed Reality (MR) via an Optical See-Through Head-Mounted Display (OST HMD). MR guidance is particularly advantageous for these procedures as it maintains the dexterity and tactile feedback of freehand drilling while eliminating the need for bulky and expensive robotic systems, such as the Globus Medical ExcelsiusGPS Robotic Navigation Platform, costing approximately $1.5 million USD [24]. Modern HMDs can serve as tracking devices, ensuring the accuracy of a fully navigated approach without requiring external optical trackers, which are difficult to position to maintain a clear line-of-sight [25]. Despite the advancements in trajectory planning and the use of MR for guidance to reduce navigation errors during K-wire placement, challenges remain. K-wires often bend unintentionally due to the force exerted on them while maintaining a constant insertion angle on an irregularly shaped bone surface. This bending means that real-time tracking of the surgical drill’s pose may not align with the K-wire’s actual trajectory, as illustrated in Fig. 1.a. There is an unmet clinical need for a surgical guiding system that ensures effective and accurate K-wire insertion in complex anatomy. To the best of our knowledge, no previous studies on tracking and navigation systems have investigated K-wire bending during insertion, nor have they addressed limiting K-wire bending or tracking it to compensate for this.

In this paper, we introduce the StraightTrack, an MR system designed for orthopedic guiding wire placement that prevents K-wire bending in complex anatomy (Fig. 1.b, 1.c). StraightTrack consists of a rigid access cannula equipped with a marker body that mitigates wire bending despite soft tissue and bone contact. Integrated with an OST HMD with on-device optical tracking, our system offers real-time surgical tracking and guidance, including a user interaction paradigm for mitigating difficulties in spatial navigation and reducing errors in entry point perception. Finally, we evaluate StraightTrack for traditional K-wire placement tasks in phantoms with two experienced orthopedic surgeons. Our results demonstrate that the proposed system effectively addresses the K-wire bending problem, reducing wire displacement errors in translation and rotation at the entry, mid, and endpoint, thereby improving the overall accuracy of K-wire placement.

2 Methodology

StraightTrack consists of two main components: a cannula body for maintaining desired trajectories and an OST-HMD (i.e., Microsoft HoloLens 2) for intra-operative tracking and navigation. For instrument tracking, we use the Time-of-Flight (ToF) depth sensor on the HMD to track retro-reflective spheres mounted on the instrumented and patient in a predefined layout[26]. We utilize the STTAR package [27] to achieve marker detection and tracking. In addition to the Kalman filter integrated within the STTAR package, we introduced smooth interpolation between time frames to improve tracking stability and reduce jitter. We implemented the user interface and navigation using Unity3D and the Mixed Reality Toolkit 2 [28].

2.1 Cannula Body Design

To prevent bending while enabling tracking during K-wire insertion, we designed a cannula body (Fig. 1.c). The cannula body is composed of three primary components: the handle, marker mount, and wire sleeve. The marker mount is designed with tilted angles to maintain a clear line-of-sight when the surgeon holds the cannula body, ensuring that the retro-reflective markers are visible on different planes during positioning. In our experiment, the cannula body was prototyped using a Stratasys F170 Fused Deposition Modeling (FDM) 3D printer with Acrylonitrile Butadiene Styrene (ABS) material. The cannula body was fabricated in three separate parts, which were subsequently assembled using screws and nuts to enhance its strength. To prevent deformation of the plastic sleeve during wire positioning in tissue, a hollow steel shaft was inserted inside the plastic sleeve.

To accurately determine the shaft axis of the cannula body and minimize errors introduced during the manufacturing phase, we conducted multiple pivot calibrations. The pivot calibration was performed by using Polaris NDI (Northern Digital Incorporated, Ontario, Canada). During each pivot calibration, a K-wire was inserted and locked in place. The K-wire was extended toward the cannula body tip in increments of 1 cm. In total, we performed six pivot calibrations, extending the K-wire by 6 cm. A regression analysis was conducted to determine the shaft axis. The pivot calibration resulted in a mean 3D RMS error of 0.66 mm and a mean error of 0.61 mm.

2.2 System Workflow

The overall system workflow consists of pre-operative trajectory planning and intra-operative guidance. The overview of the transformation chain(Fig. 2). To provide effective K-wire placement guidance, the main goal of our system is to align the cannula body shaft axis $\mathbf{n}_{\rm tip}$ to trajectory axis $\mathbf{n}_{\rm trajectory}$ in world coordinate, as shown in Equation 1.

	$\displaystyle F^{\text{W}}_{\text{H}}\cdot F^{\text{H}}_{\text{C}}\cdot\mathbf{n}_{\text{tip}}$	$\displaystyle=F^{\text{W}}_{\text{I}}\cdot\mathbf{n}_{{I_{trajectory}}}$		(1)
	$\displaystyle\mathbf{n}_{{I_{trajectory}}}$	$\displaystyle=\mathbf{p}_{{I_{entry}}}-\mathbf{p}_{I_{exit}}$		(1)

where $F^{\text{w}}_{\text{H}}$ is the transformation from World to HoloLens, $F^{\text{H}}_{\text{C}}$ is the transformation from HoloLens to Cannula body, ${n}_{tip}$ is the cannula body shaft axis in cannula body coordinate, $F^{\text{W}}_{\text{I}}$ is the transformation from World to CT image, and ${n}_{{I_{trajectory}}}$ is the trajectory axis in CT image coordinate, determined by the trajectory points $p_{\text{I}_{\text{entry}}}$ and $p_{\text{I}_{\text{exit}}}$ in CT image coordinates.

Pre-operative trajectory planning: In the pre-operative trajectory planning phase (Fig. 2.a), the primary objectives are to annotate the ideal trajectory in the patient’s CT image, denoted as $\mathbf{p}{I_{\text{entry}}}$ and $\mathbf{p}{I_{\text{exit}}}$ , and to perform the registration of the CT image to the patient coordinate system $F^{\text{P}}_{\text{I}}$ . The trajectory is either manually labeled in the CT image or determined using computer-assisted methods.

Our system utilizes the Brainlab Loop-X in conjunction with the BrainLab Curve for CT imaging. The registration process involves an external optical tracker, as represented in Equation 2.

F^{\text{P}}_{\text{I}}=(F^{\text{T}}_{\text{P}})^{-1}\cdot F^{\text{T}}_{\text{M}}\cdot F^{\text{M}}_{\text{I}}

(2)

where $F^{\text{T}}_{\text{P}}$ represents the transformation from the Optical Tracker to the Patient array, $F^{\text{T}}_{\text{M}}$ is the transformation from the Optical Tracker to the CT machine, and $F^{\text{M}}_{\text{I}}$ denotes the transformation from the CT machine to the CT image coordinate system.

To achieve the tracking of the CT machine, represented by $F^{\text{T}}_{\text{M}}$ , seven retro-reflective markers are placed on the CT machine gantry, enabling detection by the optical tracker. The pose of the CT machine is estimated using paired-point registration. The transformation $F^{\text{M}}_{\text{I}}$ is pre-calibrated by using a CT scan of an object embedded with both BB markers and optical markers arranged in a known configuration, with the registration performed using the paired-point registration.

With $F^{\text{P}}_{\text{I}}$ is determined and the ideal trajectory entry and exit points $\mathbf{p}{I_{\text{entry}}}$ and $\mathbf{p}{I_{\text{exit}}}$ are annotated, the pre-operative planning phase is completed.

Intra-operative guidance: In the intra-operative guidance phases, the system provides tracking and visual navigation for K-wire placement using the HMD, as shown in Equation 3.

	$\displaystyle F^{\text{W}}_{\text{H}}\cdot F^{\text{H}}_{\text{C}}\cdot\mathbf{n}_{\text{tip}}$	$\displaystyle=F^{\text{W}}_{\text{H}}\cdot F^{\text{H}}_{\text{P}}\cdot F^{\text{P}}_{\text{I}}\cdot\mathbf{n}_{{I_{trajectory}}}$		(3)
	$\displaystyle\mathbf{n}_{{I_{trajectory}}}$	$\displaystyle=\mathbf{p}_{{I_{entry}}}-\mathbf{p}_{I_{exit}}$		(3)

The transformation from world to HoloLens $F^{\text{w}}_{\text{H}}$ is estimated by the SLAM-based tracking system of the HMD. The pose of the patient anatomy $F^{\text{H}}_{\text{C}}$ and cannula body $F^{\text{H}}_{\text{P}}$ are tracked by HMD. CT registration $F^{\text{P}}_{\text{I}}$ and the annotated entry and exit point $\mathbf{p}_{I_{entry}}$ $\mathbf{p}_{I_{exit}}$ are obtained during pre-operative trajectory planning phase. To minimize errors introduced by SLAM drafting in the HMD, the system is designed to disable navigation if either the tool or patient momentarily loses track.

2.3 Navigation Interface

In discussions with orthopedic surgeons, several key challenges in AR navigation for percutaneous surgery were identified and addressed in our navigation system. These challenges include:

•

Spatial Alignment: Positioning surgical instruments in a 3D space demands precise alignment across six degrees of freedom. Relying solely on a surgeon’s natural hand-eye coordination and depth perception requires constant adjustment and transformation between different views. This complexity makes it challenging to align the instruments accurately with both the skin entry point and the desired trajectory within the body during percutaneous surgery.
•

Incision Point Identification: Percutaneous surgery does not provide a direct view of the desired trajectory on the bone within the body. Misalignment of the skin incision with the desired trajectory can cause the K-wire to pass through soft tissues at an unfavorable angle. Further adjustment may bend the K-wire as it encounters resistance from soft tissues and bone, increasing the risk of deviation from the planned trajectory.
•

Clear Surgical Field: Excessive visual information on navigation interfaces may increase the surgeon’s cognitive load and obstruct the surgical site. The proper navigation interface should provide essential guidance cues without causing unnecessary distraction or occlusion.

To address these spatial alignment challenges during human-in-the-loop K-wire placement, we incorporated two features to provide real-time visual guidance: an Error Indicator and a Surface Marker (Fig. 3). These design elements were inspired and refined through feedback from surgeons.

•

Error Indicator: To ensure the correct spatial alignment of the K-wire and have a clear surgical site view, our system generates visual cues in the form of dotted circles and lines at the designated entry and exit points. This approach is similar to the method described by [29], but with modifications to eliminate unnecessary visual elements that may cause distraction or occlusion. The HMD simultaneously tracks the pose of the cannula body and the patient reference array, estimating the error by comparing the tracked insertion path to the desired trajectory. The radius of each dotted circle represents the deviation between the current and desired paths, while the direction indicated by the line suggests the necessary adjustment. The user aims to minimize the size of the circles to achieve optimal alignment.
•

Surface Marker: To accurately display the insertion point and prevent visual occlusion of the surgical site, the system visualizes the surface insertion point on the skin using the HMD’s built-in depth camera. In our system, the HoloLens generates patient surface point clouds from its Articulated HAnd Tracking (AHAT) depth sensor. The position of the surface marker is calculated by identifying the closest point projected onto the desired trajectory. The orientation of the surface marker plane is determined by applying Singular Value Decomposition to the closest surrounding 500 points. During K-wire insertion, the user makes an incision at the surface marker and inserts the cannula at this location.

During the proposed wire placement workflow, desired trajectories are obtained via registration with a pre-operative plan or using an intra-operative workflow such as [20, 19]. The surgeon makes an initial incision in the soft tissue at the surface marker point and then inserts the cannula body until it reaches the bone surface. If necessary, the system can also function as a trocar for sharp instruments. Once in contact with the bone surface, the cannula body is aligned to the desired trajectory based on the error indicators. A sharp edge on the cannula allows it to grip the bone surface, preventing the K-wire from "skating" out of alignment before gaining purchase. Finally, the surgeon inserts the K-wire through the wire sleeve and begins the drilling process.

3 Experiments and Result

We evaluate our fully integrated system with the surgeon in the loop. First, we validate the end-to-end tracking error after the pre-operative planning phase, using a touch-point evaluation. To evaluate the full system, we conduct a phantom study with experienced orthopedic surgeons performing a K-wire insertion procedure.

3.1 Quantitative Assessment of End-to-End System Accuracy

To quantitatively evaluate the overall registration and navigation accuracy of our MR system without human-in-the-loop, we perform an end-to-end system accuracy assessment (Fig. 4) using the method described below.

3.1.1 Experimental Setup

We place seven 1.5 mm metal bearing balls (BBs) landmarks on a pelvic phantom target. Following the proposed preoperative trajectory planning workflow (Fig. 2.a), we obtained a CT image and manually annotated their positions $\mathbf{p}_{\rm I}$ . Using Equation 4, we obtained CT registration $F^{\text{P}}_{\text{I}}$ as well as landmark position $\mathbf{p}_{\text{I}}$ and send them to the HMD:

F^{\text{P}}_{\text{I}}=(F^{\text{T}}_{\text{P}})^{-1}\cdot F^{\text{T}}_{\text{M}}\cdot F^{\text{M}}_{\text{I}}

(4)

To measure the end-to-end system accuracy, we use a 3D-printed pointer to sequentially touch the annotated landmarks. The pointer, with a 1.5 mm diameter sphere-shaped groove tooltip, ensures repeatable and precise placement on the metal BBs. To obtain the tip position $\mathbf{p}_{\rm tip}$ , we performed a pivot calibration using the Polaris NDI. Since the pointer marker array has the same layout as the cannula body, the tracking error observed in the pointer marker can be correlated to the tracking error in the cannula body. The pointer is tracked by the HoloLens to locate the landmarks (Equation 5).

F^{\text{w}}_{\text{H}}\cdot F^{\text{H}}_{\text{tip}}\cdot\mathbf{p}_{\text{tip}}=F^{\text{W}}_{\text{H}}\cdot F^{\text{H}}_{\text{P}}\cdot F^{\text{P}}_{\text{I}}\cdot\mathbf{p}_{I}

(5)

By comparing the landmark positions computed in the pelvic phantom target coordinates with the landmark position obtained using the pointer tip, we can assess the accuracy of our MR solution. The end-to-end error, $\mathbf{\epsilon_{system}}$ , can then be computed as:

\mathbf{\epsilon_{system}}=\left|{F^{\text{W}}_{\text{H}}\cdot F^{\text{H}}_{\text{tip}}\cdot\mathbf{p}_{\text{tip}}-F^{\text{W}}_{\text{H}}\cdot F^{\text{H}}_{\text{P}}\cdot F^{\text{P}}_{\text{I}}\cdot\mathbf{p}_{I}}\right|

(6)

This error can be broken down into several components: HMD SLAM mapping ( $\epsilon^{\text{w}}_{\text{H}}$ ), HMD optical tracking ( $\epsilon^{\text{H}}_{\text{tip}}$ , $\epsilon^{\text{H}}_{\text{P}}$ ), instrument pivot calibration ( $\epsilon_{\text{tip}}$ ), and CT registration ( $\epsilon^{\text{P}}_{\text{I}}$ , $\epsilon_{\text{I}}$ ).

3.1.2 Quantitative Results

For the pointer pivot calibration, we observed a 3D RMS error of 0.58 mm and a mean error of 0.48 mm. For the end-to-end accuracy evaluation of the system, the average system error was 2.89 mm $\pm$ 0.97 mm. This result represents the overall error in reaching a point labeled in the CT volume in real space using our MR navigation system, excluding human-in-the-loop error.

3.2 Phantom study

To investigate the end-to-end, human-in-the-loop error of the StraightTrack system, we conducted a controlled phantom study focused on K-wire insertion in the presence of soft tissue and an irregular bone surface, as is the case in navigated fracture fixation. The experimental setup for our phantom study is shown in Fig. 5.a. During the experiment, the participant used a HoloLens HMD to place wires along specified trajectories, employing various guidance interfaces and marker mount methods. The cannula-mounted interface is our StraightTrack system.

3.2.1 Phantom Preparation

We manufactured five phantom models to replicate the traditional K-wire placement procedure in patients, as illustrated in Fig. 5.b. Each phantom consists of a 3D-printed structure with nine cylindrical shapes, a cast soft tissue layer, and a holder equipped with a retro-reflective marker array. The bone was fabricated using ABS material (Stratasys F170 FDM 3D printer) with 60% material infill to better simulate the properties of human bone material and the drilling experience[30]. To better replicate the difficulty of drilling on complex anatomy, such as the pelvis, each 100 mm cylinder was oriented in a random direction. The angle of orientation was less than 15°relative to the perpendicular. Additionally, each cylinder featured an uneven shape on the contact surface to simulate an irregular bone surface. The structure was then filled with 20 mm of silicone rubber gel (Ecoflex 00-35, Smooth-On, Inc. , PA) to mimic human soft tissue and was held with laser-cut acrylic boards. The structure and the holder were connected with screws to maintain a rigid transformation.

3.2.2 Study Design

In our phantom study, we invited two experienced orthopedic surgeons ((with over 10 years of experience) to place 2.8 mm K-wires. We compared three different marker mount methods (non-tracked, drill-mounted marker, and cannula body), as shown in Fig. 6, and two navigation interfaces (error indicator with and without the surface marker). For the non-tracked method, we did not track the K-wire; instead, we displayed a line connecting the entry and exit points as the navigation interface. In total, we examined six different methods resulting from the combinations of mounting methods and navigation interfaces.

Navigation Target	Entry Point Error(mm)	Mid Point Error (mm)	End Point Error (mm)	Rotation Error (°)
Non-tracked	8.91 $\pm$ 5.35	6.75 $\pm$ 3.84	6.52 $\pm$ 2.17	3.95 $\pm$ 1.86
Drill	12.08 $\pm$ 5.29	8.69 $\pm$ 4.56	6.66 $\pm$ 2.94	4.07 $\pm$ 2.19
Cannula (ours)	5.26 $\pm$ 2.29	3.85 $\pm$ 1.10	3.57 $\pm$ 1.52	2.88 $\pm$ 1.49

Table 1: Final K-wire placement error for three MR navigation systems, including visualization only (non-tracked), a drill-mounted tracked system, and our cannula-mounted StraightTrack system.

Navigation Target	Entry Point Error(mm)	Mid Point Error (mm)	End Point Error (mm)	Rotation Error (°)
Non-tracked	10.62 $\pm$ 5.09	7.52 $\pm$ 3.21	5.21 $\pm$ 1.91	4.31 $\pm$ 2.14
Drill	9.76 $\pm$ 5.43	7.16 $\pm$ 3.48	6.02 $\pm$ 2.36	4.22 $\pm$ 3.03
Cannula (ours)	8.68 $\pm$ 5.54	6.39 $\pm$ 3.45	4.72 $\pm$ 1.79	3.35 $\pm$ 2.57

Table 2: Final K-wire placement error for three MR navigation systems when using HoloLens depth sensor to provide a surface marker for initial insertion.

$p$ -value vs:	Non-tracked	Drill mount
Entry Point Error	0.0411	0.0005
Midpoint Error	0.0199	0.0017
Endpoint Error	0.0008	0.0038
Rotation Error	0.1376	0.1336

Table 3: Statistical significance of StraightTrack.

$p$ -value vs:	Non-tracked	Drill mount
Entry Point Error	0.3808	0.6356
Midpoint Error	0.4138	0.591
Endpoint Error	0.5281	0.1438
Rotation Error	0.3285	0.4573

Table 4: Statistical significance of StraightTrack with the surface marker.

During the preoperative planning phase, we collected CT volumes from the phantoms and planned the target trajectory paths for each cylinder. By computing the spatial relationship between the planned trajectory points and the patient marker array using Equation 2, the pre-planned trajectories were made visible to the surgeon through the HMD. The surgeon performed eye-tracking calibration before starting the procedure. Before the actual experiment, the surgeon practiced multiple K-wire insertions in a test phantom to become familiar with the setup and flatten the learning effect.

Following this familiarization, each surgeon performed K-wire insertions under different MR guidance methods. Each method was tested in six drilling trials, resulting in a total of 36 trajectories. The order of navigation interfaces and mounting markers was randomized to minimize the learning effect imbalance. Post-CT scans of each phantom were acquired to measure the difference between planned and actual trajectories. We report the final error of K-wire placements relative to the ideal, annotated trajectory in terms of the translational error at the entry, mid, and endpoint of the corridor, as well as the angle deviation between them.

3.2.3 Study Results

A summary of the phantom study results is presented in Fig. 7 and Table. 1. We conducted 12 trials for each combination of marker mount methods and navigation interfaces. Among all six combinations (Table. 1.a), the method utilizing the cannula body with an error indicator achieved the lowest rotation error ( $2.88\pm 1.49$ °), as well as the lowest entry, mid, and endpoint error of $5.26\pm 2.29$ mm, $3.85\pm 1.10$ mm and $3.57\pm 1.52$ mm, respectively. As shown in Table 3, the cannula body method demonstrated statistically significant improvements in translation error compared to both the non-tracked method and the drill mount method ( $p<0.05$ ). A high error indicates a high likelihood of cortical breach, highlighting the value of StraightTrack’s cannula body for mitigating errors during the human-in-the-loop workflow.

The same results were observed when using the surface marker to indicate the entry point for incisions, with the StraightTrack cannula system achieving a low rotation error of $3.35\pm 2.57$ °, as well as low start, mid, and endpoint errors of $8.68\pm 5.54$ mm, $6.39\pm 3.45$ mm, and $4.72\pm 1.79$ mm, respectively (see Table 2). However, the effect size in this initial study is not statistically significant when using the surface marker, as in Table 4. Similarly, when comparing similar tracking strategies with and without the surface marker, we observe an inconsistent effect on the final placement, with the surface marker sometimes improving final accuracy (as in the drill-tracked solution) and sometimes reducing it (as in the non-tracked and cannula-tracked solutions, including StraightTrack). In practice, providing the surface marker is valuable for estimating the incision point, but these results suggest it may not add value when assisting with the actual placement.

4 Discussion and Limitation

Addressing wire bending is crucial for bringing MR navigation systems into percutaneous procedures. Our experimental results strongly suggest that the cannula body design improves the accuracy of wire placement significantly. In particular, during the phantom study, we observed significant bending of the K-wire with non-tracked and drill-mounted marker methods when adjusting the wire within the soft tissue, as shown in Fig. 6. In contrast, the cannula body effectively reduces the deviation caused by irregular forces during drilling, thereby ensuring that the MR system’s visual guidance remains precise and reliable. Moreover, we observed that the bending issue caused the drill-mounted MR navigation to perform worse than the non-navigation method. We hypothesize that the bent wire misled the surgeon and resulted in unnecessary adjustments. This highlights the need to integrate both accurate navigation and physical stabilization to achieve precise wire placement, as improvement in only one aspect may not yield better results. Although the statistical significance is not robust enough for definitive conclusions, our results show trends that strongly suggest the benefits of the cannula body design. We plan to conduct larger-scale studies to validate these findings and further optimize the cannula body design, enhancing the reliability of percutaneous procedures.

Furthermore, results differentiating only navigation interfaces show that the introduction of surface markers actually decreased performance. This suggests that additional visual aids should be designed to avoid introducing new sources of error. We hypothesize that uncertainty introduced by the device’s depth sensor may lead surgeons to incorrectly identify the starting point on the bone. This uncertainty may arise from the Hololens depth sensor’s native accuracy in capturing the complex surface topography of soft tissue. Specifically, the ToF depth sensor exhibits inconsistent depth estimation influenced by lighting conditions and surface characteristics such as reflectivity, texture, color, and material [31, 32]. This incorrect surface marker positioning results in the misalignment of the initial insertion angle of the K-wire in the soft tissue, leading to greater overall errors in the wire trajectory.

Although only two expert orthopedic surgeons participated in the preliminary phantom study, our main objective was to investigate whether K-wire bending leads to poor outcomes and to assess the feasibility of our system in addressing this issue. The inclusion of highly experienced (>10 years) surgeons helps minimize variability due to human factors. Despite the limited number of trials for each method, there is a clear trend indicating that the cannula body provides stable performance. These insights are crucial for refining and developing the next version of our system to ensure accurate intraoperative wire placement.

5 Clinical Applicability

Seamless integration into clinical workflows is crucial for the successful adoption of any surgical navigation system. Our proposed system requires the use of a Dynamic Reference Body (DRB) with reflective spheres attached to the patient’s body. While the DRB is commonly used in surgical navigation for accurate real-time tracking [33, 34, 35], it presents challenges such as ensuring stable fixation, maintaining sterility, and minimizing procedural interference. Further investigation is needed to evaluate the practicality of DRB attachment across different surgical scenarios and explore alternative tracking methods to optimize safety and procedural efficiency.

The accuracy of the StraightTrack system, as demonstrated in our phantom studies, shows an average translation error of less than 5.26 mm and a mean rotational error of 2.88°. In the context of percutaneous pelvic trauma surgery, such as the fixation of superior pubic ramus fractures—a common injury often associated with pelvic ring damage—the average corridor width of the superior pubic ramus is reported to be 8.2 $\pm$ 1.8 mm [36]. Given this anatomical context, our system’s current accuracy is approaching, but not yet meeting, the precision required for such procedures. For more delicate orthopedic surgeries, particularly those involving proximity to critical anatomical structures, the existing error margins still require improvement. Consequently, future efforts will focus on enhancing the system’s precision to better align with the requirements of complex surgical interventions. Moreover, additional validation in more realistic settings, such as cadaver studies, is necessary to obtain clinically relevant data for our system.

As an initial prototype, our system provides a lightweight, cost-effective, and comprehensive MR solution for guiding orthopedic wire placement. The system is designed to be modular, scalable, and integrate with current technologies, including a range of AR headsets. Although our prototype was developed on the HoloLens platform, from an engineering perspective, it is compatible with other AR headsets that incorporate time-of-flight depth sensors, such as the Magic Leap 2. As technology progresses, ensuring the StraightTrack system’s adaptability across hardware platforms is crucial.

6 Conclusions and Future Work

In this paper, we introduced StraightTrack, an MR-based trajectory-preserving navigation system for K-wire placement. StraightTrack integrates spatial navigation using an OST-HMD and a hand-held cannula body. We established the end-to-end system navigation accuracy and conducted a preliminary phantom study with experienced orthopedic surgeons. The results indicate that our system effectively addresses the wire bending issue and achieves higher wire placement accuracy, with an average translation error of less than 5.26 mm and a mean rotational error of 2.88°.

For future work, we plan to conduct a comprehensive user study involving orthopedic surgeons with varying levels of expertise to further evaluate the system’s performance and gather subjective feedback for refining the design. Additionally, we plan to perform a cadaver study to validate the system’s effectiveness in a more realistic surgical setting. We will also fabricate the prototype with higher-quality machining and a smaller radius to enable less invasive approaches. Our goal is to conduct translational research to bring our system into clinical use.

7 Conflict of Interest Statement

The authors declared that they have no conflicts of interest

8 Data Availability Statement

The research data are not shared.

References

[1] D. D. Davis, L. A. Foris, S. M. Kane, and M. Waseem, “Pelvic Fracture,” in StatPearls [Internet], StatPearls Publishing, May 2023.
[2] K. H. Chong, T. DeCoster, T. Osler, and B. Robinson, “Pelvic fractures and mortality.,” Iowa Orthop. J., vol. 17, p. 110, 1997.
[3] T. Fahrenhorst-Jones, “Cannulated screws,” Radiopaedia, Apr. 2022.
[4] M. Sen, H. E. J., D. Steinitz, P. Guy, and R. Reindl, “Anatomical risks of using supra-acetabular screws in percutaneous internal fixation of the acetabulum and pelvis.,” Am. J. Orthop. (Belle Mead NJ), vol. 34, pp. 94–96, Feb. 2005.
[5] S. Wu, J. Chen, Y. Yang, W. Chen, R. Luo, and Y. Fang, “Minimally invasive internal fixation for unstable pelvic ring fractures: a retrospective study of 27 cases,” J. Orthop. Surg. Res., vol. 16, pp. 1–10, Dec. 2021.
[6] C. Khoo, A. Haseeb, and V. A. Singh, “Cannulated Screw Fixation For Femoral Neck Fractures :,” Malaysian Orthopaedic Journal, vol. 8, p. 14, July 2014.
[7] J. Y. Kim, G. M. Kong, D. H. Park, and D. Y. Kim, “Multiple cannulated screw fixation of young femoral neck fractures,” Pakistan Journal of Medical Sciences, vol. 31, p. 1517, Nov. 2015.
[8] K.-x. Yuan, F. Yang, K. Fu, D.-y. Zhu, C.-y. Jiang, D.-x. Jin, Z.-h. Wang, X.-y. Peng, Y.-S. Gao, and P.-b. Luo, “Internal fixation using fully threaded cannulated compression screws for fresh femoral neck fractures in adults,” J. Orthop. Surg. Res., vol. 17, pp. 1–8, Dec. 2022.
[9] S. Galal and M. Nagy, “Non-parallel screw fixation for femoral neck fractures in young adults,” Journal of Clinical Orthopaedics and Trauma, vol. 8, pp. 220–224, July 2017.
[10] C. A. Dickman, K. T. Foley, V. K. Sonntag, and M. M. Smith, “Cannulated screws for odontoid screw fixation and atlantoaxial transarticular screw fixation. Technical note,” J. Neurosurg., vol. 83, pp. 1095–1100, Dec. 1995.
[11] J. Feger, “Odontoid process fixation,” Radiopaedia, Jan. 2022.
[12] “Cannulated Screw System,” Feb. 2024. [Online; accessed 12. Feb. 2024].
[13] M. Unberath, J. Fotouhi, J. Hajek, A. Maier, G. Osgood, R. Taylor, M. Armand, and N. Navab, “Augmented reality-based feedback for technician-in-the-loop C-arm repositioning,” Healthcare Technol. Lett., vol. 5, pp. 143–147, Oct. 2018.
[14] S. Andress, A. Johnson, M. Unberath, A. F. Winkler, K. Yu, J. Fotouhi, S. Weidert, G. Osgood, and N. Navab, “On-the-fly augmented reality for orthopedic surgery using a multimodal fiducial,” J. Med. Imaging, vol. 5, Apr. 2018.
[15] B. D. Killeen, S. Chakraborty, G. Osgood, and M. Unberath, “Toward perception-based anticipation of cortical breach during K-wire fixation of the pelvis,” in Proceedings Volume 12031, Medical Imaging 2022: Physics of Medical Imaging, vol. 12031, pp. 410–415, SPIE, Apr. 2022.
[16] B. D. Killeen, H. Zhang, J. Mangulabnan, M. Armand, R. H. Taylor, G. Osgood, and M. Unberath, “Pelphix: Surgical Phase Recognition from X-Ray Images in Percutaneous Pelvic Fixation,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, pp. 133–143, Cham, Switzerland: Springer, Oct. 2023.
[17] B. D. Killeen, H. Zhang, L. J. Wang, Z. Liu, C. Kleinbeck, M. Rosen, R. H. Taylor, G. Osgood, and M. Unberath, “Stand in surgeon’s shoes: virtual reality cross-training to enhance teamwork in surgery,” Int. J. CARS, vol. 19, pp. 1213–1222, June 2024.
[18] T. De Silva, J. Punnoose, A. Uneri, M. Mahesh, J. Goerres, M. Jacobson, M. D. Ketcha, A. Manbachi, S. Vogt, G. Kleinszig, A. J. Khanna, J.-P. Wolinksy, J. H. Siewerdsen, and G. Osgood, “Virtual fluoroscopy for intraoperative C-arm positioning and radiation dose reduction,” J. Med. Imaging (Bellingham)., vol. 5, p. 015005., Jan. 2018.
[19] J. Fotouhi, M. Unberath, T. Song, W. Gu, A. Johnson, G. Osgood, M. Armand, and N. Navab, “Interactive Flying Frustums (IFFs): spatially aware surgical data visualization,” International journal of computer assisted radiology and surgery, vol. 14, p. 913, June 2019.
[20] B. D. Killeen, C. Gao, K. J. Oguine, S. Darcy, M. Armand, R. H. Taylor, G. Osgood, and M. Unberath, “An autonomous X-ray image acquisition and interpretation system for assisting percutaneous pelvic fracture fixation,” Int. J. CARS, vol. 18, pp. 1201–1208, July 2023.
[21] J. Wolf, D. Luchmann, Q. Lohmeyer, M. Farshad, P. Fürnstahl, and M. Meboldt, “The HoloLens in medicine: A systematic review and taxonomy,” Int J CARS, vol. 18, pp. 1363–1371, Aug. 2023.
[22] F. Liebmann, S. Roner, M. von Atzigen, D. Scaramuzza, R. Sutter, J. Snedeker, M. Farshad, and P. Fürnstahl, “Pedicle screw navigation using surface digitization on the Microsoft HoloLens,” Int J Comput Assist Radiol Surg, vol. 14, pp. 1157–1165, July 2019.
[23] F. Li, Q. Gao, N. Wang, N. Greene, T. Song, O. Dianat, and E. Azimi, “Mixed reality guided root canal therapy,” Healthcare Technol Lett, vol. 11, pp. 167–178, Apr. 2024.
[24] A. Condon, “The state of robotics in spine surgery today: 14 observations,” Feb. 2024. [Online; accessed 12. Feb. 2024].
[25] A. Martin-Gomez, A. Winkler, K. Yu, D. Roth, U. Eck, and N. Navab, “Augmented Mirrors,” in 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 09–13, IEEE.
[26] A. Martin-Gomez, H. Li, T. Song, S. Yang, G. Wang, H. Ding, N. Navab, Z. Zhao, and M. Armand, “Sttar: Surgical tool tracking using off-the-shelf augmented reality head-mounted displays,” IEEE Transactions on Visualization and Computer Graphics, pp. 1–16, 2023.
[27] A. Martin-Gomez, H. Li, T. Song, S. Yang, G. Wang, H. Ding, N. Navab, Z. Zhao, and M. Armand, “Sttar: surgical tool tracking using off-the-shelf augmented reality head-mounted displays,” IEEE Transactions on Visualization and Computer Graphics, 2023.
[28] polar kev, “MRTK2-Unity Developer Documentation - MRTK 2,” May 2024. [Online; accessed 25. Jun. 2024].
[29] B. Jiang, L. Wang, K. Xu, M. Hossbach, A. Demir, P. Rajan, R. H. Taylor, A. Moghekar, P. Foroughi, P. Kazanzides, and E. M. Boctor, “Wearable Mechatronic Ultrasound-Integrated AR Navigation System for Lumbar Puncture Guidance,” IEEE Transactions on Medical Robotics and Bionics, vol. 5, pp. 966–977, Sept. 2023.
[30] M. Kiel-Jamrozik, W. Jamrozik, M. Pawlik, and J. Goczyla, “Impact of 3D Printing Materials on Bone Phantom Features,” in Innovations in Biomedical Engineering, pp. 179–186, Cham, Switzerland: Springer, June 2022.
[31] W. Gu, K. Shah, J. Knopf, N. Navab, and M. Unberath, “Feasibility of image-based augmented reality guidance of total shoulder arthroplasty using microsoft HoloLens 1,” Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, May 2021.
[32] C. Gsaxner, J. Li, A. Pepe, Y. Jin, J. Kleesiek, D. Schmalstieg, and J. Egger, “The HoloLens in medicine: A systematic review and taxonomy,” Med. Image Anal., vol. 85:102757., Apr. 2023.
[33] J. Y. Cho, C. K. Chan, S.-H. Lee, and H.-Y. Lee, “The accuracy of 3D image navigation with a cutaneously fixed dynamic reference frame in minimally invasive transforaminal lumbar interbody fusion,” Comput Aided Surg, Nov. 2012.
[34] R. H. Taylor and D. Stoianovici, “Medical robotics in computer-integrated surgery,” IEEE Trans Robot Automat, vol. 19, pp. 765–781, Oct. 2003.
[35] R. H. Taylor, A. Menciassi, G. Fichtinger, P. Fiorini, and P. Dario, “Medical Robotics and Computer-Integrated Surgery,” in Springer Handbook of Robotics, pp. 1657–1684, Cham, Switzerland: Springer, July 2016.
[36] H. Altınayak and O. Balta, “Is percutaneous fixation of the superior pubic ramus possible in all types of pelvis?,” Turkish Journal of Trauma & Emergency Surgery, vol. 29, p. 419, Mar. 2023.