A solution for co-locating 2D histology images in 3D for histology-to-CT and MR image registration: closing the loop for bone sarcoma treatment planning

Bone sarcomas are some of the most challenging tumours to treat [3, 4]. Treatment and management of these tumours is less well understood because they are rare and heterogeneous, consisting of a variety of cell types, sub-populations, and tissue substructures [5]. They are over-represented in younger populations and, if not fatal, often impact mobility and quality of life [6, 4]. Orthopaedic surgery is indicated for all curative treatment along with chemo radiation in most cases [7, 8]. Complex procedures, such as hemipelvectomy, may be necessary when in the pelvis or other complex anatomy [9]. Despite inclusion of indicated multi-agent systemic chemotherapy, the reported five-year overall survival rates for the most common bone sarcoma have not improved much beyond 27 percent here [4, 10].

Resection margins aim to ensure full tumour removal. However, healthy tissue loss with these margins impacts patient outcomes. The recommended curative treatment is a wide resection, which aims to include 10 mm or 20 mm of healthy tissue around the removed tumour [7, 11]. Sometimes wider margins are taken due to factors like the patient’s risk profile, sarcoma sub-type or geometry of the surgical cuts. Unfortunately, damage to muscles, nerves and blood vessels resulting from healthy tissue removal impair post-surgical mobility and body function. These complications can burden the patient, family, community, and the health system [12].

Clinical teams face difficult decisions regarding resection margins. They must remove functional healthy tissue to ensure surgical success while also attempting to minimise harm from loss of the same healthy tissue [13, 14, 15, 16]. Unfortunately there is limited data to predict the surgical success of different resection options. This forces clinical teams to make important decisions with minimal quantitative guidance.

Patient-specific technology shows promise for reducing surgical margins. For example, computer-aided design (CAD) models help sarcoma treatment teams salvage healthy tissue, especially in areas with complex anatomy [17]. However, prospective studies face challenges in evaluating custom implants, cutting guides, surgical navigation against traditional orthopaedic benchmarks due to limited sample availability [18, 19, 3].

The need for more samples presents a challenge. Traditional orthopaedic oncology research often spans decades before confidently recommending new surgical techniques [20, 21, 7]. However, rapid evolution in salvage technology from the advent of 3D printers and better access to computers highlights the need for a solution to understand the inherent risk of different surgical techniques. It has been proposed that high-fidelity data be used to assess sarcoma treatment [22]. Unfortunately, the utilisation of such data is limited from constraints in current post-surgical analysis reporting [23, 24].

Medical image segmentation and 3D modelling (Figure 1) are common techniques used across almost all patient-specific technologies [25]. Using segmented CAD models to guide resection decisions is immensely valuable to clinical teams. However, this process places significant reliance on the engineers and machines performing segmentation, and the radiologists and surgeons who check the CAD model outputs. The increasing reliance on CAD models to determine resection margins introduces a limitation, as tumour segmentation remains subjective and difficult to quantify.

Some studies have evaluated the radiological presentation of sarcoma in one dimension using the clinical standard of histopathology, the diagnosis of disease by microscopic evaluation [26, 27, 28, 29]. While these radiologic evaluation techniques are clinically relevant, they only capture a fraction of information contained in histology. Furthermore, no histology registration studies have been found suitable for tissue of the pelvis or other complex, curved anatomy. Areas where resection margin assessment could greatly benefit patients.

Co-location of 2D histology images in 3D radiology volumes has proven to be effective for validating tissue boundaries and tumour margins for soft tissue [30]. Many novel techniques have been developed to co-register ex-vivo histology and in-vivo MR [31, 32, 33]. However, few published techniques are appropriate for bone, with the most notable requiring cryogenically frozen mice [34]. Cryogenesis is not always practical for larger clinical sarcoma.

In contrast, this paper outlines a solution to co-locate histology images and register them to medical image volumes for bone sarcoma. The proposed solution accommodates variations in shape, size, and tissue heterogeneity, and can be performed with minimal disruption to most clinical pathology workflows.

Histopathology is the clinical standard for tumor diagnosis and is used to guide post-surgical treatment [38, 39]. It employs histology techniques to assess tumour boundaries. Despite its relatively low cost, histology provides a substantial amount of high-fidelity data and is expected to remain a cornerstone of pathology [40, 39].

Histology techniques involve dissecting resected specimens, fixing them to prevent autolysis, and performing several immersion steps. Thin shavings from each dissection are then stained on glass slides and analysed under a microscope.

Histology literature describes how slides should quantitatively be measured and interpreted [41]. There are a number of classification systems which describe what tissue features and margins mean for post-surgical tumour treatment. The Residual (or R) classification in the Tumour, Node, Metastasis (TNM) system is a common metric used to check surgical success [23, 18].

Increasing digitisation provides very high-fidelity images for use in treatment but also processing beyond clinical workflows [40, 38]. Over the last three decades, successful co-location of microscopic histology and registration with macroscopic radiology volumes have been performed [30]. Histology can produce a discontinuous stack of images with an effective pixel size of 5 $\mu m$ by 5 $\mu m$ for 4 $\mu m$ thick slices of tissue, creating a volume with tiny voxels if shaved in parallel [42]. In contrast, MR has a typical voxel size of 0.4 x 0.4 x 4.0 $mm^{3}$-[42].

Unfortunately spatially localising human bone sarcoma histology to MR remains unattainable with current solutions [2].

This paper employs a novel pathology processing approach to assist histology image co-location within bone sarcoma radiology [2]. It can stand alone, or be added to the start of existing open source histology image co-location solutions [42].

Registration between in-vivo imaging modalities, such as CT and MR, is well established. And augmentation with histology can make high-fidelity information available at a low cost [30]. However, accurately co-locating 2D histology images in a 3D context within established clinical environments is challenging [39]. Prior research has highlighted three key requirements of any histology co-location solution: 1) to permit specimen slicing and dissecting (’cut-up’) in line with pathology protocols; 2) to provide co-location accuracy that is robust to the variation of tissue appearance on radiology and histology; and 3) to provide a quantitative evaluation of co-location error [32].

The collection of high-quality 3D bone histology datasets should prioritise the simple adaptation of existing pathology protocols before overhauling entire systems or requiring specialised equipment. It is noted that a) pathology protocols vary due to numerous factors, such as equipment availability and staffing levels. b) bone sarcoma is highly variable in shape and size. c) bone sarcoma is rare, making it crucial for the solution to be usable in as many sarcoma centers as possible. Minimising protocol change enhances resilience by leveraging the evolved knowledge and expertise within institutions.

While previous studies have explored histology co-location in the context of the prostate, this work focuses on bone anatomy and integration with the reference sarcoma workflow (Figure 1). Large bone specimens are slow to fix and require prior dissection to ensure proper penetration of fixative agents (e.g. formalin) [38, 41, 43]. Bone specimens need further demineralisation prior to microtoming and slide preparation. These practical limitations are believed to contribute to the lack of histology co-location work with clinical bone sarcoma.

Nonparallel tissue dissection is more relevant to clinical workflows than parallel [32, 44], especially for sarcoma. Our solution uses the bisection identified on ex-vivo CT to introduce a reference surface, restoring the 3D spatial orientation of nonparallel 2D digitised histology slides, and facilitating further image volume registration.

Any histology co-location solution should collect data to support both soft tissue and bone segmentation. Emerging orthopaedic surgical technology aims to minimise damage to healthy soft tissue [3]. Confidence in image characterisation will be immediately useful in making resection decisions around nerves and blood vessels.

Previous work found the presence of both bone and soft tissue in bone sarcoma specimens to be problematic [2]. However, the rigidity of bone has been utilised to maintain tissue structure during cut-up without need for external apparatus. Additionally, bone’s lower level of shrinkage compared to other tissues [22] presents an opportunity to support robust co-location.

Clinical adoption of methods for prostate histology co-location have been slow for challenges in physical processing, lack of easy access methods for data combination, and time constraints in clinical pipelines [42]. Prostate cancer groups first shared their methodology and solutions for histology co-location [31, 45, 32] which informed subsequent clinical data collection [46, 42, 44]. Their solutions have continued to evolve and grow in recent years [47]. Our work builds on the most promising methodology for bone sarcoma by Phillips et al.. We introduce a novel computational solution, supported by proof-of-concept ex-vivo animal data.

It is important to appreciate misalignment in histology co-location [1]. We tested this by comparing laboratory and computed measurements of dissected tissue size. Figure 3 illustrates how we collected dissection size measurements, as proxies for the relative positions of dissection cuts, to understand translation and rotation errors in plane assignment.

We paired euclidean distances between original spline intersections on successive planes ($d_{1,est}$ and $d_{3,est}$) with physical vernier caliper measurements ($d_{1,phy}$ and $d_{3,phy}$) from the laboratory. Similarly, we compared Euclidean distances along a third spline ($d_{2,est}$) that contoured the bone and fibrous tissue surface on $CT$ with a third set of caliper measurements (${d_{2},phy}$) for each bisection.

We evaluated plane assignment sensitivity by adjusting the user input. The extremes of spline placement on bone or fibrous tissue in $CT_{post,1\&2}$ was determined by expert interpretation. We compared planes from calculation of input spline extremes to understand how $P_{i}$ translation and rotation responded, which indicated the solution’s sensitivity.

Our study evaluated the accuracy of plane assignment by comparing computed measurements and physically dissected tissue size. Across 114 measurements on 38 dissections (over 6 bisections), the mean error difference was 0.19  mm ($\pm$ 1.8 mm). Additionally, rotational analysis showed a mean error difference of 0.25  mm ($\pm$ 1.9 mm) for $d_{2}$ measurements and 0.4  mm ($\pm$ 1.6 mm) for $d_{1}$ and $d_{3}$ measurements, based on 38 and 76 measurements, respectively.

Different user input indicated a maximum rotational plane assignment sensitivity of 1.6°, and mean translation sensitivity of 0.08 $\pm$ 0.2 mm (n = 114).

Histology images and annotations (Figure 4) have been co-located in two ex-vivo CT volumes and subsequently registered to $CT_{pre}$ and $MR_{pre}$ (Figure 5). In the proof-of-concept specimen, histology images are oriented roughly orthogonal to the humerus bone as shown in Figure 6.

Note that histology image orientation is restricted to $P_{i}$ which can take any orientation. This only depends on tissue processing decisions and measurement in pathology.

Some studies have reported that deep learning is a superior tool for histology co-location in certain tissues [47]. However, most existing initiatives focus on histology that has been sliced in parallel. When tissue cannot feasibly be sliced in parallel, as in the case of bone sarcoma, additional orientation information is required to enable co-location and registration. The use of ex-vivo CT and laboratory measurements can provide this necessary information. The solution described in this work is freely shared as a foundation for further research to advance human sarcoma histology co-location.

While digitisation of large-format (or whole-mount) histology is rare, mosaicing techniques only require standard-size histology images. A variety of software options for this purpose are listed by Pichat et al. [30]. With our solution, we stress that proper labeling of dissection orientation is essential, particularly if more standard-sized histology is to be processed. Surgical dye, often used in many surgical workflows, has proven useful in maintaining specimen orientation during proof-of-concept bisection and dissection.

The assignment of $P_{i}$ on $CT_{post,1\&2}$ volumes and use of clinical histology labels as collected for the proof-of-concept specimen will assist histology deformation. Deformations might employ techniques such as free-form deformations RAPSODI [42]. The authors do note that non-mineralised osteoid could not be easily demarcated from mineralised osteoid in proof-of-concept histology [57] due to necessary demineralisation in pathology processing. This is expected will introduce error if deformations assume all osteoid in bone histology labelmaps is radiodense.

Single energy CT’s reliance on density for tissue identification restricts it to including only mineralised osteoid in bone segmentation. Multi-energy CT may prove promising at differentiation between areas of non-mineralised osteoid and extraneous soft tissue. However, research focusing on this issue has not been found. An intermediary quantification of deformation error with any radiodense osteoid assumption would be worthy of future investigation.

Co-location of 2D clinical histology images within 3D radiology volumes will give physicians and researchers opportunity to contribute cohesive datasets of spatially characterised, microscopic tissue labelmaps to support radiology interpretation and surgical technology development. It will make high-fidelity tissue labelmaps of a clinical standard available for training segmentation algorithms at minimal additional cost. Integrating ex-vivo bisection CT and euclidean measurements while processing proof-of-concept tissue has been shown can do this.

Datasets of co-located histology and radiology will enhance the accuracy of volumetric segmentation by making histology insights available earlier in the bone sarcoma presentation-diagnosis-planning-surgery-recovery sequence. This will support margin decisions for surgical resection of bone sarcoma with evolving, patient-specific technologies.

Collection and use of animal tissue in this work did not affect animal treatment. The Otago University Ethics Committee confirmed that animal ethics was not required prior to the study commencing.

Written informed consent was obtained from legal animal owners for use of tissue and data in research and education.

The authors have no relevant financial or non-financial interests to disclose.