\hypersetup

hidelinks ¹¹institutetext: Medipixel Inc, Seoul, Republic of Korea
¹¹email: [email protected]

MPSeg : Multi-Phase strategy for coronary artery Segmentation

Jonghoe Ku 11 0009-0003-1260-619X Yong-Hee Lee 11 0000-0001-6047-701X Junsup Shin 11 0000-0003-3280-1622 In Kyu Lee 11 0000-0001-5554-808X Hyun-Woo Kim 11 * * 0009-0003-2740-0397 **

Abstract

Accurate segmentation of coronary arteries is a pivotal process in assessing cardiovascular diseases. However, the intricate structure of the cardiovascular system presents significant challenges for automatic segmentation, especially when utilizing methodologies like the SYNTAX Score, which relies extensively on detailed structural information for precise risk stratification. To address these difficulties and cater to this need, we present MPSeg, an innovative multi-phase strategy designed for coronary artery segmentation. Our approach specifically accommodates these structural complexities and adheres to the principles of the SYNTAX Score. Initially, our method segregates vessels into two categories based on their unique morphological characteristics: Left Coronary Artery (LCA) and Right Coronary Artery (RCA). Specialized ensemble models are then deployed for each category to execute the challenging segmentation task. Due to LCA’s higher complexity over RCA, a refinement model is utilized to scrutinize and correct initial class predictions on segmented areas. Notably, our approach demonstrated exceptional effectiveness when evaluated in the Automatic Region-based Coronary Artery Disease diagnostics using x-ray angiography imagEs (ARCADE) Segmentation Detection Algorithm challenge at MICCAI 2023.

Keywords:

Coronary Artery SYNTAX Score Segmentation Cardiac Angiography

1 Introduction

Coronary artery disease (CAD) is the leading cause of mortality worldwide [3, 11, 10, 19]. The diagnosis and treatment of CAD is important within the medical domain. CAD often exhibits intricate and diverse physiological aspects, including anatomical variance, multi-vessel diseases, and complex lesions, which should be considered for optimal treatment [18, 8].

There have been some attempts to systemically evaluate CAD due to its high complexity. In 1975, the American Heart Association suggested the reporting system for evaluating CAD [1]. In the reporting system, the coronary tree segments are defined, which is modified later for the ARTS (Arterial Revascularization Therapies Study) I and II trials [15, 12]. The modified definition is utilized for calculating the SYNTAX (SYNergy between PCI with TAXUS^TM and Cardiac Surgery) Score [13]. The SYNTAX Score is an evaluation system developed to assist the risk stratification of patients by integrating the validated angiographic classifications, including the morphology and location of CAD.

The structure of the coronary artery is complex and personalized. In general, the coronary artery consists of two main branches, LCA and RCA. LCA is divided into left anterior descending (LAD) and circumflex artery (LCx) after left main, then side branches extend from the main vessels, LAD, LCx, and RCA, covering the epicardial surface. In the real world, the clinical operator confronts various vasculature from normal variants, such as right or left dominance, to anomalies, such as separated origins of the LAD and LCx [16]. The heterogeneity and the complexity of the coronary vessel structure hinder the inexperienced operator from performing robust segmentation of vessel segments [13].

Moreover, manual segmentation of vessel segments is not only time-consuming but also susceptible to intra- and inter-operator variability. These factors can lead to inconsistent analysis results and pose challenges in achieving reliability and accuracy. Therefore, there exists a compelling need for the development of automatic segmentation approaches that can provide both efficiency and precision in the segmentation of vessel structures. Such automated methods have the potential to significantly enhance the speed and consistency of medical image analysis, ultimately benefiting both patients and clinical operators.

Refer to caption — Figure 1: Visualization of the entire pipeline in the proposed methodology.

In recent years, the rapid advancements in machine learning have significantly impacted various fields, notably in the domain of medical image processing [17]. Segmentation in medical image processing has garnered substantial attention in prior studies [9, 17, 2]. UNet [9], a well-known model in the domain, is renowned for its symmetric structure and utilization of skip connections, enabling effective learning of low and high-level features. Cho et al. [2] presented a multi-stage approach for segmenting breast tumors. The authors initially trained a binary classification model to detect the presence of a tumor and subsequently focused on segmenting the identified region. Also, the authors contended that this two-stage strategy significantly reduces false positives, and this strategy aligns with our approach.

We conducted a thorough investigation of the coronary artery segmentation dataset, specifically focusing on the challenges related to the SYNTAX Score segment. This analysis guided us in streamlining the intricacies of the task, resulting in a more manageable approach. Therefore, the proposed methodology strategically addresses the complexities through a well-defined multi-stage strategy.

In the coronary tree segmentation challenge, the proposed method showcased three distinctive features:

•

Data Analysis: Leveraging extensive exploratory data analysis on the ARCADE 2023 challenge dataset, we meticulously examined the intricacies of coronary artery segmentation in the context of the SYNTAX Score segment. This analysis was pivotal in simplifying the challenges associated with the task.
•

Multi-stage Strategy: Our proposed methodology is intentionally crafted to navigate and resolve the intricacies of the complication using a thoughtfully devised multi-stage strategy. This strategic approach contributes to the efficiency and efficacy of our solution.
•

Exceptional Efficacy: The proposed method demonstrated outstanding performance in the ARCADE segmentation task in the competition, underscoring its exceptional efficacy.

The proposed method encompasses three pivotal stages: Vessel classification, SYNTAX segmentation, SYNTAX classification. In the initial vessel classification stage, the image is subject to binary classification, distinguishing between RCA and LCA. Since the images manifest distinct characteristics between RCA and LCA but not within LCA (specifically LCx and LAD), the proposed method first classifies the vessels of interest as either belonging to RCA or LCA. After the classification, the vessels are segmented based on the SYNTAX Score segments. Two separate models are employed for classification output, one for RCA and another for LCA. In the case of classified as RCA, the predicted SYNTAX Score segment directly serves as the output, given the assumption that the image of RCA is relatively less complex than LCA views. On the other hand, for the images classified as LCA, the last stage SYNTAX classification is used to classify the final prediction of the segment by utilizing the original image and the mask derived from the segmentation stage. This classification is performed specifically within the masked region, determining the SYNTAX Score segment category accurately. Detailed methodologies for each stage are elaborated in section 2.

2 Method

In this section, we present an overview of the ARCADE challenge dataset, provide an analysis of the segmentation dataset, and describe the evaluation metrics utilized in the competition. Furthermore, we elaborate on the approaches employed for SYNTAX segmentation, including the models used. Subsequently, we delve into details related to training.

2.1 ARCADE Challenge Dataset

2.1.1 Dataset

The ARCADE challenge’s coronary tree segmentation dataset consists of 1000 training images, 200 validation images, and 300 test images. This dataset has 25 SYNTAX Score classes, with 8 classes assigned to RCA and 17 classes allocated to LCA.

Figure 2: Distribution of SYNTAX Score classes in ARCADE challenge vessel tree segmentation task

2.1.2 Exploratory Data Analysis

When we analyzed the segmentation dataset, the ratio of RCA to LCA vessel types was approximately 1:2. Furthermore, when examining the distribution of SYNTAX segment classes, a significant imbalance was identified between the main arteries and side branches. In Fig. 2, there were instances where the number of side branch segments was substantially small, and certain classes had only one case present in the training data. To address this imbalance, we aimed to train separate segmentation models for each type of vessel. Vessel classification was carried out as described in Algorithm 1. While this approach effectively classified RCA and LCA, the classification of LCx and LAD proved challenging due to the presence of mixed data from both vessels, making accurate classification unfeasible. Consequently, the final choice for vessel types to be used in the classification was narrowed down to RCA and LCA.

Input: List of SYNTAX Score index labels

Output: Coronary artery classification

2if segment label contains 1, 2, 3, 4, 16, 16a, 16b, 16c then

3 Coronary artery classification

\leftarrow

RCA;

5else

6 if segment label contains 11, 12, 13, 14, 14a, 14b, 15 then

7 Coronary artery classification

\leftarrow

LCX;

9 else

10 Coronary artery classification

\leftarrow

LAD;

12 end if

14 end if

Algorithm 1 Coronary Artery Classification based on SYNTAX Score Index

2.1.3 Evaluation

To evaluate SYNTAX segmentation performance, the challenge employed mean F1 score.

The F1 score is calculated by precision and recall.

F1=2\frac{precision\cdot recall}{precision+recall}

(1)

Precision and recall can be calculated using True Positive (TP), False Positive (FP) and False Negative (FN).

precision=\frac{TP}{TP+FP}

(2)

recall=\frac{TP}{TP+FN}

(3)

The F1 score is individually calculated for each class, considering only one segment at a time. When computing the F1 score, it is based on the classes present in the ground truth. If the model predicts a class that is not included in the ground truth, that prediction is disregarded in the evaluation process. To measure the mean F1 score, we aggregated all the individual F1 scores and divided this sum by the total number of segments for which F1 scores were computed.

meanF1=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{C_{i}}\sum_{j=1}^{C_{i}}F1_{ij}

(4)

$N$ and $C$ denote the total number of images and the number of segmented classes of each image, respectively.

2.2 Proposed Method

2.2.1 Data Augmentation

Accurate capturing of the SYNTAX segment necessitates an understanding of the spatial relationships within the entire vascular structure. To ensure the preservation of these relationships, we limit our data augmentations to basic transformations such as rotation and translation. This approach allows us to make minor adjustments while avoiding more drastic modifications like mosaic or flipping, which could potentially disrupt the delicate spatial information essential for accurate segmentation.

Furthermore, coronary images are acquired in grayscale with varying brightness distribution and noise characteristics depending on the imaging equipment and environmental conditions. To address these variations, we include augmentations such as adjusting brightness and blurring to ensure reliable performance across a range of imaging conditions encountered in practice.

2.2.2 Vessel classification

The morphology of the RCA and LCA exhibits distinct differences. Therefore, it is better to train a classification model to distinguish between these two types of vessels instead of trying to predict the entire SYNTAX segment in one step during SYNTAX segmentation.

To achieve this segmentation based on the classification of vessels, a preliminary step is taken by employing a vessel classification model. This model takes a coronary image as an input and categorizes it into either the RCA or LCA, as shown in Fig. 1.

2.2.3 SYNTAX segmentation

The input for SYNTAX segmentation is an image with dimensions $(512,512,1)$ . Models trained on RCA images predict masks of size $(512,512,8)$ , while models trained on LCA images predict masks of size $(512,512,17)$ . This class separation strategy is implemented to simplify the training process and mitigate the risk of misclassification. It ensures that RCA SYNTAX segments are not predicted for LCA vessels and vice versa.

In the case of LCA, there are complexities in imaging angles and vessel structures that can lead to misclassification of side SYNTAX segments as shown in Fig. 3. To overcome this issue, an additional classification model is used to adjust the class assignments for SYNTAX segments.

On the other hand, SYNTAX segments of RCA have clear distinctions between main and side segments, and the morphology of side segments is simpler than that of LCA’s side segments. This simplification in the classification process is facilitated by the inherent characteristics of the RCA, which makes it more amenable to direct segmentation without the need of additional classification refinement.

2.2.4 SYNTAX segment classification

We trained EfficientNet-b3[14], ResNet34[4], and DenseNet121[5] for SYNTAX segment classification. SYNTAX segment classification is exclusively conducted for LCA vessels, taking inputs in the form of coronary images and SYNTAX segment masks with dimensions $(512,512,2)$ . Its purpose is to predict the LCA SYNTAX segment classes based on the image concatenated with the provided mask.

2.3 Implementation details

The YOLOv8m model was trained for vessel classification using stochastic gradient descent (SGD) as the optimizer with a learning rate of 0.01. An ensemble approach was employed for the SYNTAX segmentation model, utilizing multiple models tailored to different vessel classes. For RCA, a UnetPlusPlus[20] model with a ResNet34 encoder was used, trained with 5-fold cross-validation using 800 training images and 200 validation images out of a total of 1000 training images. Among these, the top-performing three models were selected. Additionally, another UnetPlusPlus model with a ResNet50 encoder was used and trained on the entire set of 1000 training images.

For LCA, models were trained using a ResNet34 encoder with both UnetPlusPlus and Unet architectures, and a model with a ResNet50 encoder using UnetPlusPlus. Each of these models was trained using the AdamW[7] optimizer with a learning rate of 0.001 and focal loss[6].

Finally, we performed an ensemble of these trained models to obtain the ultimate SYNTAX segmentation results on a vessel-by-vessel basis. For the SYNTAX segment classification in LCA, EfficientNet-b3, ResNet34, and DenseNet121 models were trained, and the final class was determined using an ensemble approach. The mean F1 score for the final segmentation is shown in Table 1.

Table 1: Mean F1 score with and without SYNTAX classification.

Dataset	without SYNTAX classification	with SYNTAX classification
Validation	0.482	0.490
Test	0.441	0.444

3 Result

We evaluated the results of the validation and test dataset. As shown in Table 2, the vessel classification model achieved an exceptionally high accuracy. Upon inspecting misclassified data, we observed a tendency that our model makes incorrect predictions, when the guide wire was positioned in a location similar to the blood vessel. Such data were not abundant in the training dataset, and due to the resemblance in pixel value and appearance between the wire and the blood vessel, the model often misinterpreted the wire as a blood vessel, leading to classification failures.

Table 2: Accuracy of vessel classification in the validation dataset.

Model	Accuracy
YOLOv8m	$0.990$

In the case of SYNTAX segmentation, the ensemble model had a notable difference of 0.11 points compared to using a single model for predicting the validation data (Table 3). This observation underscores the significant advantage of ensemble predictions, where the average predictions from multiple models contribute to more robust detection, ultimately enhancing the accuracy and reliability of the segmentation results.

Table 3: Mean F1 scores of each vessel in the validation dataset.

Vessel type	Model	mean F1 score
RCA	Unet++(resnet50)	$0.592$
RCA	Unet++(resnet34)-1	$0.569$
RCA	Unet++(resnet34)-2	$0.518$
RCA	Unet++(resnet34)-3	$0.547$
RCA	Ensemble	$0.629$
LCA	Unet++(resnet50)	$0.386$
LCA	Unet(resnet34)	$0.385$
LCA	Unet++(resnet34)	$0.387$
LCA	Ensemble	$0.419$

For LCA, combining predictions from the SYNTAX segmentation with the SYNTAX segment classification model, which has an accuracy of 86%, as reflected in Table 4, did not lead to a significant improvement in the final mean F1 score. The decision to integrate the SYNTAX segment classification model was driven by the aim of enhancing the classification accuracy of the side branch. Unfortunately, the SYNTAX segmentation model did not generate predictions specifically for the side branch, leading to an outcome where the anticipated performance improvement was not achieved.

Table 4: Accuracy of SYNTAX segment classification in the validation dataset.

Model	Validation
EfficientNet-b3	$0.847$
ResNet34	$0.823$
DesnseNet121	$0.843$
Ensemble	$0.863$

When comparing the performance across different vessel types, the mean F1 score of RCA outperformed the score of LCA by 0.21 points based on the validation set. As previously mentioned, RCA had a simpler vascular structure and more consistent shooting angles in imaging, making it easier for the model to train and predict. In contrast, the segmentation of LCA poses unique challenges due to wide variations in the positions and complex structures of vessels, even within the same class segment. Additionally, in certain cases, labels were not accurately assigned to images with less complex vascular structures. These various factors collectively presented significant difficulties in training the model for LCA vessels, resulting in the observed performance gap between RCA and LCA.

In Fig. 6 and 5, most of the predicted masks are relatively smaller in size compared to the ground truth masks. While predictions are not made in areas without contrast agents, there is a tendency to make less accurate predictions in regions with contrast agents present. Interestingly, no overlapping regions between SYNTAX segments were found, but there are noticeable gaps or inaccuracies in delineating the boundaries between individual SYNTAX segments.

To address this problem, we conducted training a model that takes masked images of blood vessels as inputs to predict SYNTAX segmentation. When training the model to predict the SYNTAX segment of LCA using ground truth masks, we observed a relatively higher score of 0.61 on the validation dataset compared to the previous 0.42 score.

Subsequently, we attempted to train a segmentation model for vessel mask prediction by consolidating SYNTAX segment masks into a single representation as the complete vessel mask. Despite efforts to optimize, the model’s performance yielded a low intersection over union (IoU) of 0.45. This suboptimal outcome was primarily attributed to the absence of masks for the entire vasculature in the training data. This issue causes disruptions in the model training process. We anticipate that training the segmentation model with data that includes the entire vessel mask will lead to enhanced performance in LCA segmentation.

4 Conclusion

In this paper, we introduced a multi-stage methodology that takes into account the intricate structure of the cardiovascular system for conducting coronary tree segmentation based on the SYNTAX Score in coronary angiography. The proposed method initially classifies vessels into LCA and RCA, which possess morphologically distinct characteristics and hence cannot be grouped into identical classes. Following this classification, dedicated ensemble models for each LCA and RCA carry out the segmentation task. In the final step, considering the inherent complexity of LCA compared to RCA, a refinement model makes an additional round of class predictions for segmented areas. Our approach not only exhibits potential for application to various future segmentation models but also promises enhanced performance levels. Most notably, our approach has showcased exceptional performance and efficacy in the ARCADE-Segmentation Detection Algorithm.

References

[1] Austen, W.G., Edwards, J.E., Frye, R.L., Gensini, G., Gott, V.L., Griffith, L.S., McGoon, D.C., Murphy, M., Roe, B.B.: A reporting system on patients evaluated for coronary artery disease. report of the ad hoc committee for grading of coronary artery disease, council on cardiovascular surgery, american heart association. Circulation 51(4), 5–40 (1975)
[2] Cho, S.W., Baek, N.R., Park, K.R.: Deep learning-based multi-stage segmentation method using ultrasound images for breast cancer diagnosis. Journal of King Saud University-Computer and Information Sciences 34(10), 10273–10292 (2022)
[3] Dalen, J.E., Alpert, J.S., Goldberg, R.J., Weinstein, R.S.: The epidemic of the 20th century: coronary heart disease. The American journal of medicine 127(9), 807–812 (2014)
[4] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
[5] Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks pp. 4700–4708 (2017)
[6] Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection pp. 2980–2988 (2017)
[7] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
[8] Members, W.C., Virani, S.S., Newby, L.K., Arnold, S.V., Bittner, V., Brewer, L.C., Demeter, S.H., Dixon, D.L., Fearon, W.F., Hess, B., et al.: 2023 aha/acc/accp/aspc/nla/pcna guideline for the management of patients with chronic coronary disease: a report of the american heart association/american college of cardiology joint committee on clinical practice guidelines. Journal of the American College of Cardiology 82(9), 833–955 (2023)
[9] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. pp. 234–241. Springer (2015)
[10] Roth, G.A., Abate, D., Abate, K.H., Abay, S.M., Abbafati, C., Abbasi, N., Abbastabar, H., Abd-Allah, F., Abdela, J., Abdelalim, A., et al.: Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the global burden of disease study 2017. The Lancet 392(10159), 1736–1788 (2018)
[11] Sanchis-Gomar, F., Perez-Quilis, C., Leischik, R., Lucia, A.: Epidemiology of coronary heart disease and acute coronary syndrome. Annals of translational medicine 4(13) (2016)
[12] Serruys, P.W., Onuma, Y., Garg, S., Vranckx, P., De Bruyne, B., Morice, M.C., Colombo, A., Macaya, C., Richardt, G., Fajadet, J., et al.: 5-year clinical outcomes of the arts ii (arterial revascularization therapies study ii) of the sirolimus-eluting stent in the treatment of patients with multivessel de novo coronary artery lesions. Journal of the American College of Cardiology 55(11), 1093–1101 (2010)
[13] Sianos, G., Morel, M.A., Kappetein, A.P., Morice, M.C., Colombo, A., Dawkins, K., van den Brand, M., Van Dyck, N., Russell, M.E., Mohr, F.W., et al.: The syntax score: an angiographic tool grading the complexity of coronary artery disease. EuroIntervention 1(2), 219–227 (2005)
[14] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks pp. 6105–6114 (2019)
[15] Valgimigli, M., Serruys, P.W., Tsuchida, K., Vaina, S., Morel, M.A., van den Brand, M.J., Colombo, A., Morice, M.C., Dawkins, K., de Bruyne, B., et al.: Cyphering the complexity of coronary artery disease using the syntax score to predict clinical outcome in patients with three-vessel lumen obstruction undergoing percutaneous coronary intervention. The American journal of cardiology 99(8), 1072–1081 (2007)
[16] Villa, A.D., Sammut, E., Nair, A., Rajani, R., Bonamini, R., Chiribiri, A.: Coronary artery anomalies overview: The normal and the abnormal. World journal of radiology 8(6), 537 (2016)
[17] Wang, R., Lei, T., Cui, R., Zhang, B., Meng, H., Nandi, A.K.: Medical image segmentation using deep learning: A survey. IET Image Processing 16(5), 1243–1267 (2022)
[18] Wilensky, R.L., Selzer, F., Johnston, J., Laskey, W.K., Klugherz, B.D., Block, P., Cohen, H., Detre, K., Williams, D.O.: Relation of percutaneous coronary intervention of complex lesions to clinical outcomes (from the nhlbi dynamic registry). The American journal of cardiology 90(3), 216–221 (2002)
[19] World-Health-Organization: Cardiovascular diseases. https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds), accessed: 2023-09-28
[20] Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4. pp. 3–11. Springer (2018)