22institutetext: Learning Approaches for Vascular Analysis Group, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany
Segmentation of the Carotid Lumen and Vessel Wall using Deep Learning and Location Priors
Abstract
In this report we want to present our method and results for the Carotid Artery Vessel Wall Segmentation Challenge. We propose an image-based pipeline utilizing the U-Net architecture and location priors to solve the segmentation problem at hand.
Keywords:
Vessel Wall Segmentation Magnetic Resonance Imaging Deep Learning1 Introduction
One of the leading causes of death worldwide is atherosclerosis, a disease which is defined as the build-up of plaque deposits and formations narrowing the lumen, affecting medium and large-sized arteries [2, 4]. This is especially critical for the carotids, which supply the brain and face together with the vertebral arteries. Occlusion in those vessels can thus cause severe irreversible damage if not treated immediately. Therefore, the diagnosis of atherosclerotic vessels is of crucial importance. The vessel wall imaging (VWI) done with magnetic resonance imaging (MRI) is a modality designed to identify and localize stenosis of the atherosclerotic kind. In particular, the fast 3D carotid black blood MRI sequence (3D Motion Sensitized Driven Equilibrium prepared Rapid Gradient Echo, short 3D-MERGE) is designed to visualize blood vessels with sub-millimeter isotropic resolution and at the same time atherosclerotic malformations, their severity, size, and the overall stenosis [1]. Despite the good visibility using the 3D-MERGE sequence, expert knowledge is still required to read such an image. As this is a tedious and time consuming task, the Vessel-Wall-Segmentation Challenge hosted by the Vascular Imaging Laboratory from the University of Washington has been organized to foster and encourage automated methods addressing the segmentation of the outer and the inner (lumen) wall of the left and right external (ECAL, ECAR) and internal carotids (ICAL, ICAR). In this report, we present our approach for the automated vessel wall segmentation and discuss the achieved results.
2 Methods
The following section describes the methods we used to solve the segmentation task at hand. First, we propose a processing pipeline utilizing the U-Net [5] architecture which receives the MRI slices and returns the contours based on the segmentation. Furthermore, we describe all hyperparameters of the network and its training. Secondly, we describe certain characteristics in the application of said pipeline and introduce two concepts our pipeline follows to increase the segmentation accuracy.
2.1 Pipeline
The illustration in Fig. 1 shows both, training and testing phase of the proposed pipeline. During training, a 2D U-Net model receives a region of interest (RoI) of the input slices and predicts a binary segmentation mask of the carotids, the lumen, and wall in separate channels. The RoI extraction is described in Sec. 2.2. The internal and external carotid arteries were treated separately with two different models but the same architecture. All ground truth (GT) masks were given as contour formats in the CASCADE file format [3], thus it is necessary to convert the contours into a binary mask format, yielding the ground truth mask. This mask is used to supervise the architecture with the binary cross-entropy loss . During the inference and thus testing, the predicted binary mask is transferred back into a contour-based format in a post-processing step. This format is used for the final evaluation. As described later in Sec. 2.2, the model predicts both sides (left and right) separately.
The U-Net architecture, consisting of 4 downsampling and hence 4 upsampling steps, beginning with 64 channels and doubling channels up to the deepest stage of 1024 channels, was trained using Adam for 1500 epochs using a learning rate of with a batch size of 32. Max-pooling was used for the downsampling path and transposed Convolutions for the upsampling path.
2.2 Concepts
Contour-Mask Consistency
All ground truth masks were given as contours in the CASCADE file format [3]. As described in 2.1 the pipeline we propose is image-based, which means, the contours are predicted based on a binary segmentation of the internal and external carotid inner and outer walls. As the ground truth masks need to be converted to masks, and the predictions vice-versa into contours, it is hence of importance that both transforming routines are cycle consistent, explained exemplary in the following. Suppose some contour is given and the task is now to transform that contour into its mask representation using a function Contour2Mask. If this mask is now used to be transformed back into the contour representation using a function Mask2Contour returning contour , the Contour-Mask Consistency states that . To enforce this consistency contour points are forced to lie on the image grid, before converting them into binary masks.
Reduction to Region of Interest (RoI)
To counteract class imbalance, as it is inherent in the problem at hand, we propose to reduce the input image to the region where carotids are expected. This can be done for the left and right carotids respectively. To evaluate this region, the smallest bounding box was determined containing all ground truth segmentation of one side. This bounding box was additionally enlarged to ensure outliers in the test data are covered as well, to a final size of visualized as RoI in 1. The U-Net segments both bounding boxes, left and right, separately from each other. This has the advantage that either one side can be flipped always to the opposite side making left and right boxes visually look the same, since a symmetrical anatomy can be assumed, or, to augment and flip left and right randomly. Small experiments have shown the latter results in better validation performances.
3 Evaluation
The final evaluation was done based on 20 datasets with 720 voxels in height (x), width (y), and depth (z). Five datasets had 640 voxels in all dimensions instead. In total 25 datasets were therefore available. Due to technical issues, the initial submission was not robust against resolutions different to 7203 voxels, thus failed for five datasets in total 107 slices. This has been fixed and the evaluation restarted without retraining, which leads to the performance shown in Tab. 3.
| Metric |
|
|
||||
|---|---|---|---|---|---|---|
| Dice Score | 0.672±0.248 | 0.707±0.238 | ||||
| Lumen area difference | 0.142±0.239 | 0.110±0.208 | ||||
| Wall area difference | 0.162±0.224 | 0.139±0.205 | ||||
| Normalized wall index difference | 0.141±0.212 | 0.135±0.233 | ||||
|
1.095±4.813 | 1.058±4.705 | ||||
|
0.810±2.953 | 0.845±3.007 | ||||
| Quantitative score | 0.633±0.335 | 0.691±0.309 | ||||
| Number of unmatched slices | 371 | 264 |
4 Summary
For the Carotid Artery Vessel Wall Segmentation Challenge, we have proposed a U-Net-based pipeline that automatically performs a segmentation of the internal and external carotids on both sides based on 3D-MERGE MRI sequences. The network is supervised in the image domain, while the transfer from- and to contours is cycle consistent. To counteract class imbalances in the segmentation, the pipeline automatically, and dynamically extracts bounding boxes, at the regions where the carotids are expected. In the final test evaluation, our approach achieved a dice score of 0.707 and a quantitative measure of 0.691.
Author Contribution Statement
The authors confirm contribution to the work as follows. Conception and management: F.T., F.D., Manuscript draft: F.T., Manuscript review: F.D., L.R., C.V., A.M., Preparation and recording of presentations: F.T., Training of models: F.D., Pipeline implementation: F.D., L.R., F.T., F.K., C.V., Deployment: F.T., L.R., Evaluations: F.T., F.D., Various experiments: F.D., F.K., F.T., L.R., C.V.
References
- [1] Balu, N., Yarnykh, V.L., Chu, B., Wang, J., Hatsukami, T., Yuan, C.: Carotid plaque assessment using fast 3d isotropic resolution black-blood mri. Magnetic resonance in medicine 65(3), 627–637 (2011)
- [2] Herrington, W., Lacey, B., Sherliker, P., Armitage, J., Lewington, S.: Epidemiology of atherosclerosis and the potential to reduce the global burden of atherothrombotic disease. Circulation research 118(4), 535–546 (2016)
- [3] Kerwin, W., Xu, D., Liu, F., Saam, T., Underhill, H., Takaya, N., Chu, B., Hatsukami, T., Yuan, C.: Magnetic resonance imaging of carotid atherosclerosis: plaque analysis. Topics in Magnetic Resonance Imaging 18(5), 371–378 (2007)
- [4] Mitrovska, S., Jovanova, S., Matthiesen, I., Libermans, C.: Atherosclerosis: Understanding pathogenesis and challenge for treatment (2009)
- [5] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)