¹¹institutetext: Cornell University
¹¹email: [email protected] ²²institutetext: Weill Cornell Medical College ³³institutetext: University of Pennsylvania

NeRD: Neural Representation of Distribution for Medical Image Segmentation

Hang Zhang 1122 Rongguang Wang 33 Jinwei Zhang 1122 Chao Li 1122 Gufeng Yang 11 Pascal Spincemaille 22 Thanh D. Nguyen 22 Yi Wang 1122

Abstract

We introduce Neural Representation of Distribution (NeRD) technique, a module for convolutional neural networks (CNNs) that can estimate the feature distribution by optimizing an underlying function mapping image coordinates to the feature distribution. Using NeRD, we propose an end-to-end deep learning model for medical image segmentation that can compensate the negative impact of feature distribution shifting issue caused by commonly used network operations such as padding and pooling. An implicit function is used to represent the parameter space of the feature distribution by querying the image coordinate. With NeRD, the impact of issues such as over-segmenting and missing have been reduced, and experimental results on the challenging white matter lesion segmentation and left atrial segmentation verify the effectiveness of the proposed method. The code is available via https://github.com/tinymilky/NeRD.

Keywords:

Image Segmentation Neural Representation Convolutional Neural Networks

1 Introduction

Deep convolutional neural networks (CNNs) have been the dominant approach across various tasks of computer vision and image processing. The efficient convolutional operation with spatially-invariant filters is one of the main factors for the success of CNNs. The input feature map to the convolutional layers shares these filters across all spatial positions, thereby reducing network parameters and improving the generalization ability. Many medical image applications have benefited from this, for example, multiple sclerosis (MS) lesion segmentation [28, 27], white matter (WM) lesion segmentation [16], and quantitative susceptibility mapping [30, 31].

However, we still observe severe failure cases when applying the widely used U-Net [23] to brain lesion segmentation. E.g., We can see from Fig. 4 and Fig. 4 that brain lesions close to the brain boundary (meaning that they are close to the image boundary) or close to the ventricles (meaning that they are close to the center of the image) are prone to be misclassified by the U-Net. Analyzing the network architecture, we summarize the causes as follows: 1) the padding operation in convolutional layers can lead to artefacts in feature maps [18, 1] (see Fig. 2), 2) and can shift the feature distribution across different spatial positions [10]; 3) the down-sampling operations such as max-pooling and strided convolution ignore the basic sampling theorem, resulting in breaking the property of spatial invariance for the segmentation task [32, 12].

Refer to caption — Figure 1: Visualization of a failure case on brain lesion segmentation. Green boxes indicate regions of failure. (A) An T2-FLAIR image example of a patient with heavy lesion burden. (B) Lesions labeled by a human expert (marked in red). (C) Segmentation result of U-Net. (D) Segmentation result of our proposed CIF-based network.

Various methods, such as cube padding [5], circular convolution [24], explicit boundary-based filters [9], and the max-blurring-pooling [32] have been investigated by researchers to tackle the above issues. All of these methods can lessen the negative impact in some degree, but most of these methods are ad-hoc and none of them can handle all three issues collectively. Therefore, we argue that it is imperative to develop a unified framework to solve the problem to facilitate our medical image segmentation.

The key problem is the feature distribution shifting caused by commonly used network operations such as padding and pooling. Usually, the deep network stacks many convolutional layers; thus, with the network depth increases, the consecutive operations of padding and pooling can gradually shift the feature distribution from the boundary to the center of the image. Suppose $\Omega\subset\mathbb{N}^{2}$ is the spatial domain of an input image, $\mathbf{v}=(i,j)\in\Omega$ is the spatial position vector, the final feature map before the pixel-wise classifier is $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$ , $\mathbf{x}_{\mathbf{v}}\in\mathbb{R}^{C}$ is the feature vector in position $\mathbf{v}$ , and $\phi(\theta)$ is the feature vector distribution, then we can define the following mapping function:

	$\displaystyle\theta=f(\mathbf{v}),$		(1)
	$\displaystyle\mathbf{X}_{\mathbf{v}}\sim\phi(\theta),$		(2)

where $\theta$ is the parameter of the distribution $\phi$ . Basically, the equation tells us the distribution of the feature is determined by its spatial location. If convolutional neural network is strictly spatially invariant, the $\theta$ should be equivalent for features in all locations. However, as we mentioned above, certain operations in the network can alter the distribution, resulting in varying $\theta$ depending on its location.

Current segmentation network assumes that $\theta$ is equivalent for all feature vectors, which brings trouble for our brain lesion segmentation. Thus, in this paper, we propose a Neural Representation of Distribution (NeRD) technique to approximate the mapping function $f$ in Eq. (1) to resolve the issue. The idea of NeRD is inspired by a recently developed neural implicit representation (NIR) technique [21, 19], where the NIR parameterizes a signal as a continuous function that maps the domain such as the coordinate of the signal to pixel values at that coordinate. In our study, we map the coordinates to the feature distributions. In summary:

•

We propose a Neural Representation of Distribution (NeRD) technique to bring back the spatial invariance by approximating the feature distribution based on pixel coordinates.
•

We validate the proposed method on two challenging medical image segmentation tasks, where both quantitative and qualitative results demonstrate the effectiveness of our method.

1.1 Related Works

1.1.1 Brain Lesion Segmentation

MS lesion segmentation and WM lesion segmentation are most important and difficult tasks in brain lesion analysis, as these lesions vary greatly in terms of shape, size and location. Though numerous automated approaches have been proposed, a clinically reliable technique is not yet available. The 2.5D stacked denseNet [29] is proposed to capture broader brain structure information. The folded attention network [28] applies light-weight self-attention method for richer contextual information. The geometric loss [27] is developed to regularize CNN training, which helps segmenting small lesions. The boundary loss [13] uses distance transformation mapping to tackle the data imbalance problem. All these methods have achieved reasonably good result, but bone of them takes feature distribution shifting into consideration.

1.1.2 Neural Implicit Representation

NIR is a recently developed technique, which is frequently used in representation of geometry and appearance [21, 19] in graphics. DeepSDF [21] Learns a set of continuous signed distance functions for shape representation. Later, NeRF [19] provides a more flexible way for synthesizing novel views of complex scenes. Other vision applications such as image super resolution [4], image synthesis [2] also benefit from the NIR technique. Though the NIR is blooming in many areas, we haven’t seen any method using NIR for feature distribution mapping.

1.1.3 Meta Learning

We use the proposed NeRD technique to predict network weights based on image coordinates, the process of which is one of the meta learning strategies [17]. The weights of certain modules of the network are predicted by another network module instead of directly learned. [20] proposes a dynamic parameter layer for image question answering. [7] uses box weights to predict mask weights for image segmentation. [8] achieves super-resolution of arbitrary scale factor by predicting weights of up-sampling module based on the scale factor. In our work, we use image coordinates coupled with training data to dynamically estimate the feature distribution.

2 Methodology

Recently, NIR techniques have been investigated to model continuous 3D shapes as level sets, which can efficiently map coordinates to signed distance function [11, 21] for shape reconstruction. In this work, our goal is to map the image coordinates to the feature distribution to resolve the spatial invariance issue brought by basic operations such as padding and pooling.

2.1 Neural Representation of Distribution

The practical implementation of a segmentation network consists of two parts, a encoder-decoder structure for feature extraction and a multi-layer perceptron (MLP) for pixel-wise classification. Let $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$ ( $H$ and $W$ are the spatial size of the image, and $C$ is the number of channels) be the output from the encoder-decoder structure, $\mathbf{v}=(d_{t},d_{r},d_{b},d_{l})$ be the position vector for a pixel ( $d_{t},d_{r},d_{b}$ , and $d_{l}$ indicates the distance of the pixel to the top, the right, the bottom and the left of the image), we can use the Gaussian distribution to approximate the feature distribution of a given position $\mathbf{v}$ as follows:

\mathbf{x}_{\mathbf{v}}\sim\mathcal{N}(\mathbf{\mu},\mathbf{\Sigma}),

(3)

where $\mathbf{\mu}$ is the mean vector in location $\mathbf{v}$ , and $\mathbf{\Sigma}$ is the corresponding co-variance matrix. Since the spatial invariance of CNNs has been broken by certain operations in the network, one of the MLP assumptions that the data is drawn from the same distribution no longer holds. It is expected that a simple MLP classifier is prone to fail in classifying pixels close to the boundary or the center of the image, as there exists an non-trivial discrepancy between their feature distributions (as is verified in our experiments, see Fig. 4 and Fig. 5). In this work, we use the proposed NeRD technique to resolve the issue.

We represent a continuous parameter space of the distribution function as a 4D vector-valued function, whose input is a 4D position vector $\mathbf{v}=(d_{t},d_{r},d_{b},d_{l})$ , and whose output is the mean vector $\mathbf{\mu}$ and the co-variance matrix $\mathbf{\Sigma}$ of the distribution of this position. In practice, we use another MLP $f$ to approximate this continuous 4D representation of the distribution, and optimize the weight $\mathbf{\Theta}$ of $f$ along with other network weights to map each input position vector to the corresponding mean vector and co-variance matrix. It has been shown that CNNs can linearize [3, 25] the manifold of images into an Euclidean subspace of deep features, which indicate that elements in the output feature vector are independent to each other. Thus, we further simplify the model and reduce the co-variance matrix to a vector $\mathbf{\sigma}$ . With the estimation of $\mathbf{\mu}$ and $\mathbf{\sigma}$ from the function $f$ , we can normalize and classify the output feature vector at position $\mathbf{v}$ as follows:

s_{\mathbf{v}}=\mathbf{w}^{\top}\dfrac{\mathbf{x}_{\mathbf{v}}-\mathbf{\mu}}{\mathbf{\sigma}},

(4)

where $\mathbf{\mu}$ and $\mathbf{\sigma}$ are obtained with the MLP $f_{\mathbf{\Theta}}:(\mathbf{v})\rightarrow(\mathbf{\mu},\mathbf{\sigma})$ , $s_{\mathbf{v}}$ is the final output of the network, $\mathbf{w}$ is the weight of the MLP classifier (Please note that as we use z-score [33] to normalize the input image, there is no bias term for the classifier). To be numerically stable, in practice, we estimate $1/\mathbf{\sigma}$ and $-\mu/\mathbf{\sigma}$ instead, and the final equation for normalization and classification can be described as:

s_{\mathbf{v}}=\mathbf{w}^{\top}(\mathbf{x}_{\mathbf{v}}\dfrac{1}{\mathbf{\sigma}}+(-\dfrac{\mathbf{\mu}}{\mathbf{\sigma}}))

(5)

2.2 Pixel-aligned Classifier and The Overall Framework

The overall framework of our proposed method is shown in Fig. 3. The input image goes through a U-Net to obtain the final feature map for pixel-wise classification, and in the meantime, the offset generator provides the pixel-wise position vectors, followed by an MLP as the distribution calibrator to generate the estimation of $1/\mathbf{\sigma}$ and $-\mu/\mathbf{\sigma}$ . With the distribution estimation of every pixel position, we apply Eqn. (5) to normalize and classify the feature vector of each position. The final segmentation can be obtained with another MLP and a Sigmoid function. We call the framework using the proposed feature estimation technique as NeRD.

We wouldn’t say NeRD a concrete method but an idea that can improve the performance of a CNN network by estimating the pixel-wise feature distribution. Thus, it can be expected that there are many variants using the NeRD idea. Here, we describe one that we deem as interesting. Rather than estimating the parameters of the feature distribution, we estimate a pixel-aligned classifier. That is to say, we estimate a unique linear classifier for every single pixel position, which can be described as $s_{\mathbf{v}}=\mathbf{x}_{\mathbf{v}}^{\top}\mathbf{w}_{\mathbf{v}}$ , where the $\mathbf{w}_{\mathbf{v}}$ is the weight of a linear classifier estimated by an MLP $f_{\mathbf{\Theta}}:(\mathbf{v})\rightarrow(\mathbf{w}_{\mathbf{v}})$ . We call this NeRD classifier as NeRDc and the former mean-variance estimator as NeRDm. Both NeRDc and NeRDm can be efficiently implemented using tensor operations in the modern GPU architecture.

3 Experimental Results

3.1 Datasets

White matter hyperintensities¹¹1https://wmh.isi.uu.nl (WMH) [15] is a publicly available dataset which contains 60 3D scans with 2 modalities (T1 and FLAIR weighted) acquired from multiple vendors and scanners in three different institutes. The spatial resolution goes from $0.95\times 0.95\times 3\ mm^{3}$ to $1.21\times 1\times 3\ mm^{3}$ for each volume. Manual annotations of WMH are provided for the 60 scans. In the experiments, we split this dataset into training, validation, and testing sets containing 42, 6 and 12 samples, respectively.

We also use left atrial²²2http://atriaseg2018.cardiacatlas.org (LA) segmentation challenge [26] dataset for evaluation. A total of 154 independently acquired 3D LGE-MRIs from 60 deidentified patients with atrial fibrillation were used in this challenge. The clinical images were acquired with either a 1.5T Avanto or 3.0T Verio whole-body scanner. The spatial resolution of one 3D LGE-MRI scan was $0.625\times 0.625\times 0.625\ mm^{3}$ with spatial dimensions of either $576\times 576\times 88$ or $640\times 640\times 88$ pixels. From the whole set, 108 scans were used for training, 15 for validation, and the remaining 31 for testing.

3.2 Implementation Details

3.2.1 Data pre-processing.

We slice the original images into a stack of independent 2D images. Each scan is center-cropped to size $160\times 224$ for WMH and size $290\times 240$ pixels for LA, and normalized to real values between 0 and 1. Since two modalities (T1 and FLAIR) are available for WMH, both of them are concatenated along the channel dimension before being used as input to the network.

3.2.2 Netowrk and training.

We employ U-Net [23] as the backbone architecture in our experiments. To train our model, we use Adam [14] optimizer, with an initial learning rate of $1\text{e}-3$ (weight decay of $1\text{e}-6$ ) and a batch size equal to 14. The learning rate is halved at 50%, 70% and 90% of the total training epoch (90) for optimal convergence. We use PyTorch [22] for implementation, and run the experiments on a machine equipped with an NVIDIA RTX 2080 Ti GPU with 11GBs of memory.

3.2.3 Evaluation metrics.

To quantify the performance of WMH segmentation, we use Dice similarity coefficient [6], lesion-wise Dice (LDice), lesion-wise true positive rate (LTPR), and lesion-wise positive pre-dictive value (LPPV) as metrics. LDice, LTPR and LPPV are defined as $\text{LDice}=\frac{\text{TPR}}{\text{GL}+\text{PL}}$ , $\text{LTPR}=\frac{\text{TPR}}{\text{GL}}$ , and $\text{LPPV}=\frac{\text{TPR}}{\text{PL}}$ , where TPR denotes the number of lesions in ground-truth segmentation that overlap with a lesion in the produced segmentation, and GL, PL is the number of lesions in ground-truth segmentation and produced segmentation respectively. Dice quantifies the voxel-wise overlap between the output and the ground-truth. Complementarily, LDice, LTPR and LPPV are more sensitive in measuring the lesion-wise detection accuracy. As for LA segmentation, two region-based metrics, Dice and Jaccard, are used to measure the region mismatch. Three boundary-based metrics, average surface distance (ASD), Hausdorff distance (HD), and 95% Hausdorff distance (95HD), are used to evaluate errors in the boundary.

3.3 WMH Segmentation

Table 1: Quantitative comparison with average (standard deviation over three independent runs) on white matter hyperintensities (WMH) segmentation.

Model	Filter	Dice (%) $\uparrow$	LDice (%) $\uparrow$	LFPR (%) $\downarrow$	LTPR (%) $\uparrow$
U-Net	256	76.7 (1.96)	66.9 (5.37)	30.8 (4.95)	73.9 (4.15)
NeRDc	256	78.4 (0.41)	69.7 (1.84)	27.8 (3.52)	72.2 (0.94)
NeRDm	256	77.4 (0.71)	66.9 (2.82)	33.5 (3.71)	73.9 (0.86)
U-Net	512	78.4 (0.82)	69.8 (0.55)	29.2 (3.14)	76.2 (3.07)
NeRDc	512	79.2 (0.05)	72.0 (1.07)	27.1 (0.44)	77.7 (1.83)
NeRDm	512	78.6 (0.37)	72.3 (0.32)	27.7 (1.11)	77.6 (2.43)

3.3.1 Quantitative results.

We compare our proposed NeRDc and NeRDm methods with the baseline U-Net [23] on lesion-specific metrics as shown in Table 1. We use two variants of U-Net backbone containing different set of kernel channel numbers that representing low- and high-capacity networks separately. For the low-capacity network (denoted as 256 under “filter” column in the table), we use [16, 32, 64, 128, 256] as the size for convolution filters in different layers, and we double the channel number for the high-capacity network (denoted as 512). In the low-capacity group, we observe that NeRDc outperforms both U-Net and NeRDm in most metrics, and in particular, there’s 2.8% increment for LDice and 3% reduction in LFPR compared to U-Net. NeRDc also shows substantial improvements over Dice (0.8%), LFPR (2.1%), and LTPR (1.5%) in the high-capacity group compared to U-Net. Importantly, our proposed NeRDc using low-capacity backbone achieved similar performance compared to the U-Net with high-capacity backbone. For example, the Dice score of NeRDc in low-capacity group is the same as the U-Net in high-capacity group.

3.3.2 Qualitative results.

As shown in Fig. 4, we present WMH segmentation results with a missing case and a over-segmenting case made by vanilla U-Net. We can see from the first row of Fig. 4, U-Net missed a lesion close to the ventricle/center of the image, while our NeRDm accurately located this lesion, which demonstrated the superiority of our feature distribution estimation technique. Though the end of cortex (yellow box position) exhibited similar high intensity value as lesions, and was close to the boundary, both our NeRDc and NeRDm made no mistakes in this region, while U-Net over-segmented several pixels.

3.4 LA Segmentation

Table 2: Quantitative comparison with average (standard deviation over three independent runs) on left atrial (LA) segmentation.

Model	Filter	Dice (%) $\uparrow$	Jaccard (%) $\uparrow$	HD $\downarrow$	95HD $\downarrow$	ASD $\downarrow$
U-Net	256	90.3 (0.51)	82.5 (0.81)	34.1 (3.09)	6.5 (0.52)	2.1 (0.13)
NeRDc	256	90.6 (0.26)	82.9 (0.39)	32.1 (3.55)	6.4 (0.55)	2.1 (0.12)
NeRDm	256	90.5 (0.16)	82.8 (0.25)	29.6 (2.62)	6.3 (0.03)	2.1 (0.12)
U-Net	512	90.1 (0.14)	82.2 (0.24)	34.3 (0.36)	6.9 (0.10)	2.3 (0.08)
NeRDc	512	90.7 (0.14)	83.1 (0.23)	30.6 (0.67)	6.3 (0.21)	2.1 (0.07)
NeRDm	512	90.3 (0.21)	82.5 (0.37)	35.2 (2.49)	6.5 (0.31)	2.2 (0.06)

3.4.1 Quantitative results.

We report the LA segmentation results of the baseline U-Net [23], NeRDc, and NeRDm in Table 2. We evaluated the performance of each model using a set of boundary-based metrics, such as Hausdorff distance (HD) and average surface distance (ASD). Similar to WMH segmentation, we employ CNN backbone with both low- and high-capacity for detailed performance investigation. In the low-capacity group, we can observe that both NeRDc and NeRDm show improved performance on all metrics compared to U-Net. Especially, NeRDm achieved 4.5 reduction in HD with reduced variance. On the other hand, NeRDc outperforms both U-Net and NeRDm on all metrics in the high-capacity group with a significant margin in HD. Notably, NeRDc in the low-capacity group showed substantial improvement over U-Net in the high-capacity group on all metrics. This phenomena were observed in both WMH and LA segmentation, which indicates that even with fewer network parameters and lower computational resource requirement, our proposed NeRD can achieve comparable or better performance than the counterpart without NeRD module consistently.

3.4.2 Qualitative results.

Fig. 5 shows two examples of LA segmentation results. As can be seen in the first row of the figure, similar to WHM segmentation, U-Net equipped with the NeRD module didn’t miss the pixels close to the boundary, while the U-Net did. Similarly, our proposed methods didn’t over-segment the areas close to the center of the image, while the U-Net did.

4 Conclusions

We presented a novel neural representation learning technique (NeRD) to estimate the pixel-wise feature distribution. We instantiated two variants of the NeRD, NeRDm for estimating the mean and variance of the feature distribution and NeRDc for estimating pixel-wise linear classifier. Both variants showed performance improvement over the counterpart without NeRD modules on two challenging medical image segmentation tasks, WHM segmentation that contains multiple lesions spanning the whole brain and LA segmentation that contains a single large object. We believe that the proposed NeRD technique can contribute to more medical image applications.

References

[1] Alsallakh, B., Kokhlikyan, N., Miglani, V., Yuan, J., Reblitz-Richardson, O.: Mind the pad–cnns can develop blind spots. arXiv preprint arXiv:2010.02178 (2020)
[2] Anokhin, I., Demochkin, K., Khakhulin, T., Sterkin, G., Lempitsky, V., Korzhenkov, D.: Image generators with conditionally-independent pixel synthesis. arXiv preprint arXiv:2011.13775 (2020)
[3] Bengio, Y., Mesnil, G., Dauphin, Y., Rifai, S.: Better mixing via deep representations. In: International conference on machine learning. pp. 552–560. PMLR (2013)
[4] Chen, Y., Liu, S., Wang, X.: Learning continuous image representation with local implicit image function. arXiv preprint arXiv:2012.09161 (2020)
[5] Cheng, H.T., Chao, C.H., Dong, J.D., Wen, H.K., Liu, T.L., Sun, M.: Cube padding for weakly-supervised saliency prediction in 360 videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1420–1429 (2018)
[6] Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)
[7] Hu, R., Dollár, P., He, K., Darrell, T., Girshick, R.: Learning to segment every thing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4233–4241 (2018)
[8] Hu, X., Mu, H., Zhang, X., Wang, Z., Tan, T., Sun, J.: Meta-sr: A magnification-arbitrary network for super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1575–1584 (2019)
[9] Innamorati, C., Ritschel, T., Weyrich, T., Mitra, N.J.: Learning on the edge: Investigating boundary filters in cnns. International Journal of Computer Vision pp. 1–10 (2019)
[10] Islam, M.A., Jia, S., Bruce, N.D.: How much position information do convolutional neural networks encode? In: International Conference on Learning Representations (2019)
[11] Jiang, C., Sud, A., Makadia, A., Huang, J., Nießner, M., Funkhouser, T., et al.: Local implicit grid representations for 3d scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6001–6010 (2020)
[12] Kayhan, O.S., Gemert, J.C.v.: On translation invariance in cnns: Convolutional layers can exploit absolute spatial location. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14274–14285 (2020)
[13] Kervadec, H., Bouchtiba, J., Desrosiers, C., Granger, E., Dolz, J., Ayed, I.B.: Boundary loss for highly unbalanced segmentation. In: International conference on medical imaging with deep learning. pp. 285–296. PMLR (2019)
[14] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
[15] Kuijf, H.J., Biesbroek, J.M., De Bresser, J., Heinen, R., Andermatt, S., Bento, M., Berseth, M., Belyaev, M., Cardoso, M.J., Casamitjana, A., et al.: Standardized assessment of automatic segmentation of white matter hyperintensities and results of the wmh segmentation challenge. IEEE transactions on medical imaging 38(11), 2556–2568 (2019)
[16] La Rosa, F., Abdulkadir, A., Fartaria, M.J., Rahmanzadeh, R., Lu, P.J., Galbusera, R., Barakovic, M., Thiran, J.P., Granziera, C., Cuadra, M.B.: Multiple sclerosis cortical and wm lesion segmentation at 3t mri: a deep learning method based on flair and mp2rage. NeuroImage: Clinical 27, 102335 (2020)
[17] Lemke, C., Budka, M., Gabrys, B.: Metalearning: a survey of trends and technologies. Artificial intelligence review 44(1), 117–130 (2015)
[18] Liu, R., Jia, J.: Reducing boundary artifacts in image deconvolution. In: 2008 15th IEEE International Conference on Image Processing. pp. 505–508. IEEE (2008)
[19] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. arXiv preprint arXiv:2003.08934 (2020)
[20] Noh, H., Hongsuck Seo, P., Han, B.: Image question answering using convolutional neural network with dynamic parameter prediction. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 30–38 (2016)
[21] Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: Deepsdf: Learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 165–174 (2019)
[22] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems. pp. 8024–8035 (2019)
[23] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)
[24] Schubert, S., Neubert, P., Pöschmann, J., Pretzel, P.: Circular convolutional neural networks for panoramic images and laser data. In: 2019 IEEE Intelligent Vehicles Symposium (IV). pp. 653–660. IEEE (2019)
[25] Upchurch, P., Gardner, J., Pleiss, G., Pless, R., Snavely, N., Bala, K., Weinberger, K.: Deep feature interpolation for image content changes. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 7064–7073 (2017)
[26] Xiong, Z., Xia, Q., Hu, Z., Huang, N., Bian, C., Zheng, Y., Vesal, S., Ravikumar, N., Maier, A., Yang, X., et al.: A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Medical Image Analysis 67, 101832 (2021)
[27] Zhang, H., Zhang, J., Wang, R., Zhang, Q., Gauthier, S.A., Spincemaille, P., Nguyen, T.D., Wang, Y.: Geometric loss for deep multiple sclerosis lesion segmentation. arXiv preprint arXiv:2009.13755 (2020)
[28] Zhang, H., Zhang, J., Wang, R., Zhang, Q., Spincemaille, P., Nguyen, T.D., Wang, Y.: Efficient folded attention for 3d medical image reconstruction and segmentation. arXiv preprint arXiv:2009.05576 (2020)
[29] Zhang, H., Valcarcel, A.M., Bakshi, R., Chu, R., Bagnato, F., Shinohara, R.T., Hett, K., Oguz, I.: Multiple sclerosis lesion segmentation with tiramisu and 2.5 d stacked slices. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 338–346. Springer (2019)
[30] Zhang, J., Liu, Z., Zhang, S., Zhang, H., Spincemaille, P., Nguyen, T.D., Sabuncu, M.R., Wang, Y.: Fidelity imposed network edit (fine) for solving ill-posed image reconstruction. NeuroImage 211, 116579 (2020)
[31] Zhang, J., Zhang, H., Sabuncu, M., Spincemaille, P., Nguyen, T., Wang, Y.: Bayesian learning of probabilistic dipole inversion for quantitative susceptibility mapping. In: Medical Imaging with Deep Learning. pp. 892–902. PMLR (2020)
[32] Zhang, R.: Making convolutional networks shift-invariant again. In: International Conference on Machine Learning. pp. 7324–7334 (2019)
[33] Zill, D.G.: Advanced engineering mathematics. Jones & Bartlett Publishers (2020)