Differences between human and machine perception in medical diagnosis

We experimented with the NYU Breast Cancer Screening Dataset [33] developed by our research team and used in a number of prior studies [12, 13, 14, 34, 35], and we applied the same training, validation, and test set data split as has been reported previously. This dataset consists of 229,426 screening mammography exams from 141,473 patients. Each exam contains at least four images, with one or more images for each of the four standard views of screening mammography: left craniocaudal (L-CC), left mediolateral oblique (L-MLO), right craniocaudal (R-CC), and right mediolateral oblique (R-MLO). Each exam is paired with labels indicating whether there is a malignant or benign finding in each breast. See Extended Data Figure 7 for an example of a screening mammogram. We used a subset of the test set for our reader study, which is also the same subset used in the reader study of [12]. For our DNN experiments, we used two architectures in order to draw general conclusions: the deep multi-view classifier (DMV) [12], and the globally-aware multiple instance classifier (GMIC) [13, 14]. We primarily report results for GMIC, since it is the more recent and better-performing model. The corresponding results for DMV, which support the generality of our findings, are provided in the Extended Data section.

In order to compare the perception of radiologists and DNNs, we applied Gaussian low-pass filtering to mammograms, and analyzed the resulting effect on their predictions. We selected nine filter severities ranging from unperturbed to severe, where severity was represented as a wavelength in units of millimeters on the physical breast. Details regarding the calculation of the filter severity are provided in the Methods section. Figure 1 demonstrates how low-pass filtering affects the appearance of malignant breast lesions.

We conducted a reader study in order to collect predictions for low-pass filtered images from radiologists. This reader study was designed to be identical to that of [12], except that the mammograms were randomly low-pass filtered in our case. We assigned the same set of 720 exams to ten radiologists with varying levels of experience. The images were presented to the radiologists in a conventional format, and an example is shown in Extended Data Figure 7. Each radiologist read each exam once, and for each exam, we uniformly sampled one severity level out of our set of nine, and applied it to all images in the exam. The radiologists made binary predictions indicating the presence of a malignant lesion in each breast. We describe the details of the reader study in the Methods section.

We then trained five DNNs from random weight initializations, and made predictions on the same set of 720 exams. We repeated this nine times, where the set of exams was low-pass filtered with each of the nine filter severities. We note that for each DNN, we made a prediction for every pair of exam and filter severity. In contrast, for each radiologist, we only had predictions for a subset of the possible combinations of exam and filter severity. This means that if we arrange the predictions in a matrix where each row represents a filter severity and each column an exam, the matrix of predictions is sparse for each radiologist, and dense for each DNN. This fact is visualized in Figure 2 (b–c). The sparsity of the radiologist predictions is by design; we were careful to ensure that each radiologist only read each exam once, since if they were to have seen the same exam perturbed with multiple filter severities, their predictions would have been unlikely to be independent. However, the sparsity prevents us from comparing radiologists and DNNs using evaluation metrics that use predictions for the complete set of exams. We therefore utilized probabilistic modeling to use the available radiologist predictions to infer values for the missing predictions.

We applied probabilistic modeling to achieve two purposes. The first is to study the effect of low-pass filtering on specific subgroups of lesions in isolation, after factoring out various confounding effects such as the idiosyncracies of individual radiologists and DNNs. The second is to infer the radiologists’ predictions for each pair of exam and filter severity, since some pairs were missing by design. We modeled the radiologists’ and DNNs’ predictions as i.i.d. Bernoulli random variables. Let us denote radiologist (DNN) $r$’s prediction on case $n$ filtered with severity $s$ as $\hat{y}_{r,s}^{(n)}$. We parameterized our model as

$$\hat{y}_{r,s}^{(n)}\sim\operatorname{Bernoulli}(\sigma(b_{g}+\mu^{(n)}+\gamma_{s,g}+\nu_{r,g})),$$

(1)

where $\sigma$ is the logistic function. There are four latent variables with the following interpretation: $b_{g}$ represents the bias of exams in subgroup $g$, $\mu^{(n)}$ is the bias of exam $n$, $\gamma_{s,g}$ is the effect that low-pass filtering with severity $s$ has on exams in subgroup $g$, and $\nu_{r,g}$ is the idiosyncrasy of radiologist (DNN) $r$ on exams in subgroup $g$. See Figure 3 for a graphical representation of our model. We considered several parameterizations of varying complexity, and selected the one with the maximum marginal likelihood. See the Methods section for details regarding our probabilistic model.

Using the probabilistic model, we compared how low-pass filtering affects the predictions of radiologists and DNNs, separately analyzing microcalcifications and soft tissue lesions. We performed each comparison with respect to two metrics: predictive confidence and class separability. Since the latent variable $\gamma_{s,g}$ represents the effect of low-pass filtering on each prediction, we examined its posterior distribution in order to measure the effect on predictive confidence. We sampled values of $\hat{y}_{r,s}^{(n)}$ from the posterior predictive distribution in order to quantify how low-pass filtering affects class separability. We computed the Kolmogorov-Smirnov (KS) statistic between the sampled predictions for the positive and negative class. This represents the distance between the two distributions of predictions, or how separated the two classes are. Sampling from the posterior predictive distribution was necessary for radiologists, since we did not have a complete set of predictions from them. Although such sampling was not strictly necessary for DNNs given the full set of available predictions, we performed the same posterior sampling for DNNs in order to ensure a fair comparison.

Figure 4a presents the results for microcalcifications. We only consider DNNs that are trained with unperturbed data in this section. The results for training DNNs on low-pass filtered data are discussed in the next section. The left subfigure represents predictive confidence, as measured by the posterior expectation of $\gamma_{s,g}$. Since low-pass filtering removes the visual cues of malignant lesions, we hypothesized that it should decrease predictive confidence. In other words, we expected to see $\operatorname{\mathbb{E}}[\gamma_{s,g}\mid\mathcal{D}]\leq 0$. Above the left subfigure, we report $P(\gamma_{s,g}>0\mid\mathcal{D})$ in order to quantify how much the posterior distributions $P(\gamma_{s,g}\mid\mathcal{D})$ align with this hypothesis. Small values indicate a significant negative effect on predictive confidence. We note that these values are not intended to be interpreted as the $p$-values of a statistical test. Instead, they quantify the degree to which each $\gamma_{s,g}$ is negative. We observe that for microcalcifications, low-pass filtering decreases the predictive confidence of both radiologists and DNNs. There is, however, an interesting difference in that for the range of most severe filters, the effect is constant for radiologists, while DNNs continue to become less confident.

The right subfigure of Figure 4a depicts the effect of low-pass filtering on class separability. This is quantified by the KS statistic between the predictions for the positive and negative class, where the positive class is restricted to malignant microcalcifications. Similar to our hypothesis that low-pass filtering decreases predictive confidence, we hypothesized that it should also reduce class separability. That is, we expected the KS statistics for severity $s>0$ to be smaller than those for $s=0$. This is because removing the visual cues for malignant lesions should make it more difficult to distinguish between malignant and nonmalignant cases. We tested this hypothesis using the one-tailed KS test between the KS statistics for $s=0$ and $s>0$. The $p$-values for this test are reported above the right subfigures, where small values mean that filtering significantly decreases class separability. We found that low-pass filtering decreases class separability for both radiologists and DNNs, but in substantially different ways. The radiologists’ class separability steadily declines for the range of less severe filters, while it is constant for DNNs. Meanwhile, similar to what we observed for predictive confidence, the radiologists’ class separability is constant for the range of most severe filters, while it continues to decline for DNNs. In summary, for microcalcifications, low-pass filtering decreases the predictive confidence and class separability for both radiologists and DNNs, but differences suggest that they use a different set of high frequency components.

Next, we compared how low-pass filtering affects the predictions of radiologists and DNNs on soft tissue lesions (Figure 4b). The results show that low-pass filtering degrades the predictive confidence and class separability of DNNs, while having almost no effect on radiologists. The invariance of radiologists’ impressions to filtering implies that the high frequency components used by DNNs in this context are spurious, since there exist alternative features used by radiologists that are robust to low-pass filtering. The DNNs’ reliance on such spurious features is a clear vulnerability, and must be addressed before they can be trusted to make clinical diagnoses. In the Discussion section, we describe a potential explanation for this phenomenon, as well as a plausible scenario in which this behavior would cause DNNs’ performance to degrade.

We observed that low-pass filtering decreases the predictive confidence and class separability of DNNs for all lesion subgroups. However, since the DNNs only encountered low-pass filtering during testing, it is possible that this effect is solely due to the dataset shift between training and testing. We therefore repeated the previous experiments for DNNs, where the same filtering was applied during both training and testing. We then examined whether the effects of low-pass filtering on the DNNs’ perception could be attributed to information loss rather than solely to dataset shift.

For microcalcifications (Figure 4a), training on filtered data slightly reduced the effect of low-pass filtering on predictive confidence and class separability, but the effect was still present, particularly for the most severe filters. This implies that the effect of filtering on microcalcifications can be attributed to information loss, and not solely to dataset shift. In other words, high frequency components in microcalcifications contain information that is important to the perception of DNNs.

Meanwhile, for soft tissue lesions (Figure 4b), training on low-pass-filtered data significantly reduces the effect on predictive confidence and class separability, even for severe filters. This suggests that the effect of low-pass filtering on soft tissue lesions can primarily be attributed to dataset shift rather than information loss. In fact, DNNs trained with low-pass-filtered data maintain a similar level of class separability compared to networks trained on the original data. This confirms what we observed for radiologists, which is that high frequency components in soft tissue lesions are largely dispensable, and that more robust features exist.

Our results thus far show that radiologists and DNNs use a different set of high frequency components in both microcalcifications and soft tissue lesions. Since this analysis has purely been in the frequency domain, we extend our comparison to the spatial domain by examining the degree to which radiologists and DNNs agree on the most suspicious regions of an image. Towards this end, we conducted a reader study in which seven radiologists annotated up to three regions of interest (ROIs) containing the most suspicious features of each image. 120 exams were used in this study, which is a subset of the 720 exams in the perturbation reader study. See the Methods section for details regarding this reader study. We then applied low-pass filtering to the interior and exterior of the ROIs, as well as to the entire image. Examples of the annotation and the low-pass filtering schemes are shown in Figure 2 (g–j). We made predictions using DNNs trained with the original data in order to understand the relationship between the high frequency components utilized by DNNs, and the regions of mammograms that are most suspicious to the radiologists.

We began by comparing the effect of the three low-pass filtering schemes on the DNNs’ predictions for microcalcifications (Figure 5a). We observed that filtering the ROI interior has a similar effect to filtering the entire image for mild filter severities. This suggests that for the frequencies in question, DNNs primarily rely on the same regions that radiologists consider suspicious. Meanwhile, class separability for the two ROI-based filtering schemes diverge significantly for high severities: filtering the ROI interior ceases to further decrease class separability at some threshold filter severity, whereas exterior filtering continues to degrade class separability beyond this threshold. The implication is that a range of high frequency components utilized by DNNs exist in the exterior of the ROIs deemed most important by human radioloigsts.

For soft tissue lesions (Figure 5b), filtering the ROI interior decreases class separability, but to a lesser degree compared to filtering the entire image. This means that DNNs do utilize high frequency components in regions that radiologists find suspicious, but only to a limited degree. Meanwhile, filtering the ROI exterior has a similar effect on class separability as filtering the entire image. These observations suggest that the high frequency components that DNNs use for soft tissue lesions may be scattered across the image, rather than being localized in the areas that radiologists consider suspicious.

We conducted our DNN experiments using two architectures: the Deep Multi-View Classifer [12], and the Globally-Aware Multiple Instance Classifier [13, 14]. With both architectures, we trained an ensemble of five models. A subset of each model’s weights was initialized using weights pretrained on the BI-RADS label optimization task [39], while the remaining weights were randomly initialized. For each architecture, we adopted the same training methodology used by the corresponding authors.

We applied Dirichlet calibration [40] to the predictions of DNNs used in our probabilistic modeling. This amounts to using logistic regression to fit the log predictions to the targets. We trained the logistic regression model using the validation set, and applied it to the log predictions on the test set to obtain the predictions used in our analysis. We used L2 regularization when fitting the logistic regression model, and tuned the regularization hyperparameter via an internal 10-fold cross-validation where we further split the validation set into “training” and “validation” sets. In the cross-validation, we minimized the classwise expected calibration error [40].

Low-pass filtering is a method for removing information from images that allows us to interpolate between the original image and, in the most extreme case, an image where every pixel has the value of the mean pixel value of the original image. We experimented with nine filter severities selected to span a large range of the frequency spectrum. We implemented the Gaussian low-pass filter by first applying the shifted two-dimensional discrete Fourier transform to transform images to the frequency domain. The images were multiplied element-wise by a mask with values in $[0,1]$. The values of this mask are given by the Gaussian function

$$M(u,v)=\exp\left(\frac{-D^{2}(u,v)}{2D^{2}_{0}}\right),$$

(2)

where $u$ and $v$ are horizontal and vertical coordinates, $D(u,v)$ is the Euclidian distance from the origin, and $D_{0}$ is the cutoff frequency. $D_{0}$ represents the severity of the filter, where frequencies are reduced to $0.607$ of their original values when $D(u,v)=D_{0}$. Since the mammograms in our dataset vary in terms of spatial resolution as well as the physical length represented by each pixel, we expressed the filter severity $D_{0}$ in terms of a normalized unit of cycles per millimeter on the breast. Let $\alpha=\min(H,W)$ where $H$ and $W$ are the height and width of the image, and let $\beta$ denote the physical length in millimeters represented by each pixel. Then we can convert cycles per millimeter $D_{0}$ to cycles per frame length of the image $D_{0}^{\text{img}}$ using

$$D_{0}^{\text{img}}=D_{0}\cdot\alpha\cdot\beta.$$

(3)

In order to compare humans and machines with respect to their sensitivity to the removal of clinically meaningful information, we conducted a reader study in which ten radiologists read 720 exams selected from the test set. While all radiologists read the same set of exams, each exam was low-pass filtered with a different severity for each radiologist. Except for the low-pass filtering, the setup of this reader study is identical to that of [12]. Each exam consists of at least four images, with one or more images for each of the four views of mammography: L-CC, R-CC, L-MLO, and R-MLO. All images in the exam were concatenated into a single image such that the right breast faces left and is presented on the left, and the left breast faces right and is displayed on the right. Additionally, the craniocaudal (CC) views are on the top row, while the mediolateral oblique (MLO) views are on the bottom row. An example of this is shown in Extended Data Figure 7. Among the 1,440 breasts, 62 are malignant, 356 are benign, and the remaining 1,022 are nonbiopsied. Among the malignant breasts, there are 26 microcalcifications, 21 masses, 12 asymmetries, and 4 architectural distortions, while in the benign breasts, the corresponding counts are: 102, 87, 36, and 6. For each exam, radiologists make a binary prediction for each breast, indicating their diagnosis of malignancy.

We modeled the radiologists’ and DNNs’ binary malignancy predictions with the Bernoulli distribution

$$\hat{y}_{r,s}^{(n)}\sim\operatorname{Bernoulli}(p_{r,s}^{(n)}),$$

(4)

where $n\in\{1,2,\dotsc,1440\}$ indexes the breast, $r\in\{1,2,\dotsc,10\}$ the reader, and $s\in\{1,2,\dotsc,9\}$ the low-pass filter severity. Each distribution’s parameter $p_{r,s}^{(n)}$ is a function of four latent variables

$$p_{r,s}^{(n)}=\sigma(b_{g}+\mu^{(n)}+\gamma_{s,g}+\nu_{r,g}),$$

where $c\in\{1,\dotsc,5\}$ indexes the subgroup of the lesion. We included the following subgroups: unambiguous microcalcifications, unambiguous soft tissue lesions, ambiguous microcalcifications and soft tissue lesions, mammographically occult, and nonbiopsied. We considered these five subgroups in order to make use of all of our data, but only used the first two in our analysis. We assigned the generic weakly informative prior $\mathcal{N}(0,1)$ to each latent variable. We chose the Bernoulli distribution because it has a single parameter, and thus makes the latent variables interpretable. Additionally, radiologists are accustomed to making discrete predictions in clinical practice. The posterior distribution of the latent variables is given by

$$p(\bm{b},\bm{\mu},\bm{\gamma},\bm{\nu}\mid\bm{\hat{y}})=\frac{p(\bm{\hat{y}}\mid\bm{b},\bm{\mu},\bm{\gamma},\bm{\nu})}{p\left(\bm{\hat{y}}\right)}.$$

The exact computation of the posterior is intractable, since the marginal likelihood $p(\bm{\hat{y}})$ involves a four-dimensional integral. We therefore applied automatic differentiation variational inference (ADVI) [41] in order to approximate the posterior. ADVI, and variational inference in general, optimizes over a class of tractable distributions in order to find the closest match to the posterior. For our choice of tractable distributions, we used the mean-field approximation, meaning that we optimized over multivariate Gaussians with diagonal covariance matrices.

In practice, while radiologists made binary predictions, DNNs made continuous predictions in $[0,1]$ that we then calibrated. Despite the DNN predictions not being binary, we used equivalent procedures to specify the probabilistic model for radiologists and DNNs. To see how, let $\hat{y}^{(n)}\in\{0,1\}$ denote a DNN’s unobserved binary prediction for case $n$, and let $\hat{z}^{(n)}\in[0,1]$ denote its observed real-valued prediction for the same case. In order to obtain $\hat{y}^{(n)}\in\{0,1\}$, we could treat it as a random variable $\hat{y}^{(n)}\sim\operatorname{Bernoulli}(\hat{z}^{(n)})$ and obtain values for it through sampling. We instead used $\hat{z}^{(n)}$ directly, specifying the log joint density as

$$\begin{split}\log p(\bm{\hat{y}}\mid\theta)&=\sum_{n=1}^{N}\log p(\hat{y}^{(n)}\mid\theta^{(n)})\\ &\approx\sum_{n=1}^{N}\operatorname{\mathbb{E}}_{\hat{y}^{(n)}}[\log p(\hat{y}^{(n)}\mid\theta^{(n)})]\\ &=\sum_{n=1}^{N}\operatorname{\mathbb{E}}_{\hat{y}^{(n)}}\left[\hat{y}^{(n)}\log(\theta^{(n)})+(1-\hat{y}^{(n)})\log(1-\theta^{(n)})\right]\\ &=\sum_{n=1}^{N}\Pr(\hat{y}^{(n)}=1)\log(\theta^{(n)})+\Pr(\hat{y}^{(n)}=0)\log(1-\theta^{(n)})\\ &=\sum_{n=1}^{N}\hat{z}^{(n)}\log(\theta^{(n)})+(1-\hat{z}^{(n)})\log(1-\theta^{(n)}).\end{split}$$

In order to compare humans and machines with respect to the regions of an image deemed most suspicious, we conducted a reader study in which seven radiologists read the same set of 120 unperturbed exams. The exams in this study were a subset of the 720 exams from the perturbation reader study, and also included all malignant exams from the test set. This study had two stages. In the first stage, the radiologists were presented with all views of the mammogram, and they made a malignancy diagnosis for each breast. This stage was identical to the reader study in [12]. In the second stage, for breasts that were diagnosed as malignant, the radiologists annotated up to three ROIs around the regions they found most suspicious. The radiologists annotated each view individually, and the limit of three ROIs applied separately to each view. For exams that contained multiple images per view, the radiologists annotated the image where the malignancy was most visible. The radiologists annotated the images using Paintbrush on MacOS, or Microsoft Paint on Windows. In order to constrain the maximum area that is annotated for each image, we included a $250{\times}250$ pixel blue ROI template in the bottom corner of each image to serve as a reference. The radiologists then drew up to three red ROIs such that each box approximately matched the dimensions of the reference blue ROI template.