Soft-Label Anonymous Gastric X-ray Image Distillation

This subsection shows how we preprocess the training dataset. The gastric X-ray images in our dataset have high resolutions, $e.g.$, 2,048 $\times$ 2,048 pixels. In practical applications, the high-resolution images can lead to expensive computing cost. Therefore, we divide each gastric X-ray image into $H\times W$ patches ($H$ and $W$ respectively denote the number of patches in the vertical direction and the horizontal direction) and manually label these patch-based gastric X-ray images into the following three categories:

• •

  $\mathcal{I}$: patches outside of the stomach (irrelevant),

• •

  $\mathcal{N}$: patches extracted from non-gastritis X-ray images (negative) inside of the stomach,

• •

  $\mathcal{P}$: patches extracted from gastritis X-ray images (positive) inside of the stomach.

Figures 1 and 2 show the full gastric X-ray images and patch-based gastric X-ray images, respectively.

In this subsection, we explain our anonymous gastric image distillation algorithm. When we have the patch-based gastric training dataset $\left(\mathbf{x,y}\right)$ = $\left\{x_{g},y_{g}\right\}_{g=1}^{G}$, where $G$ denotes the number of training images, $x_{g}$ and $y_{g}$ denote the image and the corresponding label, respectively. We parameterize a DCNN model as $\theta$, and let the twice-differentiable loss function $\ell\left(\mathbf{x,y},\theta\right)$ denotes the loss of this network on the entire training dataset $\left(\mathbf{x,y}\right)$. Considering the gastritis patches and non-gastritis patches may have common features, we let the distilled images $\tilde{\mathbf{x}}$ have soft labels $\tilde{\mathbf{y}}$, which contain $\mathcal{I}$, $\mathcal{N}$ and $\mathcal{P}$ [22]. In our gastric image distillation method, we distill the valid information of the entire training dataset to a tiny distilled dataset $\left(\tilde{\mathbf{x}},\tilde{\mathbf{y}}\right)$ = $\left\{\tilde{x}_{m},\tilde{y}_{m}\right\}_{m=1}^{M}$ with $M\ll G$. For each distilling step $i$, the weights are updated as follows:

where $\left(\tilde{\mathbf{x}},\tilde{\mathbf{y}}\right)$ denotes the distilled dataset, and $\tilde{\alpha}$ denotes the optimized learning rate.

With the derived new weights $\theta_{i+1}$ of the DCNN model, we evaluate it on the entire training dataset $\left(\mathbf{x,y}\right)$. Our goal is to find the optimal distilled images $\tilde{\mathbf{x}}^{*}$, distilled labels $\tilde{\mathbf{y}}^{*}$ and the optimized learning rate $\tilde{\mathbf{\alpha}}^{*}$. The objective function is defined as:

where $\ell\left(\tilde{\mathbf{x}},\tilde{\mathbf{y}},\theta_{i}\right)$ is twice-differentiable, and $\mathcal{L}\left(\tilde{\mathbf{x}},\tilde{\mathbf{y}},\tilde{\alpha};\theta_{i}\right)$ is differentiable.

To obtain the optimal distilled images, distilled labels and the optimized learning rate, we update the distilled images $\tilde{\mathbf{x}}$, distilled labels $\tilde{\mathbf{y}}$ and the optimized learning rate $\tilde{\mathbf{\alpha}}$ at each distilling step with gradient descent as follows:

where $\nabla_{\tilde{\mathbf{x}}}\mathcal{L}$, $\nabla_{\tilde{\mathbf{y}}}\mathcal{L}$ and $\nabla_{\tilde{\mathbf{\alpha}}}\mathcal{L}$ respectively denote the gradient of $\mathcal{L}$ based on $\tilde{\mathbf{x}}$, $\tilde{\mathbf{y}}$ and $\tilde{\mathbf{\alpha}}$, $\alpha$ denotes the learning rate.

Then we demonstrate the details of the anonymous gastric image distillation algorithm. Firstly, we show our input and output settings. Let $\theta_{0}$ denotes the random initial weights of the DCNN model, $M$, $E$ and $I$ respectively denote the number of distilled images, distill epochs and distill steps. Also, let $\alpha$, $K$ and $T$ are respectively denote learning rate, batch size and training steps. In addition, $\tilde{\mathbf{y}}_{0}$ is the initial value of $\tilde{\mathbf{y}}$, and $\tilde{\alpha}_{0}$ is the initial value of $\tilde{\alpha}$. And we obtain the distilled images $\tilde{\mathbf{x}}$, distilled labels $\tilde{\mathbf{y}}$ and the optimized learning rate $\tilde{\alpha}$ on the output.

Next, we explain the training process of the algorithm. First, we randomly initialize the distilled images $\tilde{\mathbf{x}}$. Then we initialize the distilled labels $\tilde{\mathbf{y}}$ and the optimized learning rate $\tilde{\alpha}$ with $\tilde{\mathbf{y}}_{0}$ and $\tilde{\alpha}_{0}$, respectively. At each training step $t$, we get a minibatch of training data $\left(\mathbf{x}_{t},\mathbf{y}_{t}\right)$ whose size is $K$. Then we get the random initial weights $\theta_{0}$ and begin the distilling process. We repeat the distilling process for $E$ times. At each distilling step $i$, we compute the updated weights with Eq. (1). Then we evaluate the objective function on the minibatch of training data with Eq. (2). Finally, we update the distilled data $\tilde{\mathbf{x}}$, $\tilde{\mathbf{y}}$, and $\tilde{\alpha}$ with Eq. (3) based on gradient descent [23, 24]. Algorithm 1 shows the entire flow of our proposed approach.

In this subsection, we explain how to estimate the label of a full gastric X-ray image based on patches. First, when we have a test gastric image, we divide it into $H\times W$ patches. Then we put the divided patches into the trained DCNN model, and we can obtain the predicted labels of these patches. Next, we respectively calculate the number of patches whose predicted labels are $\mathcal{N}$ and $\mathcal{P}$. The $\mathcal{I}$ patches that extracted from outside of the stomach are not related to the gastritis/non-gastritis prediction, and hence we do not take into account them to the probability calculation. Finally, we estimate the label of a full gastric X-ray image as follows:

where $\epsilon$ is a threshold, $\mathrm{Num}(\mathcal{P})$ denotes the number of gastritis patches, and $\mathrm{Num}(\mathcal{N})$ denotes the number of non-gastritis patches. Note that if $y^{\mathrm{test}}=1$, the estimation result of a full gastric X-ray image is gastritis, and the estimation result is non-gastritis otherwise.

This subsection shows the experimental settings of our research. The dataset used in our study contains 815 patients' (240 gastritis and 575 non-gastritis) gastric X-ray images. Each image has a ground truth (gastritis/non-gastritis), which was determined by patient diagnosis results of endoscopic examination and X-ray inspection. All of the gastric X-ray images are gray-scale and high-resolution. The training dataset contains 200 patients' (100 gastritis and 100 non-gastritis) images. Also, the rest of the patients' (140 gastritis and 475 non-gastritis) images are included in the test dataset. In the data preprocessing stage, we divided the images into patches (299 $\times$ 299 pixels), $i.e.$, $H$ = $W$ = 35, where the sliding interval was set to 50 pixels. Besides, the patches extracted from the training dataset were labeled as $\mathcal{I}$, $\mathcal{N}$ and $\mathcal{P}$ by a radiological technologist. Note that if the regions inside of the stomach were less than 1$\%$ in a patch, it was labeled as $\mathcal{I}$. In addition, if the regions inside of the stomach were more than 85$\%$ in a patch, it was labeled as $\mathcal{N}$ or $\mathcal{P}$. And we discarded the rest of the patches in the training dataset. As a result, we obtained $\mathcal{I}$, $\mathcal{N}$ and $\mathcal{P}$ whose number of patches were 48,385, 42,785 and 45,127, respectively.

In the training phase, we loaded the patch-based training dataset ($\mathcal{I}$, $\mathcal{N}$ and $\mathcal{P}$) into a DCNN model. Since the distilling process involves the calculation of a quadratic gradient, a complicated DCNN model can lead to inefficiency. Hence, we constructed the ResNet18 [25] model with PyTorch framework [26, 27], and the loss $\ell$ was cross entropy loss. In our experiments, we set the number of distilled images of each category to 1, $i.e.$, $M$ = 3. Besides, the distill epochs and steps were set to 3, $i.e.$, $E$ = $I$ = 3 (total 9 distill steps). And we initialized the soft labels with one-hot values of the original labels, which tends to have higher performance compared to the random initialization. The hard-label distillation method had the same settings as soft-label distillation, except for the labels being fixed [28]. As a result, the training process distilled each class into one image and saved the distilled images, distilled labels and the optimized learning rate. We performed 400 epochs in both of soft-label distillation and hard-label distillation and saved the results after every ten epochs for testing and evaluation. It is difficult for a DCNN model to learn from only several images, so we respectively selected 1,000, 2,000 and 3,000 images per category from the patch-based training dataset and trained three ResNet18 models until convergence.

In the test phase, we performed two experiments (Ex. I and Ex. II) to show the validity of our proposed method quantitatively. Firstly, in Ex. I, we selected the distilled soft-label images that have the best classification performance on the patch-based training dataset and evaluated the performance on the full X-ray images of the test dataset. Specifically, we evaluated the full gastric X-ray images classification performance of the ResNet18 models trained on the three random subsets. Secondly, in Ex. II, we used the hard-label distillation as a comparative method. As in Ex. I, we selected the best distilled hard-label images, which have the highest classification accuracy on the patch-based training dataset. And we utilized the distilled hard-label images to evaluate the classification performance on the full gastric X-ray images. Note that we set the threshold $\epsilon$ to 0.4 in both of Exs. I and II, which tends to have an excellent classification performance. We utilized the following sensitivity (Sen), specificity (Spe) and harmonic mean (HM) of Sen and Spe as evaluation indexes:

where TP, FN, TN and FP denote the number of true positive, false negative, true negative and false positive, respectively.

The experimental results are shown in Tables 1 and 2. Table 1 shows the full gastric X-ray images classification performance of our proposed method and ResNet18 trained on the random subsets. The ResNet18 model trained with 3,000 images per category (total 9,000 images) has an HM score of 0.823. On the other hand, our soft-label distillation method that distilled each class into only one image for training has a higher HM score of 0.877. We can see that the proposed method realizes competitive classification accuracy with a tiny distilled dataset. Table 2 shows the full gastric X-ray images classification performance of our soft-label distillation and the hard-label distillation method. The HM scores of both the two approaches exceed 0.85, but our soft-label distillation outperforms the hard-label distillation with a higher score. Furthermore, we can see that the classification performance of our method becomes more stable because of the balance of Sen score and Spe score. It means that the distilled images with soft-label can lead to better distillation results. Experimental results clearly showed the validity of our purposed soft-label distillation method.

Figure 3 shows examples of distilled hard-label images and soft-label images used in our experiments. With the distillation methods, the information of gastritis/non-gastritis patch images was extracted and merged into only one image. Note that I (base), N (base) and P (base) respectively denote the image belongs to label $\mathcal{I}$, $\mathcal{N}$ and $\mathcal{P}$ with the highest probability. From Fig. 3, we can see that the features of gastritis/non-gastritis patches cannot be distinguished, in other words, the gastric images were anonymized completely. Hence, the distilled images have no private information of patients. Figure 3 clearly showed that our proposed method can effectively compress and anonymize the medical image data.