TumorCP: A Simple but Effective Object-Level Data Augmentation for Tumor Segmentation

Deep learning (DL) models work remarkably well over the past few years in computer vision tasks, including medical image analysis. Though DL models act like de facto standard, they are notoriously data-hungry, demanding more so than ever large and well-annotated datasets to achieve robust performance [16]. However, high-quality annotated datasets require intense labor and domain knowledge, which becomes more expensive in the medical domain.

To improve data-efficient learning, several successful approaches have been proposed from different perspectives, such as leveraging unlabeled data for semi-supervised self-training [16, 1, 15] or self-supervised pre-training [19, 1, 12], distilling priors from data as explicit constraints for model training [10, 9], generating new data with the imaging of an anatomy of a different modality [8, 11], or utilizing appropriate data augmentation methods to increase data diversity [5, 14, 2, 15]. Some of them are designated for medical images. Particularly, Zhou et al. [19] designed a unified self-supervised learning framework, integrating multiple proxy tasks to exploit unlabeled medical data, and showed performance gains for downstream tasks. Xue et al. [17], and Shin et al. [13] used GANs to generate additional training data for histopathology image classification and brain tumor segmentation. The quality of the “realness” of synthesized training data dramatically affects model performance due to the risk of overfitting to fake data. Eaton et.al. [3] studied Mix-up [18] augmentation for brain tumor segmentation. However, it requires a specific patch-level operation which involves complicated strategies, e.g. sampling of small patches to be mixed up.

Distinct from the trend of using increasingly sophisticated methods like GANs, we investigate “Copy-Paste”, a straightforward augmentation technique [4, 2] that has been recently revisited and made breakthroughs in natural image instance segmentation [5]. Copy-Paste augmentation avoids costly generation processes from representation space to pixel space by simply pasting the labeled instance onto new background images as additional training data. Despite its success in natural images, such method is largely unexplored in the medical image realm. Moreover, its effectiveness for medical tasks remains doubtable since the context information tends to be ignored in Copy-Paste. For instance, in the tumor segmentation, one would argue the importance of surrounding visual clues, i.e., context, for the emergence of a tumor. Besides, one would believe the inherent anatomical structures in medical image make the context indispensable for tumor segmentation. In this work, we also aim to fill the gap of understanding the role of context in medical domain by examining the effectiveness of Copy-Paste augmentation for tumor segmentation.

We propose TumorCP, a simple but effective object-level data augmentation method based on Copy-Paste for tumor segmentation tasks. Straightforwardly, TumorCP randomly chooses a tumor from a source image and paste it onto the organs in the target image after a series of spatial, contrast, and blurring augmentations. We use kidney tumor segmentation (KiTS19 dataset [6]) and a state-of-the-art model (nnUNet[7]) as the benchmark to evaluate the proposed method. We empirically show that though TumorCP inevitably generates artifacts after Copy-Paste, it consistently provides solid gains over all different settings in our experiments. Specifically, with only rigid spatial transformation and Copy-Paste within the same patient, TumorCP can surpass the baseline by 6.24% tumor Dice. Together with inter-patient Copy-Paste and other tumor-oriented augmentations, TumorCP further outperforms the baseline by 7.12% tumor Dice. Moreover, with image-level data augmentation (ImgDA), our best version beats state-of-the-art by 2.3%. Going one step further, we also study TumorCP for extremely low-data regime, where only 10% labeled data are exploited for training. Under this setting, TumorCP with ImgDA can improve the tumor Dice by 21.87% compared with no-data-augmentation (noDA), which is unprecedented to our knowledge, convincingly demonstrating the effectiveness of TumorCP for data-efficiency learning.

The success of TumorCP is an empirical observation to support context-decoupled learning even in medical domain. We briefly discuss our understanding of the open question of the context’s role and why TumorCP works in Sec 2.2. We hope our work can provide some useful data points to our community and shed light on the importance of Copy-Paste augmentation, which is powerful but unfortunately nearly absent in the medical imaging field.

TumorCP is an online and stochastic augmentation process specified for tumor segmentation. Its implementation is easy and straightforward. As illustrated in Fig.1, given a set of training samples $\mathcal{D}$, with the probability of $(1-p_{cp})$, TumorCP does nothing; otherwise TumorCP samples a pair of images $(x_{src},x_{tgt})\sim\mathcal{D}$ and conducts Copy-Paste once. Let $\mathcal{O}_{src}$ be the set of tumor(s) on $x_{src}$, $\mathcal{V}_{tgt}$ be the set of volumetric coordinates of organ(s) on $x_{tgt}$, and $\mathcal{T}$ be the set of stochastic data transformations, each of which has a probability parameter called $p_{trans}$. To do once Copy-Paste, TumorCP first samples a tumor $o\sim\mathcal{O}_{src}$, a set of transformation(s) $\tau\sim\mathcal{T}$, and a target location $v\sim\mathcal{V}_{tgt}$, followed by centering $\tau(o)$ at $v$ to replace the original data and annotation. To fully leverage the advantage of TumorCP, we carefully design two modes of Copy-Paste for tumors: intra-patient and inter-patient Copy-Paste. Meanwhile, we enhance Copy-Paste with several object-level transformation to obtain abundant augmentations.

This key contribution of our work is the proposal and comprehensive study of TumorCP, a simple but effective object-level data augmentation for tumor segmentation. Extensive experiments confirm the remarkable effectiveness of our method. In addition to surpassing current art in kidney tumor segmentation by 2.31% in tumor Dice, we also demonstrate the potential of TumorCP for the extremely low-data regime. We prefer to call our TumorCP as a new baseline, as it does not involve any sophisticated techniques nor extensive hyper-parameter adjustment while achieving the new state-of-the-art. Besides, TumorCP does not directly handle the distribution mismatching in the inter-CP setting but still gets fabulous performance. Future works can easily extend TumorCP for other medical segmentation tasks without significant modifications, and are worth trying for further improving state-of-the-art accuracy.