Pre-trained Trojan Attacks for Visual Recognition

DNNs are vulnerable to malicious data input including adversarial attacks goodfellow2014explaining; wang2021dual; liu2019perceptual; liu2020bias; liu2023x; liu2020spatiotemporal; liu2022harnessing; zhou2023advclip and backdoor attacks gu2017badnets; liu2018trojaning; shi2023badgpt; Liang2023Badclip. Specifically, during training, the adversary injects triggers in the training set (i.e., poisoning) and implants backdoors into the model; during inference, the infected models will show malicious behavior when the input image tampers with a trigger, otherwise, behave normally.

Given an image classifier $f_{\theta}$ that maps an input image $\bm{x}$ $\in$ $\mathbf{X}_{train}$ to a label $\bm{y}$ $\in$ $\mathbf{Y}_{train}$ from the training dataset $\mathbf{D}=\{(\bm{x}_{i},\bm{y}_{i})\}_{i=1}^{N}$, a dirty-label backdoor attacks try to cheat model $f_{\theta}$ by injecting poisoned data in the training phase, so that the infected model $f_{\hat{\theta}}$ would behave maliciously when the inputs are embedded with triggers while behaving normally on clean examples. In particular, the adversary randomly selects a very small portion of clean data $\{(\bm{x}_{i},\bm{y}_{i})\}_{i=1}^{M},M<N$ from the training dataset $\mathbf{D}$. Next, the adversary generates poisoned images $\{(\hat{\bm{x}}_{i},\hat{\bm{y}}_{i})\}_{i=1}^{M}$ by adding the trigger $\bm{T}$ onto the images via function $\phi$ and modifies the corresponding label to the target label $\hat{\bm{y}}_{i}$ as follows:

In practice, the trigger generation process/function $\phi$ is different, and the $\eta$ denotes the poisoning label modification rules for backdoor attacks. For example, for the patch-based attack gu2017badnets, the adversary generates patch triggers and uses a binary mask for the pattern location; for the blend-based attack chen2017targeted, the trigger is a predefined image that is added onto the image with specific trigger transparency. Finally, the model trained on the poisoned dataset $\hat{\mathbf{D}}$ that is a combination of the rest clean dataset and poisoned samples, will be embedded with backdoors and will give target label predictions $f_{\theta}(\hat{\bm{x}}_{i})=\hat{\bm{y}_{i}}$ on test images $\hat{\bm{x}}_{i}$ containing triggers.

The ”pre-training and fine-tuning” learning paradigm is widely used in modern computer vision algorithms. This framework involves initially pre-training a backbone feature extractor using a substantial amount of labeled or unlabeled data. The pre-trained backbone is then utilized to develop predictors for various downstream vision tasks, such as detection and segmentation, using a smaller domain-specific dataset. Consequently, these computation-intensive pre-trained backbones can be publicly shared among users to facilitate the construction of downstream predictors.

Significant progress in object detection and segmentation has been made through the fine-tuning of backbones pre-trained using supervised learning on the ImageNet classification dataset deng2009imagenet. Prominent examples include R-CNN Ross2014Rich and OverFeat Pierre2013Overfeat. Subsequent advancements have further extended this paradigm by pre-training on datasets with larger image collections, such as ImageNet-5k, followed by transfer learning. Another research direction within this paradigm focuses on unsupervised learning. In contrast to supervised learning, unsupervised learning involves pre-training the image encoder using a vast amount of unlabeled data. For instance, contrastive learning chen2020simple; Hadsell2006Dimen; He2020Momentum; hjelm2019learning addresses unlabeled images by generating similar feature representations for augmented versions of the same input and dissimilar feature representations for different images. Additionally, some studies explore pre-training image encoders based on pairs of unlabeled image and text data. An example is Contrastive Language-Image Training (CLIP) pan2022contrastive, which learns both image and text encoders by maximizing similarity between positive pairs and minimizing it between negative pairs, leveraging 400 million unlabeled data from the Internet.

In particular, the pre-training and fine-tuning paradigm of PVMs mainly consists of two processes. First, the provider trains a PVM $f$ on large datasets $\mathbf{D}$ based on pre-training tasks/techniques (e.g., image classification or contrastive learning), yielding a feature extractor or backbone $f$ with a set of optimized parameters $\theta^{PT}$. For example, training by supervised learning on image classification is shown as

where $\mathcal{L}(\cdot)$ represents the training loss for PVMs.

Based on the training of large-scale data, PVMs have already obtained powerful feature extraction abilities, which are usually used as encoders or backbones to represent an input $\bm{x}$. The PVMs are then stacked with a predictor network $g$ with parameter $\theta^{FT}$ (e.g., a linear classifier for classification), which will be optimized for the downstream tasks with limited samples $\mathbf{D}^{task}$ from the target domain as

where $\mathcal{L}^{task}(\cdot)$ represents the training loss function for the downstream tasks. Therefore, the obtained model for the downstream task can be regarded as a composite function $f=g_{\theta^{FT}}\circ f_{\theta^{PT}}$.

In this paper, we focus on performing backdoor attacks targeting the pre-training and fine-tuning paradigm. Specifically, we poison a PVM, and the embedded backdoors can be inherited to subsequent downstream vision tasks. In our experiment, we conduct evaluations in both supervised and unsupervised learning settings.

In the context of backdoor attacks for PVMs, the adversary is only able to access the original training dataset $\mathbf{D}$ and generate the poisoned dataset $\hat{\mathbf{D}}$. The backbone $f_{\hat{\theta}^{PT}}$ pre-trained on the poisoned dataset $\hat{\mathbf{D}}$ will be embedded with backdoors, and the backdoor should remain effective after stacking $f_{\hat{\theta}^{PT}}$ with the predictor $g$ and fine-tuned on $\mathbf{D}^{task}$. Therefore, to perform backdoor attacks in this scenario, the problem can be formulated as

Specifically, poisoned images are generated by Eqn 1, and $\mathcal{L}^{task}(\cdot)$ represents the attacking goal’s objective function for specific downstream tasks. For example, given image ${\mathbf{x}}$ from the target domain, for image classification, $\mathcal{L}^{task}(\cdot)$ denotes the prediction likelihood of $g$ on the poisoned sample $\phi({\mathbf{x}},\bm{T})$ towards target class $\hat{{\mathbf{y}}}$; for object detection, $\mathcal{L}^{task}(\cdot)$ denotes prediction function losses of $g$ on the poisoned image $\phi({\mathbf{x}},\bm{T})$ , and the learning target is the object’s poisoned annotation including target class $\hat{{\mathbf{y}}}$ and location bounding box $\hat{\mathbf{b}}_{k}=[\hat{s}_{k},\hat{r}_{k},\hat{w}_{k},\hat{h}_{k}]$. The $\hat{s}_{k}$ and $\hat{r}_{k}$ denote center point coordinates. The $\hat{w}_{k}$ and $\hat{h}_{k}$ denote the length and width of the $k$ objects, respectively; for instance segmentation, the target of learning is the poisoned annotation of the pixel set and $\hat{{\mathbf{y}}}$ represents the incorrect labels of the modified pixels in the image. Detailed illustrations of adversarial goals can be found in the following parts.

Existing backdoors for PVMs mainly aim at attacking one specific task (i.e., image classification). However, it is challenging to directly apply these attacks to the context of pre-training and fine-tuning of visual models. We observe two challenges came from the two components (i.e.,the poisoned dataset $\hat{\mathbf{D}}$ and the non-matched annotations $\hat{\bm{y}}$/$\hat{{\mathbf{y}}}$ of $f$).

Challenge ❶: Task-specific trigger patterns are hard to activate among different downstream tasks. During the backdoor injection process, the adversary can only access the original training dataset and poison the backbone $f_{\hat{\theta}^{PT}}$ on specific tasks (e.g., image classification or contrastive learning). However, the final model $f$ is composed by stacking $f_{\hat{\theta}^{PT}}$ with function $g_{\theta^{FT}}$. Injected trigger patterns in $f_{\hat{\theta}^{PT}}$ on specific tasks may fail to remain effective for $g_{\theta^{FT}}\circ f_{\hat{\theta}^{PT}}$, since predictors for different vision tasks rely on different high-level semantics for decisions uijlings2013selective; Jiang1996Fast (e.g., object color, texture, and overlap for object detection). Therefore, triggers embedded into the backbone using poisoned dataset $\hat{\mathbf{D}}$ with task-specific features will be easily ignored by predictors targeting different tasks after fine-tuning. To address the problem, we need to introduce task-irrelevant patterns into the trigger patterns to generate poisoned dataset $\hat{\mathbf{D}}$ for training.

Challenge ❷: Shortcut connections towards the target annotations are non-matched when transferred to the target domain. Traditional poisoning process aims to embed backdoor by learning the combination of trigger patterns and clean images as Eqn 1 (e.g., patch-based or perturbation-based attacks). This training paradigm builds the shortcut in $f_{\theta^{PT}}$ by training on the poisoned samples, where triggers are placed on the clean images with backgrounds. In this way, the backdoor shortcut is primarily built from the combination of triggers and contextual backgrounds (i.e., $\hat{\bm{x}}=\phi(\bm{x},\bm{T})$) to the target class $\hat{\bm{y}}$. By contrast, the connections between the trigger itself $\bm{T}$ and the target label are comparatively weak. The shortcuts connected this way are suitable and effective for image classification since classifiers use the full image $\bm{x}$ for prediction. However, they are less useful for detection and segmentation, where the attacking label $\hat{{\mathbf{y}}}$ is the bounding-box-level or pixel-level prediction on target objects (i.e., triggers $\bm{T}$). To achieve a feasible attack, the attacker should consider training and building shortcut connections that match the annotations in different downstream vision tasks.

In this paper, we aim to embed backdoors into the PVMs so that the backdoors can be inherited for subsequent downstream vision tasks after fine-tuning. As illustrated in Section 3.1, given the clean training dataset $\mathbf{D}$, the attack poisons parts of the clean dataset with backdoor triggers; after pre-trained on the poisoned dataset $\hat{\mathbf{D}}$, the backbone $f_{\hat{\theta}^{PT}}$ is infected with backdoors; and when fine-tuned to different downstream tasks, the backdoor will still exist in the model and can be effective on subsequent tasks. Since there exist multiple different vision tasks, this paper primarily chooses the most classical and representative one as the downstream task for verification (i.e., object detection and instance segmentation). For object detection, there are several directions for backdooring Chan2022BadDet. Since these different directions generate attacks in the same attacking pipeline with differences in loss functions, this paper selects Object Generation Attack (OGA) for verification, where the trigger can generate a false positive bounding box of the target class $\hat{{\mathbf{y}}}$ surrounding the trigger pattern at a random position. Similarly, the attacker’s goal for the instance segmentation task is to generate a false positive bounding box with pixel-wise labels of target class $\hat{{\mathbf{y}}}$.

Our attack can be applicable to most pre-training and fine-tuning scenarios. Adversaries do not need to manipulate the training process or access the model parameters, and they can directly perform backdoor attacks by poisoning a very small portion of the training dataset for the target model. Downstream users are unaware of these existed backdoors, they will download these infected PVMs and fine-tune them with extra predictors to downstream tasks, where backdoors are still effective. Additionally, there also exists the possibility that the attacker cannot directly poison the training dataset and perform poison training from scratch (especially large vision models). Therefore, we also evaluate our attacks by injecting backdoors via fine-tuning a pre-trained clean PVMs on limited poisoned images (c.f. Section LABEL:sec:largemodel).

Following the common assumptions in backdoor attacks gu2019badnets; chen2017targeted, this paper considers the scenario where the adversary has no access to the model information. To inject the backdoors, the adversary has the capability to poison some of the training samples for the target model. To better simulate the real-world settings where the large-scale training samples are not accessible to attackers, we also consider a scenario where the target models have already been pre-trained on clean images and published publicly. In this scenario, the adversary downloads the PVMs and injects backdoors by fine-tuning the model on limited numbers of poisoned images; the attacker then releases the infected model on their own website, which is quite similar to the original repository and will mislead some users into downloading it.

Studies have revealed the fact that deep neural networks (DNNs) rely heavily on low-level texture features when making predictions geirhos2018imagenet, and these models can still perform well on patch-shuffled images where local object textures are not destroyed zhang2019interpreting. Therefore, we aim to introduce texture features associated with the target class into the trigger patterns to generate triggers that activate among different downstream tasks. These textures could reflect the nature of the target class, which can be better exploited and recognized by models among different tasks.

However, simply learning at the pixel level and introducing per-pixel information of the target class into trigger patterns makes it difficult to extract and capture overall perceptual textures. Therefore, we adopt the style transfer technique that aims to transfer the style (texture/color) of one image to the source image while maintaining the original content (structure/shape) johnson2016perceptual, which has been widely used in artistic image generation and texture synthesis Gatys2015Texture; Gatys2015Neural. In this paper, we develop a trigger generator $\mathcal{G}$ to generate trigger patterns $\bm{T}$ based on the textures of specific style image $\bm{x}^{s}$ (from target class $\bm{\hat{y}}$) and the structure of a content image $\bm{x}^{c}$. We will then illustrate the design and development of our trigger generator.

Specifically, to force the generation of textures/style from specific style images, we introduce the style loss that specifically measures the style differences and encourages the reproduction of texture details (e.g., colors, textures, common patterns) as