AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Models

T2I models are extensively used in various applications [22, 1, 2, 23, 24, 25, 26]. Despite their strengths, they are susceptible to adversarial attacks that modify prompts to sneak past defenses and produce NSFW content like pornography, violence, or politically sensitive imagery [19, 5, 27, 28, 29].

Present adversarial attack methods are mainly divided into white-box and black-box approaches. White-box methods primarily utilize the model’s text encoder to optimize the prompt, ensuring that the generated prompt semantically aligns closely with a target prompt containing explicit NSFW information, even without sensitive words [19]. In contrast, black-box methods perturb the prompt to find alternative tokens that can replace sensitive words [5, 30]. These methods often utilize reinforcement learning or assistance from large language models to accelerate the search process. In addition, some attack strategies target T2I models with removed concepts [27, 28]. These strategies demonstrate that T2I models can still generate NSFW images even after removing NSFW concepts.

Current adversarial methods have proven highly effective. Consequently, it is crucial to develop robust defense mechanisms to counter these attack strategies and ensure the safe and responsible use of T2I models.

Existing defense methods against attacks in T2I models fall into two categories: internal and external safeguards [7].

Internal safeguards aim to disable the model’s ability to generate NSFW images by fine-tuning the model itself. They can be divided into model editing and inference guidance. Model editing methods [8, 9, 10, 31, 32, 33, 34] aim to modify the internal parameters by training. However, these methods typically require prolonged training periods, and parameter modifications can impact the quality of the generated images. Moreover, most methods focus solely on safety against malicious prompts while ignoring the adversarial prompts. In contrast, inference guidance methods [11, 12] focus on modifying internal features during the inference stage. Unlike model editing methods, they are tuning-free and plug-in, which can be easily inserted into any model. However, these methods also fail to account for adversarial prompts, making them susceptible to targeted attacks.

External safeguards aim to filter out potential malicious samples by examining intermediate variables during the generation process. Current detection methods focus on prompts, conditional embeddings, or generated images. Image-based moderation [17, 4, 18] entails reviewing the generated images to identify NSFW samples. They incur significant inference costs since images must be generated before assessment. Prompt-based moderation [13, 35, 36, 14, 37] screens input prompts to identify those likely to generate NSFW images. Given their lower cost, they are widely used by online services like Midjourney [38] and Leonardo.Ai [39]. Nonetheless, these methods generally lack targeted defenses against adversarial attacks, making them susceptible to circumvention. Embedding-based moderation [16, 15] examines the conditional embeddings to filter out malicious samples. While this approach offers some resistance to adversarial attacks, it relies on large-scale models for classification, resulting in high costs. Additionally, it also suffers from low accuracy and remains vulnerable to adaptive attacks.

As the most widely used text encoder in T2I models, the CLIP model [40] is a significant focus of adversarial attack research. Recent studies [21, 41, 42, 43] have investigated the CLIP model to analyze its internal mechanisms. For instance, Bhalla et al. [41] found that the embeddings generated by CLIP exhibit strong linear properties and can be decomposed into combinations of various concepts. Gandelsman et al. [21] discovered that different attention heads within the CLIP model are responsible for interpreting different semantics, which are then combined to produce the final output embeddings. However, these studies primarily focus on the image domain. In this paper, we build upon existing work to further explore the properties of the CLIP model in the text domain and propose a novel interpretation method to interpret how prompts containing NSFW semantics are generated.

In this section, we explore how NSFW semantics are concealed within the conditional embeddings and revealed in the text encoder’s hidden states. Our investigation focuses on the CLIP model [40], the most widely used text encoder in T2I models and the primary target of adversarial attacks.

We classify the T2I model prompts into three categories: benign prompts, which do not generate NSFW images; regular NSFW prompts, which are manually crafted with explicit NSFW semantics; and adversarial prompts, which are generated by adversarial attacks and are challenging for humans to interpret as NSFW. Through extensive data collection, we compile a dataset consisting of approximately 25,000 benign prompts, 5,000 regular NSFW prompts, and 2,000 adversarial prompts. We utilize the CLIP model to obtain pooled embeddings of these prompts and examine their distribution with a PCA map [44], as depicted in Figure 1. Although the distributions of benign and NSFW data differ, considerable overlap makes effective differentiation challenging.

This phenomenon is primarily attributed to the architecture of the CLIP network, which comprises multiple layers and attention heads. Each layer employs a multi-head self-attention mechanism [45], where each attention head independently extracts information from the prompt. The outputs of these heads are then combined through a linear layer and passed to the next layer. The linear layers integrate information from various attention heads, making it challenging to isolate individual pieces of information. As the network’s depth increases, the outputs from different layers are accumulated through residual connections, exacerbating the entanglement of information. Consequently, benign prompts and NSFW prompts become intermixed within the pooled embedding space. As illustrated in the figure, two sentences describing the same object might appear semantically similar; however, one could be used to generate NSFW images while the other remains benign.

To more accurately delineate the boundary between NSFW and benign prompts, we examine the internal workings of the model by exploring the hidden states across different layers and attention heads. Figure 2 illustrates the distribution of benign, regular NSFW and adversarial data across several attention heads. The first column illustrates their overall distribution. In some heads, significant overlap remains between benign and NSFW data distributions. However, in other heads, a distinct boundary between these data types is evident. This reveals that attention heads within the model exhibit differing sensitivities to NSFW content. Certain attention heads concentrate on the NSFW semantics within prompts, allowing them to differentiate between benign and NSFW prompts effectively. We include the distribution maps of all attention heads in the appendix for reference.

We also find that different NSFW prompt categories are processed uniquely across attention heads. Columns two, three, and four of Figure 2 illustrate how violence, hate, and sex categories differ from benign data. In the first attention head, violence and hate data are separable from benign data, whereas sex data shows overlap. The second attention head clearly differentiates violence from benign data, with weaker distinctions in the other categories. In the third head, sex and hate data are distinctly separated from benign data, while violence data overlaps significantly. These findings suggest that attention heads specialize in handling specific NSFW content types. Even if a head struggles to differentiate between benign and NSFW data, it may still excel at distinguishing a particular type of NSFW prompt.

Based on these findings, we propose leveraging the attention heads within the text encoder to differentiate between benign and NSFW prompts effectively. By capitalizing on the diverse focus of different attention heads, we can aggregate information from all attention heads to achieve more accurate prompt classification.

Based on the findings from Section 3.1, we develop a framework for detecting and interpreting NSFW prompts. The overall architecture of this framework is illustrated in Figure 3. Initially, we identify the direction within each attention head’s hidden states that best represent NSFW semantics by analyzing the distribution differences between benign and NSFW prompts, which we call NSFW features. When a new prompt is inputted, we can assess the risk of generating NSFW images by evaluating the projection magnitude of the prompt along the NSFW feature. To interpret our assessment, we pinpoint the token most representative of NSFW semantics within the prompt, analyzing why the current prompt is classified as an NSFW prompt. Furthermore, we can progressively eliminate NSFW semantics from the hidden states and input the modified prompt embeddings to generate images. This process enables us to observe how NSFW semantics are gradually removed from the embeddings. Finally, we can conduct red team testing on the framework or monitor it in real-time post-deployment to collect prompts that bypass current defenses. By incorporating them into the training set, we achieve more accurate NSFW feature identification and detection results.

The practical application of this framework manifests in two main ways. First, it prevents NSFW image generation from the outset when malicious users attempt to employ adversarial prompts. Second, when regular users inadvertently input prompts with NSFW semantics and encounter blockages, we can interpret why the prompt is considered inappropriate. This assists users in swiftly identifying problematic tokens and modifying their prompts to generate the desired image.

The CLIP text encoder consists of $L$ layers, each with a multi-head self-attention mechanism with $H$ heads followed by an MLP block. A prompt $P$ is divided into $N-1$ tokens and projected into initial token embeddings $\{z_{i}^{0}\}_{i\in\{0,...,N\}}$, where $\{z_{0}^{0}\}$ is the BOS token and $\{z_{N}^{0}\}$ is the EOS token. These embeddings form the matrix $Z_{0}$, the initial input to the encoder. Each layer updates this input through self-attention and MLP modules with two residual steps:

$$\hat{Z}^{l}=\text{ATT}^{l}(Z^{l-1})+Z^{l-1},\quad Z^{l}=\text{MLP}^{l}(\hat{Z}^{l})+\hat{Z}^{l}.$$

(1)

In this framework, the ATT layer employs $H$ attention heads to extract information and integrates them into a vector through linear projection. The MLP layer further refines them to obtain intermediate embeddings that represent the overall information of the prompt. However, some information may be obscured or discarded during this process. Therefore, we need to utilize the original outputs from the attention heads to extract NSFW semantics effectively.

Considering that the outputs from multiple attention heads have been preliminarily integrated into $\text{ATT}^{l}(Z^{l-1})$, we need to decompose its computational process to extract the information each attention head represents. Since the CLIP model’s self-attention block employs a causal mask [40], only the EOS token holds the complete semantics of the prompt. Therefore, we concentrate solely on the EOS token. Following Elhage et al. [46] and Gandelsman et al. [21], we formalize the ATT output as a sum over $H$ independent attention heads and $N+1$ tokens:

	$\displaystyle\left[\text{ATT}^{l}(Z^{l-1})\right]_{EOS}$	$\displaystyle=\sum_{h=1}^{H}\sum_{i=0}^{N}x_{i,EOS}^{l,h},$		(2)
	$\displaystyle x_{i,EOS}^{l,h}$	$\displaystyle=\alpha_{i,EOS}^{l,h}W_{VO}^{l,h}z_{i}^{l-1}.$

where $W_{VO}^{l,h}$ are transition matrices and $\alpha_{i,EOS}^{l,h}$ are the attention weights from the $i$-th token to the EOS token. In this way, we can get the contribution of $h$-th attention head in $l$-th layer for $h\in[1,H],l\in[1,L]$, which can be expressed as $c^{l,h}=\sum_{i=0}^{N}x_{i,EOS}^{l,h}$.

Each $c^{l,h}$ is located within a $d$-dimensional representation space, and our goal is to identify the direction within this space that best represents NSFW semantics. When a new prompt is input, the more its representation in attention heads aligns with the NSFW direction, the more likely it is to contain NSFW semantics. We refer to these directions as the NSFW features of each attention head.

To calculate NSFW features, we introduce two sets of prompts: benign prompts and NSFW prompts, where NSFW prompts include regular NSFW prompts and adversarial prompts. We input these prompts into the CLIP model and obtain their outputs of each attention head, denoting the outputs of benign prompts as $\{c_{b_{k}}^{l,h}\}$ and the outputs of NSFW prompts as $\{c_{m_{k}}^{l,h}\}$. Our objective is to maximize $\langle u^{l,h},c_{m_{k}}^{l,h}\rangle$ while minimizing $\langle u^{l,h},c_{b_{k}}^{l,h}\rangle$, where $\{u^{l,h}\}$ are NSFW features we want to extract. To achieve this, we establish three optimization objectives:

$$\max\|\langle u^{l,h},\mu_{m}^{l,h}\rangle-\langle u^{l,h},\mu_{b}^{l,h}\rangle\|,$$

(3)

$$\min\sum_{k=0}^{K_{b}}(\langle u^{l,h},c_{b_{k}}^{l,h}\rangle-\langle u^{l,h},\mu_{b}^{l,h}\rangle)^{2},$$

(4)

$$\min\sum_{k=0}^{K_{m}}(\langle u^{l,h},c_{m_{k}}^{l,h}\rangle-\langle u^{l,h},\mu_{m}^{l,h}\rangle)^{2},$$

(5)

where $\mu_{b}^{l,h}$ and $\mu_{m}^{l,h}$ are the mean value of $\{c_{b_{k}}^{l,h}\}$ and $\{c_{m_{k}}^{l,h}\}$. In summary, our goal is to maximize the distance between the projected means of $\{c_{b_{k}}^{l,h}\}$ and $\{c_{m_{k}}^{l,h}\}$ on the vector $u^{l,h}$, while simultaneously minimizing their respective variances. We can employ Linear Discriminant Analysis (LDA) [47] to solve this problem, ultimately obtaining the NSFW feature $\{u^{l,h}\}$:

$$u^{l,h}=S_{w}^{-1}(\mu_{m}^{l,h}-\mu_{b}^{l,h}),$$

(6)

$$S_{w}=\sum_{k=0}^{K_{m}}\|c_{m_{k}}^{l,h}-\mu_{m}^{l,h}\|^{2}+\sum_{k=0}^{K_{b}}\|c_{b_{k}}^{l,h}-\mu_{b}^{l,h}\|^{2}$$

(7)

In text encoders other than CLIP, such as T5 [48], we can still use the method above to extract NSFW features. Any text encoder utilizing multi-head self-attention can be adapted to this approach.

By utilizing the identified NSFW features, we can detect NSFW prompts. According to previous research [21], we can consider the intermediate embeddings of prompts in the CLIP model as a linear combination of concepts. The projection of these embeddings onto each concept direction represents the contribution of that concept to the embedding. Based on this, we define the projection of the embedding onto the NSFW feature as the NSFW score of a prompt $p$:

$$Score(p)^{l,h}=Proj(c_{p}^{l,h},u^{l,h})=\frac{\langle c_{p}^{l,h},u^{l,h}\rangle}{\|u^{l,h}\|}.$$

(8)

By aggregating the NSFW Scores from all attention heads, we can obtain the final NSFW Score for the current prompt:

$$Score(p)=\frac{\sum_{l=1}^{L}\sum_{h=1}^{H}Score(p)^{l,h}}{L\cdot H}.$$

(9)

The larger the $Score(p)$, the more likely the current prompt contains NSFW semantics.

Theoretically, if $\text{Score}(p)>0$, the prompt $p$ contains NSFW semantics and should be classified as an NSFW prompt. Conversely, if $\text{Score}(p)<0$, the prompt should be classified as benign. However, experimental results show that while setting the threshold to zero allows AEIOU to achieve high accuracy, optimal classification performance requires a slight threshold adjustment. We hypothesize that this is due to the distribution of the training set not fully representing the actual distribution of NSFW prompts, which introduces bias in the NSFW features derived during training. In our experiments, we determine the classification threshold by selecting the one that yields the highest F1 Score on the training set. For different text encoders, the final offset ranges from 1% to 3%.

The approach above treats NSFW as a single comprehensive category. Suppose there is a need to subdivide it further or to identify specific categories of NSFW prompts, such as sex or violence. In that case, we can categorize NSFW prompts in the training set based on labels. This enables the calculation of NSFW features for each subcategory, facilitating the determination of NSFW scores for each. If we also need to detect NSFW prompts across all categories, we can aggregate the NSFW scores from all subcategories and use their maximum value as the final NSFW score.

Moreover, we can collect adversarial prompts that successfully bypass detection by employing adaptive attacks during red team testing. Incorporating these into the training set for data augmentation allows us to achieve more accurate NSFW feature extraction and detection results. Since adaptive attacks require significant time and have a low success rate, generating a large volume of red-teaming data is challenging. However, we can increase their impact during training by assigning greater weight to these data. Specifically, by weighting the target prompt’s $c_{m}^{l,h}$ when calculating $u_{m}^{l,h}$ and $S_{w}$, we can amplify their influence on the resulting NSFW feature. Experiments have shown that optimizing the NSFW feature through data augmentation can effectively reduce the success rate of adaptive attacks.

After the detection process, we further interpret NSFW prompts through a two-module framework. First, we propose a novel interpretative method that uses NSFW features and the prompt’s hidden states to identify the tokens most representative of NSFW semantics. Second, we investigate the generation mechanism of NSFW semantics in conditional embeddings by gradually attenuating the NSFW features in the hidden states. Using the resulting embeddings to generate images, we gain insights into how NSFW semantics can be progressively eliminated.

First, we conduct an overall evaluation of the effectiveness of AEIOU. We deploy it across three different text encoders and trained models using three distinct settings. AEIOU is then compared against seven baselines on one benign dataset and eight NSFW datasets. We randomly select half of each dataset as the training set and use the remaining half as the test set.

Table II presents the overall evaluation results across all datasets, with the best performance for each metric highlighted in bold. As shown in the table, AEIOU consistently outperforms previous classification approaches across almost all metrics. It demonstrates high detection accuracy across the CLIP-L, CLIP-G, and T5 models, proving its applicability to various text encoders.

In Table III, we further present the accuracy of various classifiers on each dataset. The first six columns represent regular NSFW datasets, and the last three correspond to adversarial datasets. AEIOU maintains high accuracy across all datasets, whereas the performance of other methods is inconsistent. The accuracy of most methods is low on the I2P dataset. This is likely because they need to detect text across various applications, making it hard to handle prompts used in T2I models specifically. On other regular NSFW datasets, which typically contain human-recognizable NSFW semantics. Most classifiers achieve relatively high accuracy but still lag behind AEIOU. Notably, on the 4chan dataset, three classifiers achieve higher accuracy than AEIOU. However, the difference is very minimal. Finally, regarding adversarial prompts, AEIOU significantly outperforms all other methods. Even AEIOU${}_{\text{ua}}$, which is trained without adversarial datasets, still achieves remarkably high accuracy.

Additionally, we assess each method’s efficiency by measuring the average time per query, presented in the last column of Table II. AEIOU requires significantly less time than other methods, primarily because they often utilize large models for detection. In contrast, AEIOU only incorporates multiple matrix operations during the text encoder’s inference process. On the smallest CLIP-L model, AEIOU’s efficiency improves at least tenfold compared to other models. Even on larger models like CLIP-G and T5, AEIOU’s efficiency surpasses that of all other models.

This section examines AEIOU’s ability to defend against unknown adversarial attacks. In Tables II and III, we present the performance of AEIOU trained solely on benign and regular NSFW datasets, denoted as AEIOU${}_{\text{ua}}$. Despite never encountering adversarial prompts, the experimental results indicate that AEIOU${}_{\text{ua}}$ can still effectively identify adversarial NSFW prompts, with accuracy only slightly lower than the standard AEIOU. We attribute this effectiveness to AEIOU’s focus on the semantic information embedded in the hidden states. Although adversarial and regular NSFW prompts may appear different to the human eye, their semantic information is similar, allowing AEIOU${}_{\text{ua}}$ to recognize them accurately.

NSFW serves as an overarching descriptor for harmful prompts, and it can be decomposed into more specific categories. In this section, we follow the I2P dataset to classify NSFW prompts into seven particular categories: sexual, hate, self-harm, violence, shocking, harassment, and illegal. Adhering to the methodology described in Section 3.4, we identify features representing these concepts and derive the multi-categories AEIOU by integrating the NSFW scores of each category. Tables II and III compare the overall performance of AEIOU${}_{\text{multi}}$ with the standard AEIOU. Although the detailed AEIOU${}_{\text{multi}}$ exhibits slightly worse overall performance, the difference is minimal. Furthermore, Table IV compares their accuracies within each category. AEIOU${}_{\text{multi}}$ generally achieves higher accuracy in most categories, but in some, it underperforms compared to the standard AEIOU.

We attribute the lack of superiority in AEIOU${}_{\text{multi}}$ to two main reasons. First, prompts from different categories often share overlapping features. Common sensitive words like “f**k” appear across multiple categories, which limits AEIOU${}_{\text{multi}}$’s ability to capture shared features when trained individually on each category. As a result, it demonstrates higher accuracy in more distinct categories like self-harm but lower accuracy in more ambiguous categories such as hate and harassment. Secondly, discrepancies in data quality exist among different categories. In our datasets, sexual prompts have broad coverage and the highest quality, while the quality of other categories’ prompts varies significantly. This leads to suboptimal performance of AEIOU${}_{\text{multi}}$ on some categories. To improve AEIOU${}_{\text{multi}}$’s performance, we need to train it using higher-quality datasets.

In this section, we discuss the impact of training data size on the performance of AEIOU. We set the training data size for AEIOU to 10, 50, 100, 500 and 1000. Half of the training data is randomly selected benign data, while the other half is randomly selected NSFW data. Table V presents the experimental results of AEIOU${}_{\text{CLIP-L}}$. The experiments demonstrate that AEIOU maintains high accuracy even with only 10 training samples. When the sample size reaches 500, its performance is comparable to AEIOU trained with a full dataset. As the number of training samples increases further, there is no significant improvement in performance, indicating that improving the quality and coverage of training samples is a better strategy than simply increasing the quantity. In the appendix, we present detailed experimental results on the other two text encoders, which yield conclusions similar to CLIP-L. Figure 5 provides additional insights with different training data sizes across various datasets, which aligns with the results in Table V. These experiments confirm AEIOU’s exceptional performance in few-shot scenarios.

In our approach, we utilize NSFW features from all layers and all attention heads for detection. In this section, we will discuss the impact of using NSFW features from only a single layer of text encoder. We conduct experiments on three text encoders. Figure 6 presents the accuracy, TPR, and FPR when using each layer for detection. More detailed experimental data are presented in the appendix.

In all three text encoders, even when using attention heads from a single layer, many layers still achieve high accuracy. For CLIP-L and CLIP-G, the middle layers tend to have higher accuracy, while the early and final layers show lower accuracy. Conversely, in the T5 model, the later layers exhibit higher accuracy. This highlights the distinct characteristics of the two types of text encoders.

Although using attention heads from a single layer can achieve high accuracy, we recommend using the original AEIOU method for the highest precision in detection.

In the black-box scenario, we assume the attacker has no knowledge of the model’s details but can choose prompts and query the model to obtain output results. We integrate AEIOU into the text encoder of Stable Diffusion. When a potential NSFW prompt is detected, the model will refuse to generate the image. We select 100 prompts with clear NSFW semantics from NSFW200 as the target prompts and use SneakyPrompt to attack the models. The experimental results are shown in Table VI.

When attacking the bare model, SneakyPrompt achieves a high bypass rate and attack success rate. ESD significantly reduces the success rate of attacks, yet it cannot entirely prevent the generation of NSFW images. However, after incorporating AEIOU, SneakyPrompt is entirely thwarted by our defenses, unable to generate any NSFW images, and the bypass rate drops to 0. This is because SneakyPrompt only replaces a few tokens in the prompt, which does not effectively neutralize its overall meaning.

In a white-box scenario, we assume attackers can only generate images through queries. However, they possess a local copy of the text encoder identical to the target model and are aware of the AEIOU defense strategy. The attacker can target AEIOU by modifying the loss function to conduct a specific attack. The MMA attack’s loss aims to make the conditional embeddings of the adversarial prompt and the target prompt as similar as possible. To effectively attack AEIOU, we incorporate $Score(p)$ as $L_{\text{AEIOU}}$ into the original loss function. This transforms the objective of the loss function to minimize the NSFW Score while ensuring that the semantics of the adversarial prompt closely align with the target prompt.

$$L=L_{\text{MMA}}+\lambda\times L_{\text{AEIOU}}$$

(16)

$$L_{\text{AEIOU}}=Score(p),$$

(17)

Where $\lambda$ is a weighting factor that balances between the two components. Based on this foundation, we implement a target truncation strategy. Specifically, once $L_{\text{AEIOU}}$ exceeds the threshold by a small margin, we stop optimizing it and shift our primary focus to optimizing $L_{\text{MMA}}$. This enables adversarial prompts to approximate the semantics of the target prompt as closely as possible while avoiding detection. Consequently, the final loss is formulated as:

$$L=L_{\text{MMA}}+\lambda\times\max(L_{\text{AEIOU}},\tau-\epsilon),$$

(18)

Where $\tau$ is the threshold and $\epsilon$ is the margin. When updating the best prompt, we ensure that the $L_{\text{AEIOU}}$ surpasses the threshold. We conduct the attack using the default settings of MMA and evaluate it on 100 target prompts. The experimental results are presented in Table VI.

As a white-box attack, MMA demonstrates stronger capabilities than SneakyPrompt on the bare model. However, when it targets AEIOU, AEIOU exhibits robust performance, significantly reducing the MMA bypass rate and success rate. Meanwhile, the average CLIP score of successfully generated adversarial prompts also decreases, indicating that MMA’s optimization of these prompts is not as successful as in the bare model. Moreover, although AEIOU itself does not increase the memory usage of the text encoder, the adaptive attack against it must optimize hidden states across all attention heads, significantly increasing the memory requirements. While standard MMA operates on less than 10GB of memory, adaptive MMA demands nearly 50GB, restricting its execution to commercial-grade GPUs. This makes adaptive attacks on AEIOU more challenging.

We can also combine AEIOU with other defense methods. For instance, by integrating ESD, we can reduce the success rate of MMA attacks to zero. By implementing a comprehensive defense strategy that addresses other parts of the T2I model, we can enhance the overall defensive performance and make adaptive attacks more challenging.

To further enhance AEIOU’s resilience against adaptive attacks, we incorporate adversarial prompts that successfully breach AEIOU’s defenses into the training set for additional training. The following section will provide a detailed discussion of this process.

We conduct red team testing using a white-box adaptive attack and collect adversarial prompts that successfully bypass AEIOU and generate NSFW images. These prompts are added to AEIOU’s training dataset for data augmentation, aiming to enhance AEIOU’s ability to resist corresponding adaptive attacks. When training, we assign them greater weight to ensure they significantly influence the model even the sample size is limited.

Table VI presents the performance of the data-augmented AEIOU. We include 25 adaptive adversarial prompts in the training set with a weight of 50. This results in AEIOU${}_{\text{DA}}$, demonstrating significantly improved defense against adaptive attacks, reducing the bypass rate to 42% and the success rate to just 7% for MMA. Additionally, the CLIP score experiences a further decline. This underscores the effectiveness of further optimizing AEIOU with data augmentation.

In the practical application of AEIOU, we can also collect adversarial prompts that successfully bypass defenses through manual screening. These prompts can then be added to the training dataset with appropriate weighting, allowing for continuous updates of the defense model.

In the text modality, our interpretation aids users in understanding the semantics of a prompt by identifying tokens containing NSFW semantics. After obtaining the interpretation result $E(p)_{i}$ for each token in the prompt $p$, we sequentially remove the corresponding tokens from $p$ in descending order of $E(p)_{i}$ and observe the changes in the NSFW score. We compare our interpretation method with the random removal of tokens on 100 randomly selected NSFW prompts, and the results are shown in Figure 7. Compared to randomly removing, removing tokens based on the interpretation results more quickly weakens the NSFW semantics of the prompt, demonstrating that AEIOU can effectively identify NSFW tokens within a prompt.

Additionally, we conduct further experiments on image generation using the 100 prompts above. We will compare the images generated from the original prompt with those generated from the prompt after removing the NSFW token. For original prompts, 87 out of the 100 generated images are NSFW. After removing NSFW tokens, only 5 prompts result in NSFW images. This further demonstrates the effectiveness of our interpretation method.

Directly removing tokens with NSFW semantics can prevent the generation of NSFW images. However, it significantly disrupts the original intent of the prompt. To address this drawback, we manipulate the embeddings of each token in the image-based interpretation process, gradually eliminating NSFW semantics and observing changes in the generated images. We use the same 100 prompts as in the previous section. We evaluate the degree to which the generated images contain NSFW semantics and how closely they align with the original prompt semantics.

Figure 8 illustrates the variation in the number of NSFW images generated and the semantic similarity between the images and prompts as the parameter $\beta$ changes. The semantic similarity is evaluated using the CLIP Score. As $\beta$ increases, the number of NSFW images decreases progressively. Although the CLIP Score also shows a decreasing trend, the overall deviation from the original image’s CLIP Score remains minimal. This indicates that the NSFW semantics of the prompt are effectively mitigated while preserving other semantic information as much as possible.

However, because this method directly modifies semantics within the hidden states, the resulting conditional embeddings deviate from the normal distribution, often leading to lower-quality images. Therefore, this approach is supposed to aid in understanding the representation of NSFW semantics within the text encoder. It cannot be directly applied to erase specific concepts from images.