Avengers Ensemble! Improving Transferability of Authorship Obfuscation

Stylometry is the analysis of an author’s writing style that helps distinguish them from other authors. For example, writeprints [2] is a well-known stylometric feature set that has been used to analyze the writing style for the sake for authorship attribution. The primary goal of authorship obfuscation is to evade attribution by concealing such stylometric features in the document while retaining its original meaning. Early approaches such as Anonymouth [35] highlighted the distinctive stylometric properties of the text that could then be modified by the user to evade attribution. Follow up work at PAN-CLEF aimed to automatically obfuscate documents using simple predefined rules. For example, Mansoorizadeh et al. [34] used WordNet to identify synonyms for the most commonly used words by the author and replaced them with similar words. Castro et al. [9] used sentence simplification techniques, such as replacing contractions with their expansions, to obfuscate a document. Keswani et al. [29] used round-trip translation ($English\rightarrow German\rightarrow French\rightarrow English$) to obfuscate a document. While these automated approaches managed to evade attribution, they severely compromised the obfuscated text’s semantics. These approaches, rather unsuccessfully, navigate the trade-off between evading attribution and preserving semantics [33].

Recent work such as A⁴NT [43], Mutant-X[33], and ParChoice [22] employ more sophisticated adversarial obfuscation to evade authorship attribution classifiers. Their threat model assumes that the obfuscator can query the adversary’s attribution classifier to guide obfuscation. For example, A⁴NT uses a generative adversarial network (GAN) for obfuscation that requires white-box access to the adversary’s attribution classifier. Mutant-X uses a genetic algorithm for obfuscation that requires black-box access to the adversary’s attribution classifier. ParChoice uses combinatorial paraphrasing for obfuscation that requires black-box access to adversary’s attribution classifier. While these obfuscation approaches achieve a better tradeoff between attribution evasion and preserving semantics, they all assume white/black box access to the the adversary’s attribution classifier. This key assumption limits their effectiveness in the real world because the adversary’s attribution classifier might be different or unknown. For example, the evasion effectiveness of Mutant-X drops drastically when the adversary uses a different attribution classifier than assumed by Mutant-X [33]. Similarly, an adversarially retrained attribution classifier is resistant to obfuscation by ParChoice using the original classifier [22]. This lack of transferability to unseen attribution classifiers has major ramifications in the real world as the obfuscator’s effectiveness is questionable when the adversary happens to use a different attribution classifier. Figure 1 provides an overview of this threat model involving the obfuscator and multiple unseen adversaries.

The obfuscator seeks to obfuscate the stylometric properties of an input document $D$ of author $A$ by modifying its text to produce an obfuscated document $D^{\prime}$ such that the attributor incorrectly classifies the obfuscated document to another author $A^{\prime}\neq A$. The state-of-the-art authorship obfuscators mainly consist of two components: a generator and an internal authorship attribution classifier $C$. The generator modifies the input document based on some rules and queries the internal classifier to predict whether these modifications would degrade the likelihood of successful authorship attribution. The two components work in tandem for $M$ iterations to progressively obfuscate the input document by generating new obfuscation samples and measuring the degradation in authorship attribution by $C$. It is noteworthy that the adversary might use a different authorship attribution classifier $C^{\prime}\neq C$. There could, in fact, be multiple adversaries in this setting, with each using a different attribution classifier $C^{\prime}$ than the obfuscator’s internal classifier $C$. The primary goal of the obfuscator is to obfuscate an input document $D\rightarrow D^{\prime}$ using its internal classifier $C$ such that it evades attribution by the adversary classifier $C^{\prime}$. This problem is also referred to as transferability in the field of adversarial machine learning.

The outputs of the base classifiers are then polled by the ensemble through either a majority vote or by training another classifier (a technique called stacking) [16]. A majority vote gives uniform weight to the output of each base classifier whereas stacking causes the weights to vary, as it can learn to downplay classifiers that are inaccurate more often. While these base classifiers might not be very accurate on their own, the ensemble together can capitalize on their knowledge and make more accurate predictions.

We construct our ensemble using the feature subspace method and describe it as follows. A subspace is a subset of the entire universal set of features that are available to the classifier. We train the base classifiers of the ensemble on different subspaces of the feature set. The goal of using a subspace of features is to train a base learner that is specialized in that distinct and local set of features. This is motivated by our findings of feature importance and decision boundaries which we discuss later in Section 4.2. The subspaces can be selected randomly [47], through sampling [50], or through feature selection techniques [17].

Figure 2 illustrates the architecture of our proposed ensemble. The original feature space is divided into multiple subspaces which are then used to train the base classifiers. The outputs from these base classifiers are then aggregated to produce the final classification of the ensemble for the given input.

The Extended Brennan Greenstadt Corpus [7] comprises of writing samples submitted by various authors through Amazon’s Mechanical Turk (AMT) platform. The corpus is unique because it was collected expressly for the purpose of adversarial stylometry in text and was vetted against a strict set of guidelines imposed by AMT and the authors themselves to ensure quality. The guidelines required that the submissions be professionally written, be free of anything other than the writing itself (i.e., citations, URLs, headings, etc.), and contained at least 6500 words. The imposition of these strict guidelines ensured that the submissions were of high quality, reflected the author’s particular writing style, and that there was sufficient data to train an attribution classifier. Out of the 100 submissions, the authors selected the 45 that most closely followed the guidelines and then split them into nearly 500 word passages, averaging 15 documents per author, to create the final corpus. In our experiments, we divide the corpus into groups of 5 authors based on document length and report results on these groupings by further splitting them into a 80% training and 20% testing set.

The fitness function for Mutant-X takes into account the detection probability of a given attribution classifier $C$ and the semantic similarity between the original and the obfuscated document. We train the two classifiers that were originally used for Mutant-X, a random forest classifier and a support vector machine, on the Writeprints feature set to serve as baselines for comparing the performance of the ensemble.

To assess the transferability of the obfuscated samples to other classifiers, we train a series of classifiers and measure the performance of the baselines and the ensemble. Specifically, we train multiple classifiers that use different types of techniques to measure the cross-technique transferability [38] of the method used to generate the samples. These classifiers are also trained on the Writeprints feature set and are as follows: k-nearest neighbors (KNN), naive-bayes (NB), multilayer-perceptron (MLP), logistic regression (LR) in addition to the already trained random forest classifier (RFC), support vector machine (SVM), and the ensemble (Ens) itself. Additionally, we incorporate counter-measures for the findings by Gröndahl et al. [22] that using an internal classifier results in highly specific transformations which are not only non-transferable to other different classifiers but also to the same classifier if it is retrained. We accomplish this by training multiple versions of the classifiers that exhibit randomness (RFC, Ensemble, and MLP) during training and then report the average transferability result in their respective columns.

In addition to these Writeprints based classifiers, we also measure the transferability of the samples to JGAAP [26], a well-known system for authorship attribution that provides a wide-array of features and classifiers, and another MLP model trained on the Basic-9 feature set[35]. This setting allows us to explore the performance of the ensemble against an adversary that uses a different feature set and classifier implementation. We borrow the configuration recommended by Juola et al [27] for JGAAP. The final configuration for JGAAP is listed in Table 1.

We conduct several experiments with each corresponding to a particular internal classifier that is used by Mutant-X. In each of these experiments, all Mutant-X parameters are kept consistent and only its internal classifier is replaced. We configure Mutant-X parameters based on the findings from the original pilot experiments performed on the EBG 5 dataset. The values for the different Mutant-X parameters are specified in Table 2. For each experiment, we report the average METEOR score of the obfuscated documents, the transferability rate across our fixed set of adversaries, and the overall attack success rate of that technique.

In its default setting, Mutant-X stops obfuscation once the mutated document has been misclassified by its attribution classifier. We alter this behavior to allow Mutant-X to continue obfuscation until all $M$ iterations have been performed. This alteration is partly inspired by the idea that stopping early because of one successful misclassification is detrimental to the overall goal of transferability to a wider set of adversaries.

Originally, Mutant-X was evaluated through two metrics that measured the safety and soundness of the obfuscation. While effective for measuring obfuscation across one adversary, they fail to quantify the transferability of the obfuscated samples to multiple adversaries. To alleviate this, we utilize a third metric called the Attack Success Rate (ASR) to capture this information and slightly modify the safety metric to accommodate it. The final metrics for evaluation are as follows.

1. 1.

   Evasion Effectiveness: An obfuscated document generated from an internal attribution classifier effectively evades an adversary if it is misclassified by that particular adversary. We refer to this property of the internal classifier as its transferability to an adversary and report it as the percentage of obfuscated documents produced by that classifier that were misclassified by the adversary. For an adversary $i$ that misclassifies $a_{i}$ out of $n$ obfuscated documents generated by the internal classifier, we measure transferability as:

   $$T_{i}=\frac{a_{i}}{n}\times 100$$

   (1)

2. 2.

   Attack Success Rate: The attack success rate [11] measures the overall transferability of the obfuscated documents across the entire set of adversaries. It is an average of all the transferability scores reported for that specific internal classifier. Given a fixed set of adversaries $m$, the attack success rate of classifier $A$ can be reported as:

   $$ASR_{A}=\frac{\sum_{i=1}^{m}T_{i}}{m}$$

   (2)

3. 3.

   Semantic Similarity: An obfuscated document has to maintain semantic similarity to the original document. As originally used to evaluate Mutant-X, we use the METEOR score [14] to assess this similarity. The score lies in the range [0, 1] with 1 indicating perfect similarity and 0 indicating the opposite. The final score reported is the average METEOR score of the obfuscated documents, where a higher score implies that the final documents were similar to the original ones.

The main results of the experiments are presented in Table 3. The rows correspond to the internal classifier used by Mutant-X and the subsequent columns correspond to the classifier used by the adversary and the feature set they are trained on. The cell values are the percentage of documents generated using that method that were misclassified by the adversary’s classifier, i.e., the transferability of that method. The final two columns contain the attack success rate (mean of transferability to adversaries) and the mean METEOR score of the technique. Additionally, we reiterate the counter-measure discussed in Section 3.3 and train multiple versions of classifiers that exhibit randomness during training. As such, the columns for the Ensemble, MLP, and RFC report the average transferability to different versions of the classifiers.

We notice that the cases where the baselines perform better are where the internal classifier and adversary’s classifier are the same ($SVM\rightarrow SVM,RFC\rightarrow RFC$); cases which are fairly trivial and advantageous to Mutant-X. In fact, the 93.7% in the case of $SVM\rightarrow SVM$ is unrealistic because the adversary’s classifier is exactly the same as Mutant-X, whereas $RFC\rightarrow RFC$ and $Ensemble\rightarrow Ensemble$ scenarios have been normalized by training multiple instances of the adversary and reporting their average (see Section 3.3). Regardless, the ensemble still manages to outperform the other baseline in these trivial cases.

The effects of re-training the ensemble and RFC are also evident in these cases. The average transferability of $RFC\rightarrow RFC$ is quite low at 28.2%, corroborating the findings of Gröndahl et al. [22]. However, this doesn’t appear to be true for the ensemble as it still reports a fairly high transferability at 71.9% indicating its robustness.

In the non-trivial cases where the adversary’s classifier is different from the internal classifier, the ensemble fares far better than the baselines. In the case of KNN, the ensemble achieves a transferability of 41.6% compared to the RFC at 19.4%. In the case of NB, it achieves a transferability of 52.9%, which is 32.3% higher than the SVM at 20.6%. On average, the ensemble achieves 21% higher transferability than the SVM and 13.8% higher than the RFC across the set of adversaries where the internal classifier is different.

A comparison of the overall performance (trivial and non-trivial) between the ensemble and the baselines shows that the ensemble outperforms both across the wide range of adversarial settings. The overall attack success rate of the ensemble at 38.0% is the best compared to the baselines, which is 1.7$\times$ higher than the transferability of the RFC at 21.7% and 2.1$\times$ higher than the transferability of the SVM at 18.3%.

The ensemble does not perform as well as the other methods when we compare the METEOR scores. The samples generated by RFC and SVM retain better semantic similarity to the original documents with an average METEOR score of 0.42 and 0.40, respectively. In contrast, the ensemble reports an average score of 0.36 indicating that the generated samples differed more from their original selves. We attribute this lower score to the problem of balance between protecting the author’s identity and being true to the original content. We believe that the effort to ensure transferability requires more substantial changes to be made to the document which leads to lower similarity between the source and the obfuscated text, and consequently a lower average METEOR score.

The superior effectiveness of the ensemble as an internal classifier is undeniable when compared to the baselines. The high attack success rate and a comparable METEOR score make it a reliable alternative to other conventional classifiers for use alongside an obfuscator like Mutant-X.

Within the JGAAP setting, the SVM does not perform as well as the other two settings. It surprisingly performs the worst in the $SVM\rightarrow SVM$ case, only achieving a transferability of 5%. We attribute this to the difference between the kernel functions of the two SVMs; as opposed to a linear kernel used in the internal classifier, JGAAP’s default setting uses a RBF kernel. In comparison, the ensemble and the RFC achieve higher degrees of transferability yielding an attack success rate of 34.3% and 24% respectively with the ensemble outperforming the RFC by 10.3%.

We see similar results in the Basic-9 setting, where the ensemble achieves a transferability that is 6% higher than RFC and almost twice as high the SVM. This affirms the idea that the ensemble performs just as well against adversaries trained on a different feature set and outperforms other conventional classifiers.

We now study the various decisions we’ve made concerning the ensemble and try to understand their impact on its inner workings.

To measure the diversity of an ensemble, we use the non-pairwise metric of entropy $E$ [30]. The entropy value of an ensemble lies in the range $[0,1]$, a value closer to 0 means that the individual classifiers mostly agree, and a value closer to 1 means that they mostly disagree with one another. This measure assumes that a diverse set of classifiers will disagree with one another as opposed to correlated classifiers which will agree more often.

We train multiple ensembles and control for their diversity by selecting appropriate base classifiers. Specifically, we create 4 bins of entropy values $E\in\{0,0.25,0.5,0.75\}$ and train 10 ensembles for each bin, each ensemble having approximately the same entropy as the bin it is assigned to. We then conduct 40 experimental runs of Mutant-X, and in each run use one of these ensembles as the internal classifier to obfuscate documents and measure the attack success rate of these documents against our set of adversaries.

Figure 3 shows the results from these experiments. The y-axis represents the attack success rates while x-axis ticks represent the entropy value of the bin. Contrary to our intuition, we notice that ensembles with higher values of entropy (more diverse) had lower transferability while ensembles with lower entropy (less diverse) performed comparatively better. We explain our interpretation of these results as follows.

Recalling that Mutant-X uses the confidence score of the internal classifier to make decisions regarding obfuscation, we investigate the impact of diversity on the accuracy of the ensemble and the confidence of the classifier in its classifications. To increase the diversity of an ensemble, we need to promote disagreement between the individual classifiers that comprise it. Consequently, this makes the ensemble overall less confident in its final classification even if it is correct. Since Mutant-X uses this confidence score as an indicator of attribution, a lower score leads to poorer decision making on Mutant-X’s part and reduces the quality of obfuscation. We note that this problem is likely unique to how Mutant-X operates and not necessarily an artefact of using ensembles. In future work, it will be interesting to explore how diversity impacts ensemble based transferability in different settings that are not bound by this restriction.

At a higher level, the Writeprints feature set incorporates lexical and syntactic features that are qualitatively distinct. This distinction indicates the presence of a contextual subspace within the feature set. More specifically, there are 9 distinct subspaces and are as follows: frequency of special characters, letters, digits, parts-of-speech tags and punctuation, most common letter bigrams and trigrams, percentage of certain function words, and the ratio of unique words (hapax ratio) [2]. We train the base classifiers on this division of subspaces and measure the attack success rate of the resultant ensemble and note that, in contrast to the random subspace method, this yields base classifiers that have different values of $L_{s}$.

Additionally, we explore feature selection techniques to construct the subspaces. Using a one-way ANOVA test [18], we measure the dependency between the features and the author label, and use the highest-ranking features to train the first base classifier. We repeat this for the next base classifier by considering the remaining set of features and so on for the rest of the classifiers. This yields base classifiers of varying performance as the initial ones are highly accurate but the accuracy gradually drops as the remaining features are insufficient predictors . For this particular experiment, we set $I_{c}=8$ and $L_{s}=20$ for a consistent distribution of features.

The results of the experiments are as follows: the contextual subspace ensemble yields an ASR of 37.1% whereas the feature selection subspace ensemble yields an ASR of 34.7%. We see that the results for the subspace ensemble are comparable with those of the random subspace ensemble which had an ASR of 38.0%.

Considering the security aspect of the techniques, the contextual subspaces approach is relatively more risky. Since the features composing these subspaces are easy to identify, an adversary can undo the effects of obfuscation through adversarial training: building a classifier to recognize obfuscation by training it on the obfuscated documents generated by the internal classifier [21]. In contrast, an ensemble built using random subspaces offers a good balance: achieving a commensurable degree of transferability and providing a good defence against adversarial training as its random nature is unpredictable for the adversary.

RFC is a collection of decision trees that also counts the votes of the individual outputs of the trees to make the final classification. The decision trees consist of several nodes that split the training set into subsets based on the values of certain features. Arguably, features that are used more often for splitting and split a sizable portion of the training set compared to others bear greater significance for the model. This is known as the Gini Importance of that feature [6] and it is the number of times a particular feature was used for a split weighted by the number of samples it splits.

SVM classifies the data by learning a hyperplane between the data points that separates the different class boundaries. In a linear SVM, this hyperplane represents the points at which the distance between the class boundaries is maximum. Since the coefficients of this hyperplane are associated with the features, their absolute values represent the significance of the corresponding feature relative to the other features. In a multi-class setting, the SVM has multiple hyperplanes separating each of the classes and each hyperplane has its own set of coefficients.

We assess the differences between what features are important for the RFC and SVM. Table 4 lists the top 5 features for the baseline RFC and one of the SVM hyperplanes. We note that these are different for the two classifiers; moreover, this trend holds even beyond the top 5 features. In a high-dimensional feature space such as Writeprints, this difference in feature emphasis by the classifier amounts to some features losing their relative importance and thus the obfuscator does not consider their relevance. This highlights a fundamental flaw in the obfuscator: the obfuscation will always be tuned to the features preferred by its internal classifier and fail to transfer to a different classifier emphasizing different feature.

Our approach of using feature subspaces in an ensemble alleviates this flaw to an extent; base classifiers trained on smaller random sets of features emphasize the importance of those features. The base classifier then specializes in its localized subspace of features and, while it may not be accurate, it is representative of a certain aspect of the feature space that might be of significance to an adversary’s classifier.

Taking a look at the decision boundaries of the base classifiers in Figure 4 helps explain how they improve transferability. We use Principal Component Analysis (PCA) [49] to reduce the higher-dimensional feature space to two-dimensions for the plotting the boundaries. The data points are the documents from the test set projected into the PCA dimensions. The colored regions in the background represent the decision regions of the classifier for that particular label, i.e., points that fall in those regions are classified according to that label. We stress that this two-dimensional projection is merely an approximation of the actual high-dimensional feature space so some misalignments are expected. Looking at the decision boundaries of the base classifiers, we see that they vary significantly and that some of the classifiers perform better at classifying a particular author than others. The decision boundaries also highlight the limited access the classifiers have to the entire feature space, observable by the disjoint between the same decision region. While the projection takes into account the entire feature space, the decision region is only based on the subspace the particular classifier was concerned with.

Figure 5 shows the decision boundary of the ensemble formed from these base classifiers. We see that the decision region of the ensemble more closely encapsulates the data points than the base classifiers. Since the ensemble classifies according to the majority vote of the base classifiers, its decision boundary is approximately the average of all their decision boundaries. The voting mechanism also ensures that the base classifiers are weighted equally so as to not downplay the role of a certain subspace. Therefore, the ensemble capitalizes on the individual knowledge of the base classifiers and effectively serves as a middle-ground for the obfuscator to compare against.