Adversarial Training Based Multi-Source Unsupervised Domain Adaptation
for Sentiment Analysis

In general, we assume the existence of a shared feature space between domains where the distribution divergence is small and reducing domain divergence through the adversarial training upon the deep learning framework can further exploit domain invariant features. We utilize a discriminator $D$ to implement adversarial training and obtain the corresponding loss. The objective of discriminator $D$ is:

where $\boldsymbol{d}_{i}$ is the domain label and $D(\cdot)$ can be considered as a classifier, which will output the label probabilities of domain prediction. After convergence of the adversarial training, $D$ can discriminate which domain the features coming from. The two terms signify that $D$ needs to discriminate the two parts of features separately.

It must be stressed that $D$ has three key roles. The first is to help acquire more pure shared features. The second is to help fulfill the separation of shared and private features through its discriminated ability. The last but not least is to measure how likely the target instance is coming from the corresponding source domains, which implicitly introduces the data-dependent priors of source domains.

$C$ is an usual classifier and used to classify sentiment polarities. Its objective is:

where $\boldsymbol{y}_{i}$ is the class label and $C(\cdot)$ is a function of classical classifier. This objective means the concatenated features will be sent to $C$ to calculate the final sentiment label. It is noticeable, however, that if we set all the private features to zeros or abandon them, such as previous methods did (?; ?; ?), will lead to knowledge loss and the system performance will decrease.

As discussed by ?; ?; ?, the independence of private features and shared features is beneficial for system performance. If we just utilize the discriminator to constrain the shared feature extractor, the domain-related knowledge will appear both in shared space and private space. To overcome this, ?; ? adopt the orthogonality constraints, which penalize redundant latent representations and encourages the shared and private extractors to encode different aspects of the inputs. Here, we deeply exploit $D$ to make a constraint, reserve the private characteristic of each domain in their private space and encourage moving the common information from each domain to their shared space. The objective of each private extractor $E_{p_{j}}$ is:

The private feature extractors have two aspects of roles, one is to be concatenated with shared features to contribute to the polarity classification, and the other is trying its best to retain more domain-related information for $D$ to discriminate the domain labels, which are depicted by the first term and the second term separately in Equation 4.

Through the adversarial training, the distribution divergence between different domains in the shared feature space will become a minimum and the features extracted by $E_{s}$ will be domain-invariant and can be shared across all the domains. The objective is:

The first term denotes the contribution to the sentiment classification when concatenated with the private features. The second term is the adversarial loss, which will encourage $E_{s}$ to strengthen its feature extraction ability to confuse $D$. After reaching the Nash Equilibrium Point, features extracted by $E_{s}$ can not be discriminated by $D$.

In this subsection, we will show how to integrate all modules and train them. Overall, we follow the training tricks adopted by ? (?) and update parameters in two stages. The first stage is to enhance the discriminant ability of $D$ by minimizing the adversarial loss. At the second stage, we backward the classification error and update the parameters of $E_{s}$ and $E_{p_{j}}$. The whole training process is demonstrated in Algorithm 1.

After the convergence of above training, we can directly obtain the labels estimated by the pre-trained source classifiers and do not need any further training. At inference time, $D$ will be utilized as a probability distribution estimator and measures the instance-to-domain relations (i.e. $\hat{\boldsymbol{w}}_{\boldsymbol{j}}$). The final target labels can be calculated by:

To acquire more confident pseudo labels gradually, at the beginning we just trust the labels whose confidence is greater than a dynamic threshold $\Delta$ (e.g. 0.98), which is changing while training. In the training process, we will lessen the confidence to some degree, such as every time decrease a constant $\eta=0.02$. When the number of the pseudo-label set, which we denote as $\mathcal{T}_{l}$, does not increase obviously, we will terminate the training process at the optimal confidence. For training, we use $\tau^{-1}$ and $\tau^{-2}$ to represent the pseudo-label set generated by two successive iterations and control the terminal condition. As long as the number of pseudo labels becomes less than a certain number, which is represented by $N$(e.g. 10), then the iteration will terminate. Besides, we denote $iter$ as the iteration of training and the function $labeling$ means the method of labeling by source classifiers. We assign pseudo-labels for target samples when the weighted predictions produced by all source classifiers and the predicted results of the target classifier are both bigger than $\Delta$ (i.e. $\hat{\boldsymbol{c}}_{t}>\Delta$, where $\hat{\boldsymbol{c}}_{t}$ is calculated as Equation 6). Then we use these samples to train the target feature extractor. With the acquirement of all pseudo labels, we will finetune the target feature extractor. More detailed training information is presented in Algorithm 2.

We conduct the experiments on the Amazon reviews dataset, which has been widely used for cross-domain sentiment classification. As described by ?, the pre-processed features loss the order information, which prohibits the usage of strong feature extractor (e.g. RNN or CNN). For fair comparison, we also adopt a MLP as our feature extractor and represent each review as a 5000d feature vector which are the most frequent features.

The Amazon dataset contains 2000 samples for each of the four domains: $book$, $DVD$, $electronics$, and $kitchen$, with binary labels (positive, negative), and more details are included in Table 1.

As described before, the Amazon reviews dataset has many limitations, especially the reviews are already tokenized and converted to a bag of features and lack of the order information. To further validate our frameworks, we turn to another dataset with raw review texts, which is in line with the actual application scenario. With the raw review texts, we can process them from the very start and many strong feature extractor architectures can be adopted (e.g. LSTM). This dataset has 16 different domains of reviews: $Books$, $electronics$, $DVD$, $kitchen$, $apparel$, $camera$, $health$, $music$, $toys$, $video$, $baby$, $magazine$, $software$, and $sports$, in addition to two movies review domains from the IMDb and the MR dataset. The amount of training and unlabeled data of each domain vary across domains but are roughly 1400 and 2000, respectively. In addition, each domain has a development set of 200 samples and a test set of 400 samples.

Firstly, we use the data (both labeled and unlabeled) in the source domains and the data (only unlabeled) in the target domain to complete the training of the main process in Algorithm 1. And then, we randomly select 2000 samples from the target domain to be annotated for both frameworks, while for 2ST-UDA different training mechanism is adopted (i.e. Algorithm 2) to train the target-specific private feature extractor. For both frameworks, the remaining samples in the target domain are used as the validation set and test set, and the number of samples in the validation set is the same as (Chen et al. 2018). The private feature extractor is optimized with the Adam over shuffled mini-batches. And we set the batch size 8 and the learning rate 0.0001 for the sentiment classifier and the domain classifier. Besides, we perform early stopping on the validation set during the training process.

For this dataset, we randomly select about 1400 samples out from target domain as alternative pseudo labels. And the number of samples in the validation set and test set is consistent with that in the (Chen et al. 2018). Other implementation details and the rest of the parameter settings are the same as those for the Amazon dataset.

When comparing with single-source domain adaptation methods (i.e. mSDA, DANN), the training data in the multiple source domains are combined and viewed as a single domain. All baseline methods in the comparison include:

• •

  mSDA (?): it employs marginalized stacked denoising autoencoders to learn new representations for domain adaptation. Specially, this method does not require stochastic gradient descent or other optimization algorithms to learn parameters.

• •

  DANN (?): it introduces a new representation learning approach for domain adaptation, which is based on the adversarial training.

• •

  MDAN(H-MAX), MDAN(S-MAX) (?): they are two adversarial neural models. One directly optimizes the proposed new generalization bound (i.e. H-MAX) and another is a smoothed approximation of the first one (i.e. S-MAX), leading to a more data-efficient and task-adaptive model. These two methods are in multi-source-to-one-target setting.

Table 2 reports the classification accuracy of different methods on the Amazon reviews dataset. It shows that our second framework outperforms the state of art by 1.42 percentage. But for the first framework, we just obtain a comparable average result. For this, we argue the reason is that we can not adopt strong source extractors, the integrated results will also be weak. It is worthwhile noting that all the baselines are just for one target domain, but our framework can deal with multiple target domains.

The baseline methods in the comparison include:

• •

  ASP-MTL-SC, ASP-MTL-BC (?): these two models are single channel model and bi-channel model of adversarial multi-task learning.

• •

  MAN (?): this model is a multinomial adversarial networks for multi-domain text classification, which is extended from (?)

• •

  Meta-MTL (?): it is a meta-network, which can capture the meta-knowledge of semantic composition and generate the parameters of the task specific semantic composition models.

As shown in Table 3, the performance of our frameworks performs better in 12 of 16 domains when comparing with all other algorithms. In average accuracy, we obtain 1.2 and 1.5 percent improvement for our two introduced frameworks separately.

In this subsection, we will show the accuracy of D in the training process for experiments on Amazon, which will reveal the discriminating ability of $D$ and the feature extraction ability of $E_{s}$ and $E_{p_{j}}$. As Figure 2 depicts, the accuracy of $D$ for shared feature roughly converges at $1/K$ and for private features stabilises at around 90 percentage. The results tell that in the training process the discriminated ability of $D$ will become stronger and stronger, which will force $E_{s}$ to extract more pure shared features and $E_{p_{j}}$ to retain most of the domain-related information.

Table 3 reveals the effectiveness of the weighting scheme in our frameworks in total, to further demonstrate its effectiveness, we will pick out some representative examples (i.e. some sentences) to visualize the weights assigned to each source classifier.

We use Figure 3 to show the weight assigned to each source classifier from two sentences picking from the ’toy’ and ’dvd’ domain. As demonstrated in  (?; ?), a metric named $\mathcal{A}$-distance can be utilized to well measure the relation between two domains in an unsupervised way. We measure the distances between ’toy’ and ’dvd’ with other domains, the ranking from close to not related are baby, sports_outdoors, health_personal_care and video, books, imdb respectively. And our weights tell the rankings are baby, sports_outdoors, music and video, books, magazines. The overall rankings are coincident with the $\mathcal{A}$-distance.