A Source-Criticism Debiasing Method
for GloVe Embeddings

Word embeddings are used widely in natural language processing (NLP) for a broad range of downstream tasks. However, as Caliskan et al. (2017) and Garg et al. (2018) have shown, these embeddings confirm many human social biases, some of which lead to problematic behaviors of machine learning models which rely on pretrained embeddings (Kiritchenko & Mohammad, ). Among popular embedding models are Word2Vec (Mikolov et al., 2013), GloVe (Pennington et al., 2014), and FastText (Joulin et al., 2017). All three use unsupervised learning techniques on billions of words sourced from large internet corpora. Bolukbasi et al. (2016) demonstrated that a simple analogy task could be used to reveal unwanted latent semantic associations in the embeddings; for example, the response from pretrained GloVe embeddings to the question “man is to computer programmer as woman is to X” rendered “homemaker” as the most likely answer.

Caliskan et al. (2017) introduced a standardized method of measuring biases present in word embeddings from a template of the Implicit Association Test (IAT), which is used widely in the field of sociology to measure latent human prejudices; this includes everything from morally neutral biases, such as ‘flowers are more pleasant than insects’ to more problematic ones, such as ‘European names are more pleasant than African names’. From this, they derive the Word Embedding Association Test (WEAT), which gives the probability that the observed similarity scores could have arisen with no semantic association between the target concepts and the attribute.

Some previously proposed debiasing methods happen post-training, such as Bolukbasi et al. (2016), focus on zeroing the ‘gender direction’, i.e. the projection of each word on a predefined gender direction written as $\overrightarrow{\textbf{w}}\cdot(\overrightarrow{\textbf{he}}-\overrightarrow{\textbf{she}})=0$. Other methods attempt to mitigate bias during training, such as Zhao et al. (2020)’s method of encouraging gendered information to reside in the tail end of the word vector during training and then removing the biased subset of the word vector before use in an NLP system.

With the use of influence functions, a methodology borrowed from robust statistics, it is possible to determine which inputs to a model exert the most influence over model inference at test time (Koh & Liang, 2017). Fundamentally, influence functions are an approximation of the result that would be achieved by removing one example at a time from the dataset and training a model to see its net effect on inference. Recent work in this area has introduced computationally efficient tools for performing fast influence function calculations for large models (Guo et al., 2020) and even through a multi-stage training (pre-training and fine-tuning) process (Chen et al., 2020). These tools provide insight into model behavior by directly tracking how single training examples affect downstream inference and therefore hold potential for the growing trend of explainable machine learning (Bhatt et al., 2020).

Brunet et al. (2019) apply an influence function based approximation method to word embeddings as a way of explaining learned bias. With their method, it is possible to determine the subsets or individual documents that are most responsible for disparate gendered representations in learned word vectors. They demonstrate this method by creating perturbation sets which remove a number of biasing documents and compare the predicted differential bias sets with ground truth removal and re-embedding. They show remarkable agreement between the two, giving strong evidence that their approximation function is a trustworthy proxy for differential bias.

For our experiments, we consider the effect size of two different WEAT bias word sets as introduced by Caliskan et al. (2017).The WEAT test itself measures the similarity of words $a$ and $b$ in word embedding $w$ as measured by the cosine similarity of their vectors, $\cos{(w_{a},w_{b})}$. In WEAT1, the target word sets relate to science and arts terms, with the expectation that scientific terms will cluster more with ‘male’ attributes sets and art terms will cluster more with ‘female’ attribute sets. In WEAT2, the target word sets relate to weapons and instruments, with the expectation that weaponry terms will cluster more with ‘unpleasant’ attribute sets and musical terms will cluster more with ‘pleasant’ attribute sets. A representative list of the word sets is seen in Table 1. Of the bias sets used, WEAT1 is broadly considered to reflect a problematic bias whereas WEAT2 reflects one that is more benign.

We focus on only one kind of embedding model, GloVe, in this paper, although our method could theoretically be applied to others. GloVe (Global Vectors for word representations) is essentially a log-bilinear model with a weighted least-squares objective (Pennington et al., 2014). The underlying intuition for the model is that words which occur in the same context more frequently within a given corpus are more likely to be semantically linked. The training objective of GloVe is to learn word vectors such that their dot product equals the logarithm of the words’ probability of co-occurrence.

Formally, this happens in two steps¹¹1In the following sections, we use the mathematical notion provided in Brunet et al. (2019). In the first, a sparse co-occurrence matrix $X\in\mathbb{R}^{V\times V}$ is extracted from the corpus. Each entry in the matrix, $X_{ij}$ represents a weighted count of the number of times that word $j$ occurs in the context window of word $i$. For the second step, the optimal embedding parameters $w^{*},u^{*},b^{*},$ and $c^{*}$ are learned via gradient-based optimization such that they minimize the loss:

where $w_{i}\in\mathbb{R}^{D}$ is the embedding of the $i$th word in the vocabulary. We use an embedding dimension of 75. The set of $u_{j}\in\mathbb{R}^{D}$ are the context word vectors, and $b_{i}$ and $c_{j}$ are the bias terms for $w_{i}$ and $u_{j}$, respectively. $f(x)$, the weighting function, is used to attribute more importance to common word occurrences.

Our dataset consists of a corpus constructed from a Simple English Wikipedia dump²²2https://dumps.wikimedia.org/simplewiki/, using 75-dimensional word embedding vectors. The TOP-1 analogies test (shipped with the GloVe code base) measured approximately 35%, which is certainly lower than state-of-the-art but high enough for our purposes of demonstrating proof-of-concept of our method. We note that future work would ideally consider a broader range of corpora for testing. We train an initial GloVe embedding model on this corpus; the relevant corpus statistics and GloVe training hyperparameters are listed in Table 2.

For our debiasing method, it is crucial that we know how much each document in the corpus contributes to its downstream effect size. The naive way to compute this would be to remove a single document from the corpus and completely retrain the embedding, but this is impractical and computationally intractable even for relatively small NLP corpora. However, Brunet et al. (2019) introduce a method for calculating an approximation of the resulting embedding change by applying a modified version of influence functions³³3the code for this is publicly available at https://github.com/mebrunet/understanding-bias. They approximate that the optimal word vector learned from the initial training, $w^{*}_{i}$, will change with respect to a given corpus perturbation (removing a document) as:

Where $\tilde{w_{i}}$ is the word vector learned from a perturbed corpus, $V$ is the size of the vocabulary, $X$ is the global co-occurrence matrix, and $\tilde{X}$ is the co-occurrence matrix discounting the co-occurrence matrix of document $i$. $H$ is the Hessian with respect to only word vector $w_{i}$ of the point-wise loss at $X_{i}$, and $\nabla_{w_{i}}L(X_{i},w)$ is the gradient of the point-wise loss function at $X_{i}$ with respect to only word vector $w_{i}$.

We use this method to get an approximation of the differential bias of each document in the Wiki corpus. Once that is calculated, we pass to our method the co-occurrence matrix, the trained GloVe model, the WEAT test words, and the newly created list of differential biases, where $\beta^{(k)}$ is the differential bias approximation for document $k$. The full algorithm may be seen in Algorithm 1. It bears similarity to the differential bias approximation in Brunet et al. (2019) with the crucial changes that we weight the subtractor of the co-occurrence matrix by the pre-calculated differential bias, and that we actually set the approximated vector as the vector stored in the final GloVe model. This is because we are approximating what the word vector would have been had it paid less attention to a biased document, so we update the word vector to reflect that change, rather than temporarily storing it for comparison.

The intuition for this algorithm is that WEAT words which appear in heavily biased documents will lend the model unsavory associations between those words. Instead, we maximize the strength of the connection between target and attribute words that appear together in unbiased or, better yet, de-biasing documents (those with a negative differential bias value).

Given a set of WEAT words in a document, there are three possible actions according to our method: if the document does not affect downstream bias with respect to some bias metric, the weighting factor is zero and the true co-occurrence matrix is used. If the document increases bias at test time, the co-occurrence matrix is slightly artificially decreased for the relevant words, scaled by how heavily biased it is. If the document decreases bias downstream, the co-occurrence matrix is slightly increased for the relevant words, artificially strengthening the co-occurrences of more balances terms. We find this method to be an elegant and intuitive approach to debiasing word embeddings.