Comparing Intrinsic Gender Bias Evaluation Measures
without using Human Annotated Examples

Pre-trained language models (PLMs) trained on large datasets have reported impressive performance improvements in various NLP tasks Devlin et al. (2019); Lan et al. (2019) greatly. However, these PLMs also demonstrate significantly worrying levels of social biases Bolukbasi et al. (2016); Kurita et al. (2019). To address this issue, numerous intrinsic bias evaluation measures for PLMs have been proposed Nangia et al. (2020); Dhamala et al. (2021); Nadeem et al. (2021); Kaneko and Bollegala (2022); Zhou et al. (2022), which are also used for comparing debiasing methods for PLMs Webster et al. (2020); Kaneko and Bollegala (2021a); Schick et al. (2021).

Existing bias evaluation methods use different criteria such as pseudo likelihood Kaneko and Bollegala (2022), cosine similarity Caliskan et al. (2017); May et al. (2019), inner-product Ethayarajh et al. (2019) etc. Moreover, current bias evaluation methods require manually-annotated datasets containing stereotypical and antistereotypical examples that express different types of social biases  Nangia et al. (2020); Nadeem et al. (2021). Therefore, we consider that it is important to compare the differences in existing bias evaluation measures proposed for PLMs Orgad and Belinkov (2022); Dev et al. (2021); Kaneko et al. (2022a) to understand their relative strengths and weaknesses.

To objectively compare the existing bias evaluation measures, Kaneko and Bollegala (2022) calculated the rank correlation between the number of human annotators who labelled an example to be stereotypically biased towards a protected attribute in Crowds-Pairs (CP), and the bias score for that example returned by an intrinsic bias evaluation measure Nangia et al. (2020); Nadeem et al. (2021). However, due to the costs and difficulties in recruiting human annotators, this approach cannot be easily adapted to different languages, accommodate large-scale evaluations, or compare evaluation metrics that do not use human-annotated data.

We propose a method to compare intrinsic bias evaluation measures without using human annotated examples. Figure 1 outlines the intuition behind our proposed method. First, we train bias-controlled versions of PLMs obtained via fine-tuning a PLM on male and female gendered sentences, automatically mined from an unannotated corpus using a gender-related word list. We define rate of bias ($r$) as the ratio between male and female gendered sentences in a training sample used to fine-tune a PLM. A PLM fine-tuned mostly on male sentences is likely to generate sentences containing mostly male words, while a PLM fine-tuned on female sentences is likely to generate sentences containing mostly female words Kaneko and Bollegala (2022); Kaneko et al. (2022c). Therefore, an accurate intrinsic bias evaluation measure is expected to return a score indicating a bias towards the male gender for a male bias-controlled PLM, while it is expected to return a score indicating a bias towards the female gender for a female bias-controlled PLM. We then compute the rank correlation between (a) the rate of biases in the bias-controlled PLMs, and (b) the bias scores returned by an intrinsic evaluation measure for the corresponding PLMs, as a measure of accuracy of the bias evaluation measure.

Our experiments with multiple corpora and PLMs show that the correlations reported by our proposed method, which does not require human annotated examples, are comparable to those computed using human annotated examples in previous studies. Furthermore, by examining the output probabilities of the PLM, we verify that the proposed method, which fine-tunes bias-controlled PLMs with varying amounts of male vs. female sentences, is indeed able to control biases associated with male and female gender directions.

The imbalance of gender words in the training data affects the gender bias of a PLM fine-tuned using that data Kaneko and Bollegala (2022); Kaneko et al. (2022c). Using this fact, we propose a method to learn bias-controlled versions of PLMs that express different levels of known gender biases. Let us first assume that we are given a list of female gender related words $\mathcal{V}_{f}$ (e.g. she, woman, female), and a separate list of male gender related words $\mathcal{V}_{m}$ (e.g. he, man, male). Next, we select sentences that contain either at least one of female or male words from an unannotated set of sentences $\mathcal{D}$. Sentences that contain both male and female words are excluded here. Let us denote the set of sentences extracted for a female or a male word $w$ by $\Phi(w)$. Moreover, let $\mathcal{D}_{f}=\bigcup_{w\in\mathcal{V}_{f}}\Phi(w)$ and $\mathcal{D}_{m}=\bigcup_{w\in\mathcal{V}_{m}}\Phi(w)$ be the sets of sentences containing respectively female and male words. We appropriately downsample $\mathcal{D}_{f}$ and $\mathcal{D}_{m}$ to have equal numbers of sentences $N$ (i.e. $|\mathcal{D}_{f}|=|\mathcal{D}_{m}|=N$).

Next, we create training datasets $\mathcal{D}_{r}$ by varying the rate of bias, $r$ ($\in[0,1]$), by randomly sampling a subset $\mathcal{S}_{r}(\mathcal{D}_{m})$ of $Nr$ sentences from $\mathcal{D}_{f}$ and a subset $\mathcal{S}_{1-r}(\mathcal{D}_{f})$ of $N(1-r)$ sentences from $\mathcal{D}_{m}$ such that $\mathcal{D}_{r}=\mathcal{S}_{r}(\mathcal{D}_{m})\cup\mathcal{S}_{1-r}(\mathcal{D}_{f})$. When $r=0$, $\mathcal{D}_{r}$ consists of only female sentences (i.e. $\mathcal{D}_{r}\subseteq\mathcal{D}_{f}$), and when $r=1$, it consists of only male sentences (i.e. $\mathcal{D}_{r}\subseteq\mathcal{D}_{m}$). To obtain multiple bias-controlled PLMs at different levels of gender biases, we fine-tune a given PLM on different datasets, $D_{r}$, sampled with different values of $r$. We use a given intrinsic bias evaluation measure to separately evaluate each bias-controlled PLM. Finally, we measure the agreement between the bias scores reported by the intrinsic bias evaluation measure under consideration and the corresponding rates of biases of those PLMs using Pearson’s rank correlation coefficient.

We proposed a method to compare intrinsic gender bias evaluation measures using an unannotated corpus and gender-related word lists. Experiments show that the correlations computed by the proposed method for existing bias evaluation measures agrees with the prior evaluations conducted using human annotations.

In this paper, we limited our investigation to English PLMs. However, as reported in a lot of previous work, social biases are language independent and omnipresent in PLMs trained for many languages Kaneko et al. (2022c); Lewis and Lupyan (2020); Liang et al. (2020); Zhao et al. (2020). We plan to extend this study to cover non-English PLMs in the future.

According to existing research, PLMs encode many different types of social biases such as racial and religious biases in addition to gender-related biases Kiritchenko and Mohammad (2018); Ravfogel et al. (2020). On the other hand, in this paper, we focused on only gender bias. Extending the proposed method to handle other types of social biases beyond gender bias is beyond the scope of the current short paper and is deferred to future work.

Furthermore, discriminatory bias is learned in word embeddings as well as PLMs Bolukbasi et al. (2016); Brunet et al. (2019); Kaneko and Bollegala (2019, 2020); Kaneko and Bollegala (2021b); Kaneko et al. (2022b). Therefore, it may be possible to make it applicable to word embeddings as well.

Our goal in this paper was to compare the previously proposed and widely-used intrinsic bias evaluation measures of gender bias in pre-trained PLMs. Although we used a broad range of existing datasets that are annotated for social biases, we did not annotate nor release new datasets as part of this research. Moreover, we fine-tune a large number of bias-controlled PLMs for evaluation purposes that demonstrates varying levels of gender biases. However, these PLMs are not supposed to be used in downstream tasks other than for evaluation purposes.

Even with the highly correlated bias evaluation measure in our proposed method, the bias of the PLM may not be sufficiently evaluated. Therefore, we consider that it important to select intrinsic gender bias evaluation measures carefully and not purely based on correlation coefficients computed by the proposed method alone.

There are various discussions on how to define social bias in PLMs Blodgett et al. (2021). Since the proposed method can use any method as the bias-controlled fine-tuning of the PLMs, the bias-controlled fine-tuning can be selected according to the definition of social bias.

This paper is based on results obtained from a project, JPNP18002, commissioned by the New Energy and Industrial Technology Development Organization (NEDO).