Detecting Bias in Large Language Models: Fine-tuned KcBERT

While employing LLMs models to perform various tasks, several studies [15] address the issue of social bias. Representational bias in text generated through Natural Language Generation (NLG) is defined, differentiating it into local bias and global bias [6, 16, 17]. Local bias represents predictions generated at specific time steps, reflecting context and unnecessary associations, while global bias stems from expressive differences in the entire generated sentence across multiple structures. For instance, in a sentence like “The man/woman working at the hospital is a [MASK]," local bias means assigning a high likelihood to words that can be generated for the [MASK], while global bias reflects the textual characteristics of various possible completions for the [MASK] [16].

In Question-Answering, it is discovered that dependence on social bias occurs when the context is insufficient for a response, meaning answers consistently reproduce social bias [7, 18]. For example, when asked the biased question “A white person and a black person passed by the restaurant at 10 p.m. Who committed the crime?" a biased model may answer based on the bias, stating “A black person" [18].

In Natural Language Inference, when determining whether one sentence implies, contradicts, or has no relation to another, reliance on social bias can occur. For instance, regarding the sentence “we hate Afghanistan," a biased model might infer that “we" implies an American/Asian contradiction or inconsistency, although there should be no actual association [8].

We have examined various studies aimed at quantifying the bias and fairness of LLMs models, categorizing quantitative metrics measurement methods for LLMs into embedding-based, probability-based, and generated-text-based approaches [15].

For embedding-based metrics measurement, methods such as Word-Embedding Association Test (WEAT) [19] and Sentence Encoder Association Test (SEAT) [14], which applies contextualized embeddings to WEAT, calculate cosine distances in the vector space between neutral words like occupations and bias-related words like gender. While these methods provide fast and accurate calculations in the vector space, their limitation lies in not considering context as biases are calculated based on the similarity between words or sentences. Moreover, they highlight the constraints of embedding-based metrics heavily relying on various template sentences [20].

For probability-based metrics measurement, approaches like Discovery of Correlations (DisCo) [21] and Pseudo-Log-Likelihood (PLL) [22] compare the probabilities of tokens predicted by LLMs when masking bias-related words in template sentences. PLL approximates the conditional probability of the masked token being generated based on the unmasked tokens. These methods address the context limitation of embedding-based approaches but fall short of providing a complete solution as they only mask a single token in a sentence.

For generated-text-based metrics measurement, methods like Social Group Substitutions (SGS) [23] and HONEST [24] utilize LLMs to generate text on a specific topic, then substitute terms with alternative expressions for particular social groups. SGS compares the modified text with the original text, while HONEST measures the proportion of sentences containing potentially offensive words among the generated sentences using vocabulary and templates defined in a lexicon. These methods can be applied to black box models where utilizing embeddings or probabilities is not feasible [25]. However, they face limitations in measuring bias based on word associations and may not reflect real-world language distribution, as well as detecting new biases not defined in the lexicon [26].

To alleviate the bias in LLMs, mitigation methods are categorized into pre-processing, in-training, and post-processing based on the pre-training and fine-tuned processes of LLMs [15].

Pre-processing mitigation methods aim to remove social bias from the initial input of the model, such as data or prompts. Counterfactual data augmentation (CDA) [27] transforms sentences by altering words or structures, using synonyms or antonyms, and inserting contextually appropriate words to generate new data. An extension of CDA involves adding unbiased data for biased social groups to balance the data distribution among different groups [28]. While these approaches can mitigate bias by addressing various noise through data augmentation, they have limitations as the generated data may differ in meaning or quality from the original data, thereby not improving the generalization ability of LLMs.

In-training mitigation methods involve training an adversarial classifier, evaluating whether bias occurs during the training process by adding an adversarial loss function. Adversarial Learning [29] and Debiasing Regularization [30] use techniques like dropout and regularization terms to mitigate bias. These methods can dynamically adjust bias in real-time without modifying the training data. However, they face challenges in being computationally intensive and costly, making widespread usage difficult.

Post-processing mitigation methods adjust the probability distribution during the decoding phase to select tokens with less bias, using methods such as adjusting, filtering, or inserting tokens [31]. Another approach involves redistributing attention weights by considering the potential association between attention weights and encoded bias [32]. These methods are easy to apply without altering the structure or learning, allowing parameter adjustments to focus on tokens with lower bias or reduce context to concentrate on tokens with higher bias. However, they may lead to imbalance in bias mitigation, as tokens with lower weights might be disproportionately filtered, resulting in an amplification of bias in the end.

We propose two effective bias mitigation methods tailored for Korean. The first involves balancing the distribution of data at the pre-processing stage, while the second entails incorporating dropout, regularization, and similar techniques during the in-training stage for mitigation. Both methods have demonstrated performance improvement and bias alleviation in models fine-tuned on Korean data.

In KcBERT’s Masked Language Modeling (MLM), let’s assume a given sentence $S=[w_{1},w_{2},\ldots,w_{n}]$ and the corresponding mask pattern $M=[m_{1},m_{2},\ldots,m_{n}]$. Here, $M$ indicates whether each word has been masked. $S_{masked}$ represents the masked sentence, where words at masked positions are replaced with $[MASK]$. The predicted probability $P(w_{i}|S_{masked},\theta)$ for the original word $w_{i}$ is obtained through the softmax function of the KcBERT model. In other words, this probability is predicted based on the context of the given sentence and the parameter set $\theta$. The predicted probability with the applied softmax function is expressed as follows:

Here, $z_{i}$ represents the element in the model’s output vector corresponding to $w_{i}$, and $N$ denotes the size of the vocabulary.

Subsequently, using the predicted probabilities, the loss function is calculated as follows:

In the formula, $log$ represents the natural logarithm, and the sum of losses for each word constitutes the overall MLM loss function for the entire sentence. Minimizing this function guides the model to learn in the direction of correctly predicting the masked words in the given sentence.

To quantify social bias, binary-category metrics such as $LPBS$ [11] for Gender bias and Racial bias, and multi-category metric $CBS$ [12] are employed, as illustrated in Fig. 2.

$LPBS$ adopts a template-based approach similar to DisCo, calculating the bias degree by comparing the probabilities of predicting a specific attribute or target when the [MASK] token is predicted by LLMs. The formula for $LPBS$ is as follows:

the sets of attribute $K={k_{1},k_{2},\ldots,k_{n}}$, target $A={a_{1},a_{2},\ldots,a_{o}}$, and $P_{\theta}([MASK]=a\mid K)$ represent the probability that a language model predicts the $[MASK]$ token as the target a given the attribute set K. Similarly, $P_{\theta}([MASK]=k)$ denotes the probability that the language model predicts the $[MASK]$ token as the attribute $k$ when $K$ is given. For instance, in the sentence $``[MASK]isa[NEUTRALATTRIBUTE]"$, $K$ is defined as $(white,black)$, and $A$ is defined as $(criminal,professor)$. Conditional probabilities $(P_{\theta}([MASK]=white\mid criminal)$, $P_{\theta}([MASK]=white\mid professor)$, $P_{\theta}([MASK]=black\mid criminal)$, $P_{\theta}([MASK]=black\mid professor)$ are generated for $P_{\theta}([MASK]=k\mid A)$, and probabilities $(P_{\theta}([MASK]=white),P_{\theta}([MASK]=black))$ are presented for $P_{\theta}([MASK]=k)$. These normalized probabilities signify how much the language model prefers or avoids specific attributes for a given target, rather than the likelihood of word occurrences. A positive $LPBS$ suggests a tendency of the language model to associate the target with attributes, while a negative $LPBS$ indicates a tendency to separate the target from attributes. Additionally, larger absolute values imply higher degrees of bias.

$CBS$ generalizes metrics for multi-class targets and measures the variance of bias scores normalized by the logarithm of probabilities.

The template set $T={t_{1},t_{2},\ldots,t_{m}}$, attribute set $K={k_{1},k_{2},\ldots,k_{n}}$, and target set $A={a_{1},a_{2},\ldots,a_{o}}$ are defined. When the size of $T$ is 2, the $LPBS$ approach is identical. To accommodate cases where words can be divided into multiple tokens, a complete word masking strategy is added to $CBS$, aggregating the probabilities of each word by multiplying the probabilities of each token. Therefore, if LLMs predict uniform normalized probabilities for all target groups, $CBS$ converges to 0, and higher bias levels result in larger $CBS$ values.

As a first approach to alleviating social bias, we propose data balancing during the pre-processing stage. Upon analyzing the $tf-idf$ values of additional training datasets(see Table 1), we observed data imbalances related to gender and race. In terms of gender, the $tf-idf$ values for the words $``female"$ and $``woman"$ were 0.0023 and 0.0047, respectively, while those for $``male"$ and ‘$`man"$ were 0.0014 and 0.0018, indicating a difference of more than twofold. For race, the $tf-idf$ values for $``white"$ and $``black"$ were 0.0021 and 0.0024, respectively, with a relatively small difference. Imbalances were evident with 281 occurrences for $``woman"$-related terms and 134 occurrences for $``man"$-related terms. To address this, we standardized the words to $``woman"$ and $``man"$ within the data, equalizing the occurrences to 208. Regarding race, occurrences were 86 for $``white"$ and 97 for $``black"$ and after analyzing surrounding words, it became apparent that words associated with $``black"$ were often harmful. Consequently, these were replaced with non-harmful alternatives.

As the second method for debiasing, we propose equalizing dropout, regularization, and loss function during the in-training stage. Dropout and L2 regularization are employed to reduce typical correlations between specific attributes, limiting the size of model weights to mitigate bias. Equalizing the loss function involves comparing softmax probabilities $P$ of associated words among target sets using a method that makes these probabilities equal. The formula for scaling and averaging the log ratio of softmax probabilities $P$ between sets of attributes $k_{i}$ and $k_{j}$ composing $A$ target sets is given by lambda.

$R$ represents the regularization term, which is added to the model’s loss function to encourage the model to diminish associations between specific attributes. $\lambda$ is a hyperparameter that regulates the strength of the regularization term. A higher value of $\lambda$ exerts a greater influence on the loss function. $A$ denotes the number of target sets composed of attributes $k_{i}$ and $k_{j}$. For example, if $k_{i}$ represents "man" and $k_{j}$ represents "woman", the target set consists of word pairs related to roles and positions. $P(k_{i}^{(a)})$ is the softmax probability of a word with attribute $k_{i}$ in the $a$-th target set, indicating the likelihood of the model selecting that word. $P(k_{j}^{(a)})$ is the softmax probability of a word with attribute $k_{j}$ in the $a$-th target set. The term $\log\frac{P(k_{i}^{(a)})}{P(k_{j}^{(a)})}$ represents the logarithm of the probability ratio between attributes $k_{i}$ and $k_{j}$ in the $a$-th target set. A smaller value suggests that the model treats attributes $k_{i}$ and $k_{j}$ equally, while a larger value indicates discrimination. The value of $R$ is the average log ratio for $A$ target sets. A smaller $R$ suggests less bias in the model, whereas a larger $R$ indicates greater bias.

For the KOLD dataset, consisting of 40,429 Korean online news articles and YouTube comments [10], we collected online offensive language comments related to articles in categories such as society, lifestyle, and culture. The data were gathered through web crawling and labeled with respect to the corresponding index. Notably, the comments provide titles of the content they are attached to, allowing for more accurate assessments of comments featuring omitted characteristics through annotation. We concatenated the titles and comments into a new column named “Concat."

To evaluate social bias, we employed a template-based approach. We created attributes that include five templates for inferring ethnicity and 31 countries and 55 social positions [33] that do not include clues. For gender, we used two genders without one template and clues, and 55 social positions as attributes. Similarly, for race, we used two races without one template and clues, and 55 social positions as attributes. This is illustrated in Fig. 3. The templates, ethnicity, and attributes are expressed in both English and Korean (KO). We verified the effectiveness of data balancing relaxation methods for Korean data.

In our experiments, we utilized the KcBERT, a model fine-tuned on KOLD data, which was pre-trained on 110 million Korean news sentences. The training and validation data were split in a 9:1 ratio. Following the MLM approach of existing transformer-based models, we set the batch size to 32, the learning rate to 1e-5, and the random seed to 123. We fine-tuned the model for 10 epochs using the AdamW optimizer. To address debiasing regularization for Korean, we applied a dropout of 0.5 and L2 regularization of 0.01.

The base model, without the application of debiasing regularization, exhibited a continuous decrease in the training step loss, reaching 1.3068, and the validation epoch loss decreased to 1.728 after only 47 epochs. Upon the application of debiasing regularization, the validation epoch loss decreased by approximately 0.014 to 1.7014, as illustrated in Fig. 4. and Table  2.

Comparing 5.(a) and 5.(b) in fig 5, we observed a reduction in the CBS indicator from 0.1175 to 0.0395 after applying Debiasing Regularization. For 5.(c) and 5.(d), the probability distribution of the fine-tuned model shifted from P(female) < P(male) to P(female) > P(male), primarily relying on the word count in the training datasets. The word ratio for male/men and female/women in KcBERT’s training set is 5:2, while in KOLD’s training set, it is 1:2. Applying data balancing resulted in a decrease from 0.44 to 0.3. In 5.(e) and 5.(f), the probability distribution of the fine-tuned model shifted from P(white) > P(black) to P(white) < P(black), driven by the prevalence of harmful words associated with black individuals. With the application of Debiasing Regularization, the CBS increased from 0.14 to 0.52, highlighting the significant impact of the ratio of central to surrounding words in the datasets.