Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyond

To analyze generative inequality at the word level, where keywords (e.g., “Mother-to-child transmission” in the sentence “Mother-to-child transmission is the primary cause of HIV-1 infection in children worldwide.”) may account for a limited proportion of the overall uncertainty within the current response, we first assess the semantic relevance of each word by measuring the textual similarity between the query-answer pairs before and after removing the evaluated word. A lower similarity score signifies a significant semantic variation, indicating that the word carries more semantic information within the current textual sequence (i.e., a keyword).

Following SAR [23], we evaluate textual similarity utilizing a cross-encoder model provided by the SentenceTransformers library [44], with RoBERTa-large [45] as the backbone. The model processes sentence pairs and generates similarity scores. However, relying solely on an external language model for textual similarity evaluation is unreliable and lacks explainability, because embeddings of sentences encoded by the model, in which all semantic information is mixed in fixed-length vectors, are limited in the semantic representation [46]. Inspired by bi-directional entailment [22], we leverage a Natural Language Inference (NLI) classifier, DeBERTa-large-mnli [47], for this task. The model takes sequence pairs as the input and predicts scores (logits) for three classes of semantic relationship: entailment, neutral, and contradiction. We employ the probability of entailment as the similarity measure.

For simplicity, we define ${\textbf{s}}_{i}\setminus{w}_{ij}$ as the representation for removing the $j$-th word from the $i$-th response and $\cup$ as the concatenation of the prompt and answer. The measurement of textual similarity is formulated as:

where ${f}_{\textit{ce}}\left(\cdot\right)$ represents the utilization of the cross-encoder model to compute the textual similarity score between two sequences directly, ${f}_{\textit{ent}}\left(\cdot\right)$ refers to obtaining the probability of entailment extracted from the logit vector, which falls within the range of 0 to 1 after being scaled by the $softmax$ function, and $c$ is leveraged to control the smoothness of the logit vector.

Given that the employed language models [45, 47] are not specifically pre-trained for the medical domain, consistently high similarity can lead to low semantic relevance for all words within the current textual sequence, thereby failing to capture keywords. We adopt a conservative strategy by selecting the smaller value from the two measures in Eq. (5), which mitigates potential instability arising from extreme similarity quantification and task-specific limitations. Then, the word-level semantic relevance score of the $j$-th word within the $i$-th response can be formulated as:

In the end, we assign the same relevance score to all tokens in ${w}_{ij}$ as the word itself (i.e., the token-level semantic relevance score), to maintain the consistency of semantic relevance within a single word:

Formally, it can be observed that if the $i$-th textual sequence exhibits significant semantic variation before and after removing the $j$-th word, then the semantic relevance score of all tokens in ${w}_{ij}$ are deemed to be high.

In open-ended medical QA tasks, we generate multiple (i.e., $K$) responses to the same query to estimate the uncertainty of the current QA process, and there can be many irrelevant responses with limited semantic information. However, PE, as described in Eq. (3), calculates the average of the sequence-wise entropy of all responses within the generated set. To investigate this issue, we define the semantic relevance at the sequence level.

Building on the self-consistency hypothesis³³3Self-consistency hypothesis states that a repetitively sampled response is viewed as a form of consistency linked to higher confidence in the response. [48], we suggest that responses, which maintain strong semantic consistency with others among the set of $K$ candidate responses, are more trustworthy. We employ the identical approaches described in Eq. (5) to measure the textual similarity between any two textual sequences. Then, the sequence-level relevance score of ${\textbf{s}}_{i}$ is formulated as the accumulation of the textual similarity scores, re-weighted by the generative probability of the compared responses:

where $S\left(\cdot,\cdot\right)$ represents the smaller similarity score obtained from the two measurements in Eq. (5), and $\textbf{s}_{l}$ denotes the $l$-th textual sequence that differs from $\textbf{s}_{i}$ in the $K$ generated responses. A higher probability of $\textbf{s}_{l}$ (i.e., $p\left({\textbf{s}}_{l}\right)$) augments the persuasiveness of textual similarity between ${\textbf{s}}_{i}$ and ${\textbf{s}}_{l}$.

As mentioned previously, the token-wise entropy reflects the uncertainty committed by each token (i.e., ${\textit{E}}_{T}\left({z}_{t}\right)$ in Eq. (1)), and the overall uncertainty of the $i$-th response can be calculated by aggregating the token-wise entropy of all words within the entire textual sequence (i.e., ${\textit{E}}_{S}\left({\textbf{s}}_{i}\right)$ in Eq. (2)). To ascertain how much uncertainty is induced by individual words, we compute the word-wise entropy of ${w}_{ij}$ based on Eq. (1):

where $p\left({z}_{ijk}\right)$ refers to the probability of generating ${z}_{ijk}$ as the $k$-th token in the $j$-th word within the $i$-th response. Then we calculate the ratio of the word-wise entropy and the sequence-wise entropy to determine the proportion of uncertainty stemming from the $j$-th word within the $i$-th response.:

Similar to the word-wise situation, we formulate the uncertainty proportion of the $i$-th response in the set of $K$ generated responses (i.e., $\mathbb{S}$) as:

To characterize generative inequality in open-ended medical QA tasks, we employ the MedMCQA dataset, with LLaMA-2-7B-Chat-HF serving as the generator. Given each medical query, we generate five responses (i.e., $K=5$), and the max length of each sequence is set to 128 (i.e., ${T}_{i}\leqslant 128$). We first leverage Eq. (6) and Eq. (7) to outline the distributions of word-level and sequence-level semantic relevance scores. Results are depicted in Fig. 2. Within the generated set, a considerable proportion of words exhibit low semantic relevance (i.e., irrelevant), and only a limited subset of words conveys the primary semantic information. At the sequence level, the prevalence of irrelevant responses significantly outweighs those with meaningful content.

When conducting UQ, we should prioritize keywords and reliable textual sequences. To explore the issue of generative inequality, we analyze the correlation between semantic relevance and uncertainty proportion. We divide relevance scores into ten equal intervals. Within each interval, we calculate the sum and average uncertainty of all words or sequences. Results at both the word and sequence levels are illustrated in Fig. 3. Irrelevant words contribute significantly to the overall uncertainty. At the sequence level, both the mean and sum uncertainty of irrelevant sequences are prominent.

Given the substantial proportion of irrelevant words and textual sequences within the generated set, this can introduce unexpected biases and instability when measuring the uncertainty of LLMs-generated answers in real-world open-ended medical QA applications. To address these issues, we propose a novel UQ method in the following text.

Since the semantic information carried by each word (token) differs, treating all tokens equally, as described in Eq. (2), will lead to biased measurements of the predictive uncertainty within each textual sequence. To address this, we highlight tokens in keywords by directly multiplying the token-wise entropy by the token-level semantic relevance score:

where ${\textit{E}}_{T}\left({z}_{ijk}\right)$ refers to the token-wise entropy of the $k$-th token in the $j$-th word within the $i$-th response. Then, the calibrated word-wise entropy of $w_{ij}$ can be formulated as:

To quantify the overall uncertainty of $\textbf{s}_{i}$, we sum the calibrated (weighted) uncertainty of all words within this textual sequence:

Finally, the word-level WSE is defined as the arithmetic mean uncertainty of these $K$ candidate responses, following PE:

By employing the word-level WSE, we capture and highlight keywords carrying the main semantic information within the current textual sequence, thereby calibrating the predictive uncertainty of each candidate response.

As noted in Section 3.2, responses, which are semantically consistent with others in the set of $K$ candidate responses, are more trustworthy. We reduce the uncertainty associated with the $i$-th textual sequence by adding its generative probability to its semantic relevance score, after dividing by a constant $d$, to obtain the calibrated sequence-wise entropy of ${\textbf{s}}_{i}$:

where $d$ serves to regulate the extent to which the semantic relevance score influences the generative probability. Operating in the same way as Eq. (15), the sequence-level WSE is formulated as:

By employing the sequence-level WSE, we enlarge the generative probability of more reliable responses by assessing their semantic relevance, thereby calibrating the overall uncertainty of the current QA process.

Given the direct mathematical relation between the probability and entropy of ${\textbf{s}}_{i}$, as defined in Eq. (4), we replace $p\left({\textbf{s}}_{i}\right)$ in Eq. (16) with ${e}^{-{\textit{U}}_{S}\left({\textbf{s}}_{i}\right)}$, where ${\textit{U}}_{S}\left({\textbf{s}}_{i}\right)$ represents the calibrated uncertainty of ${\textbf{s}}_{i}$ described in Eq. (14). Since the sequence-level semantic relevance score of ${\textbf{s}}_{i}$ (i.e., ${\textit{R}}_{S}\left({\textbf{s}}_{i}\right)$) is determined by the probability of compared sequences, we replace $p\left({\textbf{s}}_{l}\right)$ as defined in Eq. (8). Then, the combined WSE, which calibrates the uncertainty at both the word and sequence levels, is formulated as:

where ${{p}_{i}}^{\prime}$ and ${{p}_{l}}^{\prime}$ refer to the replacements of the generative probability, and ${S}_{li}$ represents the semantic textual similarity between ${s}_{l}$ and ${s}_{i}$. Moreover, The pseudocode of the combined WSE is summarized in Algorithm 1.

In terms of computational complexity, we first analyze PE. As described in Eqs. (2)-(3), its computational complexity is $\mathcal{O}\left(T+K\right)=\mathcal{O}\left(\max\left(T,K\right)\right)$, where $T$ is the number of tokens within the response. For SAR, since it assesses the relevance of each token and sentence within the $K$ generated responses, its computational complexity is $\mathcal{O}\left(KT+K^{2}\right)=\mathcal{O}\left(\max\left(T,K\right)K\right)$. Here, $KT$ refers to assessing the relevance of individual tokens within these $K$ responses, and $K^{2}$ refers to analyzing the similarity between every pair of responses. WSE assesses semantic relevance as both the word and sequence levels, with the computational complexity of $\mathcal{O}\left(KN+K^{2}\right)=\mathcal{O}\left(\max\left(N,K\right)K\right)$, where $N$ is the number of words within the response, and $KN$ refers to measuring the semantic variation of the $K$ responses before and after removing each word. Since a single word can be composed of multiple tokens (i.e., $N\leq T$), WSE has a lower computational complexity compared to SAR.

By strategically calibrating the uncertainty proportion of keywords and elevating the generative probability of semantically analogous (i.e., more trustworthy) responses, WSE focuses more on significant words and responses when estimating the uncertainty of free-form answers generated by LLMs, effectively mitigating biases caused by generative inequality. In the latter part of the experiments, we denote word-level WSE, sequence-level WSE, and combined WSE by ${\textit{WSE}}_{W}$, ${\textit{WSE}}_{S}$, and ${\textit{WSE}}_{C}$, respectively.

Following Semantic Entropy (SE) [22, 49] and SAR [23], we evaluate WSE by framing UQ as the problem of predicting whether to trust a model generation for a given medical query. We employ the widely used area under the receiver operating characteristic (AUROC) curve for the binary event that a given response is incorrect, which captures both precision and recall, ranging from 0 to 1, with 1 representing a perfect classifier and 0.5 representing a random estimator. This metric evaluates whether WSE can effectively distinguish between correct and incorrect answers across various uncertainty thresholds. Additionally, since there can be unrealistic uncertainty thresholds, we employ deep AUROC [50], which measures performance in multiple groups of predicted risk, or groups of true positive rate or false positive rate.

We adopt two standard metrics to evaluate the correctness of responses: Rouge-L Similarity (RS) [51] and Sentence Similarity (SS) [23]. RS measures the longest common subsequence between the output and reference answer, serving as a fuzzy matching criterion. For SS, we utilize the cross-encoder model mentioned in Section 3.2.1, with DistillRoBERTa [52] as the backbone. SS corresponds to the semantic textual similarity denoted by ${S}_{C}$ in Eq. (5). We consider the generation correct if either the RS or SS exceeds the predefined threshold of 0.5. Notably, we employ the most likely generation, as introduced in Section 4.1.6, as the object to evaluate the correctness of the current QA. In Section 4.2.2, we will analyze the sensitivity of WSE to these threshold values.

We conduct experiments on seven open-source “off-the-shelf” LLMs provided by the Hugging Face platform, including both pre-trained LLMs (e.g., LLaMA-7B [8]) and instruction-tuned LLMs (e.g., LLaMA-2-7B-Chat [24], Mistral-v0.1 [53], Zephyr-7B-Alpha [26], Vicuna-7B-v1.5 [54], WizardLM-7B [55], StableBeluga-7B [24, 25]) with the model size of 7B.

We utilize five free-form medical QA datasets: COVID-QA [27], Medical Meadow MedQA [28], PubMedQA [29], MedMCQA [30] and MedQuAD [31]. COVID-QA consists of 2,019 query-answer pairs related to COVID-19, and we employ all query-answer pairs within the maximum sequence length allowed by the language model. MedMCQA is a large-scale, multiple-choice QA dataset for medical entrance exams, and we select all samples where a question has only one correct option and begins with “what” or “which” (1895 in total). Medical Meadow MedQA is a free-form multiple-choice OpenQA dataset for solving medical problems, collected from the professional medical board exams. MedQuAD covers 37 question types associated with diseases, drugs, and other medical entities such as tests. For Medical Meadow MedQA and MedQuAD, we randomly select 2000 test samples from the validation set. PubMedQA is a novel biomedical QA dataset collected from PubMed abstracts and we employ the full test set (1000 question-answer pairs).

Unlike COVID-QA and PubMedQA, Medical Meadow MedQA, MedMCQA, and MedQuAD do not provide contextual information, and we randomly select five fixed query-answer pairs from each dataset to form the few-shot prompts, enabling LLMs to follow the instructions.

We compare our method with PE [16], Semantic Entropy (SE) [22], Lexical Similarity (LS) [56], Token-level SAR (Token-SAR), Sentence-level SAR (Sent-SAR), and SAR [23]. PE quantifies uncertainty as described in Section 3.2. SE considers semantic equivalence and calculates the cluster-wise entropy. LS computes the mean semantic similarity score of responses in the set of $K$ generated responses. Token-SAR and Sent-SAR reallocate the uncertainty weights of tokens and sentences based on their relevance, respectively. SAR combines Token-SAR and Sent-SAR. For domain-specific medical QA, ${\textit{WSE}}_{W}$ highlights keywords in each response by assessing the semantic relevance of each word based on semantic variation, which addresses the issue of inconsistent semantic relevance within individual words that SAR encounters. Additionally, ${\textit{WSE}}_{S}$ leverages a more reliable and explainable measure for semantic textual similarity and enlarges the generative probability of more trustworthy responses based on self-consistency. Similar to SAR, ${\textit{WSE}}_{C}$ is an orthogonal combination of ${\textit{WSE}}_{W}$ and ${\textit{WSE}}_{S}$, which calibrates uncertainty at both the word and sequence levels.

Given each medical query, LLMs generate five free-form responses (i.e., $K=5$) via multinomial sampling, which are then employed for UQ. For the correctness evaluation of the current QA, we employ greedy search to obtain the most likely generation [22, 23]. The temperature is fixed at 0.5 for all LLMs, and the max length of each generation is set to 128 tokens. The coefficient $c$ in Eq. (5) is set to 1.0 by default, and the denominator $d$ in the relevance-controlled quantity in Eq. (18) is empirically set to 0.001.

Given the integrated measurement of semantic textual similarity described in Section 3.2.1, we compare ${\textit{WSE}}_{W}$, ${\textit{WSE}}_{S}$, and ${\textit{WSE}}_{C}$ with six baseline methods, utilizing SS as the criterion for correctness evaluation. As summarized in Table 1, all the three WSE variants outperform the baseline methods significantly, with ${\textit{WSE}}_{C}$ achieving the highest overall AUROC of 0.6235. By highlighting keywords and addressing the inconsistency of semantic relevance within each word, ${\textit{WSE}}_{W}$ surpasses Token-SAR by $2.3\%$ in AUROC overall, particularly on the MedQA dataset, where it exceeds Token-SAR by $3.17\%$. By employing a more reliable measure of semantic similarity, ${\textit{WSE}}_{S}$ surpasses Sent-SAR by $1.32\%$ in AUROC overall, especially on the MedMCQA dataset, where it exceeds Sent-SAR by $2.53\%$. These enhancements highlight the superior adaptability of WSE for open-ended medical QA.

In the MedQuAD task, each few-shot prompt comprises multiple question-answer pairs with a similar structure, without providing any contextual information to the language models. Additionally, ground truth and generated responses exhibit notably greater length than other tasks. To address these challenges, we calculate normalized semantic relevance scores at the word level and assign them to individual tokens. This strategy enhances the connectivity between each word and the entire sequence, effectively mitigating biases induced by sequence length. As a result, ${\textit{WSE}}_{W}$ achieves the highest AUROC of 0.626, significantly outperforming six baseline methods.

Given that RS depends on the length of the longest common subsequence, and semantically equivalent textual sequences can be syntactically or lexically distinct, tasks involving long reference answers and responses may result in no generations meeting the correctness criterion. Table 2 presents the comparative results, excluding tasks with an accuracy of 0 from our analysis. Despite the inherent evaluation limitations of RS, ${\textit{WSE}}_{W}$ and ${\textit{WSE}}_{C}$ demonstrate remarkable superiority. By assessing semantic relevance at the word level rather than evaluating individual tokens independently, ${\textit{WSE}}_{W}$ achieves the second-highest average AUROC of 0.6498, outperforming Token-SAR by $2.24\%$, while ${\textit{WSE}}_{C}$ attains the highest average AUROC of 0.6555. Notably, comparable baselines exhibit unstable uncertainty estimation under rigorous correctness evaluation conditions (e.g., Sent-SAR obtains an AUROC of 0.3647 on the PubMedQA dataset in the Vicuna-7B-v1.5 setting), while WSE consistently performs reliably, indicating significant potential for practical medical QA applications in the domain of healthcare.

We also evaluate WSE on the COVID-QA dataset utilizing a more stringent deep AUROC metric and correctness evaluation criteria. As shown in Table 3, all the three variants of WSE consistently outperform the corresponding three variants of SAR. For instance, ${\textit{WSE}}_{S}$ outperforms Sent-SAR by $3.2\%$ in deep AUROC at the RS setting, ${\textit{WSE}}_{W}$ surpasses Token-SAR by $12.06\%$ at the SS setting, and ${\textit{WSE}}_{C}$ achieves the highest deep AUROC of 0.7115, exceeding SAR by $10.04\%$, with SS as the correctness metric.

Overall, WSE demonstrates superior accuracy and stability in quantifying the uncertainty of LLMs-generated responses compared to six baseline methods, utilizing both RS and SS as correctness evaluation criteria across five popular open-ended medical QA tasks.

To investigate the impact of various thresholds for two correctness metrics on ${\textit{WSE}}_{W}$, ${\textit{WSE}}_{S}$, ${\textit{WSE}}_{C}$, and five baseline methods, we utilize LLaMA-2-7B-Chat and generate ten responses (i.e., $K=10$) to each medical query on the COVID-QA dataset. As is shown in Fig. 4 and Fig. 5, each uncertainty measure is influenced to varying degrees by the threshold. Generally, as the evaluation criteria become more stringent, WSE consistently outperforms five baseline methods. Notably, when utilizing RS, ${\textit{WSE}}_{C}$ achieves the highest AUROC of 0.7315, while using SS results in an AUROC of 0.877. When the threshold for SS is set to 0.1, all answers are identified as correct, and we exclude this scenario from our analysis.

Given that entropy-based methods integrate responses within the candidate set, we explore how the number of responses (i.e., $K$) impacts the performance of UQ. As illustrated in Fig. 6 and Fig. 7, ${\textit{WSE}}_{W}$ and Token-SAR exhibit sensitivity to variations in $K$. Nevertheless, ${\textit{WSE}}_{W}$ ultimately surpasses the baselines and achieves the second-highest AUROC score under both correctness evaluation criteria. When employing RS as the correctness metric, ${\textit{WSE}}_{S}$ generally outperforms the baseline methods and achieves the second-highest AUROC of 0.6161 leveraging only 6 generated sequences, which is generation-efficient,. It is noteworthy that ${\textit{WSE}}_{C}$ consistently outperforms comparable methods under both correctness evaluation criteria.