CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering

Psychological counseling techniques offer significant support for individual mental health and quality of life, facilitating individuals in identifying, resolving, and coping with psychological issues and dilemmas Meier and Boivin (2010). Cognitive Behavioral Therapy (CBT), one of the most widely used approaches Beck (1979), focuses on the interplay of cognition, emotion, and behavior in influencing mental health. Therapists work with clients to identify and challenge harmful thought patterns, also known as cognitive distortions, promoting healthier coping mechanisms. Acceptance and Commitment Therapy (ACT), representing the third wave of cognitive behavioral therapies Hayes et al. (2003), emphasizes psychological flexibility and embracing present experiences while acting in line with personal values. On the other hand, humanistic psychotherapy, such as Carl R. Rogers’ client-centered therapy Rogers (1951), centers on individuals’ self-actualization and growth, with therapists providing unconditional positive regard, empathy and genuineness to support self-discovery and personal development. Dialectical Behavior Therapy (DBT), developed by Linehan (2014) as a modification of CBT for treating borderline personality disorder, integrates cognitive-behavioral techniques with mindfulness practices from Buddhist traditions, aiming to balance acceptance and change to improve emotional regulation and interpersonal effectiveness.

LLMs have gained significant attention across various research domains, including medicine Thirunavukarasu et al. (2023), education Dan et al. (2023), and finance Wu et al. (2023). In the field of mental health support, the use of LLMs is an emerging and valuable research area Dhingra et al. (2023). Recent studies have shown that ChatGPT, in particular, excels in mental health analysis and model interpretability when compared to traditional neural network approaches Yang et al. (2023). To address the limited availability of mental health data, researchers have created the ExTES emotion support dialogue dataset, while the SMILE approach extends single-turn dialogues to multi-turn interactions, enriching the data source for mental health support Zheng et al. (2023); Qiu et al. (2023). Additionally, a framework called Psy-LLM has been proposed to provide real-time feedback to mental health professionals by combining pre-trained LLMs with psychological forum Q&A Lai et al. (2023). Furthermore, the rapid development of LLM has promoted psychological counseling, which mainly includes motivational interviewing Min et al. (2022); Welivita and Pu (2023) and cognitive behavioral therapy Ding et al. (2022); Maddela et al. (2023); Sharma et al. (2023), where LLMs offer new methodologies for delivering interventions and support.

Given a cognitive psychology question-answer dataset PsyQA Sun et al. (2021) with questions and their descriptions and a set of CBT prompts represented by $\mathcal{P}_{\text{CBT}}$, we employ ChatGPT in conjunction with $\mathcal{P}_{\text{CBT}}$ to generate CBT-oriented answers. For each question $q_{i}$ and its description $d_{i}$ from PsyQA, the CBT response is derived as $a_{i}=\text{ChatGPT}(q_{i},d_{i},\mathcal{P}_{\text{CBT}})$. Assembling the questions, descriptions, and generated answers results in the CBT QA dataset, represented as $\mathcal{D}_{\text{CBT QA}}=\{(q_{i},d_{i},a_{i})\}$. This dataset subsequently undergoes instruction fine-tuning to cultivate the specialized CBT-LLM. The overarching aim is to utilize the insights from the PsyQA dataset in tandem with $\mathcal{P}_{\text{CBT}}$ through the intervention of ChatGPT to derive a language model proficient in CBT question-answering.

CBT is a well-established psychological intervention for a wide range of psychological disorders. Despite the importance of CBT in mental health practice, current mental health support datasets for the development of language models do not yet adequately cover this area. To address this gap, we turned to PsyQA Sun et al. (2021), a well-known mental health Q&A dataset. The dataset is derived from the Chinese online mental health support forum Yixinli¹¹1https://www.xinli001.com/qa. The dataset encapsulates a broad spectrum of pertinent and complex questions. Given the extensive variety of queries in PsyQA, crafting individual professional CBT responses is not feasible. We draw inspiration from the Alpaca study Taori et al. (2023), which demonstrated the high quality of data generated by ChatGPT. Building upon this precedent, we meticulously designed CBT-centric prompts to guide ChatGPT in providing CBT-informed responses to the questions in PsyQA. To ensure robust generative capabilities and manage context length effectively, we utilized OpenAI’s gpt-3.5-turbo-16k model²²2https://openai.com/blog/chatgpt.

The foundational aspects of CBT are outlined in Beck (1979) research. The primary objective of CBT is to identify and comprehend an individual’s automatic thoughts and core beliefs, which play a crucial role in shaping their emotional and behavioral disturbances. CBT places great emphasis on challenging distorted cognitions, aiming to rectify any cognitive biases that contribute to psychological distress. To facilitate lasting change, CBT incorporates behavioral experiments and skill training, encouraging individuals to implement and practice new strategies in real-world settings. Recognizing the essential elements of CBT, it is crucial to adopt a comprehensive approach when applying these methodologies and strategies in therapeutic contexts.

In alignment with the core principles of CBT, we have restructured and formulated the response mechanism into five pivotal components, specifically adapted to suit the single-turn dialogue response format:

1. 1.

   Validation and Empathy: Show understanding and sympathy for the patient’s feelings or issues, creating a sense of safety.

2. 2.

   Identify Key Thought or Belief: Through the problem description, identify potential cognitive distortions or core beliefs.

3. 3.

   Pose Challenge or Reflection: Raise open-ended questions, encouraging the patient to reconsider or reflect on their initial thoughts or beliefs.

4. 4.

   Provide Strategy or Insight: Offer practical strategies or insights to help them deal with the current situation.

5. 5.

   Encouragement and Foresight: Motivate the individual to employ the suggested strategy, underscoring that this is merely an initial step and additional support may be warranted.

Based on the aforementioned structure, we crafted the prompt depicted in Fig. 3, aiming to steer ChatGPT towards generating responses that are congruent with CBT methodologies. To verify the consistency of the prompt outputs, we randomly selected a question with its description and submitted it to GPT-4 via the CBT Prompt, procuring a prototype CBT response. This example was then incorporated into the prompt to inform the generation of subsequent responses. This iterative refinement ensures that our prompts effectively direct ChatGPT to produce answers that are consistent with CBT principles.

In our research on automating Cognitive Behavioral Therapy (CBT) question-answer tasks, we chose to utilize Language Models (LMs) as the foundational framework, specifically leveraging the power of Large Language Models (LLMs) due to their exceptional performance across various Natural Language Processing (NLP) tasks. However, to better tailor the LLMs to the specific requirements of CBT Q&A, we incorporated advanced fine-tuning strategies, instruction tuning Wang et al. (2022) and LoRA Hu et al. (2021).

To fully exploit the potential of LLMs for complex tasks like CBT Q&A, we incorporated instruction tuning and LoRA fine-tuning strategy. Instruction tuning is a method that provides explicit task directives to the model during training, guiding its generation process to align more closely with the specific task requirements. We adopted the approach proposed by Wang et al. (2022) for instruction tuning. Moreover, LoRA is a fine-tuning strategy that enhances model performance by augmenting each layer of the model with additional parameters. These parameters are represented using low-rank matrices, effectively adjusting the output of each layer. Specifically, for a pre-trained weight matrix $W$, its parameter update $\Delta W$ can be calculated as:

Here, $A$ and $B$ are low-rank matrices, while the original parameter $W$ remains unchanged throughout the training process.

In addition, for training the CBT-LLM, we used cross-entropy loss, a commonly used loss function for language modeling tasks. The cross-entropy loss measures the dissimilarity between the predicted probability distribution and the true distribution of the next word. The loss can be computed using the following equation:

In summary, by incorporating instruction tuning and LoRA fine-tuning strategy, we enhance the adaptability and performance of the LLMs to better meet the requirements of CBT Q&A tasks.

We selected the CBT QA dataset and randomly split it into a training set (90%) and a test set (10%). To meet the data format requirements for instruction-based fine-tuning, we concatenated the question and description to form a single passage as input, using the CBT response as output. We further incorporated the instruction: "You are an experienced therapist specializing in cognitive behavioral therapy. Please answer the following questions in the capacity of a psychotherapist," forming a triad format of {"instruction": , "input": , "output"}.

• •

  LLaMA-Chinese-7B: The LLaMA-Chinese-7B Cui et al. (2023) is derived from the LLaMA-7B Touvron et al. (2023) model, enhanced with a series of optimizations specific to Chinese processing. Researchers expanded the model’s vocabulary, adding 20,000 Chinese tokens, culminating in a Chinese LLaMA tokenizer with a vocabulary size of 49,953. Subsequently, the model underwent secondary pre-training on 20GB of Chinese data, and by adopting the LoRA approach, freezing weights and incorporating low-rank matrices, the training efficiency was improved.

• •

  Alpaca-Chinese-7B: Building on the foundation laid by LLaMA-Chinese-7B, the Alpaca-Chinese-7B model Cui et al. (2023) underwent further refinement to specialize in instruction-following capabilities. This development utilized an instruction dataset ranging from 2M to 3M entries, aiming to improve the model’s performance in executing user-specified tasks.

• •

  Qwen-7B: With a training corpus exceeding 2.4 trillion Chinese tokens, Qwen-7B Bai et al. (2023) encompasses English and other languages. The model exhibits outstanding performance in a variety of downstream tasks. Qwen-7B utilizes a vocabulary size of approximately 150,000 tokens. Compared to mainstream open-source Chinese-English vocabulary models, it is better suited for multilingual processing, allowing users to enhance and extend the processing capability for specific languages without the need to expand the vocabulary.

• •

  Baichuan-7B: Engineered with the Transformer architecture, Baichuan-7B Baichuan (2023) leverages 7 billion parameters and was trained on a 1.2 trillion token corpus. It uniquely supports extended text sequences with a context window of 4096 and excels in Chinese language processing while maintaining effective bilingual (Chinese and English) support. Baichuan-7B has shown exemplary results on the C-EVAL benchmark Huang et al. (2023), making it a highly suitable model for CBT Q&A applications, particularly for tasks requiring extensive comprehension and generation in Chinese. On the authoritative Chinese benchmark C-EVAL Huang et al. (2023), Baichuan-7B has demonstrated superior performance in Chinese language tasks. With its unique design characteristics, Baichuan-7B is considered an ideal choice for CBT Q&A implementations.

In this experiment, we utilized the NVIDIA V100 32G GPU for model training. During training, we set the gradient accumulation steps to 4, meaning that the gradients from every 4 batches would be accumulated and then used for a single parameter update. The learning rate was set to $5\times 10^{-5}$, and we adopted the cosine-type learning rate scheduler to adjust the learning rate throughout the training process. The entire training will span across 3 epochs. To accelerate the training and enhance model performance, we also enabled the use of 16-bit half-precision floating point numbers. It’s noteworthy to mention that the fine-tuning implementation for this model is based on LLaMA Factory hiyouga (2023), an efficient model tuning toolset.

To comprehensively evaluate the model’s performance on mental health support Q&A tasks, we adopted a series of automatic evaluation metrics. BLEU Papineni et al. (2002) mainly assesses the precise matching between model outputs and reference answers by comparing n-gram co-occurrences. METEOR Banerjee and Lavie (2005) goes beyond precise matching, incorporating matches at the level of synonyms, stems, and morphological variations, offering a more holistic assessment of semantic similarity. CHRF Popović (2015), a character-based metric, primarily gauges the alignment at the character level between model outputs and reference answers. Both BLEURT Sellam et al. (2020) and BERTSCORE Zhang* et al. (2020) utilize pre-trained representations from the BERT Devlin et al. (2019) model to evaluate deep semantic alignment. While BLEURT is tailored for assessing outputs from machine translation and text generation tasks, BERTSCORE calculates the cosine similarity between BERT embeddings of the model output and references. To ensure the accuracy and consistency of our evaluation, we employed the July toolkit Cavusoglu et al. (2023) for the computation of these automatic evaluation metrics. The results of the automatic evaluation are depicted in Table 4.

Our experiments demonstrate the superiority of our CBT-LLM model in the domain of mental health support Q&A tasks, outperforming three advanced benchmark models. Firstly, LLaMA-Chinese-7B, as a Chinese-optimized version of LLaMA-7B, underwent various optimizations for Chinese processing. However, its performance on tasks related to CBT structured interventions still lagged behind CBT-LLM. This can be attributed to the fact that its training and optimization strategies were not specifically tailored for mental health support tasks. In contrast, Alpaca-Chinese-7B, which builds on the foundation of LLaMA-Chinese-7B, shows improved performance due to its fine-tuning geared towards adherence to CBT principles. Nevertheless, it still slightly lagged behind our CBT-LLM, underscoring the tailored model’s specialized effectiveness. Furthermore, while Qwen-7B possesses advantages in multilingual processing, its performance on tasks related to CBT structured interventions was slightly inferior to CBT-LLM. This might be because Qwen-7B’s training data and optimization strategies lean more toward broad multilingual processing.

To deeply evaluate the quality of the model’s responses generated based on CBT in psychotherapy counseling, we designed a comprehensive manual evaluation framework. To ensure the selection was representative, we randomly extracted 100 samples from the test set. Our evaluation team consisted of four senior psychology students and an experienced psychotherapist to ensure accuracy and professionalism. During the evaluation process, each entry consisted of three parts: a question title, a description, and an answer text. Evaluators were required to score each answer based on the following three metrics: (1) Relevance Measure, focusing on the association degree between the answer and the question, with scores ranging from 0-2, representing from not relevant to fully relevant; (2) CBT Structure Measure, assessing whether the answer adheres to specific CBT structures and principles, with scores ranging from 0-2, indicating from not adhering to structure to fully adhering; and (3) helpfulness Measure, assessing the applicability and usefulness of the answer from a psychotherapy perspective, with scores ranging from 0-2, representing from low applicability to highly applicable. These three metrics aim to comprehensively and deeply assess the model’s application performance in the field of psychotherapy counseling. The results of the automatic evaluation are shown in the table 5.

The findings shown in Table 5 indicate that Baichuan-7B marginally outperforms Alpaca-Chinese-7B in all aspects, particularly in adhering to CBT frameworks and providing helpful responses in a psychotherapeutic context.

As depicted in Figure 4, the CBT-LLM case study delved into addressing a user’s query concerning social isolation and loneliness. The CBT-LLM model, grounded in cognitive-behavioral therapy principles, adopts a multidimensional approach. It begins by acknowledging and empathizing with the user’s emotions and offering emotional support. Subsequently, the model identifies potential cognitive biases, such as overgeneralization and all-or-nothing thinking, and provides practical strategies, emphasizing the significance of social engagement and recommending professional psychological counseling. In conclusion, the model offers positive encouragement, emphasizing the individual’s potential to overcome challenges. This case underscores the unique strengths of the CBT-LLM model in comprehending, guiding, and motivating users, further validating its effectiveness in delivering psychological advice and support.