Enhancing AI-Driven Psychological Consultation: Layered Prompts with Large Language Models

The rapid development of deep learning has brought significant changes and advancements to the fields of computer vision [6, 7] and natural language processing [8]. Large Language Models (LLMs) [9] have significantly advanced the field of natural language processing (NLP) due to their ability to understand and generate human-like text. Recent surveys and overviews highlight the architectural innovations, training strategies, and applications of LLMs such as GPT-3, GPT-4, PaLM, and LLaMA [10, 11]. These models have demonstrated remarkable capabilities across various tasks, including text generation, summarization, and translation, by leveraging vast amounts of training data and sophisticated neural architectures [12, 13, 14].

The impact of LLMs extends to domains such as information retrieval, where they enhance the precision and expressiveness of user queries and improve retrieval efficiency [15, 16, zhou, 17]. Additionally, LLMs are being utilized in optimization tasks, serving as powerful tools for mathematical problem-solving and decision-making processes [18]. Their versatility is further evident in multimodal applications, where models integrate visual and textual data to perform complex tasks [12, 19].

Despite these advancements, LLMs face challenges related to ethical considerations, such as bias, toxicity, and privacy concerns [20]. Researchers are actively exploring methods to align LLMs with ethical standards and mitigate these risks, ensuring their safe and responsible deployment [20, 21].

The application of LLMs in psychological consultation is a growing area of research, driven by the need to address mental health issues efficiently and effectively. Traditional psychological counseling methods, which rely heavily on human interaction, face challenges such as scalability, accessibility, and privacy concerns [1, 3]. The integration of LLMs into psychological services aims to overcome these limitations by providing scalable and accessible mental health support.

Several studies have proposed frameworks and models that leverage LLMs for psychological consultation. For instance, CPsyCoun focuses on reconstructing and evaluating multi-turn dialogues to create a high-quality dataset for training LLMs in the context of psychological counseling [22]. Similarly, Psy-LLM and ChatCounselor utilize LLMs to provide online mental health support, enhancing the accessibility and affordability of psychological services [1, 3].

Innovative approaches such as K-ESConv and BianQue inject professional knowledge into LLMs, improving the quality and diversity of responses in emotional support dialogues [23, 24]. These methods highlight the potential of LLMs to offer empathetic and contextually appropriate support, addressing the emotional and psychological needs of users effectively.

Overall, the integration of LLMs into psychological consultation presents a promising solution to the growing demand for mental health support, offering scalable, accessible, and effective services. However, ongoing research is necessary to address ethical considerations and ensure the safe deployment of these technologies in sensitive contexts such as mental health.

To ensure a diverse and comprehensive dataset, we collected psychological consultation dialogues from multiple online platforms and mental health forums. The data collection process involved the following steps:

• •

  Source Selection: We identified reputable sources of psychological consultation dialogues, including well-known mental health platforms and forums where professionals and users interact. These sources were chosen based on their high user engagement and the quality of their content.

• •

  Data Extraction: Using web scraping techniques and API access where available, we extracted dialogues that covered a wide range of mental health issues, such as anxiety, depression, stress, relationship problems, and more. This extraction process ensured that we captured a variety of interaction types and user concerns.

• •

  Anonymization and Cleaning: To maintain privacy and confidentiality, all personal identifiers were removed from the dialogues. The data was then cleaned to eliminate any irrelevant or sensitive information, ensuring that the dataset adhered to ethical standards for data usage.

• •

  Format Standardization: The collected dialogues were formatted uniformly to facilitate ease of use in training and evaluation. Each dialogue was structured to include metadata such as the topic, user demographics (where available), and the context of the consultation.

The resulting dataset comprises thousands of dialogues, each providing a rich context for training and evaluating large language models in the domain of psychological consultation.

Traditional evaluation metrics for natural language generation, such as BLEU and ROUGE, focus on lexical similarity and do not adequately capture the nuanced requirements of psychological consultation, such as empathy, relevance, and contextual appropriateness. To address this limitation, we employed GPT-4 as a judge for evaluating the performance of our model.

The motivation for our prompt engineering approach is rooted in the need to address several key challenges in AI-driven psychological consultation. Traditional LLMs often lack the nuanced understanding and empathetic response generation required for effective psychological support. Our goal is to create prompts that not only gather comprehensive user information but also encourage the LLM to generate emotionally intelligent and contextually appropriate responses. By designing layered, empathy-driven, and scenario-based prompts, we aim to enhance the LLM’s ability to engage users in meaningful dialogues, provide accurate advice, and maintain a supportive tone throughout the interaction.

Our prompt engineering approach involves creating a series of dynamic and layered prompts that guide the LLM through various stages of the consultation process. Each prompt is designed to fulfill a specific role, from initial information gathering to providing tailored advice and empathetic support.

The effectiveness of our prompt engineering method lies in its ability to create a structured yet flexible dialogue flow that mimics the nuances of human psychological consultation. By incorporating layered prompts that adapt based on user input, we ensure that the LLM gathers comprehensive information and provides responses that are both contextually relevant and emotionally supportive. The use of empathy-driven prompts enhances the LLM’s ability to connect with users on an emotional level, while scenario-based prompts improve its practical application of psychological knowledge. This approach not only enhances the user experience but also ensures that the advice provided is accurate, empathetic, and tailored to the user’s unique needs.

In summary, our method combines advanced prompt engineering techniques with the powerful capabilities of LLMs to deliver a more effective and empathetic psychological consultation service. The structured prompts guide the LLM through various stages of the consultation process, ensuring comprehensive information gathering, deep contextual understanding, and supportive engagement. This innovative approach holds significant promise for improving the accessibility and quality of mental health support provided by AI-driven systems.

We conducted our experiments using the following models and methods:

• •

  ChatGPT Baseline: Standard prompts without specific tailoring for psychological consultation.

• •

  GPT-4 Baseline: Standard prompts using the GPT-4 model.

• •

  Chain-of-Thought (CoT) Prompting: A method where the LLM is guided to think step-by-step before generating a response.

• •

  Proposed Method: Our layered and empathy-driven prompt engineering approach applied to both ChatGPT and GPT-4.

Each model was evaluated based on its performance in generating responses to a diverse set of psychological consultation scenarios. We measured the effectiveness of the responses using several criteria, including relevance, empathy, context understanding, and overall user satisfaction.

To quantitatively assess the performance of each method, we used the following metrics:

• •

  Relevance: The degree to which the response addresses the user’s specific concerns.

• •

  Empathy: The extent to which the response shows understanding and compassion for the user’s feelings.

• •

  Context Understanding: The ability of the model to maintain context and provide consistent advice throughout the conversation.

• •

  User Satisfaction: Overall satisfaction of users based on feedback collected after each interaction.

The results of our experiments are summarized in Table 1. Our proposed method consistently outperformed the baseline and CoT methods across all metrics.

The experimental results clearly indicate that our proposed method significantly enhances the performance of LLMs in psychological consultation tasks. The improvements in relevance, empathy, context understanding, and user satisfaction highlight the effectiveness of our layered, empathy-driven, and scenario-based prompts.