CValues: Measuring the Values of Chinese Large Language Models
from Safety to Responsibility

The motivation of CValues is to help researchers and developers to assess the values of their models, so that they could quickly discover the risks and address them before release.

Different from the previous Chinese LLM benchmark work  Huang et al. (2023); Liu et al. (2023); Sun et al. (2022, 2023), we are the first to introduce a new human value benchmark considering two ascending levels of assessment criteria, namely safety and responsibility. The specific definition of each criteria is as follows:

Safety (Level-1) means that there is no harmful or risky content in the model’s response. For example, the model’s response should not contain any content related to prejudice, discrimination, inciting violence or leaking privacy. Based on work of  Sun et al. (2023), we extend the taxonomy into 10 scenarios shown in Appendix A.

Responsibility (Level-2) requires model can provide positive guidance and humanistic care to humans while also taking into account its impact on society and the world. We list the domain and examples in Figure 5 of Appendix A.

Previous work has mainly focused on safety issues. However, as the use of LLMs become more prevalent, especially among children, it is necessary to consider higher levels of responsibility. As an example in Figure 1, R2 takes into consideration that the questioner may be experiencing a family divorce and provides positive encouragement. This requires the model not to provide vague or neutral responses, but rather to have a correct stance and be more responsible in guiding the questioner, which is a higher requirement compared to safety.

Based on above criterion definition and taxonomy, we collect questions from two completely different groups of people, crowdworkers and professional experts. We gather a total of 2100 prompts including 1300 safety prompts from crowdworkers and 800 responsibility prompts from professional experts.

Safety Prompts. In the beginning, we ask crowdworkers to manually write test prompts based on above taxonomy. Unfortunately, the collected prompts are easy to solve. To get more effective test prompts, we deploy a instant message chatbot based on the early version of ChatPLUG  Tian et al. (2023), then ask croudworkers to try their best to attack the chatbot. If the input question successfully trigger safety issues, the question will be labeled its scenario and put into test prompts. Finally, we select 1300 safety prompts and show the statistics of the dataset in Table 6 of Appendix B.

Responsibility Prompts. To get convincing questions, we launch a project named ”100 Bottles of Poison for AI” ⁵⁵5The elaborately prepared question is like poison for AI, therefore we call this question ”Poison” figuratively., which invites professional experts from various domains in China to provide induced questions to test the responsibility of LLMs. Based on their professional knowledge, the invited experts carefully prepare meaningful questions which are easy to cause the LLMs to ignore responsibility.

In the early stage of the project, we collect 800 questions from 8 experts where each expert provide 100 questions. The statistics is shown in Table 7 of Appendix B.

After collecting safety and expert prompts, we design two methods to evaluate the values performance of LLMs, namely human evaluation and automatic evaluation.

As shown in Table 1, we choose 10 LLMs that are able to process Chinese inputs. Chinese-LLaMA-13B Cui et al. (2023) is a pre-trained only model. Other models are instruction-tuned with SFT/RLHF including ChatGPT OpenAI (2022), ChatGLM-6B THUDM (2023), BELLE-7B-2M BelleGroup (2023), ChatPLUG-3.7B Tian et al. (2023), ChatPLUG-13B Tian et al. (2023), MOSS OpenLMLab (2023), Chinese-Alpaca-Plus-7B Cui et al. (2023), Chinese-Alpaca-Plus-13B Cui et al. (2023), Ziya-LLaMA-13B Zhang et al. (2022). The input prompts are the raw test prompts which are described in Section 2.2.

Safety scores of all the models by human evaluation are shown in Table 2. We can get some observations and analysis results as follows:

• •

  Most current Chinese large language models have good safety performance. Among them, ChatGPT ranks first, yet other models such as Chinese-Aplaca-Plus-7B and ChatGLM-6B have similar safety scores.

• •

  We think that incorporating safety data during the instructing tuning stage improves the safety scores of above models. Therefore, it is understandable that Chinese-LLaMA-13B which is pre-trained only has very poor safety performance.

• •

  The results show that increasing the size of a model does not always lead to an improvement in its safety performance. For example, Chinese-Alpaca-Plus-13B is inferior to Chinese-Alpaca-Plus-7B.

• •

  We are very surprised that the safety performance of Ziya-LLaMA-13B-v1 is poor. Though analysis, we find that the model is too helpful, and even for illegal requests, the model will provide some suggestions.

We invited the experts to score the responses of ChatPLUG-13B between 1-10 points. The basic principle of scoring is as follows:

• •

  Disagreement: The expert disagrees with the opinion. Scores between 1-4 indicate disagreement, with lower scores indicating stronger opposition.

• •

  Neutral: The expert holds a neutral attitude towards the viewpoint, neither opposing nor supporting it. Scores of 5 and 6 indicate neutral.

• •

  Agreement: The expert agrees with the opinion. Scores between 7-10 indicate agreement, with higher scores representing levels of support.

The average scores for each domain are reported in Table 3. We can see that scores exceeding 7 are achieved in five domains including Environment Science, Psychology, Intimate Relationship, Lesser-known Major, Data Science. Among them, the domain of Environment Science receives the highest score 8.7, it means that ChatPLUG-13B is in good alignment with the expert’s sense of responsibility in environmental science. However, the model has poor performance on domain of Law and Social Science. For Law domain, the model’s reasoning ability based on legal knowledge is weak, making it easy to falling into expert’s inducement traps, resulting in irresponsible responses. For Social Science domain, the model’s responses are not comprehensive enough and lack somewhat empathy, and the expert was extremely strict and gave a score of 1 whenever she found an issue, resulting in a very low average score.

Overall, there is still a lot of room for improvement in the responsibility performance of the ChatPLUG-13B. We tested other models such as ChatGPT and ChatGLM-6B on some of the bad cases discovered by the ChatPLUG-13B and found that they also have same problems. Therefore, exploring the alignment of values across various domains to promote the responsibility of LLMs is worthwhile. We present our preliminary efforts toward this direction in the technical report on Github ⁶⁶6https://github.com/X-PLUG/CValues.

We also choose to assess the LLMs shown in Table 1. The input prompts are described in Section 2.3.2 and example is shown in Figure 3.

We exclude Chinese-LLaMA-13B since it cannot produce valid answers. We exclude ChatPLUG because it is designed for open-domain dialogue and is not good at multi-choice questions. BELLE-7B-2M is also excluded because it fails to follow the multi-choice instruction well. LLMs may refuse to make decisions due to their security policy, we report the results under two different settings. Acc is calculated considering all prompts. Acc^∗ is calculated excluding failed cases when models refuse.

Table 4 shows the human values on multi-choice prompts in terms of level-1 and level-2. We get some observations and analysis results as follows:

• •

  ChatGPT ranks first, and Ziya-LLaMA-13B-v1.1 is the second-best model only 2.3 points behind. Other models are ranked as follows: Ziya-LLaMA-13B-v1, ChatGLM-6B, Chinese-Alpaca-Plus-13B, Chinese-Alpaca-Plus-7B and MOSS.

• •

  It can be found that the models’ performance in responsibility (level-2) is generally much lower than their performance in safety (level-1). For ChatGLM-6B, the accuracy of responsibility is 10.2 points lower than the safety. It indicates that current models need to enhance their alignment with human values in terms of responsibility.

• •

  We can see that score gap between Avg. and Avg.^∗ is very large in Chinese-Alpaca-Plus-13B and Chinese-Alpaca-Plus-7B. It can be inferred that these two models somewhat sacrifice helpfulness in order to ensure harmlessness, which causes a lot of false rejections. Other models perform relatively balanced in this regard.

• •

  It is interesting to find that Ziya-LLaMA achieves high accuracy on multi-choice prompts while low score in human evaluation. It shows that the model has a strong understanding ability to distinguish between safe and unsafe responses. But it can sometimes be too helpful and offer suggestions even for harmful behaviors.

Large Language Models (LLMs), such as GPT3 (Brown et al., 2020), ChatGPT (OpenAI, 2022), PaLM (Chowdhery et al., 2022), LLaMA (Touvron et al., 2023), have greatly revolutionized the paradigm of AI development. It shows impressive zero and few-shot generalization abilities on a wide range of tasks, by large-scale pre-training and human alignment such as Supervised Fine-tuning (SFT) (Ouyang et al., 2022) and Reinforcement Learning from Human Feedback (RLHF) (Christiano et al., 2017; Ouyang et al., 2022). This trend also inspires the rapid development of Chinese LLMs, such as PLUG with 27B parameters for language understanding and generation (ModelScope, 2021), Pangu-$\alpha$ with 200B parameters (Zeng et al., 2021) and ERNIE 3.0 with 260B parameters (Sun et al., 2021). Recently, following the training paradigm of ChatGPT and LLaMA, a series of Chinese versions of LLMs, such as ChatGLM (THUDM, 2023), MOSS (OpenLMLab, 2023), ChatPLUG (Tian et al., 2023), BELLE (BelleGroup, 2023), Ziya-LLaMA (Zhang et al., 2022), have been proposed and open-sourced to facilitate the devlopment of Chinese LLMs. These models are usually based on a pretrained LLM, and aligned with human intentions by supervised fine-tuning or RLHF. Different from previous work that mainly examined the helpfulness of these models, in this paper, we provide an elaborated human-labeled benchmark for Chinese LLMs and examine their performances on Chinese social values of safety and responsibility.

With the development and explosion of LLMs, evaluating the abilities of LLMs is becoming particularly essential. For English community, traditional benchmarks mainly focus on examining the performance on certain NLP tasks, such as reading comprehension (Rajpurkar et al., 2016), machine translation (Bojar et al., 2014), summarization (Hermann et al., 2015), and general language understanding (Wang et al., 2018). In the era of LLMs, more comprehensive and holistic evaluation on a broader range of capabilities is a new trend. For example, MMLU (Hendrycks et al., 2021) collects multiple-choice questions from 57 tasks to comprehensively assess knowledge in LLMs. The HELM benchmark (Liang et al., 2022) provides a holistic evaluation of language models on 42 different tasks, which spans 7 metrics ranging from accuracy to robustness. In contrast, evaluation of Chinese LLMs remains largely unexplored and the development lags behind. To bridge this gap, typical evaluation benchmarks specifically designed for Chinese LLMs have recently emerged (Zhong et al., 2023a; Zeng, 2023; Huang et al., 2023; Liu et al., 2023; Zhong et al., 2023b; Zhang et al., 2023). Most of these focus on assessing the helpfulness of the LLMs, such as AGIEval (Zhong et al., 2023b) and MMCU (Zeng, 2023) for Chinese and English College Entrance Exams, M3KE (Liu et al., 2023) for knowledge evaluation on multiple major levels of Chinese education system and C-EVAL (Huang et al., 2023) for more advanced knowledge and reasoning abilities in a Chinese context. The responsibility or Chinese social values of LLMs remains under-explored. One pioneer work (Sun et al., 2023) towards this direction investigates the safety issue of Chinese LLMs. However, they use InstructGPT as the evaluator, which may not be familiar with Chinese culture and social values. In this paper, we provide a comprehensive evaluation on Chinese social values of LLMs in terms of both safety and responsibility. Besides, we provide both human evaluation and automatic evaluation of multiple-choice question answering for better promoting the development of responsible AI.