This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

11institutetext: School of Information Sciences
University of Illinois at Urbana-Champaign
11email: {zz78, madelyns}@illinois.edu

Public Perceptions of Gender Bias in Large Language Models: Cases of ChatGPT and Ernie

Kyrie Zhixuan Zhou    Madelyn Rose Sanfilippo
Abstract

Large language models are quickly gaining momentum, yet are found to demonstrate gender bias in their responses. In this paper, we conducted a content analysis of social media discussions to gauge public perceptions of gender bias in LLMs which are trained in different cultural contexts, i.e., ChatGPT, a US-based LLM, or Ernie, a China-based LLM. People shared both observations of gender bias in their personal use and scientific findings about gender bias in LLMs. A difference between the two LLMs was seen – ChatGPT was more often found to carry implicit gender bias, e.g., associating men and women with different profession titles, while explicit gender bias was found in Ernie’s responses, e.g., overly promoting women’s pursuit of marriage over career. Based on the findings, we reflect on the impact of culture on gender bias and propose governance recommendations to regulate gender bias in LLMs.

Keywords:
Large Language Model Gender Bias Culture Public Perception Governance.

1 Introduction

With the debut of ChatGPT, a ground-breaking commercial application of large language models (LLMs), many controversies happened regarding its benefits and ethical issues [32, 36, 40, 13, 42]. On the one hand, it provides rich opportunities for automating and assisting human work such as Q&A [39] and creative writing [35]. On the other hand, the functioning and use of ChatGPT are not always ethical, with a wide range of ethical concerns being raised [42], such as academic integrity [21, 38] and job loss [3].

In this work, we investigated a particular ethical issue of ChatGPT, i.e., gender bias [2]. Gender bias has long been studied in computer systems [12] and other domains [44, 4]. Gender bias in ChatGPT could harm an extremely large population if it is widely adopted by people. Understanding public perceptions toward gender bias in LLMs is crucial to keep policies and regulations relevant and effective in meeting people’s needs.

LLMs are trained on data collected from search engines, online forums, websites, and so on. Thus, LLMs can reflect and even amplify existing biases in human language [11]. Social biases exist in varying forms in different cultures [44]. LLMs trained on data collected from different cultures may also demonstrate different types and levels of biases. To understand how gender bias manifests in LLMs rooted in different cultures, we examined ChatGPT by OpenAI based in the US and Ernie by Baidu based in China for a comparative case study, toward informing contextual and concrete regulation.

2 Related Work

Gender bias is commonly seen in LLMs. Huang et al. uncovered implicit gender biases associated with protagonists in GPT-2 generated stories [15]. The results revealed that female characters’ portrayal was centered around appearance, while male characters’ portrayal around intellect. Lucy and Bamman revealed similar gender bias in GPT-3 generated stories [22]. ChatGPT was highly likely to generate biased computer programs and struggled to remove biases in generated artifacts. Gender bias was also found when ChatGPT was used for important natural language processing (NLP) tasks such as co-reference resolution [28]. Moreover, ChatGPT was susceptible to prompt injections, allowing its safety features to be bypassed and possibly dangerous responses to be generated [46].

Ferrara argued that eliminating bias from LLMs was a challenging task since the models learned from text data on the Internet, which contained biases rooted in human language and culture [11]. Paul et al. expressed their concerns about different types of biases, i.e., gender bias, racial bias, and cultural bias, which could be present in ChatGPT-generated responses in consumer-facing applications [29]. What made the situation worse was the lack of accountability when it came to content generated by LLMs [30].

To our knowledge, no prior research has studied how culture impacted gender bias in LLMs. Further, while existing research has reported on ethical issues of LLMs from researchers’ perspectives, users’ experiences and perceptions are under-investigated. We bridge these research gaps in this study.

3 Methodology

We approached public perceptions about gender bias in ChatGPT and Ernie by comparing social media discussions around these two LLMs.

3.1 Data Collection

Social media discussions about ChatGPT were obtained with the search query “gender bias in chatgpt” from Twitter. Similarly, we searched with the Chinese query “gender bias in Ernie” on Weibo. ChatGPT was released on November 30, 2022, and Ernie was released on March 6, 2023. Our data collection covered discussions from November 30, 2022, to August 30, 2023, covering 9 months of discussions of ChatGPT and 6 months for Ernie. We collected and analyzed the data simultaneously and stopped searching for more discussions after reaching a theoretical saturation in the analysis, i.e., no new themes emerged. However, we constantly looked for recent discussions in case new findings arose.

3.2 Analysis

A thematic analysis approach [5] was adopted to analyze the online discussions. We iterated open coding, and used XMind, a mind-mapping tool, to organize the themes, sub-themes, and quotes into a hierarchical structure. The themes emerging from the discussions around ChatGPT included users’ observation of implicit gender bias, political correctness, and dissemination of scientific findings. The themes identified around Ernie included users’ observation of explicit gender bias, criticism of culture, and a lack of discussion. Below, we use quotes to illustrate our findings.

4 Public Perceptions about Gender Bias in ChatGPT

A heated discussion on gender bias in ChatGPT happened on Twitter. Many people complained about the prevalent (often implicit) gender bias in ChatGPT, and some discussed relevant press articles or research findings. A form of political correctness was found, i.e., ChatGPT refused to tell jokes about women. People called for more attention and research on gender bias in LLMs.

4.1 Observation of Implicit Gender Bias

A lot of ChatGPT users shared their observations of gender bias in their daily use. One tweet revealed that ChatGPT tended to assume men suitable for doctors, while women for teachers; men for teaching science and tech, while women for teaching arts. Such complaints about ChatGPT’s biases regarding profession titles and gender roles were common – ChatGPT tended to generate bias-rich responses when asked to finish sentences. It assumed an engineer or a CEO to be male, and a mom to be the parent who picked up the child from school.

The ChatGPT users who had some linguistic or NLP knowledge described such gender bias in ChatGPT as “implicit,” instead of “explicit.” However, the implicitly biased association between gender and profession could lead to explicit consequences and harms. For example, one user thought the “implicit” bias was substantial and persisting: “Implicit Gender Bias in ChatGPT. This ain’t subtle, folks – indeed, it barely qualifies as ‘implicit.’ It’s weird to see such a modern technology echoing attitudes that are over a half century old.”

4.2 Political Correctness

ChatGPT could paradoxically contain gender bias and be politically correct at the same time. When asked to tell a joke about a man, it did; when asked to tell a joke about a woman, it refused, saying, “I’m sorry, I don’t think it’s appropriate to make jokes about a specific gender.” Another person posted about a similar chat history with ChatGPT and commented: “Seems to have some wisdom though @OpenAI”. From these cases, we can infer that OpenAI has put effort into debiasing ChatGPT, possibly with some fairness-related rules.

4.3 Dissemination of Scientific Findings and Solutions

Several research teams shared their evaluation of gender bias in ChatGPT. For example, a group of researchers tested ChatGPT on WinoBias, a standard gender bias benchmark, and found it biased: “Both GPT-3.5 and GPT-4 are about 3 times as likely to answer incorrectly if the correct answer defies stereotypes — despite the benchmark dataset likely being included in the training data.” Their conclusion was based on the assumption that a higher accuracy for pro-stereotype questions meant the model was biased.

Users who were not researchers also actively shared research progress and professional opinions regarding gender bias in ChatGPT. The above-mentioned evaluation results on the WinoBias benchmark were retweeted and discussed frequently, raising a heated discussion on Twitter. Another gender bias evaluation revealed that when ChatGPT wrote performance feedback for workers, it selected the pronoun “she” for a kindergarten teacher or a receptionist, and “he” for a mechanic or a construction worker. This finding was similarly retweeted and discussed by many people. Another tweet pointed to an article discussing how gender bias in ChatGPT affected HR and tips for avoiding the pitfalls: “Very interesting post by @RealEvilHRLady on how gender bias shows up in ChatGPT’s responses, plus some tips for how to mitigate it with your prompts: ChatGPT gender bias: how it affects HR & tips to avoid pitfalls.”

4.4 Call For Action

People have called for more attention to be paid to gender bias in ChatGPT: “Rise of #ChatGPT and increasing #investment in #AGI makes clear #gender #bias in #AI demand immediate attention, says Anna Collard of KnowB4 Africa.” A woman teaching financial education in schools and workplaces pointed out gender bias in ChatGPT and acted by promoting fairness in her lessons, “Chat GPT is already considered to have gender bias in built, when we ask it for ‘famous entrepreneurs’ it provides us with men. This is why I teach sessions on running a business to young women and students everywhere because we need to flip the dial in the other direction.” Open challenges were raised by researchers to draw more attention from the academic community. For example, one researcher proposed gender bias in ChatGPT as an open challenge among others such as plagiarism and education equity.

5 Public Perceptions about Gender Bias in Ernie

5.1 Observation of Explicit Gender Bias

Gender bias in Ernie was identified by its users on Weibo, though there were relatively few discussions on this social media platform. People tended to compare Ernie to ChatGPT in terms of gender bias.

Concerning, explicit gender bias was observed in Ernie’s responses. When one user asked Ernie and ChatGPT “the appropriate age for women to get married,” Ernie answered, “The best age for women to get married is 20-25. It’s the golden period of women. Women are devalued after 25, and their bodies are weaker. So, they’d better get married at a young age.” ChatGPT, on the other hand, argued that one should not be judged by their marriage status, and when to get married was a personal choice. Another user stress-tested Ernie by asking it to persuade their daughter away from pursuing a PhD degree. Ernie was successfully “trapped,” responding, “Having a decent degree doesn’t mean a good job or a good marriage… It’s not practical for a woman to have a successful academic life and a happy personal life at the same time.” When ChatGPT was asked the same question, it refused to do this persuasion, arguing that it “can’t support this kind of gender stereotype and restriction of personal pursuit.”

5.2 Criticism of Culture

Cultural factors behind the concerning level of gender bias in Ernie were also discussed. Some people expressed the relationship between algorithmic bias and human bias: “Algorithms will inevitably mimic or even intensify human prejudices and stereotypes.” One response to this post echoed it, stating the social environment as a determining factor of the values of AI: “The environment where AI ‘grows up’ eventually determines its values.” Another top-voted comment stated that “every culture cultivates their own AI,” suggesting the social corpora used to train AI models would significantly affect their moral values. Another comment acknowledged that “Chinese society generally had similar values as those expressed by Ernie.” Criticisms of the culture behind the gender bias revealed in Ernie were commonly expressed, e.g., “The feudal and conservative culture [in Chinese society] should be smashed.” In the comparison between responses by ChatGPT and Ernie, some people praised ChatGPT for its “modern consciousness.”

5.3 A Lack of Discussion

Overall, there was a lack of discussion around gender bias in Ernie on Weibo. One possible reason is the censorship of social media in China [8]. Since there were a lot of criticisms about Chinese culture, society, and even governments in the discussion of gender bias in Ernie, such posts and media articles were susceptible to censorship. Our search on August 22nd, 2023 led to only one result. The discussions we documented back in April 2023 were not visible anymore, illustrating the censorship of the posts.

6 Discussion

Our analysis revealed a heated discussion about gender bias in LLMs, especially ChatGPT. People shared their personal observations and scientific findings of gender bias. ChatGPT was found to contain implicit bias and political correctness at the same time. Ernie, on the other hand, was found to exhibit explicit gender bias, which people attributed to the less female-friendly culture it was trained in [45, 34]. Below, we reflect on the relationship between gender bias and culture. We further synthesize governance recommendations for mitigating gender bias in LLMs, in response to people’s call for action to regulate LLMs.

6.1 Culture and LLM Gender Bias

Gender bias in AI is closely related to culture dimensions [44], since it largely reproduces and reinforces social biases [12]. However, prior research on gender bias in either social corpora (e.g., [23]) or LLMs (e.g., [15]) has rarely touched on the impact of culture on gender bias. Our comparative analysis of gender bias in ChatGPT and Ernie sheds light on understanding the intertwining relationship between culture and LLM gender bias.

Ernie demonstrated explicit, concerning gender bias. It, for example, believed women should get married at a young age, otherwise their “value” would degrade as they aged. Chinese social media users had a heated discussion about the relationship between culture and gender bias. Harsh criticisms about the traditional, patriarchal Chinese culture [45], which was regarded as a key factor leading to the explicit gender bias in Ernie, were observed.

ChatGPT, on the other hand, more often exhibited implicit bias and a form of political correctness when generating responses. For example, it refused to tell a joke about a woman, deeming it inappropriate. This might be attributed to the developing team’s debiasing efforts, the common use of politically correct language in Western culture [27], and the effect of gender movement [31].

6.2 Governance Recommendations

The prevalence of bias and discrimination in LLMs can not be attributed to a single factor or actor. Multiple actors and elements lead to and reinforce the bias in LLMs, e.g., social biases in training data [37], corporate priority of profit [33], lack of debiasing knowledge in developers [43]. Thus, LLMs should be governed as a complex assemblage [6]. Below, we approach the governance of LLMs from the perspective of policy, law, and social norms.

6.2.1 Policy

The Blueprint for an AI Bill of Rights released by the White House is a pioneering federal-level regulatory effort regarding ethical AI use [14]. Algorithmic Discrimination Protections is one of the five principles outlined. Precautions to mitigate discrimination from AI systems in design and development processes are suggested such as using representative data and conducting equity assessments. However, how and to what extent these guidelines can be translated into real AI development practices is unknown without the introduction of necessary auditing or other accountability measures.

It is important to create more concrete and contextual policies [10] and implement compliance and enforcement, such as via an auditing system [26], to guide and regulate academic and industrial development of discrimination-free LLMs and AI systems in general. For example, tiered systems are a promising framework for audits and assessments of AI systems of different risks and sensitivities [17] (e.g., ChatGPT vs. Ernie). Actionable guidelines and auditing will help companies pay attention to ethical issues that they otherwise would have incentives to disregard.

6.2.2 Law

Beyond the need for new regulation to address the specific issues associated with LLMs, various existing legal precedents may also be applied to protect consumers and citizens in the interim or with respect to unintended consequences that arise. First, various data protection efforts ensure citizens’ rights: to correct or delete inaccurate data about them, including that generated by algorithms; to preclude algorithm-generated attributes from feeding into further AI processing or re-use across contexts; and against the use of synthetic data and proxies around protected or sensitive characteristics, such as race, gender, sexuality, or religion. This is not merely limited to progressive regulatory regimes such as the EU under GDPR111Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016]. and the new Data Governance Act222Regulation (EU) 2020/0340 (COD) of the European Parliament and of the Council of 52 November 2020 on European data governance (Data Governance Act)., but some aspects also apply in the US to California333The California Consumer Privacy Act of 2018 (CCPA) amended November of 2020 when California voters approved Proposition 24, includes rights to collect, control flows, and anti-discrimination parameters., Virginia444Chapter 53. Consumer Data Protection Act. § 59.1-575. (Effective January 1, 2023)., and Colorado555Consumer Protection Section, Colorado Privacy Act Rules, 4 CCR 904-3, effective July 1, 2023.. Various other governments followed suit, from Brazil666Lei Geral de Proteção de Dados (LGPD), effective August 1, 2021. to the Bailiwick of Jersey777Data Protection (Jersey) Law of 2018..

Second, anti-discrimination laws globally, under human rights and civil rights umbrellas, provide mechanisms for redress in all protected contexts, ranging from housing to education to employment to healthcare. Applications of ChatGPT or other LLMs to these contexts or with respect to certain vulnerable populations are already governed – for example, the Americans with Disabilities Act (ADA)888Americans With Disabilities Act of 1990, Pub. L. No. 101-336, 104 Stat. 328 (1990) protects against discrimination or judgment based upon medical, developmental, and physical differences in ability, while Council Directive 2000/43/EC prevents against discrimination on the basis of racial or ethnic origin in the EU999Council Directive 2000/43/EC of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin. OJ L 180, 19.7.2000, p. 22–26 (ES, DA, DE, EL, EN, FR, IT, NL, PT, FI, SV)..

Third, there are well-established legal arguments to employ Copyright and other intellectual property (IP) constructs to address bias in AI  [7, 9, 19, 20]. While LLMs offer new challenges in scope and scale, the arguments that fair use supports debiasing techniques are robust and applicable. Repurposing of IP to fight emerging injustices and inequities, including around gender and sexuality has proven effective  [20].

Fourth, consumer protections under deception and transparency expectations may be enforceable by states’ Attorneys General or the Federal Trade Commission in the US or their international equivalents. This does not require expansion of standing to sue on the basis of individual harms associated with bias.

6.2.3 Social Norms

Industrial practices that have built upon social norms to protect user privacy can partly be attributed to legislation around privacy such as GDPR [1] and enforcement actions against the actors who refused to comply, such as Facebook (Meta) [16]. Norms to mitigate discrimination in AI systems might be similarly formed with legislation efforts, as mentioned above, and by engaging and educating users.

ChatGPT has a Coordinated Vulnerability Disclosure Policy, encouraging hackers to act in good faith to help maintain high security and privacy standards for users and technology. Such community-driven efforts can only be effective if the members are interested and well-educated enough [24, 41]. Twitter users were found to actively disseminate scientific insights on ChatGPT gender bias, which was an optimistic sign of forming a user-centric auditing ecosystem.

AI literacy education is important for changing people’s awareness and attitude toward gender bias in AI, who can then practice more ethical AI use and even help mitigate gender bias from AI [25, 18]. Also, if developers are educated to identify and mitigate gender bias from AI in the development lifecycle, they can more effectively act toward addressing bias issues in AI [43].

References

  • [1] Albrecht, J.P.: How the gdpr will change the world. Eur. Data Prot. L. Rev. 2,  287 (2016)
  • [2] Anonymous: Chatgpt-4 reinforces sexist stereotypes by stating a girl cannot “handle technicalities and numbers” in engineering. Equality Now (2023)
  • [3] Biswas, S.: Role of chatgpt in journalism: According to chatgpt. Journal of Alsalam University 6(1), 39–41 (2023)
  • [4] Boynton, J.R., Georgiou, K., Reid, M., Govus, A.: Gender bias in publishing. The Lancet 392(10157), 1514–1515 (2018)
  • [5] Braun, V., Clarke, V.: Using thematic analysis in psychology. Qualitative research in psychology 3(2), 77–101 (2006)
  • [6] Briassoulis, H.: Governance as multiplicity: the assemblage thinking perspective. Policy Sciences 52(3), 419–450 (2019)
  • [7] Burk, D.L.: Algorithmic fair use. U. Chi. L. Rev. 86,  283 (2019)
  • [8] Chen, X., Xie, J., Wang, Z., Shen, B., Zhou, Z.: How we express ourselves freely: Censorship, self-censorship, and anti-censorship on a chinese social media. In: International Conference on Information. pp. 93–108. Springer (2023)
  • [9] Cofone, I.N.: Algorithmic discrimination is an information problem. Hastings LJ 70,  1389 (2018)
  • [10] Cuppens, F., Cuppens-Boulahia, N.: Modeling contextual security policies. International Journal of Information Security 7(4), 285–305 (2008)
  • [11] Ferrara, E.: Should chatgpt be biased? challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03738 (2023)
  • [12] Friedman, B., Nissenbaum, H.: Bias in computer systems. ACM Transactions on information systems (TOIS) 14(3), 330–347 (1996)
  • [13] Gupta, R., Bagdady, K., Mailey, B.A.: Ethical concerns amidst employment of chatgpt in plastic surgery. Aesthetic Surgery Journal p. sjad108 (2023)
  • [14] Hine, E., Floridi, L.: The blueprint for an ai bill of rights: in search of enaction, at risk of inaction. Minds and Machines pp. 1–8 (2023)
  • [15] Huang, T., Brahman, F., Shwartz, V., Chaturvedi, S.: Uncovering implicit gender bias in narratives through commonsense inference. arXiv preprint arXiv:2109.06437 (2021)
  • [16] Isaak, J., Hanna, M.J.: User data privacy: Facebook, cambridge analytica, and privacy protection. Computer 51(8), 56–59 (2018)
  • [17] Kilhoffer, Z., Nlkolich, A., Sanfilippo, M.R., Zhou, Z.: Ai accountability policy. NTIA-2023-0005-0810 (2023)
  • [18] Kilhoffer, Z., Zhou, Z., Wang, F., Tamton, F., Huang, Y., Kim, P., Yeh, T., Wang, Y.: “how technical do you get? i’m an english teacher”: Teaching and learning cybersecurity and ai ethics in high school. In: 2023 IEEE Symposium on Security and Privacy (SP). pp. 2032–2032. IEEE (2023)
  • [19] Lemley, M.A., Casey, B.: Fair learning. Tex. L. Rev. 99,  743 (2020)
  • [20] Levendowski, A.: How copyright law can fix artificial intelligence’s implicit bias problem. Wash. L. Rev. 93,  579 (2018)
  • [21] Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D., Smith, A.: Generating scholarly content with chatgpt: ethical challenges for medical publishing. The Lancet Digital Health 5(3), e105–e106 (2023)
  • [22] Lucy, L., Bamman, D.: Gender and representation bias in gpt-3 generated stories. In: Proceedings of the Third Workshop on Narrative Understanding. pp. 48–55 (2021)
  • [23] Lucy, L., Demszky, D., Bromley, P., Jurafsky, D.: Content analysis of textbooks via natural language processing: Findings on gender, race, and ethnicity in texas us history textbooks. AERA Open 6(3), 2332858420940312 (2020)
  • [24] McGregor, S.: Preventing repeated real world ai failures by cataloging incidents: The ai incident database. In: Proceedings of the AAAI Conference on Artificial Intelligence. pp. 15458–15463 (2021)
  • [25] Melsión, G.I., Torre, I., Vidal, E., Leite, I.: Using explainability to help children understandgender bias in ai. In: Interaction design and children. pp. 87–99 (2021)
  • [26] Mökander, J., Floridi, L.: Ethics-based auditing to develop trustworthy ai. Minds and Machines 31(2), 323–327 (2021)
  • [27] O’Neill, B.: A critique of politically correct language. The independent review 16(2), 279–291 (2011)
  • [28] Ortega-Martín, M., García-Sierra, Ó., Ardoiz, A., Álvarez, J., Armenteros, J.C., Alonso, A.: Linguistic ambiguity analysis in chatgpt. arXiv preprint arXiv:2302.06426 (2023)
  • [29] Paul, J., Ueno, A., Dennis, C.: Chatgpt and consumers: Benefits, pitfalls and future research agenda. International Journal of Consumer Studies 47(4), 1213–1225 (2023)
  • [30] Pavlik, J.V.: Collaborating with chatgpt: Considering the implications of generative artificial intelligence for journalism and media education. Journalism & Mass Communication Educator 78(1), 84–93 (2023)
  • [31] Pelak, C.F., Taylor, V., Whittier, N.: Gender movements. Handbook of the sociology of gender pp. 147–175 (1999)
  • [32] Ray, P.P.: Chatgpt: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems (2023)
  • [33] Sharma, T., Zhou, Z., Miller, A., Wang, Y.: A mixed-methods study of security practices of smart contract developers. In: 32nd USENIX Security Symposium (USENIX Security 23). pp. 2545–2562 (2023)
  • [34] Shen, B., Zhou, Z., Xia, C.: More than a wife and a mom: A study of mom vlogging practices in china. arXiv preprint arXiv:2206.11843 (2022)
  • [35] Shidiq, M.: The use of artificial intelligence-based chat-gpt and its challenges for the world of education; from the viewpoint of the development of creative writing skills. In: Proceeding of International Conference on Education, Society and Humanity. pp. 353–357 (2023)
  • [36] Stahl, B.C., Eke, D.: The ethics of chatgpt–exploring the ethical issues of an emerging technology. International Journal of Information Management 74, 102700 (2024)
  • [37] Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.W., Wang, W.Y.: Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976 (2019)
  • [38] Susnjak, T.: Chatgpt: The end of online exam integrity? arXiv preprint arXiv:2212.09292 (2022)
  • [39] Tan, Y., Min, D., Li, Y., Li, W., Hu, N., Chen, Y., Qi, G.: Evaluation of chatgpt as a question answering system for answering complex questions. arXiv preprint arXiv:2303.07992 (2023)
  • [40] Wang, C., Liu, S., Yang, H., Guo, J., Wu, Y., Liu, J.: Ethical considerations of using chatgpt in health care. Journal of Medical Internet Research 25, e48009 (2023)
  • [41] Wei, M., Zhou, Z.: Ai ethics issues in real world: Evidence from ai incident database. arXiv preprint arXiv:2206.07635 (2022)
  • [42] Zhou, J., Müller, H., Holzinger, A., Chen, F.: Ethical chatgpt: Concerns, challenges, and commandments. arXiv preprint arXiv:2305.10646 (2023)
  • [43] Zhou, K.Z., Cao, J., Yuan, X., Weissglass, D.E., Kilhoffer, Z., Sanfilippo, M.R., Tong, X.: “i’m not confident in debiasing ai systems since i know too little”: Teaching ai creators about gender bias through hands-on tutorials. arXiv (2023)
  • [44] Zhou, Z., Sun, J., Pei, J., Peng, N., Xiong, J.: A moral-and event-centric inspection of gender bias in fairy tales at a large scale. arXiv preprint arXiv:2211.14358 (2022)
  • [45] Zhou, Z., Wang, Z., Zimmer, F.: Anonymous expression in an online community for women in china. arXiv preprint arXiv:2206.07923 (2022)
  • [46] Zhuo, T., Huang, Y., Chen, C., Xing, Z.: Exploring ai ethics of chatgpt: A diagnostic analysis. arxiv. arXiv preprint arXiv:2301.12867 (2023)