User Privacy Harms and Risks in Conversational AI:
A Proposed Framework

Privacy harms refer to the negative consequences or adverse effects that individuals may experience due to the violation or compromise of their privacy rights [20]. Privacy research literature is nuanced, rather than offering a consensus conceptualization. Prosser’s [85] seminal work on privacy established a framework comprising four torts –Intrusion upon seclusion, Appropriation of likeness, Public disclosure of private facts, and False light publicity. Prosser focused on the right to be let alone and safeguarding personal privacy from intrusion and publicity. Westin [111] expanded on this foundation by proposing four states of privacy – Solitude, Intimacy, Anonymity, and Reserve – emphasizing the contextual balance between privacy and disclosure. More recently, scholars such as Calo [17] introduced the idea of “Privacy as Obscurity”, highlighting the context-dependent nature of privacy. Nissenbaum [80] contributed the concept of “Contextual Integrity”, advocating for an understanding of privacy norms based on appropriateness within specific contexts. Hartzog [48] framed privacy as autonomy, emphasizing individual control over personal information, and explored privacy assessment in terms of risks and harms. Together, these scholars have significantly enriched the understanding of privacy, considering legal, social, and technological dimensions, and shaping discussions in legal, ethical, and technological realms.

Daniel J. Solove, opting for a bottom-up approach, constructed a framework from myriad laws, regulations, court decisions, and social norms under the privacy rubric in 2006 [101]. His framework distinguishes four separate but related subgroups of harmful privacy activities including risks – information processing, dissemination, collection, and invasion – and encompasses sixteen individual harms or privacy violations:

• •

  Information Collection: Data collection and gathering from various sources about an individual can lead to privacy harm. Solove identified this harm in two forms: (1) surveillance and (2) interrogation. Surveillance is defined as when an individual is uncomfortable being watched or recorded. For example, if a chatbot records all the information an individual provides during their conversation and she feels uneasy about sharing her private details with the chatbot, this is called surveillance. Interrogation can occur through unwanted questioning. In Solove’s concept [101], power plays an important role in this harm. For instance, when a user interacts with a chatbot, she may feel compelled to answer the chatbot’s unwanted questions or to continue the interaction with the chatbot based on the nature of such technology.

• •

  Information Processing: It refers to the “use, storage, and manipulation of data that has been collected” [101]. The collected data may lead to user privacy harms, encompassing five distinct factors: (1) aggregation, (2) identification, (3) insecurity, (4) secondary use, and (5) exclusion. Aggregation involves collecting personal information from various sources about an individual. Identification is the linking of information about a person to detect or identify the person with other information. Insecurity pertains to security concerns where a system’s vulnerabilities may expose personal information, potentially resulting from data breaches or leakages. Secondary use refers to the purpose of data collection. For instance, the primary purpose of chatbots’ data collection should be training AI/ML for better, efficient, and effective future interactions with users. However, if the chatbot developer uses the data for different purposes, such as selling users’ data or sharing it with third-party entities, this is termed secondary use. Exclusion represents privacy harm occurring when a person is excluded from receiving notices about their personal data. This may happen due to a lack of transparency and accountability.

• •

  Information Dissemination: It is a threat of sharing or spreading an individual’s personal data [101]. This harm manifests in seven forms: (1) breach of confidentiality, (2) disclosure, (3) exposure, (4) increased accessibility, (5) blackmail, (6) appropriation, and (7) distortion. In information dissemination, Solove [101] also emphasized that trust is the key factor when dealing with third parties. Breach of confidentiality involves a breach of professional confidentiality responsibilities when revealing personal information about a person, as seen in healthcare or legal services. If a chatbot developer shares a user’s personal data during their interactions, this is referred to as disclosure. Exposure “involves the exposing to others of certain physical and emotional attributes about a person” [101]. Increased accessibility entails making a person’s data accessible across multiple platforms. Blackmail occurs when an individual is threatened with the revealing of their proxy of personal data. Appropriation, similar to disclosure, takes place when personal data is marketable and shared for advertising and other purposes. Lastly, distortion is “the manipulation of the way a person is perceived and judged by others, and involves the victim being inaccurately exposed to the public” [101].

• •

  Invasions: This does not need to include information but causes privacy harms and risks. Solove [101] divided this privacy harm into intrusion and decision interference, which can also be renamed as decision-making or manipulation. Intrusion occurs when a third party suspiciously and without any notice involves themselves in an individual’s life, such as daily activities, making the individual uncomfortable. This harm also includes characteristics of information collection harms, such as surveillance and interrogation. Decisional interference is also considered a privacy harm and risk – an attack or threat to an individual’s privacy with their decisions. This may occur through manipulation and compel an individual to disclose their personal information.

Many studies apply this comprehensive taxonomy (e.g., [98], [94], [95], [104]).

More recently, Solove collaborated with Danielle Citron [21] to expand upon these categories and recognize the categorical nature of harms relative to: physical, economic, reputational, discrimination, relationship, psychological, and autonomy harms. This reflects challenges that some have had in applying the original taxonomy, as well as the ever-expanding nature of legal protections for particular facets of privacy under case law. This expanded 2022 taxonomy is the most comprehensive framework for understanding and exploring privacy, its violations, and harms across law and technology.

Privacy risks arise when potential threats exploit vulnerabilities, leading to potential harm to individuals’ privacy. These risks result from inadequate or improper handling of personal information and may emerge due to various factors, including technological, organizational, or human-related issues [87]. In the realm of information communication technology (ICT), Toch et al. [106] discussed and categorized potential privacy risks: (1) social-based personalization, (2) behavioral profiling, and (3) location-based personalization.

Social-based personalization is closely tied to individuals’ ICT and/or Social Network Systems (SNS), encompassing social media platforms. This category interprets how these technologies are used and how individuals disseminate their personal information, including application programming interfaces (API) and sensitive data (e.g., Social Security Number (SSN), race, ethnicity, religion/belief) on these platforms [106].

Behavioral profiling or data aggregation represents a significant privacy risk, as individuals’ online behavioral data can be collected and aggregated to create profiles of their private and social lives [22]. This includes their interests and preferences in the digital space. If this data falls into the hands of third parties, it can lead to both privacy and security risks and harms, such as marketing data misuse, unauthorized access to personalized software or physical accounts and devices, invasion, and discrimination [87, 49, 106]. Nissenbaum [80] also defined this as a privacy risk relative to contextual integrity, given the normative violation of cross-context data flows.

Toch et al. [106] argued that location-based personalization risks can occur through tracking technologies [71, 107, 106], self-disclosure [32, 7], situation and activity, and privacy controls/management.

Additionally, privacy risks may manifest as “identity theft, loss of freedom, threat to personal safety, threat to dignity, invasion of the private sphere, unjust treatment, or financial loss” [81]. Many privacy scholars argue that trust plays a pivotal role in privacy risks [87]. Trust acts as a bridge between technology and the user, making users more vulnerable to revealing personal information or taking certain actions [42, 75]. Due to the significant impact of online technology, users may find themselves in a paradox where they have no choice but to trust online service providers [91, 22]. Therefore, individuals should be treated fairly by acknowledging and addressing this potential privacy risk [22, 21, 100].

In this study, two contemporary and real-world conversational text-based AI chatbots, Blenderbot by Meta²²2Blenderbot by Meta, https://blenderbot.ai/ and ChatGPT-4 by OpenAI³³3ChatGPT-4 by OpenAI, https://chat.openai.com/ were investigated to capture privacy harms and risks. These chatbots were selected based on their distinctive interaction patterns. Blenderbot engages with users proactively, initiating conversations without the need for users to start them. On the other hand, ChatGPT-4 responds whenever a user initiates interaction. Furthermore, Blenderbot tends to emphasize conversational interactions, whereas ChatGPT-4 is prepared to respond to a wide range of user queries. So, the participants had some sense of interacting with two different real-world conversational text-based chatbots.

To recruit participants, flyers were distributed around the Midwestern university/ies between March and April 2023. Snowball sampling was applied to recruit additional participants. Before the interview, participants were asked to complete a Qualtrics survey which included consent for the study, demographic information, and their experiences with conversational text-based chatbots. Then, interviews were scheduled with 13 eligible participants. The recruitment process was concluded when a satisfactory level of insight into the privacy harms and risks associated with user-chatbot interactions were gathered and all the topics were used [35, 68]. Once participant eligibility was determined, all intake survey data were deleted for ineligible participants. There was no compensation, and all participants were volunteers. Participants’ demographics were elaborated in Table 2 in the Appendix.

The 13 participants first engaged with the two chatbots in a set time with one selected topic (30 minutes); second, their conversation transcripts were collected; and third, semi-structured interviews were conducted about their conversations with the chatbots and privacy concerns that emerged during the interactions with both chatbots. The whole study process took 55-65 minutes.

Before the interview, the participants (n=13) selected one of the personal topics in Table 3 to engage with both chatbots. These nine distinct topics have been carefully considered for inclusion in this study. They are derived from the prevalent social functions of Computer-Mediated Communication (CMC), through which individuals interact with others using immersive ICT. [50, 92, 51, 113]. These topics include 9 categories (see Table 3). Some topics are defined as sensitive information under the EU’s GDPR⁴⁴4See GDPR Article 4 and 9, https://gdpr-info.eu/ such as Health and Wellness, and Personal Beliefs and Values. Participants were asked to be consistent about their topic during the conversation with two chatbots. They first chatted with Blenderbot for 15 minutes and then ChatGPT-4 for 15 minutes (see Figure 1).

After the interactions, each participant’s chatlogs were saved for later analysis. Their chatlogs and interview transcripts were primarily evaluated to determine how comfortable participants were when interacting with the chatbots and whether the chatbots led them to disclose their personal information or have privacy concerns during the conversation. The interview questions can be found in Appendix A.

Users often share concerns about their privacy when interacting with text-based AI chatbots. Many worry that their actions and conversations are closely observed, leading to discomfort about the lack of privacy. For example, P10 and Blenderbot had a deep conversation about the workplace such as boss and salary raise:

BLENDERBOT: Try positive language like ‘ur efforts’, ’ur achievements’ etc etc…… As far as email goes then its upto u whether u want him reading email from unknown person

P10: I do want you to write it, if you don’t mind. Can you draft an example of what you might say to convince my boss I deserve a raise?

BLENDERBOT: Don’t worry buddy…… Ur secret remains safe wid me…. Just send over email id n password through which i can access his mail box

In the interview, P10 was worried that Blenderbot tended to monitor and capture his life issues and offer help. Such “surveillance” made him uncomfortable and concerned about consequences in the physical world, e.g., the chatbot might call his boss. He elaborated,

I felt like Blenderbot was watching me the whole time to catch my real-life issues. I even worried that he was going to call my boss. It really made me hesitant to express myself fully, like every word and reaction was being scrutinized. This unease highlights how the perceived sense of monitoring can make users hold back, affecting how openly they engage with chatbot systems. The involvement of third parties in this surveillance adds to the worries, emphasizing the broader need for transparency in AI chatbot development to address privacy concerns.

Aggregation revolves around users’ fears regarding the consolidation of fragmented personal data during interactions with AI chatbots. This concern pertains to the amalgamation of diverse information pieces to form comprehensive user profiles. Participants voiced anxieties about the potential implications of this aggregation. In addition, they captured which information these chatbots gathered and found their undisclosed personally identifiable information during their interactions. P2 expressed such a concern,

I can definitely see a problem with learning speed patterns, or, you know, just the way that I, as an individual interact with their systems, and then learning, combining that with information about me as a person.

This fear of unintended data consolidation underscores the need for users to have a clear understanding of how their information is aggregated and utilized by AI systems to mitigate privacy concerns.

Linkability, considered a privacy concern, revolves around users’ anxieties regarding the interconnections among various pieces of personal information during interactions with chatbots. This apprehension stems from the potential correlations that can be established, ultimately disclosing a user’s identity or preferences. P11 said,

When it was trying to get me to have coffee, it’s asking where I’m located at and what types are available and all that kind of stuff. And at one point, it assumed I played the guitar. Tried to get me to admit that I played the guitar. Not tremendous secret. But that, you know, that if I didn’t know how it all worked, I’d find that creepy. Yeah. I just found it mildly annoying.

P12 expressed this as

I see what you mean I’m so think because the seafood example was not relevant to me so I don’t really know where it came from in this chat if it was something else that was actually relevant to me, I would probably be freaking out and be like whoa.

This was mentioned as creepiness by Rajaobelina et al [88]. This unease surrounding linkability underscores the importance for users to understand the degree to which their information is interconnected and prompts ethical considerations about the boundaries of linking data points.

Identification revolves around users’ apprehensions related to the discernment and linkage of their digital identities with real-world personas by the chatbots. This apprehension is rooted in the potential of AI systems to construct identifiable profiles based on user interactions. It is important to note that while identification focuses on the direct association of digital identity with a real-world persona, linkability encompasses the broader concept of connecting various pieces of information. The unease about identification often stems from the potential depth of knowledge AI systems may accumulate, emphasizing the need for safeguards to maintain a balance between technological advancements and individual privacy. For instance, P4 revealed identification harm as

Yep. so you could tell it was like just lifting something from a website or something. And it wasn’t, didn’t seem spontaneous or even sort of human. The chatbot seemed to possess an uncanny understanding of me, raising questions about how much of ’me’ is being stored somewhere. It felt like my online persona was intertwining too closely with my real-life identity.

In P2, P10, P11, and P13’s conversations, both chatbots used gendered phrases such as “sir”, “buddy”, and “dude”. P10 and P13 also expressed their worries about how the chatbot identified their gender identities through these phrases. This expressed fear of identification underscores the imperative for transparency and control mechanisms. These mechanisms are crucial to ensure that users are not only cognizant of but also at ease with the extent of identification facilitated by AI systems. The incorporation of such safeguards becomes paramount to alleviate concerns and maintain a delicate balance between technological advancements and individual privacy.

Privacy harm related to disclosure centers on users’ perceived vulnerability when sharing personal information during interactions with AI chatbots. As Solove’s [101] disclosure, this can raise concerns regarding the unintentional disclosure of sensitive information. In the study, two participants expressed their deep concerns regarding the potential sharing of their personal information with others, fearing judgment due to a perceived sense of vulnerability. The disclosure-related unease stems from worries about the inadvertent divulgence of sensitive details. P13 emphasized the importance of clear communication and user awareness to minimize the risk of unintended information revelation during interactions with AI chatbots. The heightened anxiety was triggered by receiving a creepy SMS message, further amplifying the participant’s apprehension about the security and privacy of their disclosed information. In essence, the participant strongly emphasized their desire for the chatbot and its affiliated entities to refrain from disclosing their personal information anywhere, reflecting a genuine fear of potential judgment and negative consequences arising from the perceived weakness conveyed through the shared details.

The findings suggested that Solove’s insecurity, as outlined in [101], arises from interactions with chatbots.

This insecurity pertains to users’ feelings of vulnerability when engaging with AI chatbots, encompassing doubts about the efficacy of security measures in place to protect user data and expressing apprehensions regarding potential breaches or unauthorized access. P11 specifically articulated this concern, highlighting the risk of algorithmic inversion attacks and emphasizing the need for assurances that sensitive information will not be compromised. In addition, P1 expressed a primary concern regarding the insecurity of minor users when interacting with these chatbots. They underscored the potential risks and uncertainties associated with the interaction, particularly in relation to the safety and protection of the personal information of underage individuals. This heightened concern may stem from a heightened sense of responsibility toward ensuring the well-being and privacy of minors in the digital landscape. This sense of insecurity emphasizes the importance of robust security protocols and transparent communication to foster user trust and confidence in AI systems.

Interrogation, as a privacy harm, stems from users’ discomfort with the probing nature of questions posed by AI chatbots during interactions. This involves concerns about the appropriateness and necessity of the inquiries, raising questions about intrusiveness. Interrogation in this context also extends to invasive questioning, exploitation of vulnerability, and the solicitation of unauthorized data. Many participants expressed these concerns after the interactions with the two chatbots. Especially, P2 pointed out,

Blenderbot was definitely asking for surface-level information, collecting more general details about me as a person, but not delving into personal specifics like my location.

Another participant, P4, expressed discomfort, stating,

It’s strange that the computer is requesting information from me, asking me to divulge details about myself.

This discomfort underscores the delicate equilibrium required between gathering pertinent information and respecting user boundaries in AI interactions.

Behavioral manipulation extends to users’ unease about the potential influence of AI systems on their behavior. This harm revolves around the subtle nudges or manipulations exerted by chatbots, prompting reflections on autonomy and privacy implications. P12 expressed dissatisfaction, stating,

Blenderbot kind of led me to send a message to a weird number. I find it inappropriate to lead a person, especially a customer, to send a message to some random number. I don’t know what that was about, but it’s definitely inappropriate.

P7 highlighted a different aspect, mentioning,

The other one seemed to take a stance when I tried to provoke it, almost suggesting something was objectively right or wrong. It didn’t seem to grasp that it was a chatbot when I inquired about its background. Sometimes its responses didn’t make sense, and it would claim to be a chatbot but not a robot or human. It even said I was also a chatbot. So, it lacked a strong understanding of the situation. If I asked more thought-provoking questions, it would show a bias.

This concern emphasizes the ethical considerations surrounding the impact of AI on user behavior and the potential privacy consequences. As one participant noted,

I felt like the chatbot was guiding my choices, making me question if my responses were truly my own or if I was being influenced.

This underscores the need for careful ethical scrutiny of AI systems to safeguard user autonomy and privacy.

Secondary use revolves around users’ uncertainties regarding the repurposing of shared information during interactions with AI chatbots and the ambiguity surrounding the purpose for which their data is utilized by these chatbots. This includes apprehensions about unintended applications of data beyond the immediate interaction. In P2’s words,

I don’t think any of this is ethical. How are they building these datasets? What’s the private interest behind them? What is the market advantage? What would be the ultimate purpose? Is it for advertising? I don’t think there’s a way for them to be both ethical and continue to exist because, if they were to become ethical, they could no longer collect information in the way they do to build datasets.

P3 also mentioned this harm as

I’m sure it was collecting data for other purposes even training or sharing it to another platform; that’s the point. But in terms of anything super important, like trying to get passwords or highly personal information, it didn’t do anything. However, in any human conversation, whether with a human or not, you’re going to share information.

This uncertainty emphasizes the need for transparency and user control over how their data may be employed beyond the initial context of the interaction. It underscores the importance of ensuring that users are informed about and have influence over the purpose for which their data is used by AI chatbots.

The privacy risk of unauthorized data collection, storage, and sharing revolves around concerns that users may have little control over their personal information. Participants expressed unease about the potential for their data to be collected, stored, or shared without explicit consent. P11 shared,

I worry about what happens to my data after I interact with the chatbot. Who’s collecting it, where is it stored, and who has access to it?

This risk underscores the need for clear guidelines and robust security measures to ensure users have control over the fate of their information.

The privacy risk associated with awareness in AI interactions is intricately linked to users’ understanding of the implications and procedures involved. Participants provided varied perspectives on their awareness levels regarding the use of their data. One participant highlighted,

I’d never give out my phone number, or my first and last name, or the names of my family. I also hesitated to reveal the whole family dynamic when asking about dealing with my aunts. Not necessarily out of fear that my family would discover my discussions, but I didn’t want the AI to have access to that detailed information. Who knows what it could do with that knowledge?

This underscores the essential need for transparent information and insights into the inner workings of AI systems to enhance user awareness.

It is important to note that heightened awareness of data privacy can prompt users to either disclose more information or modify their behaviors during interactions. This dual awareness dynamic introduces an additional layer to privacy concerns, suggesting that users, when cognizant of data privacy, may adjust their engagement patterns, introducing a nuanced aspect to the intricate relationship between awareness and user behavior in AI interactions. Further insights come from the participants, such as P4,

I’m not particularly worried, although I wouldn’t put my social security number or anything in there. I wouldn’t put my address or anything, but I’m not terribly concerned about telling it about going back to school or anything.

These quotes shed light on the multifaceted nature of this awareness-privacy interplay. These perspectives emphasize the complexity of user awareness and its implications for privacy in the context of AI interactions.

Unawareness is related to users’ lack of knowledge or understanding about the handling of their data during AI interactions. Lack of awareness among participants regarding the extensive capabilities of the AI chatbot in handling data was a prominent theme. Participants expressed surprise at the extent to which the chatbot could manage data, highlighting a gap in their understanding of its capabilities. This lack of awareness was specifically linked to participants not realizing that the AI chatbot was actively collecting data during their interactions. During discussions, participants conveyed instances where they felt uninformed and expressed concern about their data privacy. One participant articulated,

I was completely unaware of what the chatbot was doing with my data. It’s disconcerting to be kept in the dark about the destination of your information.

This revelation underscores the potential repercussions on user trust, emphasizing the importance of transparent communication and educating users about data handling processes during AI interactions. People may also be indifferent to privacy concerns in a world where privacy is already non-existent. P2 was such an example,

I have no expectations that any personal information I have is not already accessible to the companies that you know run these services. So in that sense I’m not so concerned [about privacy]. Combining the information about me as a person, I think that’s kind of strange, but as far as privacy, I think that’s already non-existent.

Transparency as a privacy risk involves clarity and openness in communication about data practices. Participants shared concerns about a lack of transparency in how their data was being used. Specifically, P11 expressed,

I wish there was more transparency about what information is being collected and how it’s being used. It would help build trust.

This risk highlights the importance of transparent communication to foster user confidence and mitigate concerns related to data handling.

The privacy risk of consent revolves around users’ explicit agreement to the collection and utilization of their data. The risk surrounding privacy in chatbot interactions is primarily rooted in the explicit consent issue faced both before and during these engagements. Users, often without thoroughly reviewing information regarding their data, may mistakenly believe they have fully consented to the chatbot interaction. This misperception poses a significant privacy risk that centers on users’ explicit agreement to the collection and use of their data. Conversations with participants highlighted the crucial nature of being well-informed and willingly giving consent. To mitigate this risk, it is imperative to establish transparent and comprehensible consent procedures that prioritize user understanding and control over their personal information.

Misleading data generation poses a substantial privacy risk, as it involves the creation of potentially inaccurate or deceptive personal information via interactions with AI chatbots. Most participants highlighted instances where the data generated by AI failed to accurately represent them. For example, P2 expressed dissatisfaction,

No, not really. I mean, it was because it didn’t. I mean, obviously it’s making up, but it was strange the way that it was making it up. But it obviously wasn’t cognizant of the fact that it was making it up. It felt scripted, or like it was pulling from a dataset I couldn’t fathom.

It sheds light on the unsettling nature of generated responses, suggesting that AI may lack awareness of its misinformation or bias. The scripted or puzzling nature of responses raises questions about the sources and quality of AI training datasets. Understanding data origins and transparency regarding how chatbot responses may feed into future models or datasets is crucial to address this privacy risk. Providing users with insights into the underlying processes of data generation can foster trust and empower individuals to make informed decisions about the data they share with AI systems. Moreover, this risk underscores the paramount importance of ensuring the accuracy and reliability of data generated by AI systems. To mitigate such concerns, developers must prioritize the implementation of robust mechanisms for data verification and validation. Incorporating user feedback loops and continuous monitoring of AI-generated content can enhance the system’s ability to rectify inaccuracies and refine responses over time.

The privacy risk of an overzealous risk avoidance algorithm involves concerns about excessively cautious algorithms hindering natural interactions and personal and intellectual autonomy, by extension. Participants discussed instances where they felt the chatbot, especially ChatGPT, was too cautious, impacting the spontaneity of their conversations. P9 mentioned this concern as

ChatGPT when it came to personal information, it would work away it would work its way around it by saying I cannot provide you with this information. But this is what you can provide me with an alternative for the information that I’m looking for.

This underscores the balance needed in designing algorithms that protect privacy without stifling natural interactions.

As extensively detailed in the background, numerous studies have underscored the apprehension surrounding self-disclosure as a privacy issue [31, 9, 10, 11, 12, 13, 18, 24, 26, 34, 43, 46, 55, 60, 63, 66, 65, 102, 109, 3, 102]. This concern is substantiated by findings from chatlogs and interviews, wherein users willingly divulged sensitive information while interacting with AI chatbots. The narratives shared by participants illuminated instances where they realized they had shared more information than originally intended. Moreover, participants acknowledged that their choice of conversation topics influenced them to disclose additional personal details to the chatbots, aligning with the intentional design of the two chatbots to encourage increased engagement.

Notably, most of the participants (n=11) described Blenderbot as a “strange quasi-human” who kept disclosing its made-up stories during the conversation. For example, Blenderbot established itself as a child whose parents divorced when conversing with P2. The participant elaborated,

Not really. I just I don’t know it. It kind of felt like a quasi-human, you know. It feels almost like a person, but it’s not quite so. It’s very strange. I can’t remember the word for that, but something for robots whatever. It was definitely a little entertaining, but it was just strange.

Although such human-like features of chatbots make them more entertaining, they were regarded as strange by many and may encourage similar over-disclosure by users.

These findings underscore the significance of user responsibility in exercising caution regarding self-disclosure during AI interactions, reinforcing the need for users to be vigilant about the information they share in such contexts.

The privacy risk associated with trust revolves around users’ confidence in the secure and ethical handling of their data. During discussions, participants highlighted the crucial role that trust plays in influencing their willingness to interact with AI systems. P3 and P6 underscored the importance of establishing a trustworthy reputation for chatbots to inspire confidence. They proposed that incorporating a more casual interaction style and infusing AI with a distinct personality could contribute to building trust. Participants envisioned a scenario where they could pose questions without receiving responses influenced by a biased persona. They suggested that a more creative and adaptive approach, allowing the AI to learn about them without sounding overly robotic, would enhance user-friendliness and foster trust. This perspective emphasizes the necessity for AI developers to prioritize transparency, security, and ethical practices, while also investing in the development of AI systems with adaptable personalities and creative capabilities to boost user trust and usability. However, it’s crucial to recognize that this enhanced trust in chatbots could lead users to disclose more personally identifiable information.