Virtual Co-presenter: Connecting Deaf and Hard-of-hearing Livestreamers and Hearing audience in E-commerce Livestreaming
Abstract.
Deaf and Hard-of-Hearing (DHH) individuals are increasingly participating as livestreamers in China’s e-commerce livestreaming industry but face obstacles that limit the scope and diversity of their audience. Our paper examines these challenges and explores a potential solution for connecting the hearing audience to sign language (SL) livestreaming teams with DHH members in e-commerce livestreaming. We interviewed four SL livestreaming team members and 15 hearing audience members to identify information and emotional communication challenges that discourage the hearing audience from continuing to watch SL livestreaming. Based on these findings, we developed a virtual co-presenter demo, which targets SL livestreaming teams with DHH members as users, through a design workshop with six designers, incorporating voice broadcasting with animations. Follow-up evaluations with previous participants provided positive feedback on the virtual co-presenter’s potential to address these challenges. We summarize design suggestions on its functionality and interaction design for further refinement to assist SL livestreaming teams with DHH members in reaching a broader hearing audience.
1. Introduction
Livestreaming has become a dominant form of social media entertainment worldwide, creating employment opportunities across various sectors(Tang et al., 2022; Lu et al., 2019), including for individuals with disabilities(Johnson, 2019; Jun et al., 2021; Rong et al., 2022; Anderson and Johnson, 2022). In e-commerce livestreaming, which relies heavily on verbal communication and real-time interaction(Zhang et al., 2023), Deaf and Hard of Hearing (DHH) livestreamers face unique challenges(Cao et al., 2023). They primarily communicate through Sign Language (SL)(yan, 2021), which inherently limits their audience to the DHH community, reducing the hearing audience reach and earning potential of DHH livestreamers.
Prior research has explored SL conversion technologies to facilitate communication between DHH and hearing individuals(Ahmed et al., 2021, 2018; Camgoz et al., 2018, 2020). However, e-commerce livestreaming requires not only accurate information transmission but also engaging presentation skills, which SL conversion technologies struggle to provide. Recently, virtual livestreamers have gained popularity among younger audiences due to their enhanced expressiveness (Miao et al., 2022; Liew and Tan, 2018; Gao et al., 2023), yet little research has examined how virtual human technology could support DHH livestreamers. Inspired by this, we explore the potential of virtual human technology in assisting DHH livestreamers in SL e-commerce livestreaming.
To comprehensively examine this issue, our research adopts a bilateral perspective, investigating communication barriers in SL e-commerce livestreaming from both the perspectives of SL livestreaming teams with DHH members and hearing audience. Specifically, we address two key research questions: (1) What are the primary challenges SL livestreaming teams with DHH members face in connecting with hearing audience? (2) What factors deter hearing audience from sustained engagement in SL livestreaming? Through semi-structured interviews with four SL livestreaming team members and 15 hearing audiences, followed by inductive coding and thematic analysis, we identified communication gaps in information conveyance and emotional resonance between these two groups, informing our assistive system design.
Building on these findings, we conducted a professional design workshop with six designers to explore potential solutions. Through collaborative efforts, we developed a virtual co-presenter system which targets SL livestreaming teams with DHH members as users and is designed to bridge communication gaps in livestreaming to hearing audiences. To evaluate the design, we conducted follow-up assessments with both SL livestreaming team members and hearing audience, gathering feedback on system functionality and interaction design. These evaluations provided insights into the system’s practical applications and future development directions in real livestreaming scenarios.
Our research offers two main contributions: (1) An in-depth qualitative investigation of the challenges faced by SL livestreaming teams with DHH members and hearing audience, examining their perspectives on communication barriers in e-commerce livestreaming; (2) The design and preliminary evaluation of a virtual co-presenter demo aimed at supporting SL livestreaming teams with DHH members in reaching a broader hearing audience.
2. Sign language E-commerce livestreaming
Sign language e-commerce livestreaming is a specialized form of e-commerce livestreaming where DHH livestreamers promote products or services using SL and textual prompts while interacting with audience in real time(yan, 2021; Cao et al., 2023). A typical SL livestreaming team, primarily composed of DHH individuals, consists of three key roles: livestreamer, coordinator, and assistant (Fig. 1a). Each team includes at least one DHH livestreamer, while others may be DHH or hearing individuals proficient in SL.
The preparation and livestreaming process follow a fixed and structured workflow. Before the livestream, the team collaborates with brands to finalize product details, promotional strategies, and a detailed script that the team will follow in livestreaming. During the livestream, DHH livestreamers adhere to a defined sequence (Fig. 1b). It includes introducing products, explaining pricing and promotions, guiding audience through purchasing, and providing real-time sales updates while engaging with the audience. Meanwhile, the coordinator manages the livestreaming process behind the stage and interacts with audience via live comments, while the assistant handles teleprompting tasks.
3. Related Works
3.1. DHH content creators on social media
With the rise of social media, DHH individuals have gained opportunities to participate as content creators(Schäfer and Miles, 2023; of Technology, 2024; Choi et al., 2022), but their engagement remains limited. Their contributions primarily involve asynchronous content, such as uploading edited videos. In these videos, DHH creators use captions, AI-generated voices, and other accessibility tools to communicate with a broader audience(Tang et al., 2023; Cao et al., 2024). Beyond asynchronous content, some DHH individuals have begun participating in livestreaming as livestreamers(Cao et al., 2023).
However, livestreaming, which relies on real-time interaction and dynamic expressiveness(Liu and Cheng, 2025; Lu et al., 2018; Zhang et al., 2023), poses challenges for DHH content creators(Cao et al., 2023). Methods commonly used for asynchronous content creation are often insufficient for livestreaming, particularly in e-commerce livestreaming, where strong real-time engagement is crucial(Wang et al., 2022; Chen et al., 2020; Joo and Yang, 2023; Wang et al., 2024).
Despite the high demands for real-time interaction and expressiveness in e-commerce livestreaming, its pre-structured and relatively fixed interaction processes(Kang et al., 2021; Liu and Cheng, 2025) still offer design opportunities for enhancing communication in SL livestreaming teams with DHH members. Therefore, our study focuses on this unique scenario, exploring how to better support these teams by leveraging the structured nature of e-commerce livestreaming to improve engagement with a wider scope audience.
3.2. Virtual human supporting DHH communication
The rapid development and widespread adoption of virtual human (VH) let it emerge as a possible solution for providing accessible information for DHH individuals by generating SL animations(Haimson and Tang, 2017; Hamilton et al., 2014; Do Amaral and De Martino, 2010; Llamazares de Prado, 2021). For instance, the Beijing Winter Olympics AI signer launched in 2022(Federation, 2021), and some applications include AI SL streamers like Nai Nai introduced by iQiyi in 2018 (News, 2021), as well as the SignPal Kit in 2021 (SEVICES, 2023). These applications developed by leading technology companies primarily focus on helping DHH individuals as consumers of audio-visual media with significantly high costs. However, limited research has focused on the use of VH to help DHH individuals as creators express their content. Our work is inspired by the VH applications and tries to bridge this gap in SL e-commerce livestreaming. We use VH to bridge SL livestreaming team with DHHs and hearing audiences to support DHH’s livestreaming career.
4. Interviews for Understanding Challenges
In this section, we aimed to understand the communication barriers between SL livestreaming teams with DHH members and hearing audience in SL e-commerce livestreaming to inform the following design workshop.
4.1. Methods
Following approval from the university’s Institutional Review Board, we recruited four SL livestream team members (DL1-DL4) by sending recruitment messages on Kuaishou livestream app and snowball sampling in local special education institutions. Three are DHH livestreamers and one is a hearing individual working in an SL livestream team with DHH livestreamers. We also recruited 15 hearing audiences (HV1-HV15), aged 18-40 (M=25.33, SD=4.35; 9 males and 6 females), who were familiar with e-commerce livestreaming and watched livestreams at least twice a week for more than 30 minutes. Full demographic details are provided in Table 1 and Appendix A Table 2.
Semi-structured interviews were conducted with participants’ consent. In interviews with DLs, we explored their livestreaming experiences and the challenges they faced in expanding the scope and diversity of their audience. In interviews with HVs, we showed two SL livestream recordings and asked about their viewing experience. All interviews were conducted via WeChat and Tencent Meeting. Since we did not involve SL interpreters, interviews with DL1 and DL3 were conducted through text messages on WeChat, while DL4, who used hearing aids, and all hearing participants completed the interviews via online meetings. The interviews with DLs lasted between 1 hour to 3 hours and the interviews with HVs lasted between 18 minutes to 33 minutes excluding the time spent watching SL livestream recordings. All text messages were documented, and all meetings were audio-recorded and transcribed verbatim in Chinese.
After finishing the interviews, we analyzed the transcripts using iterative inductive coding (Huberman et al., 2014) and reflexive thematic analysis (Braun and Clarke, 2006). First, we reviewed the transcripts to familiarize ourselves with the data. The first and second authors performed iterative open coding for multiple rounds to identify themes. All authors then met to discuss and finalize the codebook. After open coding, the first and second authors followed the codebook to complete the coding and translated the selected quotes. The coding results and interpretations were double-checked by the other two authors.
| ID | Gender | Platforms |
|
|
Streaming Topics | Hearing Status | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DL1 | F |
|
1 | 4 | Electronic products |
|
||||||
| DL2 | M | Kuaishou | 3 | <10 |
|
Hearing | ||||||
| DL3 | F | Kuaishou | 3 | <10 |
|
|
||||||
| DL4 | M |
|
2 | 8 |
|
|
4.2. Challenges of reaching hearing audience in sign language livestreaming
In this phase of the interview, we revealed that the absence of hearing audience in SL livestreaming by team with DHHs is due to hearing audience not establishing the intention to continue watching. The reasons for this phenomenon are the challenges in conveying and comprehending product information and the challenges in emotional expression and resonance.
Our interviews revealed that the absence of hearing audience in SL livestreaming by teams with DHH members is primarily due to hearing audience’s lack of intention to continue watching. This issue stems from challenges in presenting and receiving product information, as well as challenges in emotional expression and resonance.
4.2.1. Lack of intention to continue watching
Hearing audience typically enter SL livestreams conducted by DHH livestreamers out of curiosity or compassion, but these motivations are insufficient to sustain long-term engagement. HV5 said, ”Because of curiosity, I might click to take a look, but just for a couple of minutes.” HV3 noted his willingness to support, ”I would support as much as I can because the livestreamer is Deaf.” However, HV6 pointed out that such purchases are often one-time acts of goodwill. She stated, ”After expressing my goodwill, I may not watch it again.” The following two sections outline the key reasons why hearing audience lose interest quickly.
4.2.2. Challenges in Presenting and Receiving Product Information
Hearing audience struggle to access product information presented by DHH livestreamers in SL e-commerce livestreaming due to both visual overload and ineffective auditory cues.
Difficulty in obtaining SL information. Both DHH livestreamers and hearing audiences identified SL as a major communication barrier. DL2 stated, ”Hearing audience might find it meaningless and leave immediately because they can’t hear any speech or understand SL.” HV2 added, ”I don’t expect to get product information from SL livestreaming, and if there are hearing livestreamers selling the same products, I probably won’t buy from DHH livestreamers.” Due to the challenges of SL translation, DL2’s team abandoned plans to expand their viewer pool to hearing audience. DL4, facing the same issue, explained why collaboration with interpreters was not feasible. He said, ”There are too few SL interpreters, and they are too expensive to hire.”
Confusing and overwhelming visual information. DHH livestreamers use text prompts to provide product details for audience with varying SL proficiency. DL2 described their approach, ”We usually use a whiteboard or paper to display product qualifications and features.” While HV8, HV11, and HV12 found written product details helpful, others felt the excessive text without clear explanation made comprehension difficult. HV6 said, ”It’s too ’noisy’ to my eyes. They just wrote down lots of numbers, and I don’t know which one is the product’s price without verbal explanation.” HV1 echoed this concern, ”It’s overwhelming to watch SL gestures, look at the product, and read the text at the same time.” HV13 noted that she missed critical moments, such as livestreamers putting up the purchasing link, due to visual overload.
Disruptive auditory interferences. Although DHH livestreamers do not speak during SL livestreaming, various background noises are still present, such as sounds of body collisions during the signing, non-semantic vocalizations, and ambient sounds. These sounds were often unnoticed by DHH livestreamers, but were distracting to hearing audience. HV11 reported, ”There are lots of noises, and they are very loud to me.” HV1 added, ”Not only is the sound meaningless, but it also interferes with my viewing experience. Since I know the livestreamer won’t speak, I might directly mute the sound.” Additionally, some DHH livestreamers played unrelated background audio, which may also affect hearing audience’s understanding, to avoid platform restrictions. DL4 explained, ”We found the platform may categorize our silent livestream as ’vulgar’ and ban our account. So now we play music or narrate stories to prevent this.”
4.2.3. Challenges in emotional expression and resonance
In e-commerce livestreaming, hearing livestreamers convey emotions through tone, volume, facial expressions, and gestures, which enhance viewer engagement and purchasing intent (Bharadwaj et al., 2022; Yu et al., 2022). However, this form of emotional communication relies heavily on auditory cues, which are absent in SL livestreaming. Instead, DHH livestreamers primarily express emotions through facial expressions and signing styles, which hearing audience may not be familiar with, leading to challenges in emotional connection. Our interviews showed that both DHH livestreamers and hearing audience identified these challenges and mentioned the reasons from their perspectives.
Insufficient emotional expression of DHH livestreamers. Expressing emotions effectively during livestreaming is challenging for DHH livestreamers. Despite two years of experience, DL4 still acknowledged a gap in emotional conveyance compared to experienced livestreamers. DL1 described the reason as a lack of confidence, ”I often feel like I can’t handle it well. Although I can repeat lines like a robot to get the job done, I still feel I need more empathy.” DL2 also noted the pressure of maintaining enthusiasm despite negative viewer comments. He said, ”Livestreamers need to be mentally strong to maintain a good and positive spirit in livestreaming.”
Misinterpretation of emotions by audience. Hearing audiences found themselves struggling to interpret and resonate with the emotions expressed in SL livestreaming. They might misinterpret the positive emotions of DHH livestreamers. HV12 misread DHH livestreamers’ enthusiastic signing as aggressive and said, ”They had exaggerated facial expressions, seeming like they were arguing.” HV8 shared a similar impression: ”They seemed hurried, possibly rushing product introductions.” Even when hearing audiences correctly identified emotions, they often lacked context to relate and resonate with the emotions. HV2 noted, ”I could sense their excitement but didn’t know why. It felt dull to me.” HV6 added, ”I can’t relate to their excitement since I don’t understand SL.”
5. Virtual co-presenter demo design and evaluation
5.1. Virtual co-presenter demo design workshop
Building on the challenges identified in the previous interviews, we conducted a design workshop to develop a virtual human as a co-presenter for DHH livestreamers and to create a preliminary demo for initial evaluation to guide future design refinements. We recruited six designers with expertise in animation design and user experience design through snowball sampling in local universities. Full demographic details are provided in Appendix A Table 3.
The workshop began with a review of SL livestreaming recordings and the communication barriers identified in the interviews. Then, the designers proposed several potential solutions, including captions, a VH, and verbal broadcasts. They discussed and evaluated the advantages, limitations, and feasibility of each based on current technologies. Prior research suggests that captions may distract hearing audience from DHH livestreamers (Gugenheimer et al., 2017), while verbal broadcasts may contribute to ineffective auditory information, as noted in the interviews. Additionally, automatic Chinese SL translation remains underdeveloped (Jiang et al., 2020; Tang et al., 2023). Based on these considerations, the designers reached a consensus to use VH technology creating a prerecorded virtual co-presenter that leverages the structured nature of SL e-commerce livestreaming, which follows fixed processes and well-prepared scripts.
The design process began with selecting the virtual co-presenter’s appearance, opting for a simple, young, human-like cartoon character based on prior research (Zhang et al., 2017; Chen et al., 2024). To help hearing audience better understand DHH livestreamers’ emotions, the designers created six gesture animations (AN1–AN6) shown in Fig. 3, imitating gestures from real hearing livestreamers. The selection of gestures was informed by over 30 hours of observed e-commerce livestreaming by hearing livestreamers. AN1 and AN2 were designed to appear randomly or be repeated during the virtual co-presenter verbal broadcasts of product information to improve the anthropomorphism and naturalness of the virtual co-presenter. AN3 to AN6, paired with specific verbal broadcasts, were designed to highlight critical moments and amplify positive emotions.
After finalizing the gesture animations, the designers developed the virtual co-presenter tool’s user interface (Fig. 2) for SL livestreaming teams with DHH members. The left panel included buttons triggering verbal broadcasts paired with animations. The upper-left buttons corresponded to nine product information segments provided by DLs, while the lower-left buttons activated AN3–AN6. To enhance accessibility for SL livestreaming teams with DHH members, the designers incorporated a play reminder and red-colored captions to improve visibility and usability.
5.2. Initial evaluation
To inform the future development of a full virtual co-presenter system, we conducted an initial evaluation to gather feedback on the preliminary virtual co-presenter demo through semi-structured interviews with the original participants from Section 4. Participants reviewed the virtual co-presenter demo created in the design workshop, providing suggestions on its appearance, functionality, and potential improvements. Hearing audiences watched recordings of virtual co-presenter animations, while SL livestreaming team members evaluated the preliminary virtual co-presenter tool with user interface. All participants from the Section 4 interviews took part in this evaluation. The interviews with DLs lasted between 20 minutes to 40 minutes and the interviews with HVs lasted between 10 minutes to 31 minutes. Data analysis followed the same process as in the previous interview study.
5.3. Design suggestions for virtual co-presenter demo
The preliminary virtual co-presenter demo received mostly positive feedback. HV9 noted, ”I think virtual co-presenter will make their livestreaming stand out and be more special.” HV13 liked the virtual co-presenter demo and said, ”Using virtual co-presenter to help people with language barriers will be great. It will be a trend in the future.” In addition to positive feedback, both DLs and HVs provided design suggestions regarding the virtual co-presenter’s appearance and functionality to improve its role in SL livestreaming.
5.3.1. Appearance
Hearing audiences paid more attention to the virtual co-presenter’s appearance than SL livestreamers. Most hearing audiences agreed that the virtual co-presenter should resemble a human-like cartoon character. HV3 felt the current design met expectations, describing it as ”an approachable person and not too extravagant.” HV13 preferred a simple design and stated, ”It doesn’t need too many decorations, just something simple.” HV1 suggested, ”Its movements and speaking should be human-like, not too mechanical or fake.” Some hearing audiences also found symbolic or anthropomorphic characters acceptable. HV4 gave an example: ”Many platforms and livestreaming teams have mascots, like the human-like cat of TMall (a Chinese e-commerce platform).”
Both SL livestreamers and hearing audiences suggested incorporating elements to represent DHH identity. DL1 expressed a preference for the virtual co-presenter to include ”DHH characteristics, such as wearing hearing aids or cochlear implants.” HV6 also suggested, ”Adding DHH symbols could help hearing audience better recognize their identity if DHH livestreamers are willing to do so.”
5.3.2. Functions
Assistance in conveying information. Almost all hearing audiences expected the prerecorded virtual co-presenter to act as an interpreter to provide real-time SL translation naturally. HV5 emphasized, ”The virtual co-presenter and DHH signers need to be consistent, not only in terms of product information but also in livestreaming processes.” HV15 suggested that the virtual co-presenter could provide prompts at critical moments to prevent missing key information. HV3 proposed a more interactive role, suggesting, ”For selling food products, the virtual co-presenter could simulate eating and swallowing actions to enhance sensory engagement.”
Assistance in enhancing emotions. DL1 hoped the virtual co-presenter could help convey ”positive emotions, joy, and a sense of closeness.” Participants emphasized that the virtual co-presenter’s voice should sound natural, with variations in tone, to avoid a mechanical feel. HV5 believed that ”if the virtual co-presenter expresses emotions, it might encourage purchasing intention.” HV4 suggested using promotional phrases, such as ”limited quantities available” or ”today’s lowest price,” to create urgency and boost sales.
Controllability and usability. As primary users of the virtual co-presenter system, DLs stressed the importance of maintaining control. DL3 said, ”The virtual co-presenter should be something we can control.” This concern also extended to the virtual co-presenter’s display position. DL2 preferred that the virtual co-presenter ”not take up too much space or distract audience. It could be placed in a small box in the corner.”
Ease of use was another critical factor for DLs. DL2 emphasized, ”We should be able to operate it immediately without the need for extensive training. It would be helpful if it could be triggered directly based on SL.”
6. Conclusion and Ongoing Work
This research investigates communication barriers between DHH livestreamers and hearing audience in sign language e-commerce livestreaming, focusing on challenges in information conveyance and emotional resonance. Through an in-depth analysis of SL livestreaming characteristics, we developed a virtual co-presenter demo via design workshops, aiming to support DHH livestreamers in creating more expressive and interactive content to engage a broader hearing audience. Preliminary evaluations with participants from both communities provided valuable design insights.
Building on these insights, we are refining the virtual co-presenter into a comprehensive system optimized for SL livestreaming team collaboration. With recent advancements in text-to-3D human animation and direct animation generation technologies (Azadi et al., 2023; Kling AI, 2024), we plan to enhance customization capabilities in future system, allowing DHH livestreamers to flexibly create animated content based on livestreaming scripts. Additionally, we plan to conduct system testing on real livestreaming platforms, iteratively optimizing real-time interaction mechanisms based on feedback from SL livestreaming teams and hearing audiences to assist DHH livestreamers’ expressiveness. Our ultimate goal is to establish a design space informed by the needs of both SL livestreaming teams with DHH members and hearing audiences, providing a foundation for broader interactive design between the two communities in e-commerce livestreaming.
Acknowledgements.
We acknowledge the contributions of Yun Wang and Weiwei Zhang in facilitating the workshop and interview. We also appreciate the valuable suggestions from Dr. Hernisa Kacorri.References
- (1)
- Ahmed et al. (2021) Mohamed Aktham Ahmed, BB Zaidan, AA Zaidan, Abdullah Hussein Alamoodi, Osamah Shihab Albahri, ZT Al-Qaysi, Ahmed Shihab Albahri, and Mahmood M Salih. 2021. Real-time sign language framework based on wearable device: analysis of MSL, DataGlove, and gesture recognition. Soft Computing 25, 16 (2021), 11101–11122.
- Ahmed et al. (2018) Mohamed Aktham Ahmed, Bilal Bahaa Zaidan, Aws Alaa Zaidan, Mahmood Maher Salih, and Muhammad Modi Bin Lakulu. 2018. A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017. Sensors 18, 7 (2018), 2208.
- Anderson and Johnson (2022) Sky LaRell Anderson and Mark R Johnson. 2022. Gamer identities of video game live streamers with disabilities. Information, Communication & Society 25, 13 (2022), 1900–1915.
- Azadi et al. (2023) Samaneh Azadi, Akbar Shah, Thomas Hayes, Devi Parikh, and Sonal Gupta. 2023. Make-an-animation: Large-scale text-conditional 3D human motion generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 15039–15048.
- Bharadwaj et al. (2022) Neeraj Bharadwaj, Michel Ballings, Prasad A Naik, Miller Moore, and Mustafa Murat Arat. 2022. A new livestream retail analytics framework to assess the sales impact of emotional displays. Journal of Marketing 86, 1 (2022), 27–47.
- Braun and Clarke (2006) Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101. https://doi.org/10.1191/1478088706qp063oa arXiv:https://www.tandfonline.com/doi/pdf/10.1191/1478088706qp063oa
- Camgoz et al. (2018) Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. 2018. Neural sign language translation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7784–7793.
- Camgoz et al. (2020) Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. 2020. Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10023–10033.
- Cao et al. (2023) Beiyan Cao, Changyang He, Muzhi Zhou, and Mingming Fan. 2023. Sparkling Silence: Practices and Challenges of Livestreaming Among Deaf or Hard of Hearing Streamers. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15.
- Cao et al. (2024) Jiaxun Cao, Xuening Peng, Fan Liang, and Xin Tong. 2024. ”Voices Help Correlate Signs and Words”: Analyzing Deaf and Hard-of-Hearing (DHH) TikTokers’ Content, Practices, and Pitfalls. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 34, 18 pages. https://doi.org/10.1145/3613904.3642413
- Chen et al. (2024) Hongquan Chen, Bingjia Shao, Xuemei Yang, Weiyao Kang, and Wenfang Fan. 2024. Avatars in live streaming commerce: The influence of anthropomorphism on consumers’ willingness to accept virtual live streamers. Computers in Human Behavior 156 (2024), 108216.
- Chen et al. (2020) Yifei Chen, Feiyan Lu, and Siyu Zheng. 2020. A study on the influence of e-commerce live streaming on consumer repurchase intentions. International Journal of Marketing Studies 12, 4 (2020), 48.
- Choi et al. (2022) Dasom Choi, Uichin Lee, and Hwajung Hong. 2022. “It’s not wrong, but I’m quite disappointed”: Toward an Inclusive Algorithmic Experience for Content Creators with Disabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 593, 19 pages. https://doi.org/10.1145/3491102.3517574
- Do Amaral and De Martino (2010) Wanessa Machado Do Amaral and José Mario De Martino. 2010. Towards a transcription system of sign language for 3D virtual agents. In Innovations in Computing Sciences and Software Engineering. Springer, 85–90.
- Federation (2021) China Disabled Person’s Federation. 2021. CCTV News Client: Meet the Winter Olympics, listen with ”AI”! CCTV News AI Sign Language Anchor Unveiled.
- Gao et al. (2023) Wei Gao, Ning Jiang, and Qingqing Guo. 2023. How do virtual streamers affect purchase intention in the live streaming context? A presence perspective. Journal of Retailing and Consumer Services 73 (2023), 103356. https://doi.org/10.1016/j.jretconser.2023.103356
- Gugenheimer et al. (2017) Jan Gugenheimer, Katrin Plaumann, Florian Schaub, Patrizia Di Campli San Vito, Saskia Duck, Melanie Rabus, and Enrico Rukzio. 2017. The Impact of Assistive Technology on Communication Quality Between Deaf and Hearing Individuals. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW ’17). Association for Computing Machinery, New York, NY, USA, 669–682. https://doi.org/10.1145/2998181.2998203
- Haimson and Tang (2017) Oliver L. Haimson and John C. Tang. 2017. What Makes Live Events Engaging on Facebook Live, Periscope, and Snapchat. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 48–60. https://doi.org/10.1145/3025453.3025642
- Hamilton et al. (2014) William A. Hamilton, Oliver Garretson, and Andruid Kerne. 2014. Streaming on Twitch: Fostering Participatory Communities of Play within Live Mixed Media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 1315–1324. https://doi.org/10.1145/2556288.2557048
- Huberman et al. (2014) A Huberman et al. 2014. Qualitative data analysis a methods sourcebook. Thousand Oaks, Califorinia SAGE Publications, Inc.
- Jiang et al. (2020) Xianwei Jiang, Suresh Chandra Satapathy, Longxiang Yang, Shui-Hua Wang, and Yu-Dong Zhang. 2020. A survey on artificial intelligence in Chinese sign language recognition. Arabian Journal for Science and Engineering 45 (2020), 9859–9894.
- Johnson (2019) Mark R Johnson. 2019. Inclusion and exclusion in the digital economy: Disability and mental health as a live streamer on Twitch. tv. Information, Communication & Society 22, 4 (2019), 506–520.
- Joo and Yang (2023) Eunsin Joo and Jing Yang. 2023. How perceived interactivity affects consumers’ shopping intentions in live stream commerce: roles of immersion, user gratification and product involvement. Journal of Research in Interactive Marketing 17, 5 (2023), 754–772.
- Jun et al. (2021) Joonyoung Jun, Woosuk Seo, Jihyeon Park, Subin Park, and Hyunggu Jung. 2021. Exploring the experiences of streamers with visual impairments. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–23.
- Kang et al. (2021) Kai Kang, Jinxuan Lu, Lingyun Guo, and Wenlu Li. 2021. The dynamic effect of interactivity on customer engagement behavior through tie strength: Evidence from live streaming commerce platforms. International journal of information management 56 (2021), 102251.
- Kling AI (2024) Kling AI. 2024. Kling AI: Text to Animation Platform. Kling AI. https://klingai.com/ An AI-powered text-to-animation platform.
- Liew and Tan (2018) Tze Wei Liew and Su-Mae Tan. 2018. Exploring the effects of specialist versus generalist embodied virtual agents in a multi-product category online store. Telematics and Informatics 35, 1 (2018), 122–135.
- Liu and Cheng (2025) Qian Liu and Wei Cheng. 2025. “I’m telling you”: The use of interactional metadiscourse in Chinese live streaming commerce. Journal of Pragmatics 237 (2025), 14–29.
- Llamazares de Prado (2021) Jose Enrique Llamazares de Prado. 2021. Sign language teaching technological advances and differences in international contexts. The International Journal of Information and Learning Technology 38, 5 (2021), 433–453.
- Lu et al. (2019) Zhicong Lu, Michelle Annett, Mingming Fan, and Daniel Wigdor. 2019. ”I Feel It is My Responsibility to Stream”: Streaming and Engaging with Intangible Cultural Heritage through Livestreaming. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300459
- Lu et al. (2018) Zhicong Lu, Haijun Xia, Seongkook Heo, and Daniel Wigdor. 2018. You watch, you give, and you engage: a study of live streaming practices in China. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–13.
- Miao et al. (2022) Fred Miao, Irina V. Kozlenkova, Haizhong Wang, Tao Xie, and Robert W. Palmatier. 2022. An Emerging Theory of Avatar Marketing. Journal of Marketing 86, 1 (2022), 67–90. https://doi.org/10.1177/0022242921996646 arXiv:https://doi.org/10.1177/0022242921996646
- News (2021) China Deaf Net News. 2021. AI Technology: Converting Speech-to-Text into Live Sign Language to Enable Information Sharing for the Hearing Impaire. http://www.cndcm.cn/html/hot/news/14201_1.html
- of Technology (2024) Rochester Institute of Technology. 2024. InfoGuides: LEAD 301 Social Media Communication and Leadership: Deaf Social Media Influencers. https://infoguides.rit.edu/LEADsocialmedia/textbook. Accessed: 2024-10-25.
- Rong et al. (2022) Ethan Z Rong, Mo Morgana Zhou, Zhicong Lu, and Mingming Fan. 2022. “It Feels Like Being Locked in A Cage”: Understanding Blind or Low Vision Streamers’ Perceptions of Content Curation Algorithms. In Proceedings of the 2022 ACM Designing Interactive Systems Conference. 571–585.
- Schäfer and Miles (2023) Karolin Schäfer and Felix Miles. 2023. Social media use and mental health in deaf or hard-of-hearing adults—Results of an online survey. Frontiers in Communication 8 (2023), 1175461.
- SEVICES (2023) HUAWEI APP SEVICES. 2023. About the Service. https://developer.huawei.com/consumer/cn/doc/introduction-0000001172994921?ha_source=hms1
- Tang et al. (2022) Ningjing Tang, Lei Tao, Bo Wen, and Zhicong Lu. 2022. Dare to Dream, Dare to Livestream: How E-Commerce Livestreaming Empowers Chinese Rural Women (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 297, 13 pages. https://doi.org/10.1145/3491102.3517634
- Tang et al. (2023) Xinru Tang, Xiang Chang, Nuoran Chen, Yingjie (MaoMao) Ni, RAY LC, and Xin Tong. 2023. Community-Driven Information Accessibility: Online Sign Language Content Creation within d/Deaf Communities. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 50, 24 pages. https://doi.org/10.1145/3544548.3581286
- Wang et al. (2024) Lei Wang, Ru-Shui Zhang, and Cheng-Xin Zhang. 2024. Live streaming E-commerce platform characteristics: Influencing consumer value co-creation and co-destruction behavior. Acta psychologica 243 (2024), 104163.
- Wang et al. (2022) Ye Wang, Zhicong Lu, Peng Cao, Jingyi Chu, Haonan Wang, and Roger Wattenhofer. 2022. How live streaming changes shopping decisions in E-commerce: A study of live streaming commerce. Computer Supported Cooperative Work (CSCW) 31, 4 (2022), 701–729.
- yan (2021) yan. 2021. New media empowerment of deaf people in Taobao Sign Language live streaming. new 035, 003 (2021), P.49–51.
- Yu et al. (2022) Miao Yu, Hailiang Chen, and Yifan Dou. 2022. Just the right emotion: Exploring the effects of verbal and facial emotions in Live Streaming E-commerce. Available at SSRN 4141860 (2022).
- Zhang et al. (2023) Xiaoping Zhang, Xusen Cheng, and Xiaowen Huang. 2023. “Oh, My God, Buy It!” Investigating Impulse Buying Behavior in Live Streaming Commerce. International Journal of Human–Computer Interaction 39, 12 (2023), 2436–2449. https://doi.org/10.1080/10447318.2022.2076773 arXiv:https://doi.org/10.1080/10447318.2022.2076773
- Zhang et al. (2017) Yulei Gavin Zhang, Yan Mandy Dang, Susan A Brown, and Hsinchun Chen. 2017. Investigating the impacts of avatar gender, avatar age, and region theme on avatar physical activity in the virtual world. Computers in Human Behavior 68 (2017), 378–387.
Appendix A Hearing Viewers in Study 1
| ID | Gender | Age | Platform |
|
|
Viewing Topics | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| HV1 | F | 29 | Taobao & Douyin | ¿8 | 10-30 | Daily Necessities, Food, Cosmetics, Clothes | ||||
| HV2 | M | 23 | Douyin & Kuaishou | ¿8 | 10-30 | Daily Necessities, Food, Cosmetics, Electronic Products, Clothes | ||||
| HV3 | M | 19 | Douyin | ¿8 | 10-30 | Daily Necessities, Food, Cosmetics, Clothes | ||||
| HV4 | F | 26 | Taobao & Douyin & Kuaishou | 2-8 | 10-30 | Daily Necessities, Food, Cosmetics, Electronic Products | ||||
| HV5 | M | 23 | Taobao & Douyin | ¿8 | 10-30 | Daily Necessities, Food | ||||
| HV6 | F | 28 | Taobao & Douyin & Xiaohongshu | ¿8 | 10-30 | Daily Necessities, Food, Cosmetics | ||||
| HV7 | M | 22 | Douyin & Kuaishou | ¿8 | 30-60 | Daily Necessities, Food, Cosmetics, Electronic Products | ||||
| HV8 | M | 20 | Douyin & Kuaishou | ¿8 | ¿60 | Daily Necessities, Food, Maternal and Child Products, Electronic Products | ||||
| HV9 | M | 22 | Taobao & Douyin & Xiaohongshu | ¿8 | ¿60 | Daily Necessities, Food, Maternal and Child Products, Electronic Products | ||||
| HV10 | F | 33 | Douyin & Kuaishou | ¿8 | ¿60 | Daily Necessities, Food, Cosmetics, Clothes, Maternal and Child Products, Electronic Products | ||||
| HV11 | M | 28 | Taobao & Douyin | ¿8 | 30-60 | Daily Necessities, Food, Cosmetics, Clothes, Electronic Products | ||||
| HV12 | M | 20 | Taobao & Douyin & Kuaishou | ¿8 | 30-60 | Daily Necessities, Food, Cosmetics, Sports Products, Electronic Products, Others | ||||
| HV13 | F | 32 | Taobao & Douyin | ¿8 | 30-60 | Daily Necessities, Food, Cosmetics, Electronic Products | ||||
| HV14 | M | 25 | Taobao & Douyin | ¿8 | 30-60 | Daily Necessities, Food, Cosmetics, Clothes | ||||
| HV15 | F | 30 | Taobao | ¿8 | 10-30 | N/A |
”Platform” describes the applications that hearing viewers use to watch livestreaming. ”Frequency (times/month)” describes the average frequency of hearing viewers watching livestreaming in their daily life. ”Duration (minutes)” describes the average time of hearing viewers watching livestreaming each time in their daily life. ”Viewing Topics” describes the categories that hearing viewers watch livestreaming for in their daily life.
| Designer | Gender | Age | Expertise |
|
||
|---|---|---|---|---|---|---|
| D1 | M | 27 | 6 years in Animation Design | 4 hours | ||
| D2 | F | 24 | 3 years in User Experience Design | 56 hours | ||
| D3 | M | 21 | 3 years in Animation Design | 45 hours | ||
| D4 | F | 35 | 10 years in User Interface Design | more than 6 hours | ||
| D5 | F | 22 | 4 years in User Experience Design | about 5 hours | ||
| D6 | F | 21 | 3 years in Animation Design | more than 5 hours |
Appendix B Virtual co-presenter’s animations and corresponding quotes
