Personalized Affect-Aware Socially Assistive Robot Tutors Aimed at Fostering Social Grit in Children with Autism

Zhonghao Shi [email protected] https://orcid.org/0000-0002-2855-5863 University of Southern CaliforniaLos AngelesUSA90007 , Manwei Cao [email protected] University of Southern CaliforniaLos AngelesUSA90007 , Sophia Pei [email protected] University of Southern CaliforniaLos AngelesUSA90007 , Xiaoyang Qiao [email protected] University of Southern CaliforniaLos AngelesUSA90007 , Thomas R Groechel [email protected] University of Southern CaliforniaLos AngelesUSA90007 and Maja J Matarić [email protected] https://orcid.org/0000-0001-8958-6666 University of Southern CaliforniaLos AngelesUSA90007

(2018)

Abstract.

Affect-aware socially assistive robotics (SAR) tutors have great potential to augment and democratize professional therapeutic interventions for children with autism spectrum disorders (ASD) from different socioeconomic backgrounds. However, the majority of research on SAR for ASD has been on teaching cognitive and/or social skills, not on addressing users’ emotional needs for real-world social situations. To bridge that gap, this work aims to develop personalized affect-aware SAR tutors to help alleviate social anxiety and foster social grit–the growth mindset for social skill development–in children with ASD. We propose a novel paradigm to incorporate clinically validated Acceptance and Commitment Training (ACT) with personalized SAR interventions. This work paves the way toward developing personalized affect-aware SAR interventions to support the unique and diverse socio-emotional needs and challenges of children with ASD.

Human-Robot Interaction, Socially Assistive Robotics, Affective Computing

^†^†copyright: acmcopyright^†^†journalyear: 2018^†^†doi: 10.1145/1122445.1122456^†^†conference: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’21 Companion); March 08–11, 2021; virtual^†^†booktitle: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’21 Companion), March 08–11, 2021, virtual^†^†price: 15.00^†^†isbn: 978-1-4503-XXXX-X/18/06^†^†ccs: Computer systems organization Robotics

1. INTRODUCTION

Socially assistive robot (SAR) learning companions and tutors have great potential to effectively promote cognitive development of social skills for children with ASD (Saleh et al., 2020). Past research on SAR for ASD has been mainly focusing on augmenting or automating interventions for specific cognitive impairments of ASD, such as social interaction deficits and stereotyped behaviors (Ismail et al., 2019). This further validates the potential of autonomous SAR to provide more cost-effective and accessible clinical treatments for children with ASD, so the economic burden of their families can be alleviated (Lavelle et al., 2014). However, individuals with ASD also face significant emotional challenges such as social anxiety due to deficits in social-emotional reciprocity due to ASD (Bellini, 2006). While cognitive impairments can be reduced with evidence-based persistent clinical interventions and personal practice (White et al., 2007), Sumiya et al. (2018) showed that children with ASD tend to have low social motivation and grit, making it especially challenging for them to persistently practice social skills in daily social interactions, despite having previously learned and practiced those skills with therapists (Briot et al., 2020). As shown in a study conducted by Bellini (2006), children with ASD are at high risk of suffering social anxiety, so they can particularly demonstrate this lack of generalization of learned social skills into their daily social life. SAR therefore also has the potential to provide support for addressing the socio-emotional challenges faced by children with ASD.

In this work, we propose a novel paradigm that incorporates clinically validated Acceptance and Commitment Training (ACT) methods (Eifert and Forsyth, 2005) with personalized SAR interventions to help children with ASD alleviate socio-emotional challenges and foster social grit, so they can become more emotionally competent and more likely to positively engage in social activities.

2. Methods

Refer to caption — Figure 1. (A) Six areas of focus of ACT methods (Hayes, [n.d.]); (B) Our autonomous SAR system for children with ASD

ACT Areas of Focus(Eifert and Forsyth, 2005)	ACT Design Principles (Eifert and Forsyth, 2005)	ACT-Based SAR Interventions (Eifert and Forsyth, 2005)
Acceptance	Increasing the participants’ social behaviors replacing avoidance/escape behavior in the presence of aversive social stimuli in order to contact a desired reinforcer.	A SAR intervention is designed to walk the participant through the “Tug-of-war with an Anxiety Monster” metaphor (Eifert and Forsyth, 2005). The SAR dialog is scripted and programmed to help uer user fully embrace their aversive thoughts and feelings in the presence of aversive social stimuli in a non-judgemental and empowering way.
Defusion	Weakening the participants’ rigid rule-following behaviors and restrictive cognitive patterns, and increasing a skill repertoire of flexible responding to social stimuli that were previously rigidly responded to.	A SAR intervention is designed to engage the user in the ”Hand” exercise (Eifert and Forsyth, 2005) and ”Silly Voices” exercise (Eilers and Hayes, 2015). The goal of the SAR tutor is to help the user foster their ability to create space and distance between themselves and their negative emotions.
Present moment	Strengthening the user’s skill repertoire of actively attending to social stimuli in the present moment, while weakening the skill repertoire of attending to thir own emotions/memories/thoughts with respect to past/future/imagined events.	A SAR intervention is designed to guide the user through the ”Soles of the Feet” mindfulness exercise (Singh et al., 2003), where the SAR tutor teaches the user about how to shift attention flexibly to focus on the present moment, instead of overthinking about the negative past experience or uncertainty of the future.
Self-as-context	Weakening the user’s rigid concepts regarding self image and self identities, and increasing their flexible perspective-taking skills of relational responding to I/you, here/there, now/then.	A SAR intervention is designed to guide the user through the ”Bus Driver” exercise (Eifert and Forsyth, 2005). In this exercise, the SAR tutor’s goal is help the user understand that their actions are defined by their values rather than by the external environment.
Values	Enhancing the user’s skill repertoire of deriving and following rules that associate previously aversive stimuli with a value/motivation, or increasing effectiveness of existing value/motivation.	A SAR intervention is designed to guide the user through the ”Three Wishes” exercise (Eifert and Forsyth, 2005) with the goal of increasing the user’s ability to identify their intrinsic motivation and life values as the reason to engage in meaningful social activities.
Committed Action	Increasing the user’s response repertoire of meaningful social behaviors that are in line with their values.	A SAR intervention is designed to guide the user through the ”Choice Point” exercise (Bailey et al., 2014), so that the user is likely to perform actions that move them closer toward their values and goals, and away from experiential avoidance.

Table 1. Example SAR interventions based on the design principles of ACT (Niles et al., 2014; Eifert and Forsyth, 2005; Hayes, 2020)

We propose a novel paradigm to design personalized SAR interventions based on clinically validated acceptance and commitment training (ACT) methods (Eifert and Forsyth, 2005) to help children with ASD with their socio-emotional challenges and lack of social grit. ACT is a behavioral treatment that has been clinically validated and is commonly conducted in clinical settings for the treatment of socio-emotional challenges (Dalrymple and Herbert, 2007; Niles et al., 2014). ACT aims to promote mindfulness, acceptance, and cognitive defusion (learning to detach from current thoughts and emotions) with the ultimate goal of increasing psychological flexibility and promoting behavior change that aligns with one’s life values (Hayes, [n.d.]). Our goal of incorporating ACT with SAR interventions is to not only support the socio-emotional needs of children with ASD but also to foster social grit and encourage behavior changes toward more actively engaging in positive social activities.

Specifically, we adapt the ACT treatment format outlined in a prior clinical study by Niles et al. (2014) and a clinical manual by Eifert and Forsyth (2005) to inform the design of our personalized SAR interventions. As detailed in Figure 1(A) and Table 1, by following the design principle of the six areas of focus in ACT, we first select ACT exercises that can be potentially effectively adapted and performed by a SAR tutor, and then design SAR interactions based on those interventions. Our design involves weekly intervention sessions wherein the child engages in a triadic interaction with the robot and caregiver to complete ACT-based SAR interventions. The affect-aware SAR tutor is programmed to conduct interventions while providing personalized feedback to keep the participants engaged and motivated in the exercises. The caregiver helps the child when the child’s needs are outside of the scope of the SAR’s functionalities. More details about the SAR system and experiment design are found in the next section.

3. Experiment and System Setup

3.1. Study Design

The goal of this project is to study how effectively personalized affect-aware SAR can deliver ACT-based interventions to children with ASD outside clinical settings. We will evaluate the effectiveness of the SAR-supported interventions to help alleviate social anxiety and foster social grit for children with ASD. Our study design follows the single-subject withdrawal design embedded in a multiple-baseline design commonly used in ASD studies (Cardon and Azuma, 2011). The study consists of three phases: pre-treatment (3-5 weeks), in-treatment (6-10 weeks), and post-treatment (2-4 weeks). Each participant stays in the pre-treatment (baseline) phase for 3 weeks or longer and then enters the in-treatment phase at different points of time. Consequently, a comparison measurement between participants’ baseline level of performance is established, and the potential maturation effect caused by passage of time can be controlled (Cardon and Azuma, 2011). The in-treatment phase involves the deployments of ACT-based personalized SAR interventions. We plan to conduct weekly sessions to cover the six areas of focus in ACT treatment summarized in Sec 2. Since the socio-emotional needs of each child are unique, the types and total number of treatment sessions are personalized based on the child’s behavioral improvements during the in-treatment phase. The in-treatment phase can be extended with more sessions if the SAR decision-tree-based personalization algorithm, co-designed by the therapist and researcher, determines the child needs more support to reach their therapeutic goal. The final post-treatment phase allows the participants’ behavior changes to be re-assessed and studied to learn whether observed improvements can be sustained after the removal of the intervention.

3.2. Personalized SAR Intervention System

As shown in Figure 1(B), we plan to develop the SAR tutor with a touchscreen tablet that will serve as a shared collaborative space for the child and robot. Each participant will take part in a first pilot session, whose multimodal data will be annotated and used to pre-train personalized affective models of arousal and valence using supervised domain adaptation we have already validated in our previous work (Shi et al., 2021). Using real-time predictions of arousal and valence, we plan to apply an affective reinforcement learning approach in order to personalize the robot’s responses to keep the child engaged in the ACT interventions. The autonomous SAR system will use dialog to further encourage social grit and a growth mindset grounded in prior work (Park et al., 2017; Rhew, 2017) in order to keep the participants motivated throughout the intervention.

3.3. Participant Information

In this study, we plan to recruit 14 participants with the following inclusion criteria: 1) age between 7 and 12 years old; 2) good physical, sensory (hearing, vision), and medical health conditions; 3) English as the primary language; 4) clinical diagnosis of ASD as described in the Diagnostic and Statistical Manual of Mental Disorders–Version 5 (Van Bourgondien et al., 1992; Kanne et al., 2008; Dover and Le Couteur, 2007; Baird et al., 2003); 5) at high risk of suffering social anxiety based on the Anxiety Disorders Interview Schedule for DSM-IV (ADIS-IV) (Grisham et al., 2004); 6) at high risk for low social grit based on the assessment of acceptance and action questionnaires (Hayes et al., 2006). The detailed study plan including the recruitment procedure, consent process, participant privacy and data confidentiality protocol is currently being developed, and will be submitted to Institutional Review Board of the University of Southern California for review.

3.4. Assessment

To assess the participants’ behavior changes during the in-treatment phase, we plan to administer two validated assessment tools: 1) Acceptance and Action Questionnaire (16-item version) developed specifically to measure the proposed mechanisms of change in ACT treatment (Hayes et al., 2004); and 2) Self-Statements During Public Speaking Questionnaire (SSPS) developed to measure negative and positive self-statements in the context of public speaking in social situations (Hofmann and DiBartolo, 2000). These measurements also inform the personalization of the SAR tutor, so the duration and design of in-treatment phase can be personalized to each individual child’s needs.

To validate the effectiveness of the personalized SAR intervention for emotional challenges and social grit, we plan to compare the participants’ outcome measures between the pre-treatment and post-treatment phases. We will use three widely used validated outcome assessment tools for social anxiety: 1) the Liebowitz Social Anxiety Scale–Self-Report (LSAS-SR): a 24-item measure that assesses fear and avoidance of social interaction and performance situations (Fresco et al., 2001); 2) the Social Interaction Anxiety Scale (SIAS): a 20-item measure of cognitive, affective, or behavioral reactions to social interaction in dyads or groups (Mattick and Clarke, 1998); and 3) the Social Phobia Scale (SPS): a 20-item measure describing situations or themes related to being observed by others (Mattick and Clarke, 1998). Moreover, to study the effects of SAR interventions on social grit and motivation, we also plan to conduct direct measurements of the frequency of the social activities engaged by the participants during the study. Semi-structured interviews with participants’ family members (parents, teachers, friends) will also be conducted weekly to collect indirect measures. Statistical analysis with multilevel modeling (Kenny et al., 1998; Niles et al., 2014) will then be conducted to evaluate the significance of behavior changes between the pre-treatment and post-treatment phases.

4. ACKNOWLEDGMENTS

This work is supported by the NSF Expeditions in Computing grant on Socially Assistive Robotics, IIS-1139148.

References

(1)
Bailey et al. (2014) Ann Bailey, Joseph Ciarrochi, and Russ Harris. 2014. The weight escape: How to stop dieting and start living. Shambhala Publications.
Baird et al. (2003) Gillian Baird, Hilary Cass, and Vicky Slonims. 2003. Diagnosis of autism. Bmj 327, 7413 (2003), 488–493.
Bellini (2006) Scott Bellini. 2006. The development of social anxiety in adolescents with autism spectrum disorders. Focus on autism and other developmental disabilities 21, 3 (2006), 138–145.
Briot et al. (2020) Kellen Briot, François Jean, Ali Jouni, Marie-Maude Geoffray, Ly-Le Moal, Daniel Umbricht, Christopher Chatham, Lorraine Murtagh, Richard Delorme, Manuel Bouvard, et al. 2020. Social Anxiety in Children and Adolescents With Autism Spectrum Disorders Contribute to Impairments in Social Communication and Social Motivation. Frontiers in psychiatry 11 (2020), 710.
Cardon and Azuma (2011) Teresa A Cardon and Tamiko Azuma. 2011. Deciphering single-subject research design and autism spectrum disorders. The ASHA Leader 16, 7 (2011), online–only.
Dalrymple and Herbert (2007) Kristy L Dalrymple and James D Herbert. 2007. Acceptance and commitment therapy for generalized social anxiety disorder: A pilot study. Behavior modification 31, 5 (2007), 543–568.
Dover and Le Couteur (2007) Clare J Dover and Ann Le Couteur. 2007. How to diagnose autism. Archives of disease in childhood 92, 6 (2007), 540–545.
Eifert and Forsyth (2005) Georg H Eifert and John P Forsyth. 2005. Acceptance and commitment therapy for anxiety disorders: A practitioner’s treatment guide to using mindfulness, acceptance, and values-based behavior change. New Harbinger Publications.
Eilers and Hayes (2015) Heidi J Eilers and Steven C Hayes. 2015. Exposure and response prevention therapy with cognitive defusion exercises to reduce repetitive and restrictive behaviors displayed by children with autism spectrum disorder. Research in Autism Spectrum Disorders 19 (2015), 18–31.
Fresco et al. (2001) DM Fresco, ME Coles, Richard G Heimberg, MR Liebowitz, S Hami, Murray B Stein, and D Goetz. 2001. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychological medicine 31, 6 (2001), 1025–1035.
Grisham et al. (2004) Jessica R Grisham, Timothy A Brown, and Laura A Campbell. 2004. The Anxiety Disorders Interview Schedule for DSM-IV (ADIS-IV). (2004).
Hayes ([n.d.]) Steven C Hayes. [n.d.]. Acceptance and commitment therapy. American Psychological Association: Washington, DC.
Hayes (2020) Steven C Hayes. 2020. A liberated mind: How to pivot toward what matters. Avery.
Hayes et al. (2006) Steven C Hayes, Jason B Luoma, Frank W Bond, Akihiko Masuda, and Jason Lillis. 2006. Acceptance and commitment therapy: Model, processes and outcomes. Behaviour research and therapy 44, 1 (2006), 1–25.
Hayes et al. (2004) Steven C Hayes, Kirk Strosahl, Kelly G Wilson, Richard T Bissett, Jacqueline Pistorello, Dosheen Toarmino, Melissa A Polusny, Thane A Dykstra, Sonja V Batten, John Bergan, et al. 2004. Measuring experiential avoidance: A preliminary test of a working model. The psychological record 54, 4 (2004), 553–578.
Hofmann and DiBartolo (2000) Stefan G Hofmann and Patricia Marten DiBartolo. 2000. An instrument to assess self-statements during public speaking: Scale development and preliminary psychometric properties. Behavior Therapy 31, 3 (2000), 499–515.
Ismail et al. (2019) Luthffi Idzhar Ismail, Thibault Verhoeven, Joni Dambre, and Francis Wyffels. 2019. Leveraging robotics research for children with autism: a review. International Journal of Social Robotics 11, 3 (2019), 389–410.
Kanne et al. (2008) Stephen M Kanne, Jena K Randolph, and Janet E Farmer. 2008. Diagnostic and assessment findings: A bridge to academic planning for children with autism spectrum disorders. Neuropsychology review 18, 4 (2008), 367–384.
Kenny et al. (1998) David A Kenny, Deborah A Kashy, Niall Bolger, D Gilbert, Susan T Fiske, and Gardner Lindzey. 1998. The handbook of social psychology. , 233–265 pages.
Lavelle et al. (2014) Tara A Lavelle, Milton C Weinstein, Joseph P Newhouse, Kerim Munir, Karen A Kuhlthau, and Lisa A Prosser. 2014. Economic burden of childhood autism spectrum disorders. Pediatrics 133, 3 (2014), e520–e529.
Mattick and Clarke (1998) Richard P Mattick and J Christopher Clarke. 1998. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behaviour research and therapy 36, 4 (1998), 455–470.
Niles et al. (2014) Andrea N Niles, Lisa J Burklund, Joanna J Arch, Matthew D Lieberman, Darby Saxbe, and Michelle G Craske. 2014. Cognitive mediators of treatment for social anxiety disorder: Comparing acceptance and commitment therapy and cognitive-behavioral therapy. Behavior therapy 45, 5 (2014), 664–677.
Park et al. (2017) Hae Won Park, Rinat Rosenberg-Kima, Maor Rosenberg, Goren Gordon, and Cynthia Breazeal. 2017. Growing growth mindset with a social robot peer. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction. 137–145.
Rhew (2017) Emily A Rhew. 2017. The effects of a growth mindset intervention on self-efficacy and motivation of adolescent special education students. (2017).
Saleh et al. (2020) Mohammed A Saleh, Fazah Akhtar Hanapiah, and Habibah Hashim. 2020. Robot applications for autism: a comprehensive review. Disability and Rehabilitation: Assistive Technology (2020), 1–23.
Shi et al. (2021) Zhonghao Shi, Thomas R Groechel, Shomik Jain, Kourtney Chima, Maja J Matarić, et al. 2021. Toward Personalized Affect-Aware Socially Assistive Robot Tutors in Long-Term Interventions for Children with Autism. arXiv preprint arXiv:2101.10580 (2021).
Singh et al. (2003) Nirbhay N Singh, Robert G Wahler, Angela D Adkins, Rachel E Myers, Mindfulness Research Group, et al. 2003. Soles of the feet: A mindfulness-based self-control intervention for aggression by an individual with mild mental retardation and mental illness. Research in Developmental Disabilities 24, 3 (2003), 158–169.
Sumiya et al. (2018) Motofumi Sumiya, Kazue Igarashi, and Motohide Miyahara. 2018. Emotions surrounding friendships of adolescents with autism spectrum disorder in Japan: A qualitative interview study. PloS one 13, 2 (2018), e0191538.
Van Bourgondien et al. (1992) Mary E Van Bourgondien, Lee M Marcus, and Eric Schopler. 1992. Comparison of DSM-III-R and childhood autism rating scale diagnoses of autism. Journal of Autism and Developmental Disorders 22, 4 (1992), 493–506.
White et al. (2007) Susan Williams White, Kathleen Keonig, and Lawrence Scahill. 2007. Social skills development in children with autism spectrum disorders: A review of the intervention research. Journal of autism and developmental disorders 37, 10 (2007), 1858–1868.