Towards RehabCoach: Design and Preliminary Evaluation of a Conversational Agent Supporting Unsupervised Therapy after Stroke

Giada Devittori¹, Mehdi Akeddar¹, Alexandra Retevoi¹, Fabian Schneider², Viktoria Cvetkova³, Daria
Dinacci³, Antonella Califfi³, Paolo Rossi³, Claudio Petrillo³, Tobias Kowatsch^2,4,5,6 and Olivier Lambercy^1,6 This work was supported by the Competence Centre for Rehabilitation Engineering and Science (RESC) of ETH Zurich, and the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) program.¹Rehabilitation Engineering Laboratory, ETH Zurich, Switzerland [email protected]²Department of Technology, Management, and Economics, ETH Zurich, Switzerland³Clinica Hildebrand Centro di riabilitazione Brissago, Switzerland⁴Institute for Implementation Science in Health Care, University of Zurich, Switzerland⁵School of Medicine, University of St. Gallen, Switzerland⁶Future Health Technologies program, Singapore-ETH Centre, Campus for Research Excellence and Technological Enterprise (CREATE), Singapore

Abstract

Unsupervised therapy after stroke is a promising way to boost therapy dose without significantly increasing the workload on healthcare professionals. However, it raises important challenges, such as lower adherence to therapy in the absence of social interaction with therapists. We present the initial prototype of RehabCoach, a novel smartphone-based app with conversational agent to support unsupervised therapy. RehabCoach is designed to increase patients’ engagement and adherence to therapy and to provide information (e.g., about stroke, health) in an interactive and user-friendly manner. We report on the design and usability evaluation of the first prototype of RehabCoach, assessed by four stroke patients and five healthcare professionals, who interacted with the app in a single testing session. Task completion time and success rates were measured for 15 representative tasks, and participants assessed usability via questionnaires and a semi-structured interview. Results show that it was feasible for stroke patients to successfully interact with RehabCoach (task success $\geq$ 93 %) without requiring extensive training. Participants positively rated the usability of RehabCoach (mean mHealth App Usability Questionnaire score: 1.3 for primary users, 1.4 for healthcare professionals, on a scale from 1 (positive evaluation) to 7). The feedback collected in this work opens the door to further enhance RehabCoach as an interactive digital tool to support unsupervised rehabilitation.

I INTRODUCTION

People after a stroke often do not fully recover upper limb function [1], which negatively impacts independence and quality of life. Increasing the dose (i.e., task repetitions and therapy time) of upper limb therapy may enhance functional outcomes or prevent the long-term loss of the improvements gained during the early rehabilitation phase [2, 3]. However, the current rehabilitation model, mainly based on one-to-one therapy sessions, and the limited resources available (e.g., low therapist-to-patient ratio [4]) make increasing therapy dose challenging.

Unsupervised therapy, defined as patients training without any direct external supervision, might be a way to increase dose without further straining the healthcare system. Especially when applied in the home setting, unsupervised therapy could help decrease the reliance on hospital stays and visits, thus ensuring access to rehabilitation services even when the healthcare system is under pressure, as exemplified during the COVID-19 pandemic [5, 6]. Furthermore, studies have shown that unsupervised therapy can be comparable to standard care [7].

The literature describes a wide range of tools that may support unsupervised therapy, such as booklets of exercises [8], virtual reality systems [9, 10], and actuated robots [11, 12]. However, unsupervised therapy raises important challenges, such as low engagement and motivation to train, poor adherence to therapy schedule, as well as lack of social interaction and feedback on progress [13, 14, 7]. Attempts to address these challenges include weekly communication with a therapist [15] or the implementation of multiuser therapy exercises [9], which can increase adherence but rely on the presence of external persons. Integrating a gaming environment into rehabilitation may also increase motivation [16], but it does not solve the issue of lack of social interaction.

Digital interventions may play a key role in addressing these gaps [17]. To explore this, we are proposing RehabCoach, a novel chatbot-based mobile application designed to complement and support existing methods for unsupervised therapy. RehabCoach is based on the MobileCoach framework [18, 19, 20], an open-source platform for health interventions that relies on the establishment of a personal relationship between the user and a digital coach, mainly through a chat function, to promote long-term compliance with the intervention [21]. RehabCoach is not meant to deliver therapy directly but takes advantage of MobileCoach’s digital coaching system, for instance to send messages to encourage patients to adhere to predefined therapy plans or to give feedback, instead of only implementing conventional reminders [22, 23]. MobileCoach-based interventions have already been applied and validated in different populations and applications [24, 25], but never in people after stroke, who might present additional challenges regarding the use of digital tools.

Our long-term vision is to use the RehabCoach app in combination with one or more robotic devices in order to increase patients’ adherence and motivation to train unsupervised (in the clinic or at home). As an initial step towards this, we developed a first version of RehabCoach to be presented to primary (i.e., stroke patients) and secondary (i.e., healthcare professionals) users in a single evaluation session. This first version is not yet connected to a rehabilitation device, but still allows users to be involved early in a first iteration of the RehabCoach app design, to collect data and feedback on usability to be incorporated into the next version, promoting its acceptance in the future [26]. Furthermore, testing the app separately from a robotic device avoids introducing patients to many different technologies at the same time, limiting potential confounding factors when evaluating usability.

In this paper, we describe the interdisciplinary development and preliminary usability assessment of RehabCoach. The primary goal is to assess whether individuals after a stroke can successfully interact with such a digital tool after minimal instructions. The secondary goal is to collect quantitative and qualitative feedback on the usability of RehabCoach and gather suggestions for improvements from both primary and secondary users, which could help drive the design of future digital interventions supporting rehabilitation after stroke.

II METHODS

II-A RehabCoach design

The RehabCoach app builds on MobileCoach, an open-source platform for smartphone-based ecological momentary assessments and health interventions that focus on health-promoting behavior and psychoeducation [18, 19, 20]. MobileCoach supports the implementation of a chat-based user interface with a conversational agent and pre-defined answer options. The use of a smartphone-based conversational agent in individuals after stroke raises additional challenges, as this population is often not used to technology and may suffer from cognitive and sensorimotor impairments that could hinder the use of such an app.

For the first prototype of RehabCoach (version 0), the existing MobileCoach features previously applied to other populations were combined into a therapy coaching concept, collaboratively discussed by experts in neurorehabilitation, computer science, digital health, and information system. Also, existing literature on mobile apps and means to boost motivation to train for elderly and stroke patients was consulted and combined with prior experience with MobileCoach-based interventions in other populations. This resulted in the definition of a first set of key features relevant to neurorehabilitation and, in particular, to unsupervised therapy (Figure 1).

Refer to caption — Figure 1: Screenshot of the main menu of the RehabCoach app.

Profile: When opening the RehabCoach app for the first time, the patient must enter some information (e.g., name, ability to type on the phone, ability to walk) which could then be used to personalize the intervention (i.e., the set of interactions with RehabCoach aimed at supporting unsupervised therapy). The key information to enter was discussed and approved by healthcare professionals. Furthermore, the patient can choose between two avatars both representing a digital coach.

Chat: The digital coach can send messages to the user via a chat interface. These messages are designed to increase adherence to pre-defined therapy schedules, for example, by reminding the user to start a planned training session. To increase motivation, messages containing feedback about the daily results can be sent [27]. During the day, the digital coach can also send messages which are not strictly relevant to therapy but relate, for instance, to the emotional or physical state of the user (e.g., “How was your day?”). This aims at strengthening the personal relationship with the user, promoting long-term interaction with RehabCoach [21], and compensating to a certain extent for the lack of direct social contact during unsupervised therapy.

Learn: Another detrimental factor to patients’ wellbeing is the lack of understanding towards stroke and secondary stroke prevention [28]. RehabCoach offers a learning section where videos on stroke, health, and the importance of rehabilitation can be accessed. The goal of this section is to increase patients’ awareness about their disease and health status, which may lead to higher adherence to therapy, the adoption of a healthier lifestyle, and act as a secondary stroke prevention intervention [29, 27].

Checklist: The overview of the daily goals (e.g., training sessions) which have already been achieved or still need to be completed is provided in the form of a checklist. Clearly stating the daily goals should increase users’ engagement and, thus, adherence to the therapy schedule [30].

“I want to train”: The virtual coach sends messages related to a training session based on a pre-defined schedule. The “I want to train” feature allows the user to start a training session spontaneously, in addition to the training program. Since unsupervised therapy could lead to patients not training consistently, this feature allows the user of the app to perform a training session even if they skipped a scheduled one, or to train more.

It is envisioned that the features described above are exploited differently throughout a typical day of using the app to support unsupervised therapy (Figure 2). The first time the user opens the app, RehabCoach explains how the different interactions and sessions work (welcome interaction). Then, every morning, a planning interaction is performed to plan the daily training and learning sessions at a time suitable for the user, depending on their daily schedule. Therefore, during this interaction, the number of training sessions and the time for the training and learning sessions can be set. If the user does not answer the messages sent by the digital coach during this interaction, the sessions are set to a default time. At the set time, the training and learning interactions start with a message from the digital coach reminding the user to perform the therapy exercises or to watch an informative video. Users can either confirm that they will start the session or postpone it. The digital coach then asks for feedback on the session, providing the opportunity to collect subjective information on the quality of a therapy session. Finally, in the evening, RehabCoach summarizes the user’s daily performance (summary interaction). When the app is not open, messages appear on the smartphone as push notifications. If users do not respond within a certain time frame (i.e., 10 minutes), the interaction is tagged as incomplete, and the app proceeds to the next interaction scheduled. All the answers inputted by the user are transmitted over an encrypted channel and saved automatically on a secure web server.

II-B Preliminary usability evaluation

The goal of this preliminary usability evaluation was to identify potential technical or accessibility issues and to gather feedback on the app to decide on its further development before combining it with a rehabilitation device and conducting a larger feasibility study at home. The study protocol was approved by the ethics commission of ETH Zurich (2022-N-29). For each participant, the usability evaluation consisted of a single study session of about 45 minutes, simulating an entire day of using the app. The evaluation included individuals after a stroke and healthcare professionals working with stroke patients. Healthcare professionals were included with the objective to gain additional feedback on the app, for example, on further features that could be implemented in upcoming versions. This group would further serve as a baseline regarding feasibility and performance in the given tasks with the app. To be included in the study, participants had to be older than 18 years and sign the informed consent. An additional inclusion criterion for primary users was a diagnosis of stroke, while healthcare professionals had to regularly interact with individuals post-stroke. Persons with major cognitive and/or communication deficits, or major comprehension and/or memory deficits reported in their clinical records were excluded. The usability evaluation session consisted of three phases:

Familiarization: The researcher described the aim of the project, answered eventual questions, and checked that participants understood and signed the informed consent form. Then, the researcher presented the RehabCoach app on a test smartphone and explained how to interact with it. This was then followed by three minutes of familiarization, where the participant could freely interact with the app and ask questions.

Testing: Participants were asked to perform specific tasks (Figure 3a) considered essential for correct daily interactions with the app. The time taken to complete each task and task success were recorded. Task success could be rated as: successfully completed, completed with an input from the researcher, completed with an error, or not completed.

Evaluation: Participants were asked to fill in the mHealth App Usability Questionnaire (MAUQ, [31]) and to rate four custom statements. Both the MAUQ and the custom statements were rated on a seven-point Likert scale ranging from 1 (strongly agree, i.e., positive opinion about the app and high usability) to 7 (strongly disagree), with the possibility of leaving out an item in case it was not applicable (I don’t know). Items 1-5 of the MAUQ are specific to ease of use (MAUQ_E), items 6-12 to interface and satisfaction (MAUQ_I), and items 13-18 to usefulness (MAUQ_U). This was followed by a semi-structured interview that touched topics related to first impressions about RehabCoach, likes and dislikes, additional features that participants would like to have, design, and informative videos.

II-C Data analysis

Boxplots were used to represent the time taken to complete the tasks for the group of primary and secondary users separately. Given the small sample size of this preliminary study, no statistical testing was reported. Task success was represented with a heat map to visualize tasks potentially causing more difficulties. MAUQ scores were calculated as the mean of the scores given to the single items, excluding the items where participants answered I don’t know.

III RESULTS

Four subacute stroke patients between 45 and 78 years old (mean age: 63.75 years) and five healthcare professionals between 28 and 51 years old (mean age: 40.2 years) took part in the study. Among the healthcare professionals, one was a speech therapist, one a neuropsychologist, two were physiotherapists and one had a double specialty in physiotherapy and neuropsychology.

III-A Task success and completion time

All healthcare professionals and two of the primary users could complete all tasks (Figure 3b). P1 completed the summary interaction (T14) with an error, as an empty message was sent instead of typing an answer to one of the questions. During the planning interaction (T10), P2 accepted the proposed time for the training session (i.e., 2 pm) instead of changing it to 3 pm and therefore did not successfully complete the task. The input given by the researcher (n=2) consisted in encouraging participants to carefully look at one specific section of the app interface to find the icon of interest after the participants expressed difficulties in finding it. For ten tasks, the time taken by the primary users was similar to the healthcare professionals (difference of medians: <6 seconds) (Figure 3c). However, for Tasks 3, 9, 12, 13, and 14, the time taken by the stroke patients was found to be longer (difference of medians: 14-32 seconds).

III-B MAUQ and custom questionnaire

The mean overall MAUQ score for the primary users was 1.3 (range: 1-1.8), while for the healthcare professionals it was 1.4 (range: 1-1.8). Mean MAUQ_E scores were 1.1 and 1.6 for primary users and healthcare professionals, respectively, MAUQ_I scores were 1.3 and 1.4, and MAUQ_U scores were 1.9 and 1.5. The mean scores assigned to the custom statements are listed in Table I.

TABLE I: Custom statements with mean (range) scores assigned by primary users (P, n = 4) and healthcare professionals (H, n = 5). Possible scores ranged from 1 (strongly agree) to 7 (strongly disagree).

Statement	P	H
1: The font size and the colors of the app	2.3 (1-3)	2.4 (1-4)
are adequate
2: The size of the icons is adequate	1 (1-1)	2.2 (1-4)
3: The questions asked by the virtual coach	1 (1-1)	1.8 (1-4)
are easy to understand
4: It is easy to answer the questions of the	1 (1-1)	1.4 (1-2)
virtual coach in the chat with the buttons

III-C Semi-structured interview

Eight participants specifically stated that RehabCoach is easy to use. Regarding the chat function, a participant mentioned that it humanizes the interaction. One patient mentioned that more choices for the avatar of the digital coach should be provided for increased personalization. Similarly, it was requested to enhance the chat feature so that users can initiate interactions themselves, for instance, by asking common questions such as “How are you?”. One patient mentioned that the empathy of RehabCoach should be improved, for example, by adding interactions that aim at strengthening the attachment bond even more, even though they might not be directly linked to therapy.

Regarding the chat interface, three participants mentioned that they liked the buttons with predefined answers. Still, two healthcare professionals and one primary user suggested adding more options, especially when the possible answers have opposite meanings and do not cover a broad spectrum of possible responses. For two secondary users more settings should be added to the personalized profile, especially for the chat background, the font size, choosing between capital or lowercase letters, and asking for aphasia. Furthermore, most participants generally liked the idea of having the informative videos. Two healthcare professionals pointed out that the information given should be personalized, for example according to the type of impairment.

IV DISCUSSION

Here we present the first prototype of RehabCoach, a coaching app designed to increase motivation and adherence during unsupervised therapy after stroke, and its preliminary usability testing. RehabCoach is chatbot-based and thought of as a scalable social actor to support and later interface with different devices for unsupervised rehabilitation, depending on the needs of the single patient. Despite the many existing apps for people after stroke [32], the potential of chatbot-based apps connected to rehabilitation devices still needs to be investigated. Our results show that primary users could correctly interact with the basic features of RehabCoach after only a short familiarization phase, as demonstrated by high task success and perceived usability. This is aligned with the findings of Burns et al. [32], who reviewed 49 articles related to mobile health app interventions for people after stroke, with most of the articles evaluating feasibility concluding that the tested digital interventions were feasible.

RehabCoach was perceived as user-friendly and the chat messages were easy to understand, which is a critical requirement when designing a digital health intervention for the elderly with potential cognitive impairments, as our target population. The tested tasks were representative of the expected interactions during a day of use of RehabCoach and the different features implemented in this prototype. For most of the tasks, primary users did not take longer than healthcare professionals to complete them. Two out of the five tasks that resulted in noticeably longer execution time by the primary users required typing (T3) or reading long text messages (T9), which is not surprising as literature shows that elderly read and text slower than young adults [33, 34]. This suggests that short messages and predefined answers may be preferred and more suitable for our population. However, in these tasks, the priority was not to be fast but to correctly interact with the app or interpret the information. Therefore, the average additional time needed by primary users to complete all the tasks (+111 seconds in total) is acceptable, also considering that in the intended use case, this additional time would be split over the entire day.

Overall, the MAUQ results were very positive despite the currently limited functionalities of the prototype, which supports the need for developing a more advanced version of RehabCoach in the future. The results of the semi-structured interviews will be used to define key features to implement in the next prototype. For example, while the predefined answers were appreciated as they provided an easier and faster way to answer than typing, three participants stated that the offered predefined answers should be further developed to cover a broader range of possible opinions. Indeed, for most questions we opted to provide only two possible answers, so as to reduce the number of options to read and increase usability. Issues with predefined answer options were also identified in a previous work involving stroke patients interacting with a chatbot [35].

As the objective was to evaluate as early as possible the feasibility of using the app in persons with stroke and its usability, the interaction with the app was limited to a single short session (representing an accelerated day), and users did not actually engage in the scheduled therapy sessions. This may influence the perception of the app by the participants. While the number of participants in this usability evaluation is limited, it still allowed us to collect essential preliminary data to show feasibility and to improve the app in the next design iteration, which would be tested in a more extensive study including more participants undergoing unsupervised rehabilitation sessions and interacting with the app over multiple days. This will allow defining whether the positive evaluation of this first prototype can translate into high motivation to train and adherence to unsupervised therapy. In conclusion, the positive results gained during this work underline the feasibility of using chatbot-based interventions in individuals after stroke and open the door to the use of RehabCoach as a bridging digital tool to support different devices for unsupervised therapy after stroke.

ACKNOWLEDGMENT

The authors would like to thank the participants of the usability evaluation study and the team of the Clinica Hildebrand in Brissago for their support during this work.

CONFLICT OF INTEREST

FS and TK are affiliated with the Centre for Digital Health Interventions (CDHI), a joint initiative of the Institute for Implementation Science in Health Care, University of Zurich, the Department of Management, Technology, and Economics at ETH Zurich, and the Institute of Technology Management and School of Medicine at the University of St.Gallen. CDHI is funded in part by CSS, a Swiss health insurer, MTIP, a Swiss investor, and Mavie Next, an Austrian healthcare provider. TK is also a co-founder of Pathmate Technologies, a university spin-off company that creates and delivers digital clinical pathways. However, neither CSS, MTIP, Mavie Next nor Pathmate Technologies were involved in this research.

References

[1] E. S. Lawrence et al., Estimates of the Prevalence of Acute Stroke Impairments and Disability in a Multiethnic Population, Stroke, vol. 32, no. 6, pp. 1279-1284, 2001.
[2] J. McCabe et al., Comparison of robotics, functional electrical stimulation, and motor learning methods for treatment of persistent upper extremity dysfunction after stroke: A randomized controlled trial, Arch. Phys. Med. Rehabil., vol. 96, no. 6, pp. 981-990, 2015.
[3] N. S. Ward et al., Intensive upper limb neurorehabilitation in chronic stroke: Outcomes from the Queen Square programme, J. Neurol. Neurosurg. Psychiatry, vol. 90, no. 5, pp. 498-506, 2019.
[4] T. S. Jesus et al., Human resources for health (and rehabilitation): Six Rehab-Workforce Challenges for the century, Hum. Resour. Health, vol. 15, no. 1, 2017.
[5] A. Bersano et al., Stroke care during the COVID‐19 pandemic: experience from three large European countries, European Journal of Neurology, vol. 27, no. 9, pp. 1794, 2020.
[6] D. Aguiar de Sousa, Maintaining stroke care in Europe during the COVID-19 pandemic: Results from an international survey of stroke professionals and practice recommendations from the European Stroke Organisation, European Stroke Journal, vol. 5, no. 3, pp. 230, 2020.
[7] S. J. Olney, A randomized controlled trial of supervised versus unsupervised exercise programs for ambulatory stroke survivors, Stroke, vol. 37, no. 2, pp. 476-481, 2006.
[8] D. K. Zondervan et al., Home-based hand rehabilitation after chronic stroke: Randomized, controlled single-blind trial comparing the music glove with a conventional exercise program, J. Rehabil. Res. Dev., vol. 53, no. 4, pp. 457-472, 2016.
[9] K. O. Thielbar et al., Home-based Upper Extremity Stroke Therapy Using a Multiuser Virtual Reality Environment: A Randomized Trial, Archives of Physical Medicine and Rehabilitation, vol. 101, no. 2, pp. 196-203, 2020.
[10] F. Wittmann et al., Self-directed arm therapy at home after stroke with a sensor-based virtual reality training system, Journal of NeuroEngineering and Rehabilitation, vol. 13, no. 75, 2016.
[11] R. Ranzani et al., Design, characterization and preliminary usability testing of a portable robot for unsupervised therapy of hand function, Frontiers in Mechanical Engineering, vol. 8, 2023.
[12] G. Devittori et al., Unsupervised robot-assisted rehabilitation after stroke: feasibility, effect on therapy dose, and user experience, preprint (Version 1) available at Research Square, 26 December 2023.
[13] Y. Chen et al., Home-based technologies for stroke rehabilitation: A systematic review, International Journal of Medical Informatics, vol. 123, pp. 11-22, 2019.
[14] A. Gelineau et al., Compliance with Upper Limb Home-Based Exergaming Interventions for Stroke Patients: A Narrative Review, Journal of rehabilitation medicine, vol. 54, jrm00325, 2022.
[15] S. L. Wolf et al., The HAAPI (Home Arm Assistance Progression Initiative) Trial: A Novel Robotics Delivery Approach in Stroke Rehabilitation, Neurorehabilitation and Neural Repair, vol. 29, no. 10, pp. 958-968, 2015.
[16] R. Karamians et al., Effectiveness of Virtual Reality- and Gaming-Based Interventions for Upper Extremity Rehabilitation Poststroke: A Meta-analysis, Archives of Physical Medicine and Rehabilitation, vol. 101, no. 5, pp. 885-896, 2020.
[17] S. Lang et al., Do digital interventions increase adherence to home exercise rehabilitation? A systematic review of randomised controlled trials, Arch. Physiother., vol. 12, no. 1, pp. 1-12, 2022.
[18] Center for Digital Health Interventions @ ETH Zurich & University of St. Gallen, MobileCoach, 2022. https://www.mobile-coach.eu/.
[19] A. Filler et al., MobileCoach: A novel open source platform for the design of evidence-based, scalable and low-cost behavioral health interventions: Overview and preliminary evaluation in the public health context, in 14th annual Wireless Telecommunications Symposium USA, New York 2015, IEEE Computer Society, pp. 1-6.
[20] T. Kowatsch et al., Design and Evaluation of a Mobile Chat App for the Open Source Behavioral Health Intervention Platform MobileCoach, in Designing the Digital Transformation. Lecture Notes in Computer Science (LNCS10243), Springer: Berlin; Germany, 2017, pp. 485-489.
[21] T. Bickmore and R. Picard, Establishing and maintaining long-term human-computer relationships, ACM Transactions on Computer-Human Interaction (TOCHI), vol. 12, no. 2, pp. 293-327, 2005.
[22] M. Jamieson et al., ForgetMeNot: Active reminder entry support for adults with acquired brain injury, in Conf Hum Factors Comput Syst - Proc 2017, May 2015, pp. 6012-6023.
[23] N. Micallef et al., Time to Exercise! An Aide-Memoire Stroke App for Post-Stroke Arm Rehabilitation, in Proc 18th Int Conf Human-Computer Interact with Mob Devices Serv MobileHCI 2016, 2016, pp. 112-123.
[24] T. Kowatsch et al., Conversational Agents as Mediating Social Actors in Chronic Disease Management Involving Health Care Professionals, Patients, and Family Members: Multisite Single-Arm Feasibility Study, J Med Internet Res, vol. 23, no. 2, e25060, 2021.
[25] J. N. Kramer et al., Which Components of a Smartphone Walking App Help Users to Reach Personalized Step Goals? Results From an Optimization Trial, Annals of Behavioral Medicine, vol. 54, no. 7, pp. 518-528, 2020.
[26] J. Van Der Linden et al., A blended design approach for pervasive healthcare: Bringing together users, experts and technology, Health Informatics J., vol. 18, no. 3, pp. 212-218, 2012.
[27] K. Oyake et al., Motivational Strategies for Stroke Rehabilitation: A Delphi Study, Archives of Physical Medicine and Rehabilitation, vol. 101, no. 11, pp. 1929-1936, 2020.
[28] K. Faiz et al., Patient knowledge on stroke risk factors, symptoms and treatment options, Vasc Health Risk Manag., vol. 14, pp. 37-40, 2018.
[29] A. Croquelois and J. Bogousslavsky, Risk awareness and knowledge of patients with stroke: results of a questionnaire survey 3 months after stroke, J Neurol Neurosurg Psychiatry, vol. 77, pp. 726-728, 2006.
[30] D. Charles et al., Virtual Reality Design for Stroke Rehabilitation, in Advances in Experimental Medicine and Biology, Springer, 2020, pp. 53-87.
[31] L. Zhou et al., The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study, JMIR Mhealth Uhealth, vol. 7, no. 4, e11500, 2019.
[32] S. Burns et al., mHealth Intervention Applications for Adults Living With the Effects of Stroke: A Scoping Review, Archives of Rehabilitation Research and Clinical Translation, vol. 3, no. 1, 100095, 2021.
[33] R. Kliegl et al., Length, frequency, and predictability effects of words on eye movements in reading, European Journal of Cognitive Psychology, vol. 16, no. 1-2, pp. 262-284, 2004.
[34] T. Krasovsky et al., Older Adults Pay an Additional Cost When Texting and Walking: Effects of Age, Environment, and Use of Mixed Reality on Dual-Task Performance, Physical Therapy, vol. 98, no. 7, 2018.
[35] K. Epalte et al., Patient experience using digital therapy “Vigo” for stroke patient recovery: a qualitative descriptive study, Disability and Rehabilitation: Assistive Technology, vol. 18, no. 2, pp. 175-184, 2020.