BalanceVR: Balance Training to Increase Tolerance to Cybersickness in Immersive Virtual Reality

The human body as an articulated and complex skeleton structure is inherently mechanically unstable PAI1997347 ; LEE2006569 . Maintaining balance (around the center of mass) is a complex process that involves multiple systems in the human body. The vestibular, proprioceptive, and visual channels are used to detect and gather balance/pose-related information and the brain integrates them to coordinate and generate the motor responses and establish the center of pressure through the muscles and joints with constant adjustments to counteract the external perturbation to the body PETERKA201827 ; Lee1998DeterminantsOT ; LAFOND20041421 .

There are numerous physical training routines to improve one’s balance by strengthening and improving the capabilities of the aforementioned subsystems Brachman . Conventional means of balance training usually involve following paper or live instructions from the second/third person point of view. Virtual reality (VR) can be an effective media offering the first-person perspective and sense of personal space in enhancing the understanding of the various training poses and work-outs 10.1145/3198910.3198917 ; Pastel . Gamification can further provide the motivation and impetus to facilitate the training process 10.1145/3562939.3567818 ; 10.1145/2909132.2909287 . However, the effect of balance training (VR-based or not) on cybersickness has not been investigated much despite the fact that they are often touted to be closely related Riccio ; Haran ; 10.1093/ptj/79.10.949 .

Cybersickness, also known as VR sickness, refers to the unpleasant symptoms when using immersive VR simulators, especially with navigational content. Typical symptoms include disorientation, headache, nausea, and ocular strains laviola2000discussion ; kennedy1993simulator . The leading explanation for cybersickness is the “sensory mismatch theory”, which attributes cybersickness to the conflicting user motion information as interpreted by between the visual and vestibular senses laviola2000discussion ; Rebenitsch . That is, the aforementioned unpleasant symptoms arise when the virtual/visual motion is perceived by the human’s visual system while the vestibular senses detect no physical motion. Note that the visual and vestibular systems are neurally coupled GRSSER1972573 .

To combat these symptoms, several studies have focused on reducing the amount of or neutralizing the visual motion information to minimize the sensory mismatch fernandes2016combating ; park2022mixing ; Keshavarz . For instance, Fernandes et al. fernandes2016combating developed a dynamic size-shifting field-of-view (FOV) in response to the speed/angular velocity of users or content. When the user motion accelerates, the FOV is reduced, which in turn reduces the extent of the visual stimulation and ultimately the sickness. In a similar vein, blurring the peripheral visual field has been proposed to minimize the visual stimulation budhiraja2017rotation ; Lin2020 ; Caputo2021 . Park et al. park2022mixing ; customizedVR have proposed neutralizing the visual motion stimuli by simultaneously presenting the reverse optical flow.

Another popular theory is the rest frame theory, which points to the absence of reference object(s) (objects in the VR content that are not moving with respect to the user), called the rest frame Harm . The rest frame is thought to help the user maintain one’s balance and be aware of the ground (or gravity) direction hemmerich2020visually ; wienrich2018virtual ; Harm . One interesting remedy to cybersickness is the inclusion of the virtual nose, which can be considered as a rest frame object wienrich2018virtual ; wittinghill2015 ; cao2017effect .

Alternatively, a potential strategy for addressing cybersickness might involve methods to alleviate the immediate symptoms and enhance users’ physical well-being, rather than directly targeting the root cause. These can include e.g. supplying a fresh breeze with a fan  igoshina2022comparing ; reliefVR or providing pleasant music or calming aural feedback keshavarz2014pleasant ; kourtesis2023cybersickness ; restingVRPoster . These measures can be regarded cognitive distraction as a way to reduce the cybersickness by preventing users from focusing on the sickness-inducing VR content kourtesis2023cybersickness .

One newer hypothesis, although not fullheartedly accepted in the research community, for the cause of cybersickness is the postural instability theory Riccio ; Ruixuan . Accordingly, postural instability, the lack of ability to maintain balance due to external factors (such as being subjected to a new unfamiliar, provocative, and thus challenging situation) can induce the sickness. Note that this does not preclude the fact that imbalance is one typical after-effect of the sickness as well. This is based on the various studies that have observed a strong correlation between one’s balancing ability (before) and the extent of the motion sickness (after) Riccio ; Ruixuan ; Smart . This theory is also in line with the rest frame theory - i.e., the lack of the rest frame object (indicating the direction of gravity and help one maintain balance) could be seen as a provocative situation for the user hemmerich2020visually ; wienrich2018virtual ; Harm .

Based on all these studies, one can posit that balance training while navigating in the immersive VR would make the user to be even more unstable and exacerbate the extent of the cybersickness. In turn, this could make the balance training itself even harder imaizumi2020virtual ; horlings2009influence . Nevertheless, given that the user can endure through the training, its effect can eventually ease and break this vicious cycle. We can further hypothesize that the immersive feedback will be an important factor, as maintaining and training for balance involves the visual channel and spatial awareness, which non-immersive and 2D-oriented media is difficult to provide fully.

On a related note, the length of time exposed to a virtual environment is known to affect the severity of cybersickness Duzmanska . Stanney et al. Stanney has found high correlations between exposure time and cybersickness, with longer exposure times increasing the risk of cybersickness. On the other hand, there is also the opposite view that people may build up a resistance or adapt over time (or by frequent exposures) to cybersickness Duzmanska . Thus, the exact relationship between extended exposure and cybersickness symptoms is not firmly established.

To our knowledge, no prior work on applying balance training as a way to train for tolerance to cybersickness has been reported. Note that similarly to any external stimulation, the mere repeated and prolonged exposure to VR in itself can certainly have the effect of insensitization or habituation to the cybersickness palmisano2022reductions . However, we expect it to be a relatively time-consuming method and quickly receding in its effect (compared to active training), and little is known about whether there is any transfer effect to other contents (for which the user was not exposed to) palmisano2022reductions ; duzmanska2018can ; adhanom2022vr ; smither2008reducing .

The main purpose of the experimental study is to confirm the effect whether the learned balance ability has an impact on developing one’s tolerance to cybersickness. The balance training may occur in either a non-immersive environment or a VR environment, using sickness-eliciting contents (e.g., navigation). We hypothesize that given the same content, the effects of the balance training on tolerance to cybersickness will be stronger if the training occurred in the VR environment compared to using the non-immersive environment (even if the given content may be different from the one used for training). On the other hand, to single out the effect, if any, of the balance training to cybersickness tolerance, from that of by the media type, the training method by mere extended exposure to the same sickness-eliciting VR contents must be tested too. Humans can become habituated, desensitized, and tolerant to cybersickness after long exposure to various stimuli by VR Fransson ; Duzmanska . Thus, the experiment was designed as a two-factor repeated measure between subjects; the first factor being the training method in three levels (as shown in Figure 1):

• •

  2DT: watching a sickness-eliciting navigation content on a 2D projection display while carrying out a balance training routine.

• •

  VRT: watching a sickness-eliciting navigation content using a VR headset while carrying out a balance training routine.

• •

  VRO: only exposure/just watching a sickness-eliciting navigation content using a VR headset, but without any balance training.

To avoid any learning effect with regard to the contents used, a between-subject experiment was chosen. As the effects of training may take time, the experiment was conducted over 2 weeks, but in two separate weekly segments: Experiment Week 1 (EW1) and 2 (EW2). Note that the same subject groups of EW1 continued to participate in EW2. Thus, the time (days) constituted the second factor. Two weeks of balance training was deemed sufficient because marked progress is usually attainable in that time frame RASOOL2007177 ; SZCZERBIK2021513 . EW1 proceeded over 4 days, and the subjects were trained while watching the VR content which induced only a relatively moderate/less degree of sickness as to start the overall training gently (not too abruptly). After a three-day break, EW2 was conducted with a duration of 5 days. Due to the possible learning effect and getting accustomed to the same content after repeated exposures, a new and more dynamic content with a relatively higher degree of sickness was used (see Figure 3). Although it is difficult to exactly quantify the difference in the sickness levels, from Figure 4 which shows the navigation motion profiles of the respective content, it is reasonably clear that the Week 2 content would induce a much more severe level of cybersickness. EW2 was administered three days after the 4-day EW1 with the treatment-wise same subjects from EW1, thus we postulate that that subjects still were possibly affected by the training given in EW1.

In summary, there were two mixed model and longitudinal experiments; each designed as two factors, 3 x 2, repeated measure between subjects. Even though experimental tasks were carried out and dependent variables measured every day during the 4-day/5-day periods for EW1/EW2, we focus only on the difference between the first and last days and data analyzed accordingly (making it a two-level study for the second factor). Figure 5 shows the overall experimental process. The hypotheses regarding the outcome of the experiment can be summarized as follows:

• •

  H1: There will be a balance training effect (i.e., significantly getting better in time) in both immersive VR (VRT) and 2D-based environments (2DT).

• •

  H2: Balance training improves tolerance to cybersickness. If so, this partly serves as evidence for the posture instability theory where imbalance can be considered the cause of cybersickness.

• •

  H3: The training effect for balancing and cybersickness will be greater with the use of immersive VR (VRT) than with the 2D environment (2DT).

• •

  H4: The training effect for cybersickness through balance training (with VR and/or non-VR) will be greater than just the extended exposure to the same content.

In both EW1 and EW2, except for the training contents used, the experimental task and procedure were the same. For 2DT and VRT, part of the experimental task was to follow a simple balance training routine called the “one leg stand” (also known as the Flamingo test Uzunkulaoglu ; Marcori ). In 2DT, the balance training was administered while watching the navigational content on the projection display (60 inches viewed from 1.5 m away), and in VRT, watching the same content through the VR headset (Oculus Quest 2 VR headset with a FOV of 104 $x$ 98 degrees). During the 3-minute experience, the balance training routine is as follows: ready/rest (30s’) - training (30s’) - rest (30s’) - training (30s’) - rest (30s’) - training (30s’). The instructions are presented through a distinct and visible user interface in the VR/2D projection system. The instructions were provided by the distinct visible UI of the VR/2D-projection system. For VRO, no balance training occurred - the subject just watched the same VR content with the VR headset standing on two feet.

As already indicated, two contents of different sickness levels and themes were used - the lesser sickness eliciting one in EW1 (flying through the forest trail) and more in EW2 (wild roller coaster ride). Figure 2 (a), (b), and Figure 3 show the example scenes from the respective contents. Experimenting with the new more difficult (sickening) content also provided the opportunity to examine the user behavior and performance after a week of training.

The navigation path contained several types of motion - forward translation and pitch/yaw, rotation/turning in varied speed and acceleration, as shown in Figure 3, - mixed up for each content to be clearly differentiated in their respective sickness level (also informally tested with no subjects). In addition to using different contents between EW1 and EW2 to prevent the learning effect, similar provisions were made within the same content within EW1 and EW2 as well. Each content was presented in slightly different versions by varying the navigation path/motion profiles and surrounding environment objects (while maintaining the same sickness level) between each day.

The training proceeded 2 times a day for four days in EW1 and likewise for five days in EW2. The subject was free to put one’s foot back down anytime if felt to be in danger of falling down (or for any reason e.g., whether not able to maintain balance or due to too much sickness) but was asked to resume and continue in one’s best way. The experiment helper stood by to prevent the subject from completely falling down. The subject was also free to stop the experiment at any time, although there was no such case.

The dependent variables of main interest were two: changes in the balance performance and cybersickness scores over time. Each of these aspects was measured in several ways.

The balancing performance was measured quantitatively by (1) the number of times the subject put back one’s foot down during the training process, and (2) computing the extent of the deviation of the body from the reference center of mass (e.g., when standing still). The former was manually counted, while the latter, was obtained off-line by analyzing the subject’s 2D pose data extracted from the recorded video using the PoseNet Papandreou_2017_CVPR ; Papandreou . In particular, the variation of the midpoint of the screen space locations of the right and left hips were used to estimate this measure. We omit further implementation details for lack of space.

In addition, to assess whether the balancing performance improved regardless of the given training content, we measured the subject’s performances of the “one leg stand with eyes closed” Bohannon on the first and last days of each week (see Figure 5).

As for the overall cybersickness level assessment, Kennedy’s Simulation Sickness Questionnaire (SSQ) score was used kennedy1993simulator . However, since the SSQ only asks for the existence of certain symptoms, it is not possible to assess their probable cause for our experiment - e.g., from the visual motion or balancing act. Thus, in addition to the original SSQ (herein referred to as the “Original”), two revised versions, called the “Visual” and “Balance”, were made and used. Questions in the revised versions ask of the same symptoms, but also of what the subjects thought the source might be, i.e., from the visual motion or the one-leg stand balancing act. The whole questionnaire comprised 48 questions (see Appendix).

Lastly, to assess whether the cybersickness tolerance improved regardless of the given content (i.e., transfer effect to another VR content), we measured the subject’s sickness levels using a completely different (from the ones used in the main experiment) sickness-inducing content; tested with transfer VR content 1 (rollercoaster ride¹¹1 The YouTube 360-degree video link is available at https://www.youtube.com/watch?v=eHAu8BV85vE., and transfer VR content 2 (space exploration) before and after EW2. [The duration of both content was 3 minutes, the same as the experimental content. Moreover, subjects in all conditions experienced the transfer content while standing and wearing a VR headset without any balance training.] We reemphasize that the rollercoaster content used to explore the transfer effect was completely different from the one used for training in EW2. These transfer effect test contents are illustrated in Figure 6.

Subjects were initially recruited through the university’s online community. The first round of subjects was surveyed for their self-reported sensitivity to motion sickness using the MSSQ-short golding2006predicting ; NESBITT20171 and familiarity or prior experiences in using the VR system. We notified the potential subjects of the need to carry out balance training (one leg stand) for about 10-15 minutes per day for two weeks and asked them to excuse themselves if they deemed it to be beyond their physical capabilities. Subjects in the extreme ends in terms of their reported sensitivity were also excluded, as our study targeted subjects in the middle of the sensitivity spectrum.

Fifteen final subjects were selected and placed in the three subject groups (5 each) for 2DT, VRO, and VRT such that their MSSQ score variations were similar and within an acceptable range (all male, aged 19 to 33, mean = 25.6, SD = 2.19). All participants had at least some experience in using VR applications (mostly game playing and video watching) but did not have any prior balance training experience. The subjects were paid 16 USD per hour for their participation (a total of about $ 120 for the whole two weeks). All 15 subjects managed to finish the experiment in two weeks without giving up in the middle.

The subjects first filled out the consent agreement form, were briefed about the [procedure of the experiment], and explained the experimental tasks. Five to ten minutes were given for the subjects to get oneself familiarized with the balancing task while watching the content through the monitor or the headset. In particular, the subjects were given detailed instructions on how to carefully respond to the three types of SSQs and to deeply think about the probable causes of the symptoms the best they could. The helper assisted the subject to position oneself in front of the monitor on the floor (with cushioned walls) or donning and adjusting the headset. The helper also stood by to prevent the subject from falling down.

On each day, the subjects for 2DT and VRT alternated between a 30-second rest/preparation period, followed by a 30-second balance training procedure, repeating this sequence three times in total. The subjects selected which foot to use to stand or lift on their own. This protocol was designed considering that the similar Y-balance (one leg with eyes closed) test typically lasted around 30 seconds on average ybalance , and our own pilot test (with four males) indicated that exceeding one minute often led to muscle strain. Although the experiments were run over two weeks and sufficient rests were taken between the treatments, minimizing subject fatigue and ensuring the best physical condition was deemed important to derive the best and credible results.

Subjects of VRO simply watched the navigation content one time for 3 minutes in a normal standing pose. After the respective treatments, subjects rested and filled out the sickness survey. Subjects were free to stop the experiment for any reason without any financial penalty. After experiencing all treatments, informal post-briefings were taken. The experiment was approved by the Institutional Review Board (IRB No. 2023-0143-01).

The experimental contents were developed using the Unity 3D game engine, specifically version 2021.3.12f, and run and displayed with the Oculus Quest 2 VR headset.

The primary focus of our study was to alleviate cybersickness caused by visual mismatch through balance training. However, we considered the possibility that physical discomfort or hardship from the balancing act could be similar to the many symptoms of the cybersickness as assessed by the SSQ (e.g. “disorientation” from trying to stand on one foot). Therefore, we administered two additional revised versions of the “Original” SSQ: namely, “Visual”, and “Balance” - for responders to distinguish between and report whether the cybersickness-like symptoms were induced by the visual mismatch and/or by the balance training. Nevertheless, we do acknowledge the subjects’ inherent difficulty (despite the survey’s kind explanation and clarification) to objectively and correctly judging the sources of the symptoms, and also of the possible interaction between these two mixed factors. Note that the subject was free to attribute a given symptom to both visual stimulation and balancing exercise. On the other hand, it is not common to see people seriously suffer from sickness-like symptoms from just doing balance exercises. Considering all these, our analysis focused on investigating the effects on the Visual SSQ scores.

Figure 7 (and also partly for just the first and last days in Table 2) illustrates the trends of the average Visual SSQ scores over the 4-day/5-day periods of EW1 and EW2 for VRT, VRO, and 2DT. The detailed statistical figures, including the pairwise comparisons, are given in Figure 8, Table 3, Table 4, and Table 5.

To relate the potential effect of balance training to increasing tolerance for cybersickness, various measures were taken over the course of the experiments (excluding the VRO subjects), such as (1) the duration of subjects’ balance maintenance along with (2) the occurrences of balance failures (instances of subjects having to place their one foot back on the ground to avoid falling to the ground) and (3) the variability in their centers of mass. The former was measured before and after the training sessions, while the latter two during.

The comparative analysis with respect to the between-subject factor of the training method is not presented for the similar reason of the different subject groups of limited number.

To further investigate the relationship between balance performance and the reduction in “Visual” sickness, a correlation analysis was conducted using the Pearson correlation coefficient test. The following null hypothesis value of correlations were made: (1) “Visual” sickness scores and balance maintenance time would be negatively correlated (-1); (2) “Visual” sickness and the number of balance fails would be positively correlated (+1); and (3) “Visual” sickness and center of mass variability would be positively correlated. The results are given in Table 6 and they are mostly consistent with our hypotheses (e.g., H2 and H3).

Statistically significant correlations were found between the improvements in the number of balance failures and center of mass variability, respectively (either by VRT or 2DT) with the “Visual” sickness scores over EW1 and EW2. [VRT showed higher correlation coefficients than the 2DT in two measures] (i.e., No. of balance failures: $r=0.612>0.267$; center of mass variability: $r=0.305>0.269$). For the balance maintenance time, there was no significant correlation found, however, it is worth noting that while VRT showed a negative correlation as expected ($r=-0.295$), 2DT only showed near zero correlation ($r=0.001$).

To summarize, as the balance performance improves, there is an increase in tolerance to the “Visual” sickness in both immersive (VRT) and non-immersive (2DT) environments. Furthermore, the correlation values indicate that the training effect in the immersive VR environment (VRT) was more pronounced than that in the non-VR (2DT).

The true test for any effect of the balance training on cybersickness would be shown by observing how the balance-trained subjects perform on completely different VR contents, namely, the “transfer” contents that were totally different from the test content (see Figure 6). The assessments were made two times, before and after EW1 using transfer VR content 1 (Rollercoaster ride), and before and after EW2 using transfer VR content 2 (Space navigation). The sickness levels were measured in the same way as those conducted during the training sessions of EW1 and EW2. The results are summarized in Figure 8, Table 3, Table 4, and Table 5.

The statistical analysis (see Table 5 and Figure 8 (d)) found a significant decrease in the cybersickness only in the VRT condition ($p<$ .05) before and after EW2. No other condition exhibited any statistically significant reduction (i.e., VRO, 2DT). This indicates the training transfer effect of balance training in VR for cybersickness tolerance, consistent with the observation that the VRO group, who received no training in EW1, showed much higher levels of sickness in the early stages of EW2 (when switched to the new training content) than VRT. Furthermore, it confirms our hypotheses (H3 and H4) that immersive media (VRT) with balance training is a more effective method than simply being exposed to the VR content for an equal amount of time.

As discussed in length in Section 2.2, there have been several theories as to why and how cybersickness occurs, such as the sensory mismatch laviola2000discussion ; Rebenitsch , lack of the rest frame Harm , and postural instability Riccio ; Ruixuan ; Smart . All such factors are plausible and debatable at the same time. While the proposal of immersive balance training for developing tolerance to cybersickness hinges on the postural instability in particular, it does not discount the effect of those other factors nor is it in conflict with them.

Our experiments have shown significant reductions in cybersickness symptoms in all the treatments. This trend was also observed, albeit to a lesser extent, in the VRO treatment that did not include training. These results indicate that repeated exposure to VR contents reduced sickness adhanom2022vr ; palmisano2022reductions , and it is difficult to deny that this effect may have influenced other conditions as well (in terms of what contributed to the reduction). On the other side, balance training may have reduced the sickness acting as cognitive distraction kourtesis2023cybersickness ; venkatakrishnan2023cog02 . However, cognitive distraction alone is difficult to explain the training transfer effect. The same is true as for the mere exposure to a particular VR content.

Palmisano et al. palmisano2022reductions have shown that repeated exposure to VR contents could significantly improve cybersickness. However, this improvement was observed only for the very content the subjects were exposed to, and it was not shown whether the effect extended to other VR contents. On the other hand, our experiment confirmed the transfer effect of the balance training to a completely different content. Only the VRT group, which engaged in the immersive balance training, significantly saw the reduction in cybersickness in the transfer contents (see Figure 8). This is the critical finding that sets forth the training (and the improved physical/mental capability) as the main culprit to the sickness reduction - more so than the exposure itself or distraction. This also signifies the potential practicality of the approach.

One representative experiment in the attempt to validate the postural instability theory by Riccio showed the decreasing sickness levels in a provocative situation by the subject making a more widened and stable stance Dennison . In contrast, the experiment in this work went to other ways, where the subjects were purposely situated to be unstable (one leg stand), leading to a possible expectation that the “sickness” should increase according to the same theory. One important difference, however, is that the subjects were also instructed to “learn” and train as to how to maintain the balance. Indeed, instead of the increased level of sickness, our results clearly show the reduction and even the transfer effects, singling out the very effect of the “training”.

Interestingly, according to Menshikova et al. Menshikova , when compared figure skaters, soccer players, and wushu fighters, figure skaters showed the most resilience to cybersickness. Thus, innate or learned balancing capability seems related to tolerance to cybersickness. Ritter et al. Ritter studied the VR-based (safe) training of balance beam performance with gymnastics beginners. Among others, the work showed that the subjects generally performed worse in VR than in the real world. This indirectly suggests that, for cybersickness improvement by balance training, the training environment (i.e., VR) will be important. Likewise, our results point similarly in the same direction, namely, VRT being more effective than 2DT and even VRO.

As for 2DT, the level of the sickness arising from the visual motion must have been less so to begin with compared to that by VR. The visual content has a substantially smaller field of view (approximately VRT: 100^∘ vs. 2DT: 60^∘) outside which objects possibly acting as reference objects are visible (e.g., walls, floor). These are aspects that can diminish the training effect in 2DT as well. On a similar note, training for a spatial task (which the balancing or even withstanding cybersickness from visual motion could be examples of) on the 2D oriented desktop environment has shown a negative transfer effect to the corresponding 3D VR environment Pausch .

Even though our study seems to show that extended exposure to VR does have an effect on building tolerance to cybersickness, in relation to the related work (see Section 2), its firm establishment is still debatable. Even if it was, we believe that its effect is weaker and not so long-lasting than that of balance training. In balance training, the user makes a conscious effort to encode the relevant information into one’s proprioceptive and muscular control system. How long the training effect can be sustained would be a topic of future research.

Our study is limited in several aspects. Cybersickness is a truly multifactorial issue, including gender, age, the nature of the tasks undertaken, type of feedback, and multimodality feng2016tactile ; peng2020walkingvibe and the types of devices used kourtesis2023cybersickness ; kim2008application ; chang2020virtual . Our work only investigated one such probable factor, i.e., balancing capability. While most factors mentioned above are known to influence the level of cybersickness in one way or another, the variance from the individual difference is relatively large tian2022review ; chang2020virtual ; howard2021meta . Balancing capability can be considered a more predictable control factor ARCIONI20193 ; doi:10.1080/10447318.2017.1286767 . Training for it is also expected to be much less dependent on the immersive training environment (content genre). Note that the training process can be further expedited by employing multimodal feedback, guidance features, and gamification 10.1145/3562939.3567818 ; Juras ; Prasertsakul . Such are subjects of future research topics.

Another limitation is the number of experimental subjects in each of the training method (between-subject) groups. The subjects were also confined to a particular group, i.e., young adult males. It is too early to generalize our claims to other subject populations. A future larger-scale experiment should not only accommodate a larger number of subjects but also employ a variety of sickness-eliciting or “provocative” contents as well. As there may be more fitting and proper balance training routines, these new VR contents may involve interaction techniques to guide such balancing acts more effectively as demonstrated in Yang .