Balancing Exploration and Cybersickness: Investigating Curiosity-Driven Behavior in Virtual Environments

User behavior in virtual environments varies widely due to individual differences. VR provides a controlled platform for experiments, where participants’ movements can be continuously recorded, enabling the collection of validated datasets for comprehensive analysis [17]. These datasets are instrumental in identifying factors that influence user behavior during virtual navigation.

For example, visual gaze analysis has been used to replace traditional controllers for object selection, offering valuable insights into interaction dynamics [18]. Similarly, head orientation is often used to estimate gaze direction in social scenarios, and combined changes in head and hand positions can reveal distinct behavioral patterns [17]. Proxemic information, derived from user positioning, is critical for quantifying behaviors such as social approach and avoidance [19]. For instance, Won et al. [20] leveraged head orientation data to analyze scanning behavior in a virtual classroom, which could indicate student anxiety levels, while Gillath et al. [21] used proxemic patterns to identify prosocial approach behaviors.

Comparisons between VR and real-world proxemic patterns have revealed both similarities and differences. Bailenson et al.[22] reported that proxemic behaviors in VR often mirror those in physical settings. Additionally, walking speeds in virtual environments generally align with real-world speeds [23], although Iryo-Asano et al.[24] found that maximum walking speeds tend to be lower in VR. Notably, Wei et al. [25] reported that users with greater technological experience exhibit higher immersion and confidence, leading to more effective navigation behaviors.

Curiosity, often defined as a desire for information gain, drives exploratory actions that enhance knowledge about the environment [26, 27, 28]. It plays a pivotal role in decision-making, particularly in immersive environments requiring active exploration, such as VR [7]. In VR settings, curiosity has been shown to motivate users to engage more deeply with their surroundings, fostering active exploration.

Empirical studies shed light on the neural and behavioral dynamics of curiosity. For instance, Jepma et al.[29] demonstrated that curiosity activates brain regions associated with conflict and arousal, while resolving curiosity triggers reward-related neural pathways. Over time, the impact of curiosity on decision-making evolves, as prolonged information gaps can diminish users’ anticipation and reduce their exploratory drive [30]. Despite its potential to introduce discomfort in uncertain situations, individuals are naturally drawn to the unknown, as highlighted by [31] and [30].

To model and investigate curiosity, various methodologies have been employed [15, 30]. The free energy principle (FEP), introduced by Karl Friston under the Bayesian brain hypothesis [32, 33, 34], frames decision-making as the optimization of both reward-seeking and curiosity-driven behaviors. While FEP provides a framework for understanding these dynamics, a limitation lies in the assumption that curiosity and rewards are weighted equally, which does not fully reflect real-world behavior. For example, Konaka and Naoki [35] noted that traditional FEP models assume constant curiosity, neglecting the dynamic nature of user curiosity, which fluctuates over time. Similarly, Millidge et al.[36] criticized the oversimplification of curiosity in bi-choice tasks, where the interplay between curiosity and rewards is more complex and changes based on context.

Machine learning (ML) and reinforcement learning (RL) offer an alternative approach to modeling curiosity [37, 38]. While RL focuses on maximizing future rewards through a series of actions, it also distinguishes between extrinsic rewards (externally set) and intrinsic rewards (self-generated by the agent). Schmidhuber [38] introduced the idea of curiosity as an intrinsic reward, driving exploration even in the absence of external incentives. RL-based models of curiosity-driven exploration are typically divided into novelty-based and prediction-error algorithms. In novelty-based models, curiosity motivates the brain to explore new, unfamiliar tasks, while prediction-error models suggest that curiosity is driven by the need to improve predictions about the environment [38]. Despite the usefulness of these models, they fall short of capturing the complexity of user curiosity in immersive VR settings, where exploration is often intentional and guided by the user’s curiosity, not a passive, random process as often modeled in RL [39].

While curiosity can enhance engagement and exploration, cybersickness poses significant challenges to user experience, often altering behaviors and diminishing immersion. Individual characteristics such as age, gender, health, and VR experience influence the susceptibility and severity of cybersickness [8, 40]. For example, Kolasinski et al. [41] reported that younger users, particularly those aged 2–12, are more prone to cybersickness, while susceptibility decreases with age. Some studies suggest gender differences, with females potentially being more susceptible than males [42], though others have contested this [43].

Experience with VR can mitigate some effects of cybersickness. For instance, gamers tend to exhibit better spatial perception and fewer sickness symptoms due to their familiarity with rapid virtual movements [44, 45]. However, cybersickness remains a critical factor in shaping user behavior and overall satisfaction. Wang et al. [46] found that cybersickness affects walking patterns during navigation, reducing users’ sense of presence. Similarly, Gabel et al. [47] demonstrated that cybersickness could impair performance in tasks such as text reading, negatively impacting retention and comprehension in educational contexts.

Quantitative models of cybersickness have made strides in predicting its onset but often fail to account for the diversity of individual user profiles, limiting their applicability [48, 49]. Understanding the interplay between curiosity and cybersickness offers valuable insights into designing VR systems that balance user engagement with comfort. By identifying patterns in user behavior, designers can mitigate the adverse effects of cybersickness while enhancing exploratory freedom and interaction quality.

The ReCU model was designed to replicate the user’s decision-making process. It assumes that curiosity and reward are factors that could influence decision-making, where curiosity is one of the reasons a user makes irrational decisions, and reward refers to the credit received after choosing one action. In this way, we could discover how VR users decide their navigation speed in an immersive environment, more specifically, balance the desire to explore the VR environment and the severity of cybersickness. The replication process involves two steps. Initially, the VR user adjusts the reward probability for each action based on their observation. Following this, the VR user decides based on reward probability and curiosity. According to the ReCU model, the VR user seeks to maximize their expected net utility, which is the sum of the expected reward and information gain. The expected net utility at trial $t$ for choosing action $a_{t+1}$ is determined by

where $c_{t}$ denotes the curiosity level at trial $t$ and $a_{t}$ denotes the action selected at trial $t$. Curiosity is modeled as a factor that could influence the participant to make irrational decisions and is assumed to fluctuate over time. In our context, reward refers to the presence of cybersickness, which can be seen as a negative reward. In addition, the action selection $a_{t}$ follows a sigmoid function

where $\beta$ controls the action selection’s randomness and $\Delta U_{t}=U_{t}(a_{t+1})-U_{t}(a^{\prime}_{t+1})$.

The free energy principle (FEP) posits that any system aims to minimize the discrepancy between predictions and actual observations through active inference [33]. This principle also suggests that systems seek to minimize free energy, which serves as an upper bound for surprise. In the model presented by [35], the primary objective is to maximize expected net utility, which corresponds to the negative of expected free energy. A detailed formulation of expected net utility can be found in Appendix A.

To decode the decision-making process of VR users, we employed the inverse Free Energy Principle (iFEP) method, which analyzes behavioral data, including user actions and observations. This decoding process quantifies internal states, such as curiosity levels, recognition of reward probabilities for various options, and confidence in those estimates. The iFEP method operates through a repetitive cycle of predicting internal states followed by corrections, necessitating prolonged trials of behavioral data for the particle filter to converge effectively [35].

User behavioral data at each trial during the navigation task is essential for modeling with the iFEP method. Such data shall include information on the speed of movement in the environment generated through the user pushing the VR handle, the actions taken, and the severity of any cybersickness experienced during the navigation task. By collecting this information, we can more accurately analyze user behavior.

A cohort of eleven local inhabitants was invited to perform a navigation task in a VR environment ¹¹1https://github.com/coreturn/CybersicknessDataset. These volunteers, averaging 24.8 years of age (SD = 9.6), included 3 females. Before the study, a preliminary health and gaming/VR familiarity survey was administered, confirming the absence of any health-related impediments to the experiment’s integrity. The experiment also received IRB approval, and informed consent was obtained from all participants before the study.

Each participant was asked to undertake the navigation task several times with different speeds across separate days to create a dataset that included more individual behavior information. Each task lasts 4 minutes, so the total duration for participants who perform four navigation tasks is 16 minutes. At the end of the experiment, subjects received gifts in recognition of their contribution. Figure 2 gives the timeline of the experiment procedure

The experiment procedure is as follows:

• •

  The participants were given a brief introduction to how to use the HTC Vive Pro handle controllers to navigate in the virtual environment. Also, they can choose to terminate the experiment early if discomfort occurs.

• •

  The participants wear an HTC Vive Pro head-mounted display on their hand (to display the virtual environment) and an Empatica E4 wristband on their arm (to collect the EDA signal at a frequency of 4Hz).

• •

  The participants enter the virtual environment and navigate using the HTC Vive Pro handle controller touchpad along the highlighted path illustrated in Figure 1. Head-tracking (head position and rotation), motion (speed and rotation), and biosignal (EDA, blood volume pulse, temperature, and heart rate) data were collected during the entire process.

• •

  The participants completed the SSQ both before and after the navigation task. The difference between post-exposure and pre-exposure contributes to the SSQ score:

  $$SSQ=SSQ_{post}-SSQ_{pre}.$$

  The classification of the severity of cybersickness based on the SSQ score is as follows [49]:

  $$SSQ=\left\{\begin{array}[]{lcl}\mathrm{Negligible}&,\mathrm{if}&0\leq SSQ\leq 5\\ \mathrm{Low}&,\mathrm{if}&5<SSQ\leq 20\\ \mathrm{Moderate}&,\mathrm{if}&20<SSQ\leq 40\\ \mathrm{High}&,\mathrm{if}&SSQ>40\\ \end{array}\right.$$

  (1)

In the navigation task, we categorize the movement of each participant into Rest and Accelerate based on the average speed every 8 seconds. In addition, we calculate the motion sickness dose value (MSDV) [50] as an estimation of the participant’s cybersickness degree using the following formula:

where $T$ is the total duration, and $a$ is the acceleration computed from speed and time. Then, based on the MSDV value, it was categorized into Cybersickness and No cybersickness, which represent feeling cybersickness or not, respectively. Also, the participants wore an Empatica E4 wristband, which could record their electrodermal activity data at a rate of 4 Hz. The extracted SCR signals from the EDA signals serve as the ground truth of the participant’s probability of cybersickness.

We performed data preprocessing to fit the iFEP model. First, we utilized the original dataset’s speed, EDA signal, and SSQ score. Second, we divided the data collected in one task into 30 intervals. Each interval represents one trial and contains data over 8 seconds. Therefore, each participant’s data can be divided into 120 trials.

The iFEP method takes the participant’s action and cybersickness state at each trial as input. It predicts the internal states at each trial, including cybersickness probability, confidence, and curiosity. The internal states model factors that influence one participant to decide whether to go forward or stay still. Therefore, by analyzing these factors, the iFEP method could decode how the participants make decisions by estimating the intensity of curiosity and ground truth reward probability.

We generated simulated EDA signals to test the ReCU model’s validity in estimating the participant’s cybersickness level. The simulation test has 1000 trials, and the ground truth of the cybersickness degree is computed as described in Section III. Given the SCR signals extracted from the simulated EDA signals, the ReCU model replicates the decision-making process and estimates the selection probability for each option (Rest and Accelerate) and the reward probability (i.e., the probability of feeling cybersickness). We assume constant curiosity changes over time and is determined by $c_{t}=4\sin(4T\pi t)$, where $t$ represents the trial number and $T$ represents the total number of trials. The learning rate $\alpha$ is 0.05, the randomness of action selection $\beta$ is 2, $P_{0}=0.8$, and $\sigma_{w}=0.4$. Moreover, the ground truth of the cybersickness degree was determined by the average of SCR signals every 8 seconds, and the probability of feeling cybersickness was assumed to be 10%, 40%, 60%, and 90%.

From Figure 3b, we can see that the ReCU model can predict the reward probability reasonably well. Also, the confidence in the estimated reward probability for one option increases when that option is selected and vice versa (Figure 3c). See Appendix B for the formulation of confidence. Additionally, the new information gained may decrease if the same option is selected consecutively (Figure 3d). Thus, it may encourage the participant to choose another option to obtain more information about the environment. This result shows the validation of the ReCU model for estimating user movement in a VR environment.

After that, we used the same simulated EDA signals to test the validity of the iFEP method in the application of decoding the user decision-making process. First, the extracted SCR signals were input into the ReCU model to estimate the action selection and whether to receive a reward. The estimated reward probability for each option was treated as the ground truth (dashed line in Figures 4c and 4d) and compared with the results from the iFEP method. In addition, the estimated action and reward were inputs for the iFEP method to decode the reward probability and intensity of curiosity. The learing rate $\alpha$ is set as 0.05, the inverse temperature $\beta$ is 2, $P_{0}=0.8$ and $\sigma_{w}=0.4$. Figure 4a shows the action selection for 1000 trials, where the shorter line indicates no reward after selecting an action, and the longer line indicates the presence of a reward after selecting an action. Moreover, we can see that the estimated reward probability for each option (red and blue solid lines in Figures 4c and 4d) correctly decodes the ground truth (dashed lines in Figures 4c and 4d). As a result, Figure 4 shows that the iFEP method could decode user decision-making processes in a VR environment.

Furthermore, we investigated the ability of the iFEP method to accurately decode user decision-making processes by applying different noise values to the intensity of curiosity. As shown in Figure 5a, the predictions of curiosity intensity with noise follow a similar pattern to the ground truth. Also, the root mean square error is minor when the value of the noise intensity is smaller (Figure 5b). Moreover, there is a positive correlation between the estimated and the ground truth of curiosity (Figures 5c and 5d). By comparing the estimation and ground truth of curiosity values, we could conclude that the prediction of curiosity intensity is robust under the influence of noise.

After validating the ReCU model and iFEP method, we applied actual virtual navigation data from the abovementioned dataset to decode how the user chooses between Accelerate and Rest resulting from varying curiosity levels. We analyzed how the iFEP method predicted the participants’ realization of their probability of feeling cybersickness. Also, we evaluated the participant’s navigation behavior in the navigation task. The behavioral data are set up as described in Section III-C, and the total behavioral data is 120 trials for each participant. Here, we demonstrate one participant’s result, whose features are commonly present in the results of other participants. However, the differences in features in other participants’ results will also be discussed.

Figure 6 shows the decoding result of one participant’s navigation data using the iFEP method. Figure 6a indicates the action selected and the presence of reward at every trial, and the line in between indicates the probability of choosing to Accelerate, which is identical to the blue line in Figure 6b. Although the estimated values do not perfectly match the ground truth value, the estimations for both actions can recognize the fluctuation in the actual cybersickness probabilities (Figures 6c and 6d). Another reason affecting the accuracy of the estimated value could be that the iFEP assumes a gradual change in reward probability instead of a sudden change [35]. The confidence in the prediction of the reward probability for choosing to Rest and Accelerate is demonstrated by Figure 6e. Generally, the intensity of curiosity is harmful to most trials, with some positive curiosity around trials 30 to 60, as shown in Figure 6f. This outcome shows the participant is relatively conservative in choosing an action during the navigation task. Despite the general negative curiosity level for most participants, one participant’s behavioral data estimates positive curiosity for all trials. According to the SSQ classification, this participant has an average SSQ score of 11.22, which is the category of low cybersickness.

In addition, we analyzed the user behavioral data to explore the correlation between the expected information gain and curiosity. We computed the correlation coefficient between the expected information gain and curiosity and between the expected information gain and the temporal derivative of curiosity. As shown in Figure 7a, the correlation between expected information gain and curiosity at time lag = 0 is very small (0.0051), indicating no significant relationship between the two variables when they are aligned in time. However, the maximum correlation coefficient of 0.6316 occurs at a time lag of -24 samples. This suggests that, while there is no immediate correlation, a positive correlation emerges when expected information gain lags behind curiosity by 6 seconds. Moreover, the regression line illustrated in Figure 7b shows a positive correlation between the expected information gain and the temporal derivative of curiosity at zero time lag. The maximum correlation for this relationship occurs at a time lag of -4 samples, with a correlation coefficient of 0.5071. When considering the overall dataset from all participants, we observed that the average maximum correlation (accounting for both positive and negative correlations) occurs at a time lag of -2.36 samples, which corresponds to approximately -0.5909 seconds. This indicates that, on average, curiosity tends to precede expected information gain by about 0.59 seconds. Thus, suggest that curiosity have a predictive role represented by expected information gain. Although the average reaction time for a target detection task is approximately 0.454 seconds [51], which is faster than the observed lag in our study, this discrepancy can be attributed to individual differences in cognitive processing. Thus, our findings provide a reasonable reaction time, especially in the context of decision-making processes related to curiosity. Therefore, we can conclude that the participant would upscale their curiosity when the expected information gain increases (i.e., the environment becomes unfamiliar).

The subjective reward method and the Q-learning method are two alternative approaches for decoding a user’s decision-making process. The subjective reward method states that the expected net utility $U_{t}$ is computed by

where $d_{t}$ indicates the desire for reward at trial $t$. So, instead of putting weight on the expected information gain in the iFEP method, the subjective reward method puts weight on the expected reward. In the context of participants performing navigation tasks in a VR environment, the desire for reward would represent whether the participant would like to experience cybersickness. Therefore, we would expect the value of $d_{t}$ to be small. However, the average value of reward intensity of 41.1953, which shows that the reward intensity did not indicate that the participant wanted to avoid cybersickness (Figure 8f).

On the other hand, the Q-learning method predicts the reward probability $Q$ for action $i$ using

where $\alpha_{t}$ indicates the learning rate at trial $t$ and $a_{i,t}$ indicates the action selected at trial $t$. Also, the action selection probability is determined by

where $\beta_{t}$ indicates the inverse temperature at trial $t$ and controls the randomness of action selection. Appendix C gives a detailed explanation of the formulation. Therefore, to understand the relationship between inverse temperature and action selection, we analyzed the correlation coefficient between the expected information gain, inverse temperature, and the derivative of inverse temperature at different time lags.

As shown in Figures 9a and 9b, there is a positive correlation between the expected information gain and inverse temperature (correlation coefficient = 0.7031 at time lag = 0) and no correlation between the expected information gain and the temporal derivative of inverse temperature (correlation coefficient = -0.1635 at time lag = 0).

The subjective reward method emphasizes the expected reward to model a reward-driven condition. In the context of virtual navigation, this desire for reward reflects participants’ inclination to experience or avoid cybersickness. We anticipated that the reward intensity would be low, as participants are likely motivated to avoid discomfort. However, the average reward intensity across all participants was 24.97, indicating a stronger-than-expected desire for reward. Notably, only three participants exhibited a reward intensity of less than 1, suggesting they were actively trying to avoid cybersickness. This finding indicates that the subjective reward method may not effectively explain user decision-making in this scenario.

In contrast, the Q-learning method employs reinforcement learning techniques to model adaptive reward-seeking behavior by adjusting time-dependent meta-parameters, focusing exclusively on maximizing expected cumulative rewards. As shown in Figure 9a, the inverse temperature correlates with expected information gain. Additionally, the majority of participants demonstrated a positive correlation between expected cumulative reward and inverse temperature. This result implies that the randomness of action selection aligns with curiosity levels, suggesting that the iFEP method is more suitable for decoding user actions in a VR environment.

By investigating the temporal balance between cybersickness and curiosity based on users’ virtual navigation behavior, we gain insights into the neural correlates of temporal variability during immersion. However, we aware that our work may have some limitations.

First, the short navigation duration may limit accuracy. The result from the simulated data revealed that longer trials yield more accurate prediction. However, longer trials might improve the estimation accuracy, but also raise the risk of severe sickness symptoms, complicating the user study [8].

Second, only one scenario was used (see Figure 1), appealing more to participants with an affinity for forest-related contexts, prompting more active navigation. Conversely, participants who were indifferent to the setting may engaged less. Users often have varying interest levels in VR scenes, highlighting scenario design’s importance in VR applications. Future studies should also incorporate additional psychological factors like attention and stress, which could enhance the precision of decision-making models and foster a more comprehensive understanding of user interaction in immersive settings.

Previous study [58] found these variables to be positively correlated with cybersickness severity. Additionally, So et al. [59] used spatial velocity to estimate cybersickness exposure. To improve our analysis, we could collect additional user behavioral data (e.g., eyeblink rate and spatial velocity) during navigation. Expanding the number of participants in future experiments will also improve the reliability of our results.