Abstract

Past reports have suggested that active visual training in virtual reality (VR) can reduce symptoms of cybersickness. Here, we adapted such a protocol to a computer-based version and compared it with a passive exposure control condition. We employed heart rate and other subjective predictors of cybersickness to try to predict the efficacy of the intervention as well as likelihood of drop out. While we found a significant decrease in heart rate across sessions, the intervention we employed did not appear to be effective at reducing cybersickness or dropout. However, a heart rate increase of 15.5 bpm from baseline, nausea self-report of 4.5 on a scale of 1–10, and dizziness self-report of 5.5 on a scale of 1–10 predicted an equal probability of experiment dropout, independent of whether participants were in the experimental or control intervention condition. Our findings suggest that a single immersion of visual training in VR or passive VR exposure may not be sufficient to provide adaptation for VR. At the same time, our findings bolster past reports suggesting the value of employing heart rate monitoring, rather than subjective reports, to monitor the onset of cybersickness.

1 Introduction

One common issue for both commercial and research applications of virtual reality (VR) has been complaints of cybersickness, also known as virtual reality (VR) sickness, which results in symptoms similar to motion sickness and simulator sickness. A previous study found that these symptoms occurred 65% more often when viewing a scene in immersive VR as compared to using a traditional external monitor (Sharples, Cobb, Moody, & Wilson, 2008). Another study reported that 78% of the participants tested experienced significant levels of cybersickness while navigating in a 3-dimensional (3D) room created via projection systems (Kim, Kim, Kim, Ko, & Kim, 2005). While theoretical papers have explored a possible relationship between cybersickness and visually induced motion sickness (Keshavarz, Riecke, Hettinger, & Campos, 2015), the exact mechanisms underlying cybersickness have remained unclear.

Being that cybersickness has been described as a subcategory of motion sickness (Keshavarz et al., 2015), conventional hypotheses for motion sickness have been applied to cybersickness. One widely accepted explanation for the cause of motion sickness, and potentially other conditions characterized by nausea, such as seasickness or cybersickness, has been the sensory rearrangement hypothesis (Golding, 2016). Sensory rearrangement posits that nausea symptoms are caused by conflict across some combination of visual, vestibular, and proprioceptive information that the brain receives (Reason, 1978). Reason (1978) posited that for symptoms to occur, this must involve one or more sensory systems conflicting, critically, with the vestibular system. Benson (2002) further explained sensory rearrangement by categorizing two types of neural mismatch as being either visual-vestibular or canal-otolith related (the otoliths being part of the vestibular system).

Exactly how this mismatch results in the symptoms of motion sickness---such as nausea---has remained unclear. Treisman (1977) suggested that motion sickness has a protective mechanism in response to neurotoxins. Variance among visual, vestibular, and proprioceptive information is produced by toxins when they are ingested. When this variance is detected, the body's response is emesis (vomiting) to remove any toxins. While this hypothesis provides a plausible explanation for why motion sickness occurs, it operates via a mechanism of mismatch between actual and expected sensory information across modalities and thus may be complementary to sensory rearrangement.

The sensory rearrangement hypothesis suggests that adaptation may be an important component of recovery from cybersickness, that is, a change in response to a sustained stimulus. According to Reason (1978), when exposure history conflicted with present sensory information, it led to nauseating symptoms until they disappeared with adaptation to the present sensory information. Oman (1990) described this process with an internal model that received motor efference and sensory afference that produced an efference copy. Sensory afference was then continuously compared to the efference copy. Differences between the two resulted in a sensory conflict that corrected motor output and caused sensorimotor learning.

Current research on the etiology of motion sickness has been consistent with this form of adaptation with the discovery of “vestibular only” neurons in Rhesus monkeys that demonstrated reafference cancellation (Cullen, Brooks, & Sadeghi, 2009). Reafference cancelation has been important in the reduction of sensory conflict when an efference copy and sensory afference are compared (Oman, 1990). Furthermore, Oman and Cullen (2014) proposed that these vestibular only neurons might project to emetic centers and might explain the adaptation seen to motion sickness through sensorimotor learning. Indeed, if the adaptation hypothesis of motion sickness holds, then it seems reasonable to predict that exposure to VR could create a sort of immunity to cybersickness. Indeed, several studies have found repeated exposure in VR and simulator environments reduced motion sickness symptoms over time (Domeyer, Cassavaugh, & Backs, 2013; Hill & Howarth, 2000; Regan, 1995; Teasdale, Lavallière, Tremblay, Laurendeau, & Simoneau, 2009). Specifically, Teasdale et al. (2009) found shorter immersion sessions to be more effective and Domeyer et al. (2013) found a delay between sessions was also effective in adapting to simulators. These conclusions have been extended to immersive VR with comparable results (Hill & Howarth, 2000; Howarth & Hodder, 2008).

In addition to exposure, other researchers have taken a more active approach by applying vision therapy principles (Fulvio & Rokers, 2018). That is, eye exercises may be effective in treating visual problems such as convergence insufficiency and improving stereoscopic skills (Rawstron, Burley, & Elder, 2005). It should be noted, however, that vision therapy has been applied to other visual deficits with mixed results. A meta-analysis found vision therapy to be effective for convergence insufficiency, which is the inability of the eyes to turn towards each other, and for vision rehabilitation after injury, but not for well-studied medical conditions like myopia or headaches (Barrett, 2009). In terms of efficacy for vision therapy in preventing motion sickness, recovery from cybersickness appears to be aided by hand-eye coordination exercises. Research by two related studies found a virtual peg-in-hole task to have effectively mitigated cybersickness symptoms post VR immersion (Champney et al., 2007; Curtis et al., 2015). According to the sensory rearrangement theory, these vision therapies serve to (re)condition the user to the sensory conflict. Sensory conflict may also be the basis for vestibular rehabilitation exercises (Boyer et al., 2008), which may improve postural stability in individuals with normal vestibular function (Hall, Heusel-Gillig, Tusa, & Herdman, 2010; Morimoto et al., 2011).

Other studies have found optical discrepancies may have a role in inducing cybersickness. One particular issue is the vergence-accommodation (VA) conflict, which is caused by focal accommodation to the HMD screen rather than the virtual scene. Since the screen remains fixed but vergence (horizontal movement of the eyes inward or outward) cues change depending on the virtual scene, this can lead to VA conflict and discomfort (Hoffman, Girschick, Akeley, & Banks, 2008; Koulieris, Bui, Banks, & Drettakis, 2017). The study by Koulieris et al. (2017) investigated various display configurations and found adjustable lenses to be more efficient in reducing discomfort than depth of field rendering or monocular vision designs. This suggests VR designs may benefit from focal accommodation that complements the virtual scene.

The present study aims to better understand cybersickness by attempting to minimize its onset with visual therapy and, at the same time, determine the factors that might make individuals more susceptible to cybersickness. Extending previous work using vision therapy, we employed a visual training exercise intervention, compared to viewing a passive video, between two sessions of exploration in a virtual environment, based on work by Rine, Schubert, and Balkany (1999) on vision therapy for motion sickness. This study was chosen because it applied visual-vestibular habituation exercises to promote adaptation to motion sickness. These exercises, although not in VR, rely on repeated visual and vestibular stimulation and are built upon the sensory conflict hypothesis as described by Rine et al. (1999). Because repeated VR exposure has been found to reduce cybersickness symptoms in the aforementioned studies (Champney et al., 2007; Curtis et al., 2015; Howarth & Hodder, 2008) by visual and vestibular stimulation with HMDs, we believe similarly applying these exercises in VR may also be suitable for cybersickness adaptation. It should be noted, however, that there are some limitations in implementing visual-vestibular exercises, as described by Rine et al. (1999), with our virtual reality interface. Namely, because participants must be tethered to a desktop computer with an HMD, experiencing a limited field of view would restrict particular movements of the Rine et al. (1999) study (see methods). These restrictions specified the required modifications to implement such exercises in VR and are further described below.

The Rine et al. (1999) therapy incorporated visual–vestibular exercises in two stages. In Stage 1, seated individuals hold an index card in front of them moving it in a horizontal and vertical direction with increasing speed. First, the card is followed with their eyes while keeping the head fixed, and then the head is moved while the card is fixed. In Stage 2, the same exercises are done while standing and then, in a seated position, both head and eyes are moved simultaneously and in opposite directions. Balance training exercises are also part of the therapy which involves walking with soft cushions to simulate a floating platform. The Rine et al. (1999) exercises are largely based on ocular-motor exercises described by Cawthorne (1945) and Cooksey (1946) with some modifications and are conducted repeatedly over long time intervals.

It should be noted that visual–vestibular exercises before and after the Rine et al. (1999) study have been shown to be an effective treatment for dizziness and balance disorders (Dai, Raphan, & Cohen, 2011; Whitney, Alghwiri, & Alghadir, 2016). Furthermore, the application of visual–vestibular exercises appear to have small but significant effects in applications to simulators but are overall effective (Bergeron, Lortie, & Guitton, 2015; Pavlou et al., 2004). However, the application of the Rine et al. (1999) study in VR has not yet been tested.

Here, we implemented a modified VR version of the Rine et al. (1999) Stage 1 smooth pursuit eye movements, Stage 2 head movements, and motion to induce vection as a substitute for balance training exercises but in a short and focused fashion. We described each of the modified stages as blocks. The key modifications to the Rine et al. (1999) study should be noted: first, the moving index card in Stage 1 was substituted with an oscillating disc; second, the index card's letters were substituted with numbers which the participants inputted via keyboard to assure the participants actually performed the task while wearing an HMD; third, numbers instead of letters were chosen to restrict hand movements and permit ease of input to minimize interface difficulties; fourth, standing and walking exercises were not included in our design because participants were tethered to a desktop; fifth, head movement exercises were implemented by substituting the index card with green squares in different positions which would disappear once participants oriented the HMD toward them; sixth, balance training exercises, as implemented by Rine et al. (1999), resulted in unexpected movement while on a floating platform and this movement was replicated in our study through radial and translational motion to induce vection.

Because we aimed to determine which therapies might be used to mitigate cybersickness, we started with ones most likely to be adopted by companies/users who might be concerned about such side-effects of VR. We hypothesized that short, focused bouts of smooth pursuit eye and head movements might be effective in mitigating cybersickness. Thus, we predicted a single immersion of visual training would better adapt the individual to the VR environment, compared to passive VR exposure alone, by reducing heart rate and cybersickness symptoms. As dependent measures, we employed responses to a battery of questionnaires to attempt to predict dropout and symptoms of cybersickness. We also collected heart rate measures by pulse oximetry and other continuous measures of cybersickness to attempt to better understand its onset.

2 Methods

2.1 Participants

Data were collected from 64 adult undergraduate students (30 female) who had a mean age of 20.91. Students were recruited from the University of California Davis Psychology Research Participation System (SONA). All participants provided informed, written consent upon arriving for the experiment. Participants received compensation in the form of extra credit in an undergraduate Psychology course. Study criteria disqualified participants with a cardiac history, intoxication, or nail polish present on fingers. Nail polish disqualified participants because it interferes with pulse oximetry readings. All had normal or corrected-to-normal vision with contact lenses. Participants who failed to finish the first part of the experiment (n=13) were excluded from analysis of variance (ANOVA) adaptation analyses because these analyses required repeated measures. This exclusion resulted in a final sample of 51 participants (22 female), with a mean age of 20.92 years, whose data were analyzed. However, for regression analyses, all 64 participants' (30 female) data only from the first part of the experiment were used with individual dropout participant performance averaged up to the point of attrition (see Analyses). All procedures were approved by the UC Davis Internal Review Board for Human Subjects Testing.

2.2 Materials

2.2.1 Virtual Reality Interface

Three virtual environments were developed in Unity 3D (version 5.0.1f1, Unity Technologies ApS, San Francisco, CA) and viewed through an Oculus Rift Development Kit 2 (Oculus VR LLC, Menlo Park, CA) head-mounted display (HMD) at a resolution of 960×1080 pixels per eye, 75-Hz refresh rate, and 100° (nominal) field of view. Stimuli were presented on the HMD using a 27′′ iMac desktop computer (Apple Inc., Cupertino, CA). Head tracking allowed the participant's head orientation to reflect in the virtual environment at a rate of 1000 samples per second. A navigation environment was developed by modifying the Unity Viking Village demo (version 1.0.1772, Unity Technologies ApS, San Francisco, CA) for the Oculus Rift. In accessible areas of the environment, 81 blue cubes of size 0.5 scale units were located randomly. Locomotion was available to participants and this created an indeterminate distance between 1 to 30 meters between the participant and cube as allowed by the virtual environment (see procedure below). A control environment consisted of a simulated computer screen playing a five-minute video of a TED talk called, “What hallucination reveals about our minds” (Sacks, 2009). The treatment environment had three blocks each implementing participant eye movement, head movement, and vection.

2.2.2 Pulse Oximetry

Heartrate was continuously measured with a Nonin 3230 wireless pulse oximeter (Nonin Medical Inc., Plymouth, MN) and recorded on a 2017 MacBook Pro (Apple Inc., Cupertino, CA). Cybersickness symptoms were measured at the end of each navigation session using a modified version of the Simulator Sickness Questionnaire (Kennedy, Lane, Berbaum, & Lilienthal,1993) that was proposed by Bouchard, Robillard, and Renaud (2007). When scored, the SSQ gives a Total score, a Nausea score, and an Ocular-Motor score. Participants' self-reported symptoms of dizziness and nausea were also recorded during each block of navigation on a scale of 1–10.

These dependent measures were chosen because of their previous use in simulator and VR studies. However, their strengths and weakness should be noted. Heartrate monitoring by pulse oximetry can detect fluctuations in heartrate and are associated with motion sickness symptoms (Dennison, Wisti, & D'Zmura, 2016; Holmes & Griffin, 2001). A study by Kim et al. (2005) found cybersickness symptoms to have a significant positive correlation with increased heartrate. However, a related study based on Kim et al. (2005) found stomach activity, blinking, and breathing rate to be more strongly correlated with cybersickness symptoms than heartrate (Dennison et al., 2016). Furthermore, the authors note that individual variability between participants for measures such as resting heartrate must be accounted for when analyzing such data. The SSQ is also used frequently in VR and simulator studies (Bouchard, Robillard, Larouche, & Loranger, 2012; Fulvio & Rokers, 2018; Solimini, 2013) and revisions have since been made to the Kennedy et al. (1993) version. It has an advantage of quantifying simulator sickness in a more standardized manner than a simple scale, but its calculation can be sluggish and unsuitable for obtaining multiple immediate data points. Furthermore, a recent study has found that the SSQ may not be applicable for measuring cybersickness because of its psychometric qualities. Rather, tailored questionnaires for cybersickness, such as the cybersickness questionnaire (CSQ) or virtual reality sickness questionnaire (VRSQ), may be more suitable (Sevinc & Berkman, 2020).

Baseline heartrate data were sampled every second for 60 seconds and then averaged for each participant. This baseline was subtracted from their average heartrate during navigation sessions to calculate their change in average heartrate. This change was made to account for differences in baseline heartrate among participants. The average change in heart rate from both navigation sessions was obtained. The last five seconds of each block in the first session were averaged for heartrate. This was done to ensure that enough data were collected for an accurate average before dropping or not dropping out.

2.2.3 Simulator Sickness Questionnaire

Simulation sickness questionnaire (SSQ) scores were obtained by subtracting the initial score from the final score for each respective session (Kennedy et al., 1993). This was done to obtain a score describing the change in symptoms from the VR experience and not from antecedent symptoms. Self-reported symptoms for nausea and dizziness were also averaged across each block.

2.3 Design

We employed a 2 (Condition: control, treatment) × 2 (Session 1, Session 2) mixed design. The independent variable, “Condition,” was manipulated between subjects and the independent variable, “Session,” was manipulated within subjects (see Figure 1). The primary dependent measures were heartrate, SSQ scores, and self-reported sickness scores. Additional dependent measures were collected, including alcohol consumption, race, ethnicity, and specific stressors: school tests; deadlines; difficult schedule; upcoming projects; illness or injury, but were not analyzed for the purposes of this study.

2.4 Procedure

Participants began the experiment by completing a background information survey and study consent form. Prior to arrival, participants were assigned to treatment or control groups randomly by a coin toss. Because of participant attrition (n=13) before Session 2, however, some assignments were adjusted to ensure a relatively balanced design (24 treatment, 27 control) as participants signed up. For each of the conditions, baseline resting heartrate data were collected. Participants who had an elevated heartrate from physical activity prior to arriving at the study site rested for an additional five minutes before collecting baseline heartrate data. Pulse-oximetry was measured from the middle finger of the right hand while participants sat still in a chair for sixty seconds.

Figure 1.

Timeline of two session experiment. Heartrate data was continuously measured throughout each session. Self-reported symptoms were collected in three blocks during each navigation session. SSQ scores were collected before and after each navigation session.

Figure 1.

Timeline of two session experiment. Heartrate data was continuously measured throughout each session. Self-reported symptoms were collected in three blocks during each navigation session. SSQ scores were collected before and after each navigation session.

After collecting baseline data, participants were informed that the experiment involved three blocks and that the goal was to test memory before being fitted with the HMD. This was done to ensure that all participants paid attention to the task. The virtual navigation environment was loaded and then the focal distance on the HMD was adjusted as needed for image sharpness. Participants were then instructed how to move around the virtual environment using a keyboard and mouse for one minute. Participants were informed their head movements would orient the first-person view camera identically (such as a pitch or yaw of the head), but body rotations would require horizontal translations of the mouse (left and right). Translational movements in VR (forward, back, left, right) were initiated by the following keystrokes: W, A, S, D; or arrow keys. Translational movements had an instantaneous velocity of 8 m/s when moving forward and an instantaneous velocity of 4 m/s when moving backward or sideways. Velocity remained constant as long as participants held a keystroke and decelerated to a stop at 20 m/s2 when a keystroke was not held. Jumping and running were not part of the controls.

After understanding the controls, participants were instructed to count as many blue cubes as possible around the map within one minute and they repeated this task three times. They were instructed not to count the same cube more than once. Participants were informed that a very small minority of the population experience eyestrain, headache, dizziness, and/or nausea. Participants were allowed to stop the experiment at any time if they felt uncomfortable. They were also informed that stopping the experiment would not impact remuneration (receiving credit). Pulse-oximetry was resumed at the start of the navigation session.

After each block, participants were asked to verbally rate how uncomfortable they felt on a scale from 1–10, with 1 being no symptoms and 10 being very uncomfortable. Participants also rated their dizziness and nausea on a scale of 1–10, with 1 being no change and 10 being severe. Participants who reported any symptoms were asked if they wanted to stop the experiment, take a five-minute break, or continue the experiment. Asymptomatic participants were asked if they wanted to take a five-minute break or continue the experiment after each block.

At the end of the navigation session, participants removed the HMD and completed the SSQ set for Session 1 in a reversed order. First, participants were instructed to complete an SSQ immediately after completing Session 1. Second, after completion of the SSQ, they were given a second SSQ but were instructed to report how they felt at the beginning of the experiment (before they were immersed in VR). This produced a single set of two SSQs (SSQ 1 for Session 1 and SSQ 2 for Session 1). The reversal in order was done to prevent priming participants from being aware of cybersickness symptoms, similar to how they were informed that the experiment was testing memory to reduce primacy effects. Although there could be disadvantages to this approach, this strategy was less likely to inadvertently “prime” participants to expect cybersickness compared to previous studies that implemented them in order. Consistent with this argument, another study found that administering SSQs at the beginning of experiments caused more cybersickness than only administering SSQs at the end of experiments, suggesting that priming may induce cybersickness symptoms (Young, Adelstein, & Ellis, 2007). A five-minute break was given to all participants before moving on to the next stage.

Participants in the treatment condition were refitted with the HMD and began an eye movement intervention based on the Rine et al. (1999) Stage 1 smooth pursuit eye exercises. In the intervention, participants were informed that an oscillating disc alternating in horizontal and vertical motion would display a series of numbers from 1–3 and instructed to correctly press the corresponding number key as fast as possible. Thirty random numbers from 1–3 were displayed before alternating direction nine times. The discs initially oscillated at one round per six seconds and accelerated to one round per two seconds. Disc presentation subtended 68° and was 1.3 meters in front of the participant. In the head movement block, participants were informed green squares would be displayed and their task was to orient the center of their field of view on the square until it disappeared. We based this block on the Rine et al. (1999) Stage 2 head movement exercises. Twenty squares were displayed with ten in fixed positions and ten in oscillating movement. Square presentation subtended 141° of visual space and thus initially occurred outside the field of view of the headset. This was done to encourage participant head movements and not eye movements.

In the vection block, participants were informed that particles would fly around them giving the sensation of movement. This block was a substitute for Rine et al. (1999) balance exercises that involved walking on soft cushions. At any given time, 100 white particles moved toward, away, and upward from the participant's field of view for one minute and twenty seconds. These particles were placed against a black background.

Participants in the control condition were refitted with the HMD and informed they were going to watch a five-minute video in VR of a TED talk (Sacks, 2009) called “What hallucination reveals about our minds.” The control video was displayed on a virtual 27” iMac desktop computer (Apple Inc., Cupertino, CA). The virtual video screen was 51.6cm×65cm, subtended 28° of horizontal visual space, and was 1.3 meters in front of the participant. Participants were not specifically encouraged to relax and were simply told to watch the video without further instruction. Both treatment and control virtual environments were designed to be similar in terms of lighting position, virtual object distance, and surrounding environment colors. All control participants watched the same video. Displaying the control group video within the HMD maintained VR immersion consistency for both treatment and control groups without having an explicit intervention designed to mitigate cybersickness.

Participants in both conditions were given a five-minute break before moving on to the second navigation session. The procedure for the second navigation session was identical to the procedure for the first navigation session except that the positions of the cubes were altered. Participants finished the experiment by completing an optional demographics survey and were debriefed.

2.5 Analyses

Data analyses were conducted in R (Version 3.4.4; RCore Team, 2018) using RStudio (Version 1.1.456; RStudio Team, 2018). The following R packages were used to carry out analyses: ggplot2 (Version 2.2.1; Wickham et al., 2009), MOTE (Version 0.0.0.9100; Buchanan, Gillenwaters, Scofield, & Valentine, 2019), bestglm (Version 0.37; Mcleod, Xu, & Lai, 2018), usdm (Version 1.1-18; Naimi, 2015), BayesFactor (Morey et al., 2018), and reshape (Version 0.8.7; Wickham, 2007).

Best subset regression was performed using the bestglm R package (Mcleod et al., 2018) with five-second heartrate, biological sex, self-reported nausea, and self-reported dizziness included in the model. Akaike Information Criteria (AIC) was used, consistent with Zhang (2016). Bayes factors were utilized for ANOVA adaptation analyses to gauge the fit of models with and without Session, Condition, and interaction effects. This factor compares the likelihood of a restricted model to a full model (H0/H1; where H0 is the likelihood of the restricted model and H1 is the likelihood of the full model). Bayes factors that are less than one suggest stronger evidence than the data that were obtained under the alternative hypothesis, that is, the full model. Conversely, Bayes factors that are greater than one suggest stronger evidence than the data that were obtained under the null hypothesis, that is, the restricted model (Jarosz & Wiley, 2014). For the regression analyses involving physiological measures, all 64 participants' (30 female) Session 1 data were used and subjects had a mean age of 20.91. Data from participants who dropped out before the third block were averaged with the previous blocks.

3 Results

3.1 Intervention Efficacy

We obtained heartrate data from the 51 participants (24 treatment). A Shapiro--Wilk normality test found the data to be normally distributed for both Session 1, W=.98, p>.250, and for Session 2, W=.98, p>.250. Levene's test for homogeneity of variance found the groups (treatment vs. no treatment) to have equal variance, F=0.27, p>.250. Session data were correlated, r(49)=.71, p<.001. Data were screened for outliers, using Mahalanobis distance with p<.001 as a criterion (De Maesschalck, Jouan-Rimbaud, & Massart, 2000; Mahalanobis, 1936). No outliers were found in the analysis. We thus proceeded to use parametric tests in the analysis of heartrate data.

Figure 2.

Heartrate change during navigation. Changes in heartrate are from participant baseline for treatment and control groups during Session 1 and Session 2. Heartrate was averaged for each session. Error bars represent mean standard error; **p = .001.

Figure 2.

Heartrate change during navigation. Changes in heartrate are from participant baseline for treatment and control groups during Session 1 and Session 2. Heartrate was averaged for each session. Error bars represent mean standard error; **p = .001.

We first conducted a 2 (Session: Session 1, Session 2) × 2 (Condition: treatment, control) mixed ANOVA on heartrate. There was a significant main effect of Session, F(1,49)=11.55, p=.001, ηp2=.04, with participants having lower average heart rate, relative to baseline, in Session 2 (M=-2.08,SD=7.93) compared to Session 1 (M=0.64,SD=4.77). No main effect of Condition was found between treatment (M=-0.92,SD=6.55) and control (M=-0.55,SD=6.80) groups, F(1,49)=0.05, p>.250, ηp2<.001. The Session × Condition interaction did not reach significance, F(1,49)=.001, p>.250, ηp2<.001 (see Figure 2).

In addition, the data were examined by estimating a Bayes factor, using R package Bayesfactor (Morey et al., 2018), to compare the likelihood of obtaining of the data with a full model versus a model restricted to exclude any interaction or main effect, using a top-down approach as suggested by Rouder, Morey, Speckman, and Province (2012). The Bayes factor for the interaction provided substantial evidence in favor of the null hypothesis for a restricted model without the interaction term, B01=3.61. Therefore, further Bayes factor analyses were conducted to assess the main effects of Session and Condition. There was strong evidence in favor of the alternative hypothesis that there was a main effect of Session; that is, there was a higher probability of obtaining these data under a full model that included Session as compared to a restricted model where Session was excluded as a predictor, B01=0.04. There was anecdotal evidence in favor of the null hypothesis that there was not a main effect for Condition; that is, these data had an anecdotally higher relative probability of being obtained under a restricted model that assumes no effect of Condition, B01=2.81. These results are consistent with the significance thresholds obtained using the p-values produced from the mixed ANOVA.

SSQ scores were obtained for 51 participants and were screened for assumptions and outliers, using Mahalanobis distance as a criterion with p<.001. No outliers were found in the analysis. We conducted a 2 (Session: Session 1, Session 2) × 2 (Condition: treatment, control) mixed ANOVA on the change in Total, Nausea, and Ocular-Motor SSQ scores. There were no significant effects. For Total change SSQ, no main effect of Condition was found between treatment (M=6.83,SD=7.55) and control (M=4.94,SD=5.21) groups, F(1,49)=1.35, p>.250, ηp2=.016. No main effect of Session was found between Session 1 (M=5.27,SD=5.13) and Session 2 (M=6.39,SD=7.57), F(1,49)=1.99, p=.165, ηp2=.007. The Session × Condition interaction did not reach significance, F(1,49)=0.90, p>.250, ηp2=.004.

For Nausea change SSQ, no main effect of Condition was found between treatment (M=4.17,SD=4.53) and control (M=3.17,SD=3.37) groups, F(1,49)=1.01, p>.250, ηp2=.016. No main effect of Session was found between Session 1 (M=3.31,SD=3.16) and Session 2 (M=3.96,SD=4.65), F(1,49)=1.66, p=.203, ηp2=.007. The Session × Condition interaction did not reach significance, F(1,49)=0.95, p>.250, ηp2=.004.

For Ocular-Motor change SSQ, no main effect of Condition was found between treatment (M=2.67,SD=3.50) and control (M=1.81,SD=2.63) groups, F(1,49)=1.20, p>.250, ηp2=.019. No main effect of Session was found between Session 1 (M=1.96,SD=2.65) and Session 2 (M=2.47,SD=3.48), F(1,49)=1.78, p=.188, ηp2=.007. The Session × Condition interaction did not reach significance, F(1,49)=0.352, p>.250, ηp2=.001 (see Figure 3).

3.2 Mitigation Analysis

We further compared SSQ scores from before and after the intervention (or control task). The SSQ data were entered into a 2 (Time: before-intervention, after-intervention) × 2 (Condition: treatment, control) mixed ANOVA to compare the effects of Condition and Time on SSQ scores. For Total SSQ, there was a significant main effect of Time, F(1,49)=12.47, p<.001, ηp2=.08, with participants having lower scores, after the intervention (M=4.49,SD=5.57) compared to before the intervention (M=7.65,SD=5.58). No main effect of Condition was found between treatment (M=5.75,SD=4.98) and control (M=6.35,SD=6.42) groups, F(1,49)=0.22, p>.250, ηp2=.003. The Time × Condition interaction did not reach significance, F(1,49)=0.08, p>.250, ηp2<.001.

For Nausea SSQ, there was a significant main effect of Time, F(1,49)=21.83, p<.001, ηp2=.11, with participants having lower scores in after-intervention (M=2.27,SD=2.99) compared to before-intervention (M=4.43,SD=3.35). No main effect of Condition was found between treatment (M=3.31,SD=2.89) and control (M=3.39,SD=3.72) groups, F(1,49)=0.01, p>.250, ηp2<.001. The Time × Condition interaction did not reach significance, F(1,49)=0.20, p>.250, ηp2=.001.

For Ocular-Motor SSQ, no main effect of Time was found between before-intervention (M=3.22,SD=3.11) and after-intervention (M=2.24,SD=3.33) measurements, F(1,49)=3.31, p=.075, ηp2=.02. No main effect of Condition was found between treatment (M=2.44,SD=2.62) and control (M=2.98,SD=3.71) groups, F(1,49)=0.55, p>.250, ηp2=.01. The Time × Condition interaction did not reach significance, F(1,49)=0.01, p>.250, ηp2<.001.

Figure 3.

SSQ scores by subscale post Session 1 and post Session 2. SSQ scores were averaged after each session for treatment and control groups. Error bars represent the standard error of the mean.

Figure 3.

SSQ scores by subscale post Session 1 and post Session 2. SSQ scores were averaged after each session for treatment and control groups. Error bars represent the standard error of the mean.

Data were also examined by estimating Bayes factors. For Total SSQ, the Bayes factor for the interaction provided substantial evidence in favor of the null hypothesis for a restricted model without the interaction term, B01=3.27. There was very strong evidence in favor of the alternative hypothesis that there was a main effect of Time, B01=0.02 and anecdotal evidence in favor of the null hypothesis that there was no main effect for Condition, B01=2.82. For Nausea SSQ, the Bayes factor for the interaction provided substantial evidence in favor of the null hypothesis for a restricted model without the interaction term, B01=3.37. There was decisive evidence in favor of the alternative hypothesis that there was a main effect of Time, B01<0.01 and substantial evidence in favor of the null hypothesis that there was not a main effect for Condition, B01=3.26. For Ocular-Motor SSQ, the Bayes factor for the interaction provided substantial evidence in favor of the null hypothesis for a restricted model without the interaction term, B01=3.53, anecdotal evidence in favor of the null hypothesis that there was not a main effect of Time, B01=1.03, and substantial evidence in favor of the null hypothesis that there was not a main effect for Condition, B01=3.26. These results are consistent with the significance thresholds obtained using p-values from the mixed ANOVAs (Figure 4).

3.3 Identifying Predictors of Dropout

An additional goal of this work was to determine whether we could predict symptoms of cybersickness based on physiological markers and responses to the cybersickness questionnaire. We conducted a logistic regression analysis with variable selection using best subset regression to predict attrition rates in participants using the following: averaged final five-second heartrate from Session 1; biological sex; weight; sleep; hours since last meal; gaming experience; previous VR use; stress on a scale of 1–10; average self-reported nausea; average self-reported dizziness (see Analyses section). We included 64 participants in this analysis (51 non-dropouts, 13 dropouts; we included dropout subjects because we were interested in attrition). An independent-samples t-test was conducted to compare attrition rates in treatment and control conditions. There was no significant difference in attrition rate for treatment (M=0.14,SD=0.36) and control (M=0.24,SD=0.43), t(49)=-0.95, p>.250. Multicollinearity between variables was found to be very low in the dataset by analyzing their variance inflation factor (biological sex, VIF=2.00; weight, VIF=1.72; sleep, VIF=1.30; hours since last meal, VIF=1.27; gaming experience, VIF=2.08; previous VR use, VIF=1.35; stress, VIF=1.78; average self-reported nausea, VIF=1.71; average self-reported dizziness, VIF=2.05) (Zuur, Ieno, & Elphick, 2010).

Figure 4.

SSQ scores by subscale immediately pre and post intervention. SSQ scores were averaged before and after the intervention for treatment and control groups. Error bars represent the standard error of the mean; ***p < .001.

Figure 4.

SSQ scores by subscale immediately pre and post intervention. SSQ scores were averaged before and after the intervention for treatment and control groups. Error bars represent the standard error of the mean; ***p < .001.

The significance of each parameter included in the full model (with factors for five-second heartrate, biological sex, self-reported dizziness, and self-reported nausea) was compared to an intercept-only, null model. Inclusion of each factor (five-second heartrate, biological sex, self-reported dizziness, and self-reported nausea) significantly improved the model fit relative to the null model, χdiff2(1)=6.08, p=.014, χdiff2(1)=6.12, p=.013, χdiff2(1)=21.51, p<.001, χdiff2(1)=23.84, p<.001, respectively.1 The overall model resulted in a McFadden R2=.74, suggesting that our full model could explain 74% of the variability in the probability that a subject would drop out of the study. Please see Table 1 for exponent B values and significance levels. The area under a receiving operating characteristic curve (AUC) was determined to quantify the ability to discriminate between participants who dropped out and participants who remained in the experiment. This produced an AUC of 0.83. An analysis of five-second heartrate change revealed an increase of 15.5 bpm resulted in a participant having an equal probability of dropping out or remaining in the experiment. An increase of 4.5 self-reported nausea and 5.5 self-reported dizziness both on a scale of 1–10, also gave a participant an equal probability of dropping out or remaining in the experiment (see Figure 5). Females were more likely to drop out (n = 10); see Table 1.
Table 1.
Exponent B and Significance Values for Predictors
PredictorBS.E.Waldp
Heartrate 0.242 0.116 4.382 0.036 
Change 
Female (yes) −4.226 2.098 4.059 0.044 
Dizziness 1.461 0.642 5.178 0.023 
Nausea 1.307 0.583 5.036 0.025 
Constant −6.834 3.660 −1.867 0.062 
PredictorBS.E.Waldp
Heartrate 0.242 0.116 4.382 0.036 
Change 
Female (yes) −4.226 2.098 4.059 0.044 
Dizziness 1.461 0.642 5.178 0.023 
Nausea 1.307 0.583 5.036 0.025 
Constant −6.834 3.660 −1.867 0.062 

Note. df = 1.

Figure 5.

Probability of dropping out based on heartrate change from baseline and self-reported symptoms.

Figure 5.

Probability of dropping out based on heartrate change from baseline and self-reported symptoms.

Gaming experience did not remain in the final model after best subset regression (see analyses section). On average, however, female participants had less gaming experience (M=1.93,SD=1.11) than male participants (M=3.59,SD=1.46). An independent samples t-test found this difference in gaming experience to be significant t(62)=-5.05, p<.001.

4 Discussion

The first goal of this work was to determine if a general replication of a vision therapy study (Rine et al., 1999) would provide better adaptivity to VR by reducing heartrate and cybersickness symptoms when compared to passive VR exposure. Second, we sought to identify reliable predictors of participant attrition from VR studies through continuous heartrate measures and questionnaires. We found there was no significant difference between our treatment intervention and a control condition in terms of heartrate and SSQ scores for both intervention efficacy and mitigation. We did, however, find a significant decrease in heart rate across sessions but no significant difference in SSQ scores across sessions. Our finding of a decrease in heartrate correlating with a decrease in motion sickness is consistent with previous studies in both VR and the real-world. These real-world studies involved the rotation of a subject off a vertical axis to induce motion sickness, and likely related increases in parasympathetic activity (Russell, Hoffman, Stromberg, & Carlson, 2014; Sang, Billar, Gresty, & Golding 2005; Stromberg, Russell, & Carlson, 2015).

In contrast to heartrate, however, the SSQ scores showed no difference across sessions suggesting the intervention had no effect (see Figure 4). This trend resembles that of Regan's (1995) study in which repeated immersions showed a decrease in cybersickness symptoms but SSQ scores failed to reach statistical significance. As suggested in Regan's (1995) study, and potentially in our data as well, this may be due to a need for continued immersive experiences to reveal the full course of adaptation to VR. Our conflicting results across heart rate and SSQ measures may be due to the single immersion design, for example, the Rine et al. (1999) patient reported improved symptoms after 10 days of repeated therapy. Another possibility, however, is that subjective reports in the SSQ were simply not as sensitive to changes in cybersickness as physiological markers such as heart rate. While it is still possible that greater immersion in VR and exposure to the intervention may have been needed to more fully mitigate cybersickness, consistent with other studies that have found multiple short immersions to be effective for VR adaptation (Domeyer et al., 2013; Howarth & Hodder, 2008; Teasdale et al., 2009), if there were no adaptation in our study, then we would expect heart rate and SSQ scores to remain constant across sessions. Future studies may wish to investigate the symptomatic effect of multiple sessions, the time between sessions, and the content of the adaptation intervention used [e.g., active mitigating approach versus natural decay (recovery from cybersickness outside of the VR environment) versus traditional vision therapy].

We also analyzed the mitigative properties of VR training and passive VR exposure by comparing the SSQ scores before and after the intervention (or control). Both VR conditions significantly mitigated Total SSQ and Nausea SSQ scores post-VR navigation of Session 1 and were marginally significant at mitigating Ocular-Motor SSQ scores. This suggested that although nausea scores and total scores improved as a function of exposure to VR, the intervention itself appeared ineffective at reducing SSQ scores in terms of Condition, which is consistent with heartrate data (see Figure 5). These results suggest there may be a mitigating mechanism that is effective at reducing symptoms associated with nausea such as salivation, sweating, and dizziness, but not ocular-motor symptoms such as eye strain or blurred vision according to the modified SSQ questionnaire by Bouchard et al. (2007). Thus, while our findings suggest that a short-term intervention is unlikely to be effective in reducing physiological symptoms of cybersickness, our data do suggest that longer exposures to VR may result in some gradual adaptations, at least in heart rate.

A secondary aim of this study was to identify reliable predictors of study–attrition due to cybersickness through the use of heartrate monitoring and questionnaires. We found heartrate, self-reported symptoms, and biological sex to be reliable predictors of attrition. Weight, sleep, hours since last meal, gaming experience, previous VR use, and stress were not found to be reliable predictors of attrition. Our heartrate findings are comparable to research by LaCount et al. (2011) where an increase in heartrate from baseline around 15 bpm is associated with strong levels of nausea. Participants with an increase in self-reported symptoms were also more likely to drop out. Female participants were also more likely to drop out, which is in agreement with some prior studies (Holmes & Griffin, 2001; Munafo, Diedrick, & Stoffregen, 2017). These results suggest heartrate, self-reported symptoms, and biological sex may be reliable predictors of attrition in VR research and may be useful in reducing attrition rates. Notably, though, female participants had less gaming experience (see Results), which may have contributed to their tendency to drop out independent of biological sex.

Together, our study revealed two important findings that we believe will be of interest in the virtual reality community. First, we found that a short intervention involving eye movements and based on a previously developed “real-world” motion sickness mitigation paradigm, did not appear to be effective. As discussed, because we did see a gradual reduction in heart rate over sessions, which in turn was related to the probability of dropping out of the study, our findings suggest that longer (but not brief) focused exposure to virtual reality may be effective in reducing symptoms of cybersickness. These findings, therefore, suggest that to effectively mitigate cybersickness, a short and focused intervention, at least like the one used here, is unlikely to be effective. Instead, perhaps to overcome vestibular mismatch, repeated and slow adaptation is necessary to avoid the sudden spikes in heart rate and other physiological precursors of nausea.

The second important finding of our study was that we could predict participant dropout based on changes in heartrate and that this measure was more sensitive than other dependent measures we collected, like the SSQ. Predicting cybersickness is an important goal, both in terms of mitigating its symptoms and avoiding more extreme reactions to virtual reality, such as vomiting, which in turn could create stronger aversions to virtual reality. Our results thus suggest that heartrate monitoring could be used in a situation in which participants might be expected to experience cybersickness, such as their first exposure to virtual reality, or novel virtual reality interfaces, like omnidirectional treadmills (Starrett, Stokes, Huffman, Ferrer, & Ekstrom, 2019).

Our study has some important limitations that are worth mentioning. The intervention we employed had not been used previously to mitigate cybersickness in the form we rendered it, and instead was based on a version employed in the real world. Nonetheless, we believe that our null finding is useful in terms of suggesting that this particular intervention, when employed for short periods, is not likely to be effective. Another limitation is that we did not collect a complete battery of physiological measures, including measures such as pupil dilation, which might improve predictive power compared to heart rate alone.

Furthermore, implementing a comparison questionnaire asking which navigation session may have induced more cybersickness in a randomized order might have also been helpful. Vergence–accommodation conflicts were also not accounted for in our experiment, although this may have a lesser effect in the treatment and control environments rather than the navigation environment because of variability in where participants picked up the cubes.

Participant motivation to take or not take 5-minute breaks between blocks of the navigation sessions was also not assessed. This might have been useful in inferring possible cybersickness influences on participant behavior, that is, finishing as fast as possible. Although the option to take breaks during navigation may prevent a perception of mandated continuity that otherwise might lead to significant cybersickness and possibly vomiting, future studies should still assess participant motivation with this design or instead implement mandatory 5-minute breaks for consistency and participant safety.

Finally, our findings of greater attrition for female participants compared to male participants should be handled with some caution. Because we did find a significant difference in gaming experience but did not calculate body–mass index, we cannot be sure that these other variables did not somehow explain our gender-related findings.

Acknowledgments

This research was supported by NSF BCS-1630296 (ADE), the UC Davis Chancellor's Award (ADE), and the Mentorship for Undergraduate Research in Agriculture, Letters, and Science (MURALS) program at the University of California, Davis (CEP).

Notes

1

The reported χdiff2 test statistics are the change in deviance between models.

REFERENCES

Barrett
,
B. T.
(
2009
).
A critical evaluation of the evidence supporting the practice of behavioural vision therapy
.
Ophthalmic and Physiological Optics
,
29
(
1
),
4
25
.
Retrieved from
https://doi.org/10.1111/j.1475-1313.2008.00607.x
Benson
,
A. J.
(
2002
).
Motion sickness
. In
K. B.
Pandolf
&
R. E.
Burr
(Eds.),
Medical aspects of harsh environments
(pp.
1059
1094
).
Washington, DC
:
Textbooks of Military Medicine
.
Bergeron
,
M.
,
Lortie
,
C. L.
, &
Guitton
,
M. J.
(
2015
).
Use of virtual reality tools for vestibular disorders rehabilitation: A comprehensive analysis
.
Advances in Medicine
,
2015
,
1
9
. https://doi.org/10.1155/2015/916735
Bouchard
,
S.
,
Robillard
,
G.
,
Larouche
,
S.
, &
Loranger
,
C.
(
2012
).
Description of a treatment manual for in virtuo exposure with specific phobia
. In
C.
Eichenberg
(Ed.),
Virtual reality in psychological, medical and pedagogical applications
(
Issue October 2014
, pp.
81
108
).
Jena, Germany
:
IntechOpen
.
Retrieved from
https://doi.org/10.5772/46417
Bouchard
,
S.
,
Robillard
,
G.
, &
Renaud
,
P.
(
2007
).
Revising the factor structure of the simulator sickness questionnaire
.
Annual Review of Cyber Therapy and Telemedicine
,
5
,
117
122
.
Retrieved from
https://doi.org/10.1007/s13398-014-0173-7.2
Boyer
,
F. C.
,
Percebois-Macadré
,
L.
,
Regrain
,
E.
,
Lévêque
,
M.
,
Taïar
,
R.
,
Seidermann
,
L.
,
Belassian
,
G.
, &
Chays
,
A.
(
2008
).
Vestibular rehabilitation therapy
.
Neurophysiologie Clinique/Clinical Neurophysiology
,
38
(
6
),
479
487
.
Retrieved from
https://doi.org/10.1016/j.neucli.2008.09.011
Buchanan
,
E. M.
,
Gillenwaters
,
A. M.
,
Scofield
,
J. E.
, &
Valentine
,
K. D.
(
2019
).
MOTE: Effect size and confidence interval calculator
.
R Foundation for Statistical Computing (Version 0.0.0.9100) [Software]. Available from
https://cran.r-project.org/package=MOTE
Cawthorne
,
T. C.
(
1945
).
The physiological basis for head exercises
.
The Journal of the Chartered Society of Physiotherapy
,
30
,
106
.
Champney
,
R. K.
,
Stanney
,
K. M.
,
Hash
,
P. A. K.
,
Malone
,
L. C.
,
Kennedy
,
R. S.
, &
Compton
,
D. E.
(
2007
).
Recovery from virtual environment exposure: Expected time course of symptoms and potential readaptation strategies
.
Human Factors: The Journal of the Human Factors and Ergonomics Society
,
49
(
3
),
491
506
.
Retrieved from
https://doi.org/10.1518/001872007X200120
Cooksey
,
F. S.
(
1946
).
Rehabilitation in vestibular injuries
.
Journal of the Royal Society of Medicine
,
39
(
5
),
273
278
.
Retrieved from
https://doi.org/10.1177/003591574603900523
Cullen
,
K. E.
,
Brooks
,
J. X.
, &
Sadeghi
,
S. G.
(
2009
).
How actions alter sensory processing: Reafference in the vestibular system
.
Annals of the New York Academy of Sciences
,
1164
(
1
),
29
36
.
Retrieved from
https://doi.org/10.1111/j.1749-6632.2009.03866.x
Curtis
,
M. K.
,
Dawson
,
K.
,
Jackson
,
K.
,
Litwin
,
L.
,
Meusel
,
C.
,
Dorneich
,
M. C.
,
Gilbert
,
S. B.
,
Kelly
,
J.
,
Stone
,
R.
, &
Winer
,
E.
(
2015
).
Mitigating visually induced motion sickness: A virtual handeye coordination task
.
Proceedings of the Human Factors and Ergonomics Society
,
1839
1843
.
Retrieved from
https://doi.org/10.1177/1541931215591397
Dai
,
M.
,
Raphan
,
T.
, &
Cohen
,
B.
(
2011
).
Prolonged reduction of motion sickness sensitivity by visual--vestibular interaction
.
Experimental Brain Research
,
210
(
3–4
),
503
513
.
Retrieved from
https://doi.org/10.1007/s00221-011-2548-8
De Maesschalck
,
R.
,
Jouan-Rimbaud
,
D.
, &
Massart
,
D. L.
(
2000
).
The Mahalanobis distance
.
Chemometrics and Intelligent Laboratory Systems
,
50
(
1
),
1
18
.
Retrieved from
https://doi.org/10.1016/S0169-7439(99)00047-7
Dennison
,
M. S.
,
Wisti
,
A. Z.
, &
D'Zmura
,
M.
(
2016
).
Use of physiological signals to predict cybersickness
.
Displays
,
44
(
2016
),
42
52
.
Retrieved from
https://doi.org/10.1016/j.displa.2016.07.002
Domeyer
,
J. E.
,
Cassavaugh
,
N. D.
, &
Backs
,
R. W.
(
2013
).
The use of adaptation to reduce simulator sickness in driving assessment and research
.
Accident Analysis and Prevention
,
53
(
2013
),
127
132
.
Retrieved from
https://doi.org/10.1016/j.aap.2012.12.039
Fulvio
,
J.
, &
Rokers
,
B.
(
2018
).
Sensitivity to sensory cues predicts motion sickness in virtual reality
.
Journal of Vision
,
18
(
10
),
1066
.
Retrieved from
https://doi.org/10.1167/18.10.1066
Golding
,
J. F.
(
2016
).
Motion sickness
. In
J. M.
Furman
&
T.
Lempert
(Eds.),
Handbook of clinical neurology
(3rd ed., Vol. 137, pp.
371
390
).
London, UK
:
Elsevier
.
Retrieved from
https://doi.org/10.1016/B978-0-444-63437-5.00027-3
Hall
,
C. D.
,
Heusel-Gillig
,
L.
,
Tusa
,
R. J.
, &
Herdman
,
S. J.
(
2010
).
Efficacy of gaze stability exercises in older adults with dizziness
.
Journal of Neurologic Physical Therapy
,
34
(
2
),
64
69
.
Retrieved from
https://doi.org/10.1097/NPT.0b013e3181dde6d8
Hill
,
K. J.
, &
Howarth
,
P. A.
(
2000
).
Habituation to the side effects of immersion in a virtual environment
.
Displays
,
21
(
1
),
25
30
.
Retrieved from
https://doi.org/10.1016/S0141-9382(00)00029-9
Hoffman
,
D. M.
,
Girshick
,
A. R.
,
Akeley
,
K.
, &
Banks
,
M. S.
(
2008
).
Vergence–accommodation conflicts hinder visual performance and cause visual fatigue
.
Journal of Vision
,
8
(
3
),
33
.
Retrieved from
https://doi.org/10.1167/8.3.33
Holmes
,
S. R.
, &
Griffin
,
M. J.
(
2001
).
Correlation between heart rate and the severity of motion sickness caused by optokinetic stimulation
.
Journal of Psychophysiology
,
15
(
1
),
35
42
.
Retrieved from
https://doi.org/10.1027//0269-8803.15.1.35
Howarth
,
P. A.
, &
Hodder
,
S. G.
(
2008
).
Characteristics of habituation to motion in a virtual environment
.
Displays
,
29
(
2
),
117
123
.
Retrieved from
https://doi.org/10.1016/j.displa.2007.09.009
Jarosz
,
A. F.
, &
Wiley
,
J.
(
2014
).
What are the odds? A practical guide to computing and reporting Bayes factors
.
Journal of Problem Solving
,
7
(
1
),
2
9
.
Retrieved from
https://doi.org/10.7771/1932-6246.1167
Kennedy
,
R. S.
,
Lane
,
N. E.
,
Berbaum
,
K. S.
, &
Lilienthal
,
M. G.
(
1993
).
Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness
.
The International Journal of Aviation Psychology
,
3
(
3
),
203
220
.
Retrieved from
https://doi.org/10.1207/s15327108ijap0303_3
Keshavarz
,
B.
,
Riecke
,
B. E.
,
Hettinger
,
L. J.
, &
Campos
,
J. L.
(
2015
).
Vection and visually induced motion sickness: How are they related
?
Frontiers in Psychology
,
6
(
472
).
Retrieved from
https://doi.org/10.3389/fpsyg.2015.00472
Kim
,
Y. Y.
,
Kim
,
H. J.
,
Kim
,
E. N.
,
Ko
,
H. D.
, &
Kim
,
H. T.
(
2005
).
Characteristic changes in the physiological components of cybersickness
.
Psychophysiology
,
42
(
2005
),
616
625
.
Retrieved from
https://doi.org/10.1111/j.1469-8986.2005.00349.x
Koulieris
,
G. A.
,
Bui
,
B.
,
Banks
,
M. S.
, &
Drettakis
,
G.
(
2017
).
Accommodation and comfort in head-mounted displays
.
Association for Computing Machinery Transactions on Graphics
,
36
(
4
),
1
11
.
Retrieved from
https://doi.org/10.1145/3072959.3073622
LaCount
,
L. T.
,
Barbieri
,
R.
,
Park
,
K.
,
Kim
,
J.
,
Brown
,
E. N.
,
Kuo
,
B.
, &
Napadow
,
V.
(
2011
).
Static and dynamic autonomic response with increasing nausea perception
.
Aviation Space and Environmental Medicine
,
82
(
4
),
424
433
.
Retrieved from
https://doi.org/10.3357/ASEM.2932.2011
Mahalanobis
,
P. C.
(
1936
).
On the generalised distance in statistics
.
Proceedings of the National Institute of Sciences of India
,
2
(
1
),
49
55
.
Retrieved from
https://doi.org/0.1007/s10551-015-2583-7
Mcleod
,
A. I.
,
Xu
,
C.
, &
Lai
,
Y.
(
2018
).
Bestglm : Best subset GLM and regression utilities
.
R Foundation for Statistical Computing (Version 0.37) [Software]. Available from
https://cran.r-project.org/web/packages/bestglm/index.html
Morey
,
R. D.
,
Rouder
,
J. N.
,
Jamil
,
T.
,
Urbanek
,
S.
,
Forner
,
K.
, &
Ly
,
A.
(
2018
).
Bayesfactor: Computation of Bayes factors for common designs
.
R Foundation for Statistical Computing (Version 0.9.12-2) [Software]. Available from
https://cran.r-project.org/web/packages/BayesFactor/index.html
Morimoto
,
H.
,
Asai
,
Y.
,
Johnson
,
E. G.
,
Lohman
,
E. B.
,
Khoo
,
K.
,
Mizutani
,
Y.
, &
Mizutani
,
T.
(
2011
).
Effect of oculo-motor and gaze stability exercises on postural stability and dynamic visual acuity in healthy young adults
.
Gait and Posture
,
33
(
4
),
600
603
.
Retrieved from
https://doi.org/10.1016/j.gaitpost.2011.01.016
Munafo
,
J.
,
Diedrick
,
M.
, &
Stoffregen
,
T. A.
(
2017
).
The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects
.
Experimental Brain Research
,
235
(
3
),
889
901
.
Retrieved from
https://doi.org/10.1007/s00221-016-4846-7
Naimi
,
B.
(
2015
).
Usdm: Uncertainty analysis for species distribution models
.
R Foundation for Statistical Computing (Version 1.1-18) [Software]. Available from
https://cran.r-project.org/web/packages/usdm/index.html
Oman
,
C. M.
(
1990
).
Motion sickness: A synthesis and evaluation of the sensory conflict theory
.
Canadian Journal of Physiology and Pharmacology
,
68
(
2
),
294
303
.
Retrieved from
https://doi.org/10.1139/y90-044
Oman
,
C. M.
, &
Cullen
,
K. E.
(
2014
).
Brainstem processing of vestibular sensory exafference: Implications for motion sickness etiology
.
Experimental Brain Research
,
232
(
8
),
2483
2492
.
Retrieved from
https://doi.org/10.1007/s00221-014-3973-2
Pavlou
,
M.
,
Lingeswaran
,
A.
,
Davies
,
R. A.
,
Gresty
,
M. A.
, &
Bronstein
,
A. M.
(
2004
).
Simulator based rehabilitation in refractory dizziness
.
Journal of Neurology
,
251
(
8
),
983
995
.
Retrieved from
https://doi.org/10.1007/s00415-004-0476-2
Rawstron
,
J. A.
,
Burley
,
C. D.
, &
Elder
,
M. J.
(
2005
).
A systematic review of the applicability and efficacy of eye exercises
.
Journal of Pediatric Ophthalmology and Strabismus
,
42
(
2
),
82
88
.
Retrieved from
https://doi.org/10.3928/01913913-20050301-02
RCore
Team
. (
2018
).
R: A language and environment for statistical computing
.
R Foundation for Statistical Computing (Version 3.4.4) [Software]. Available from
https://www.r-project.org/
Reason
,
J. T.
(
1978
).
Motion sickness adaptation: A neural mismatch model
.
Journal of the Royal Society of Medicine
,
71
(
11
),
819
829
.
Retrieved from
https://doi.org/10.1177/014107687807101109
Regan
,
E. C.
(
1995
).
Some evidence of adaptation to immersion in virtual reality
.
Displays
,
16
(
3
),
135
139
.
Retrieved from
https://doi.org/10.1016/0141-9382(96)81213-3
Rine
,
R. M.
,
Schubert
,
M. C.
, &
Balkany
,
T. J.
(
1999
).
Visual-vestibular habituation and balance training for motion sickness
.
Physical Therapy
,
79
(
10
),
949
957
.
Retrieved from
https://doi.org/10.1093/ptj/79.10.949
Rouder
,
J. N.
,
Morey
,
R. D.
,
Speckman
,
P. L.
, &
Province
,
J. M.
(
2012
).
Default Bayes factors for ANOVA designs
.
Journal of Mathematical Psychology
,
56
(
5
),
356
374
.
Retrieved from
https://doi.org/10.1016/j.jmp.2012.08.001
RStudio
Team
. (
2018
).
R Studio: Integrated development environment for R
.
RStudio, Inc. (Version 1.1.456) [Software]. Available from
https://rstudio.com/
Russell
,
M. E. B.
,
Hoffman
,
B.
,
Stromberg
,
S.
, &
Carlson
,
C. R.
(
2014
).
Use of controlled diaphragmatic breathing for the management of motion sickness in a virtual reality environment
.
Applied Psychophysiology and Biofeedback
,
39
(
3–4
),
269
277
.
Retrieved from
https://doi.org/10.1007/s10484-014-9265-6
Sacks
,
O.
(
2009
).
What hallucination reveals about our minds
. Presented at TED2009 conference,
Long Beach, CA
.
Available from
https://www.ted.com/talks/oliver_sacks_what_hallucination_reveals_about_our_minds
Sang
,
F. Y. P.
,
Billar
,
J.
,
Gresty
,
M. A.
, &
Golding
,
J. F.
(
2005
).
Effect of a novel motion desensitization training regime and controlled breathing on habituation to motion sickness
.
Perceptual and Motor Skills
,
101
(
1
),
244
256
.
Retrieved from
https://doi.org/10.2466/pms.101.1.244–256
Sevinc
,
V.
, &
Berkman
,
M. I.
(
2020
).
Psychometric evaluation of simulator sickness questionnaire and its variants as a measure of cybersickness in consumer virtual environments
.
Applied Ergonomics
,
82
(
2020
).
Retrieved from
https://doi.org/10.1016/j.apergo.2019.102958
Sharples
,
S.
,
Cobb
,
S.
,
Moody
,
A.
, &
Wilson
,
J. R.
(
2008
).
Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems
.
Displays
,
29
(
2
),
58
69
.
Retrieved from
https://doi.org/10.1016/j.displa.2007.09.005
Solimini
,
A. G.
(
2013
).
Are there side effects to watching 3D movies? A prospective crossover observational study on visually induced motion sickness
.
PLOS ONE
,
8
(
2
).
Retrieved from
https://doi.org/10.1371/journal.pone.0056160
Starrett
,
M. J.
,
Stokes
,
J. D.
,
Huffman
,
D. J.
,
Ferrer
,
E.
, &
Ekstrom
,
A. D.
(
2019
).
Learning-dependent evolution of spatial representations in large-scale virtual environments
.
Journal of Experimental Psychology: Learning Memory and Cognition
,
45
(
3
),
497
514
.
Retrieved from
https://doi.org/10.1037/xlm0000597
Stromberg
,
S. E.
,
Russell
,
M. E.
, &
Carlson
,
C. R.
(
2015
).
Diaphragmatic breathing and its effectiveness for the management of motion sickness
.
Aerospace Medicine and Human Performance
,
86
(
5
),
452
457
.
Retrieved from
https://doi.org/10.3357/AMHP.4152.2015
Teasdale
,
N.
,
Lavallière
,
M.
,
Tremblay
,
M.
,
Laurendeau
,
D.
, &
Simoneau
,
M.
(
2009
).
Multiple exposition to a driving simulator reduces simulator symptoms for elderly drivers
. In Proceedings of the Fifth International Driving Symposium on Human Factors in Driver Assessment.
Iowa City, Iowa
:
University of Iowa
.
Retrieved from
https://doi.org/10.17077/drivingassessment.1318
Treisman
,
M.
(
1977
).
Motion sickness: An evolutionary hypothesis
.
Science
,
197
(
4302
),
493
495
.
Retrieved from
https://doi.org/10.1126/science.301659
Whitney
,
S. L.
,
Alghwiri
,
A. A.
, &
Alghadir
,
A.
(
2016
).
An overview of vestibular rehabilitation
. In
J. M.
Furman
&
T.
Lempert
(Eds.),
Handbook of clinical neurology
(1st ed., Vol. 137, pp.
187
205
).
Pittsburgh, PA
:
Elsevier
.
Retrieved from
https://doi.org/10.1016/B978-0-444-63437-5.00013-3
Wickham
,
H.
(
2007
).
Reshaping data with the reshape package
.
Journal of Statistical Software
,
21
(
12
),
1
20
.
Retrieved from
https://doi.org/10.18637/jss.v021.i12
Wickham
,
H.
,
Chang
,
W.
,
Henry
,
L.
,
Lin Pedersen
,
T.
,
Takahashi
,
K.
,
Wilke
,
C.
,
Woo
,
K.
,
Yutani
,
H.
, &
Dunnington
,
D.
(
2009
).
Ggplot2: Create elegant data visualisations using the grammar of graphics
.
R Foundation for Statistical Computing (Version 2.2.1) [Software]. Available from
https://cran.r-project.org/web/packages/ggplot2/index.html
Young
,
S. D.
,
Adelstein
,
B. D.
, &
Ellis
,
S. R.
(
2007
).
Demand characteristics in assessing motion sickness in a virtual environment: Or does taking a motion sickness questionnaire make you sick
?
IEEE Transactions on Visualization and Computer Graphics
,
13
(
3
),
422
428
.
Retrieved from
https://doi.org/10.1109/TVCG.2007.1029
Zhang
,
Z.
(
2016
).
Variable selection with stepwise and best subset approaches
.
Annals of Translational Medicine
,
4
(
7
),
136
.
Retrieved from
https://doi.org/10.21037/atm.2016.03.35
Zuur
,
A. F.
,
Ieno
,
E. N.
, &
Elphick
,
C. S.
(
2010
).
A protocol for data exploration to avoid common statistical problems
.
Methods in Ecology and Evolution
,
1
(
1
),
3
14
.
Retrieved from
https://doi.org/10.1111/j.2041-210x.2009.00001.x