Abstract
Cinematic virtual reality offers 360-degree moving image experiences that engage a viewer's body as its position defines the momentary perspective over the surrounding simulated space. While a 360-degree narrative space has been demonstrated to provide highly immersive experiences, it may also affect information intake and the recollection of narrative events. The present study hypothesizes that the immersive quality of cinematic VR induces a viewer's first-person perspective in observing a narrative in contrast to a camera perspective. A first-person perspective is associated with increase in emotional engagement, sensation of presence, and a more vivid and accurate recollection of information. To determine these effects, we measured viewing experiences, memory characteristics, and recollection accuracy of participants watching an animated movie either using a VR headset or a stationary screen. The comparison revealed that VR viewers experience a higher level of presence in the displayed environment than screen viewers and that their memories of the movie are more vivid, evoke stronger emotions, and are more likely to be recalled from a first-person perspective. Yet, VR participants can recall fewer details than screen viewers. Overall, these results show that while cinematic virtual reality viewing involves more immersive and intense experiences, the 360-degree composition can negatively impact comprehension and recollection.
1 Introduction
Virtual reality (VR) screening platforms have lately grown in immense popularity for the general public, which is clearly demonstrated by the increasingly accessible screening appliances and the rapidly rising number of cinematic virtual reality content—some of which are presented at film festivals or feature well-known Hollywood actors. The term, cinematic virtual reality, encompasses immersive 360-degree and interactive film-like entertainment (MacQuarrie & Steed, 2017; Mateer, 2017). In the context of this study, the term refers to 360-degree animated or live-action movies, which allow for changes of viewing perspective of the fictional space during the screening by an individual viewer's head or full-body movements.
Accordingly, the novelty of cinematic virtual reality lies in the viewer's access to a 360-degree simulated space and the fact that spectatorship engages one's body as its position defines the momentary perspective over the surrounding simulated space. This also means that, unlike watching a screen that delimits and controls attention over the fictional environment, cinematic virtual reality induces different experiences for each viewer, where he or she may choose to which areas of the 360-degree space to pay attention and, consequently, what will fall out of the view at any moment. These qualities may have implications for viewing experiences. The purpose of this study is to explore these implications and compare cinematic virtual reality with screen-based viewing in terms of viewing experiences and the recollection of narrative events. Our objective is to draw conclusions on the differences between individual viewing of 360-degree and two-dimensional content. With this objective, we aim to provide conclusions for the design of immersive moving-image narratives and for research into the mental processes related to virtual reality experiences, such as attention, emotional engagement, comprehension, and recollection.
1.1 Viewing Experiences and the Sense of Presence
Engagement with mediated stimuli in virtual reality has been approached through a variety of aspects, including a medium's immersive quality and viewers’ sensation of presence and its effects on emotional engagement (Baños et al., 2004; Cummings & Bailenson, 2016). Immersive quality and presence intertwine in the technology and media industries’ pursuit for enhancing user effects and the effectiveness of conveying mediated content. As Slater and Wilbur (1997) defined it, presence refers to an individual experience supported by a medium's immersive quality and is dependent on audiovisual fidelity, the correspondence between the physical body and virtual actions, and the extent to which it distances a user from a physical environment. Wirth et al. (2007) note that these qualities lead to a user's sensation of presence after perceiving the virtual environment as logical and plausible and developing the sensation of being in it. Balakrishnan and Sundar (2011) extend Wirth and colleagues’ theory by users’ perception of potential interactions afforded by immersive platforms.
The mentioned definitions intertwining the evolution of virtual reality experiences correspond with the idea that a user's body gains agency by integrating motor actions with changes in the visual and auditory spectrum, which are attributed to sensorimotor contingencies that a virtual reality system supports (Slater, 2009). In relation to this, previous studies provide extensive results on the sense of user presence in virtual environments based on bodily agency (Biocca, 2002; Shin & Biocca, 2018; Slater, 2009) and interactivity (Mütterlein, 2018; Reyes, 2017) as well as measurement methods—even for narrative audiovisual experiences (Lee, 2004a, 2004b; Schwind, Knierim, Haas, & Henze, 2019).
In terms of narrative moving images in general, the condition of a viewer's sensation of presence has been identified in narrative transportation; the sensation of being transported into a fictional world by establishing an ecological connection to the depicted environment and empathizing with its characters (Bálint & Tan, 2015; Gerrig, 1993). This assumes a viewer's identification of the environment and an emotional connection to actions and characters.
Research into the specific case of cinematic virtual reality and 360-degree spectatorship, however, has yet to provide comprehensive knowledge regarding the effects of storytelling and viewers’ presence. Recent studies on 360-degree storytelling, including Chang's (2016), Dooley's (2017), Nielsen et al.’s (2016), and Mateer's (2017) works, have analyzed definitions, formats, and existing cinematic virtual reality content. Mateer (2017), for instance, highlights that 360-degree storytelling roots in the attention control of Hollywood-style feature films, while also addresses the challenges a 360-degree visual field entails. He argues that while the immersive quality of cinematic virtual reality content may be compelling, directing must take proprioceptive characteristics—defined by a viewer's body, its position, and motion—into account. In other words, Mateer calls attention to the limitations of VR storytelling, namely that a viewer's posture may affect access to narrative information, even though it implies increased immersion.
Other recent studies, including those of Bala, Dionisio, Nisi, and Nunes (2016), Bala, Nisi, and Nunes (2017), and Reyes (2018) provide strategies for telling and evaluating interactive immersive stories in virtual reality. Interactivity is assessed based on agency, decision making, and engagement with the fictional space and narrative events (Roth & Koenitz, 2016). Correspondingly, Reyes (2018) proposes that cinematic virtual reality experiences are defined by the combination of cinematic storytelling, spatial cues, like sound effects, as well as cognitive and biological effects, such as the sensation of presence, attention, or even physical discomfort.
Extended reality contents’ immersive qualities and attention control are often assessed through stimulus quality, that is, the way a sudden noise or intense light would guide one's attention in a virtual environment (Cummings & Bailenson, 2016). However, in the case of cinematic VR, it is also important to reflect on social and narrative elements (e.g., empathizing with fictional characters; Hassan, 2019) as well as how a movie's visual presentation and the lack of framing would affect engagement and comprehension, and that attention can be guided using visual and sonic diegetic cues (Gödde, Gabler, Siegmund, & Braun, 2018; Kvisgaard et al., 2019; Rothe & Hußmann, 2018).
Offering empirical findings on the matters of storytelling and viewing experiences of 360-degree and virtual reality content, Syrett, Calvi, and van Gisbergen (2016) and Van Damme, All, Marez, and Leuven (2019) measured viewers’ sense of involvement while watching a short feature film using a virtual reality headset and a 360-degree news report, respectively. Both studies concluded that the moving-image stimulus (film in VR and 360-degree news report) evokes a strong sense of involvement and empathy with the depicted subjects of the story.
In the lack of a control group (that would allow for comparing responses to other types of screening technology or content), Syrett et al.’s (2016) experiment only highlights that the majority of their participants sufficiently comprehended the story and claimed to have had an immersive experience even though the novelty of the experience was somewhat distracting to them. Van Damme et al. (2019) compared four viewing conditions for the 360-degree news item: participants watched the same footage on YouTube on a screen either with a fixed viewpoint or with enabled viewpoint changes (“drag-and-drop”), using a cardboard viewer, or an Oculus Rift virtual reality headset. The authors observed an increase in participants’ sense of presence and fidelity when watching the footage using the cardboard viewer and Oculus Rift.
In another study, Fonseca and Kraus (2016) compared the sensation of presence and a video's emotional impacts between participants who watched it using either a virtual reality headset or on a ten-inch tablet. They found that VR watching increases emotional impacts and VR viewers rate their sense of presence higher than those who used the tablet.
1.2 Recollection and the Perspective of Observation
In addition to the sense of presence and engagement with a cinematic virtual reality narrative, virtual reality viewing evokes another crucial question, that of the correlation between engagement and the perspective of observation. Film scholarship has a history of treating the camera perspective as a neutral information-projecting agent and an aesthetic feature (Bordwell & Thompson, 2001; Eisenstein, 1982; Grodal, 1997). According to this thesis, the camera perspective defines the presence/absence, arrangement, and motion of narrative elements (e.g., characters, objects) in relation to the screen's edges. Visual composition in narrative films can affect cognitive processes and viewers’ responses: a high level of synchrony has been observed across viewers in brain activation (Hasson, Furman, Clark, Dudai, & Davachi, 2008; Hasson, Nir, Levy, Fuhrmann, & Malach, 2004) and eye movements (Smith & Henderson, 2008; Smith & Mital, 2013) when watching narrative moving-image sequences.
The effects of narrative films notwithstanding, due to the active bodily agency in virtual environments, viewers may be more likely to adopt a subjective (first-person) point of view instead of a camera (observer) perspective. This is based on the idea that the correspondence between the movement of a physical body (head movements, turning around) and the effects it induces (changes in perspective) would increase the sense of being in the virtual space (Slater & Wilbur, 1997; Van den Broeck, Kawsar, & Schöning, 2017). In other words, during virtual reality viewing, bodily involvement and a viewer's role in defining the sensory scope can generate the sensation that the camera's point of view corresponds with the viewer's own perspective, thereby evoking a strong sensation of inhabiting the fictional space. This may influence how narrative events are encoded and retrieved from memory.
Previous research on autobiographical memories has investigated phenomenological characteristics (e.g., vividness and emotion) and visual perspective of personal memories—whether memory is observed from one's own or from the viewpoint of an outside observer. A first-person perspective is associated with higher emotional intensity and an increased amount of perceptual details, while an observer perspective is associated with lower emotional intensity and less contextual information (Nigro & Neisser, 1983; Rice & Rubin, 2009; Robinson & Swanson, 1993; St. Jacques, 2019). The subjective perspective and bodily control afforded by cinematic virtual reality can result in experiences encoded in memory using a first-person perspective, yielding higher emotional intensity. This may also lead to a higher sense of presence when recollecting the experience.
Research into memory assessment methods highlights the potentials of immersive moving-image content: Serino and Repetto (2018) see the advantages of 360 videos for memory studies in, among other things, their capacities of effectuating an egocentric view of the depicted environments and presenting a near-natural visual experience. According to them and others (e.g., Rubin & Umanath, 2015), this egocentric view contributes to the recollection of events through reliving them.
1.3 Research Gap and the Present Study
Previous research discussed above has provided findings on viewer responses to 360-degree moving-image content. But while Syrett et al. (2016) drew conclusions on VR film viewers’ immersion and comprehension, their study design could not put cinematic virtual reality experiences into the context of other viewing platforms. Other studies, such as those by Fonseca and Kraus (2016) and Van Damme et al. (2019), have compared 360-degree experiences to other platforms (portable screens and other screen- and headset-based 360-degree video platforms, respectively). Yet, we argue, their contributions must be extended with the aspect of film storytelling to highlight the capacities of cinematic virtual reality in comparison with stationary screen viewing that provides a frame for the visual content but lacks the options for customization mid-screening. Storytelling provides an aspect that was not directly addressed in previous empirical inquiries of cinematic virtual reality: we find it essential to address how the viewer's bodily control affects not only access and engagement with narrative information, but also the mechanisms behind recollection, attention, and viewing perspective.
The aim of the present study is to fill a gap in understanding the immersive power of virtual reality spectatorship and provide novel insights into the effects of cinematic virtual reality on viewers’ emotional engagement, sense of presence, comprehension, recollection, and memory characteristics—including the perspective of recollection. To determine the effects of virtual reality spectatorship on these factors, we recruited volunteers for an experiment. Participants in this experiment watched a short animated movie using either a VR headset or a screen, then rated their experiences and completed a performance test that measured recollection of the movie narrative.
The present study is based on the hypotheses explained below. Due to viewers’ bodily and sensorimotor involvement, cinematic virtual reality screenings likely evoke the sensation of a first-person perspective, while screen-based spectatorship induces a third-person (camera) perspective. It follows that virtual reality spectatorship increases engagement with narrative events compared to screen-based viewing, which we hypothesize to positively affect event recollection and comprehension.1
Due to the sensation of a first-person perspective, we also hypothesize that participants rate their viewing experience (i.e., emotional engagement and sense of presence) higher in the virtual reality condition. This means that they would feel more engaged with the narrative, the fictional space, and characters while being less aware of the surrounding physical space or their personal worries while using virtual reality headsets compared with those watching the movie on a screen. Correspondingly, we predict that memories of the movie and viewing thereof would be experienced more vividly and accompanied by bodily manifestations of emotions relative to screen viewers.
Recollection accuracy reveals information about the effects of immersive viewing and viewers’ spatial orientation in the fictional environment. On one hand, the increased sense of engagement and presence in the case of virtual reality viewing as well as VR viewers’ first-person perspective may positively affect recollection. This means that VR viewers would be able to remember details of the movie better. On the other hand, the 360-degree visual field may hinder VR viewers from paying attention to all details resulting in poorer performance in the recollection accuracy test.
Based on these hypotheses, this study targets to answer the following research questions:
Research question 1: How does VR viewing affect emotions, engagement, and the overall viewing experience compared with screen-based spectatorship?
Research question 2: How does VR viewing affect memory characteristics (such as how the memory is experienced in terms of visual perspective, visual imagery, spatial arrangement, and emotion) compared with screen-based spectatorship?
Research question 3: How does VR viewing affect recollection accuracy compared with screen-based spectatorship?
2 Method
2.1 Design
Aiming to isolate the effects of virtual reality (360-degree) spectatorship, in the present study, we followed a between-subjects design to reveal whether cinematic virtual reality and screen-based viewing would induce different experiences in terms of engagement, presence, emotions, memory characteristics, and recollection accuracy. Viewing condition served as the independent variable: either virtual reality viewing or screen viewing. For the two viewing conditions, we used two versions of the same film stimulus.
2.2 Film Stimulus
Stills from Pearl (Osborne, 2016) in the 360-degree format, where the viewer can define the direction of viewing within the 6 degrees of freedom (a); and the same scene in the theatrical version of the movie (b). Screenshots made by the authors of the movie available for the public on Steam (a) and YouTube (b).
Stills from Pearl (Osborne, 2016) in the 360-degree format, where the viewer can define the direction of viewing within the 6 degrees of freedom (a); and the same scene in the theatrical version of the movie (b). Screenshots made by the authors of the movie available for the public on Steam (a) and YouTube (b).
The narratological features of Pearl are the same for the VR and theatrical versions. The point of view is objective; viewers observe fictional events and characters from a fixed point of view of the camera rather than from the perspective of a character. The camera's position (point of observation) is fixed and is predominantly situated on the front passenger seat of the car in which the narrative unfolds (see Figure 1). The viewer has no direct access to characters’ thoughts or feelings, only to the verbal and non-verbal manifestation thereof. Further, the viewer is an independent observer who cannot interact with the characters or objects in the fictional universe. Narrative events unfold without the viewer's intervention.
Pearl conforms to the criteria for measuring indices of emotions, engagement, and memory containing a complex narrative with multiple characters and locations: it presents a wide range of emotional elements (e.g., happiness, nostalgia, sadness, love), everyday characters and locations, and a complex but linear storyline. The movie contains varying shot scales and it is dominated by close-ups and medium close-ups on characters—depending on how far they are from the fixed camera position. Close-ups facilitate mental processes related to the theory of mind or cognitive empathy in animated movies just as much as in live-action ones (Bálint & Rooney, 2018). This makes the movie suitable for measuring empathy and other emotional effects related to engagement with fictional characters and narratives.
Perception of animated characters’ agency can depend on representation (Mar, Kelley, Heatherton, & Macrae, 2007) including facial expressions (Tinwell, Grimshaw, Nabi, & Williams, 2011). However, it is argued that recognition of character roles and actions, fictional objects, and emotional content in a moving-image narrative is based on complex schematic and prototypical information and the construction of “mental models” that contain causal, spatial, and temporal systems of narrative elements (Busselle & Bilandzic, 2008; Zwaan, Magliano, & Graesser, 1995). This means that animated and live-action movies do not differ in terms of the comprehension of narrative themes (Grodal, 2018).
Further criteria when choosing the movie stimulus included simple and clear storytelling form and causal structures (Pearl follows a linear narrative with explicit progression of time), a minimal number of dialogues that would otherwise affect attention and comprehension (the movie contains few dialogues and the majority of verbal elements are embedded into a song), and relatively short duration to avoid the risk of severe cybersickness for VR participants as well as fatigue or loss of attention that could bias the results. Previous research studying moving-image experiences across viewing formats and platforms has demonstrated immersion (spatial and social presence) and comprehension effects using sequences between five and ten minutes (Syrett et al., 2016; Szita & Rooney, 2021; Van Damme et al., 2019). A sequence of approximately six minutes was deemed suitable for maximizing the effectiveness of measuring viewing experiences and comprehension and minimizing negative health effects and biased responses.
2.3 Participants
One hundred and sixty-five volunteers (109 males, 52 females, and 4 other), aged 16–62 (M = 30.44, SD = 9.61) participated in the experiment from a place of their choice using their own virtual reality headsets or screens. The criterion for taking part was access to the respective screening appliance (tethered virtual reality headset or a stationary screen of minimum 12 inches) and a device with Internet access to complete the survey.
Participants were recruited through online advertisements on various social media channels and word of mouth. The virtual reality version was advertised mainly in online special interest groups for virtual reality users with the aim to target potential participants who have access to headsets and are experienced with VR technology. This was to avoid novelty experiences’ biasing effects for inexperienced users. The screen version was advertised on platforms for cinema enthusiasts and general audiences. The online advertisements contained the links through which volunteers could access the study (stimulus and survey; see below).
Participants received no compensation. Participation was in accordance with the ethical guidelines stated in the Declaration of Helsinki: participation was voluntary, participants were obliged to provide their written informed consent, and participants had the opportunity to abort the study at any time and leave comments or feedback. The study did not involve direct intervention; therefore, based on national laws, ethical approval was not required.
2.4 Apparatus and Setup
In the virtual reality condition (n = 85), participants watched the 360-degree version of the movie that they accessed through their personal accounts to the Steam digital video game distribution service. Participants used their own virtual reality headsets with their personal settings and reported on the headset type they used. Only tethered virtual reality headsets that enable six degrees of freedom were allowed to be used; headsets that require an external device (e.g., smartphone) to act as a display were excluded from this study. While headset types and settings (e.g., interpupillary distance, display resolution) were not controlled, participants were instructed to use the headset and settings they were most comfortable with and that provided the best possible sound and visual experience for them personally. All but one VR participant claimed to be experienced with using VR headsets and consuming VR content.
In the screen condition (n = 80), participants gained direct access to the theatrical version of the movie on YouTube and were instructed to play it in full-screen mode on a 12-inch or larger fixed computer monitor or television set. As the study's objective is to compare 360-degree and regular-screen viewing experiences, no further restrictions on screen size were made.
2.5 Procedure
Participants gained access either to the virtual reality or screen version of the movie via Internet browser links. The links provided direct access to the movie with predefined (4K) image quality; participants only needed to start it. After providing informed consent, each participant was instructed to click Play and watch the movie alone once without intermissions.
After watching the respective sequence, participants completed a survey accessed through a separate link. The survey contained comment sections in which participants were instructed to address any issues they experienced during the completion of the study (e.g., image or sound quality). At the end of the study, participants were debriefed and informed that they cannot re-enter the study and, thus, could not participate in the other condition. The entire procedure including instructions, viewing, and completing the survey took approximately 15–20 minutes.
2.6 Measures
Participants in each viewing condition were asked to assess their memories and experience of the movie answering questions in a survey hosted on Psytoolkit, an online survey tool (Stoet, 2010, 2017). The survey was specifically created for this study combining items of surveys used in previous research (see below). The survey consisted of three sections measuring viewing experience, memory characteristics, and recollection accuracy, and an additional set of questions recorded demographic data, user habits, and technical details of participation (e.g., VR headset type).
The first section (consisting of ten items) measured emotional engagement with and sense of presence in the fictional space and the narrative, empathy toward fictional characters, awareness of the physical surroundings, and physiological reactions (e.g., nausea, dizziness) (based on Cho & Kang, 2012; Fonseca & Kraus, 2016; Huang & Hsu Liu, 2014; Qin, Rau, & Salvendy, 2009; Witmer & Singer, 1998; Zhang, 2020). Participants rated their experience on seven-point Likert scales stretching from “not at all” (1) to “completely” (7).
The second section, memory characteristics (ten items), was designed to measure recollection vividness, emotional reactions (e.g., feeling sad, moved when recalling the movie) and physical reactions (e.g., sweating, laughter), memory perspective (first-person or third-person), and the structural comprehension of the movie narrative (Berntsen & Rubin, 2006; Johnson, Foley, Suengas, & Raye, 1988; Qin et al., 2009). For these ratings too, seven-point Likert scales were used, where the higher values indicated more intense memories or reactions with one exception. Rating first- versus third-person perspectives when recalling the movie was done using a scale ranging from “inside the story world” (1) to “as an outside observer looking into the story world” (7).
The third section measured recollection accuracy based on previously established methods (Pyrczak, 1972; Syrett et al., 2016; Szita & Rooney, 2021). Here, participants were given twelve statements from the movie narrative and then they had to determine whether they were true. The statements were based on key events and themes in the movie narration and the answers to all these items were equally accessible in the two versions of the movie. The statements included information that was explicit even without focusing one's attention on a specific area (e.g., “The central element of the movie is a car.” or “The [protagonists’] car breaks down in an urban area.”); that was related to sounds (e.g., “The music featured in the movie is about home.” or “The band's song didn't make it to the radio charts.); that was presented once in one specific part of the 360 space in the VR version and viewers’ attention was guided toward it by lighting and other visual cues (e.g., “The girl catches a firefly from the car.” or “A trash bin was blown up by a gunshot.”); and that was presented once in one specific part of the 360 space in the VR version, but were not accompanied by attention-guiding sensory cues (e.g., “The father exchanges music to an office job.” or “The girl gets her first instrument for Christmas.”).
Possible answers were “true,” “false,” and “I don't know,” which were analyzed as correct, incorrect, and “I don't know” answers. The “I don't know” option was added for cases when participants would miss information based on their body posture or attention (i.e., when information was presented behind their backs) to avoid guessing the correct answer. Analysis of “I don't know” answers served as an additional way to determine whether viewing conditions would affect recollection.
3 Results
The survey measured emotions and engagement, memory characteristics, and recollection accuracy. A Shapiro–Wilk's test (p > .05) (Razali & Wah, 2011; Shapiro & Wilk, 1965), skewness and kurtosis z-values (Doane & Seward, 2011), and a visual inspection of the respective histograms, normal Q-Q plots, and box plots, determined that the values for each of the dependent variables are not normally distributed in either of the viewing conditions. Therefore, we used a Mann–Whitney U test to compare the two conditions. For the results of the survey items, see Table 1.
Mann--Whitney U Test: Mean Rank and Median Values, z-Scores, and Effect Sizes
| VR | Screen | ||||||
|---|---|---|---|---|---|---|---|
| Variable | Scale (from 1 to 7) | Mean rank | Median | Mean rank | Median | z-score | r2 |
| Viewing experience | |||||||
| 1. I felt that I was involved in the visual world of the movie | not at all–completely | 79.85 | 5 | 86.35 | 5.5 | −.90 | .005 |
| 2. I felt I was inside the story** | not at all–completely | 92.68 | 5 | 72.71 | 5 | −2.74 | .046 |
| 3. I felt I was at the places in the displayed environment*** | not at all–completely | 102.93 | 5 | 61.83 | 4 | −5.62 | .191 |
| 4. I empathized with one or more character(s) | not at all–completely | 87.65 | 6 | 78.06 | 6 | −1.35 | .011 |
| 5. The story affected me emotionally | not at all–completely | 88.43 | 6 | 77.23 | 5 | −1.54 | .014 |
| 6. I became less aware of the real world and my personal problems while watching the movie | not at all–completely | 83.51 | 6 | 82.46 | 5 | −.14 | .00012 |
| 7. When I was watching, time seemed to fly quickly*** | not at all–completely | 95.96 | 5 | 69.23 | 4 | −3.65 | .081 |
| 8. This experience was fascinating*** | not at all–completely | 97.45 | 6 | 67.65 | 4.5 | −4.08 | .101 |
| 9. I felt nauseous while watching the movie* | not at all–completely | 88.74 | 1 | 76.91 | 1 | −2.12 | .027 |
| 10. I felt dizzy while watching the movie | not at all–completely | 86.06 | 1 | 79.75 | 1 | −1.14 | .008 |
| Memory characteristics | |||||||
| 1. When I think of the movie, I can see with my mind's eye what took place* | not at all–as clearly as if I watched it now | 89.92 | 6 | 75.65 | 5.5 | 1.98 | .024 |
| 2. My memory for the movie involves sound | little or none–a lot | 84.39 | 7 | 81.53 | 7 | .44 | .001 |
| 3. The relative spatial arrangement of people and objects in my memory for the movie is*** | vague–clear | 96.64 | 6 | 68.51 | 5 | 3.93 | .094 |
| 4. At parts, the movie made me feel moved. This feeling was | weak–strong | 88.46 | 6 | 77.20 | 5 | 1.55 | .015 |
| 5. At parts, the movie made me feel sad. This feeling was | weak–strong | 86.69 | 4 | 79.08 | 4 | 1.04 | .007 |
| 6. At parts, the movie made me feel happy. This feeling was* | weak–strong | 90.54 | 6 | 74.99 | 5 | 2.13 | .027 |
| 7. The emotions I have when I recall the movie are | not at all intense–extremely intense | 87.52 | 4 | 78.20 | 4 | 1.27 | .010 |
| 8. When I recollect the movie, I have a physical/bodily reaction (e.g., palpitations, sweating, tears, laughter) | not at all–very strongly | 88.66 | 2 | 76.98 | 2 | 1.65 | .017 |
| 9. When I now recall the movie, I primarily see what happened from a perspective as seen from** | inside the story world–as an outside observer looking into the story world | 72.43 | 5 | 84.23 | 6 | −3.00 | .055 |
| 10. I was able to understand the structure and content of the story | not at all–completely | 82.81 | 6 | 83.21 | 7 | −.06 | .00002 |
| Recollection accuracy | |||||||
| 1. Percentage of correct answers** | — | 72.06 | 66.7 | 94.63 | 75 | −3.07 | .057 |
| 2. Percentage of “I don't know” answers | — | 88.61 | 16.67 | 77.04 | 16.67 | 1.59 | .015 |
| VR | Screen | ||||||
|---|---|---|---|---|---|---|---|
| Variable | Scale (from 1 to 7) | Mean rank | Median | Mean rank | Median | z-score | r2 |
| Viewing experience | |||||||
| 1. I felt that I was involved in the visual world of the movie | not at all–completely | 79.85 | 5 | 86.35 | 5.5 | −.90 | .005 |
| 2. I felt I was inside the story** | not at all–completely | 92.68 | 5 | 72.71 | 5 | −2.74 | .046 |
| 3. I felt I was at the places in the displayed environment*** | not at all–completely | 102.93 | 5 | 61.83 | 4 | −5.62 | .191 |
| 4. I empathized with one or more character(s) | not at all–completely | 87.65 | 6 | 78.06 | 6 | −1.35 | .011 |
| 5. The story affected me emotionally | not at all–completely | 88.43 | 6 | 77.23 | 5 | −1.54 | .014 |
| 6. I became less aware of the real world and my personal problems while watching the movie | not at all–completely | 83.51 | 6 | 82.46 | 5 | −.14 | .00012 |
| 7. When I was watching, time seemed to fly quickly*** | not at all–completely | 95.96 | 5 | 69.23 | 4 | −3.65 | .081 |
| 8. This experience was fascinating*** | not at all–completely | 97.45 | 6 | 67.65 | 4.5 | −4.08 | .101 |
| 9. I felt nauseous while watching the movie* | not at all–completely | 88.74 | 1 | 76.91 | 1 | −2.12 | .027 |
| 10. I felt dizzy while watching the movie | not at all–completely | 86.06 | 1 | 79.75 | 1 | −1.14 | .008 |
| Memory characteristics | |||||||
| 1. When I think of the movie, I can see with my mind's eye what took place* | not at all–as clearly as if I watched it now | 89.92 | 6 | 75.65 | 5.5 | 1.98 | .024 |
| 2. My memory for the movie involves sound | little or none–a lot | 84.39 | 7 | 81.53 | 7 | .44 | .001 |
| 3. The relative spatial arrangement of people and objects in my memory for the movie is*** | vague–clear | 96.64 | 6 | 68.51 | 5 | 3.93 | .094 |
| 4. At parts, the movie made me feel moved. This feeling was | weak–strong | 88.46 | 6 | 77.20 | 5 | 1.55 | .015 |
| 5. At parts, the movie made me feel sad. This feeling was | weak–strong | 86.69 | 4 | 79.08 | 4 | 1.04 | .007 |
| 6. At parts, the movie made me feel happy. This feeling was* | weak–strong | 90.54 | 6 | 74.99 | 5 | 2.13 | .027 |
| 7. The emotions I have when I recall the movie are | not at all intense–extremely intense | 87.52 | 4 | 78.20 | 4 | 1.27 | .010 |
| 8. When I recollect the movie, I have a physical/bodily reaction (e.g., palpitations, sweating, tears, laughter) | not at all–very strongly | 88.66 | 2 | 76.98 | 2 | 1.65 | .017 |
| 9. When I now recall the movie, I primarily see what happened from a perspective as seen from** | inside the story world–as an outside observer looking into the story world | 72.43 | 5 | 84.23 | 6 | −3.00 | .055 |
| 10. I was able to understand the structure and content of the story | not at all–completely | 82.81 | 6 | 83.21 | 7 | −.06 | .00002 |
| Recollection accuracy | |||||||
| 1. Percentage of correct answers** | — | 72.06 | 66.7 | 94.63 | 75 | −3.07 | .057 |
| 2. Percentage of “I don't know” answers | — | 88.61 | 16.67 | 77.04 | 16.67 | 1.59 | .015 |
Note. Analyses were performed using a sample size of n = 85 for the virtual reality condition and n = 80 for the screen condition. Significant at *p < .05, **p < .01, and ***p < .001.
3.1 Viewing Experience
Engagement with the movie, including emotional reactions and the sensation of presence, was expected to increase in the case of virtual reality viewing based on the immersive quality of the VR headset and 360-degree simulation and their capacity to mask the presence of a physical environment. Statistically significant differences were found for items that measure engagement with the movie and detachment from the physical world.
Participants reported an increased sense of presence in the story and the fictional environment in the virtual reality condition in the case of two items. The Mann–Whitney U test indicated that ratings for feeling like being inside the story (survey item 2) were significantly higher in the virtual reality condition (mean rank = 92.68) than the screen condition (mean rank = 72.71), U = 2577, p = .006. Similarly, ratings for feeling like being at the places in the displayed fictional environment (item 3) were significantly higher after virtual reality viewing (mean rank = 102.93) than screen viewing (mean rank = 61.83), U = 1706, p < .00001.
Measuring detachment from an individual's physical environment, ratings given to the item “when I was watching, time seemed to fly quickly” (item 7) showed significantly higher values in the case of virtual reality (mean rank = 95.96) than the screen condition (mean rank = 69.23), U = 2298.5, p = .0003.
Two additional items provided statistically different results between viewing conditions. Participants felt more fascinated (item 8) by the virtual reality experience (mean rank = 97.45) than screen viewing experience (mean rank = 67.65), U = 2172, p = .00005; but also felt more nauseous (item 9) in VR (mean rank = 88.74) than during screen viewing (mean rank = 76.91), U = 2912.5, p = .034.
Sensation of visual involvement (item 1; U = 3132, p = .37), empathy toward characters (item 4; U = 3005, p = .18), emotional effect (item 5; U = 2938.5, p = .12), awareness of personal problems (item 6; U = 3357, p = .89), and feeling dizzy (item 10; U = 3140, p = .25) did not show statistically significant differences between viewing conditions.
3.2 Memory Characteristics
To answer the second research question, we measured memory characteristics. For this part of the study, we hypothesized that virtual reality viewers would recall narrative events more vividly and more in detail than screen viewers, and this recollection is accompanied by stronger emotions. Statistically significant differences were found supporting these hypotheses. As all the items with significant differences showed similar distributions between conditions, the results of the Mann--Whitney U test can be interpreted as differences in median values.
According to the results of survey item 1, participants in the virtual reality condition were more likely to recall narrative events as clearly as if they watched it at the moment of answering (Mdn = 6) than screen viewers (Mdn = 5.5), U = 3988, p = .048. Correspondingly, the relative spatial arrangement of people and objects in participants’ memory of the movie (item 3) was rated clearer in the virtual reality condition (Mdn = 6) than in the screen condition (Mdn = 5), U = 4559, p = .000085.
Virtual reality participants reported a stronger feeling of happiness when recalling the movie (item 6) (Mdn = 6) than screen participants (Mdn = 5), U = 4040.5, p = .033.
To measure first- versus third-person perspectives when recalling the movie (item 9), participants rated their experience on a scale stretching from “inside the story world” (1) to “as an outside observer looking into the story world” (7). Supporting our hypothesis, virtual reality viewers reported recollection more from inside the story world through a first-person perspective (Mdn = 5) than screen viewers (Mdn = 6), U = 2501.5, p = .003.
The remaining items of the second survey showed no significant differences. For memory of sound (item 2; U = 3518, p = .66), feeling moved (item 4; U = 3864, p = .12) and sad (item 5; U = 3714, p = .3), the emotional intensity of recollection (item 7; U = 3784, p = .2), physical reactions to recollection (item 8; U = 3881.5, p = .1), and understanding of the structure of the movie narrative (item 10; U = 3383.5, p = .95), no effects of viewing condition were found.
3.3 Recollection Accuracy
The recollection accuracy test measured participants’ attention and memory of key narrative events and themes. We calculated the amount and percentage of correct answers for each participant as well as those of “I don't know” answers. Correct answers showed significant differences between viewing conditions. Distributions for correct answers were similar between the two conditions; therefore, median values are reported here. Statistically significant differences showed that participants in the screen condition recalled the movie more accurately with a median percentage of 75% than virtual reality participants, who reached a median percentage of 66.7% (U = 2470, p = .002). “I don't know” answers did not differ significantly between the conditions (U = 3877, p = .11).
4 Discussion
In the present study, we measured the effects of cinematic virtual reality on viewing experiences and recollection of narrative elements (memory characteristics and recollection accuracy). Our results revealed the impact of viewing conditions on viewers’ sensation of presence in the fictional space, comfort, emotional experiences, as well as the characteristics and accuracy of memories of the movie.
Our first research question asked whether virtual reality would evoke a stronger sensation of engagement with the fictional world and detachment from the physical world than screen viewing. This was partly reflected in the results. Those of the variables that showed significant differences between viewing conditions revealed stronger engagement with the movie and less awareness of the real world when watching the movie using VR headsets: virtual reality participants reported a higher sensation of being part of the story and the fictional environment than screen participants and time seemed to fly quicker for them. These findings confirm previous research that declares the high immersive quality of 360-degree and virtual reality experiences (Cummings & Bailenson, 2016; Slater & Wilbur, 1997).
Although VR viewers were found to be more immersed, they were also more likely to experience cybersickness (motion sickness during virtual reality experiences), as we predicted based on previous findings (Van Damme et al., 2019). Discomfort, such as cybersickness, may draw attention to one's physical body, thereby hindering immersion. This might be the reason for the lack of significant differences between screen and VR viewers’ emotional engagement with the narrative and empathy with characters.
The lack of previous research about the way virtual reality viewing would affect memory characteristics of a movie led us to measure how vivid viewers’ memories would be after watching a movie using a virtual reality headset in comparison to watching it on a stationary screen. Following the second research question, we found that recollection of the narrative is more vivid after virtual reality viewing: participants rated their recollection of the spatial arrangements and actions in the movie clearer. These results correspond to the effects of immersive viewing explained above. Also, increased immersive quality leads to more intense emotions that affect memory of mediated experiences. This may explain why the recollection of feeling happy during watching was rated more intense in the VR condition, although our results failed to show any differences for feeling sad or moved.
The relationship between immersive quality and memory intensity also corresponds with the perspective a mediated experience evokes. We measured the perspective from which viewers would recall the fictional events and found that virtual reality viewing would more likely induce a first-person point of view while screen viewing leads to a third-person (camera) perspective. This confirms previous findings claiming that a first-person perspective correlates with increased emotional intensity and more vivid recollection (Nigro & Neisser, 1983; Robinson & Swanson, 1993).
Although participants rated their memories of the movie clearer in the virtual reality condition, the accuracy of their recollection was poorer. This contradicts previous research by Nigro and Neisser (1983), Robinson and Swanson (1993), and Rice (2010) regarding the association between immersion, first-person perspective of recollection, and recollection accuracy. Nevertheless, these results suggest a causal relationship between the 360-degree field of simulation and attention: VR viewers need to turn their bodies to access information in the different parts of the visual field of the movie which may cause them to miss details that are presented in parts of the space that are momentarily obscured.
To fully confirm this explanation, further studies are necessary for investigating the correspondence between attention and recollection. Yet, these results draw attention to the paradox between control and immersion: while the 360-degree multisensory film experience can positively affect the sensation of presence and engagement, momentary changes in the visual field (by turning one's head or body) and the 360-degree composition of the fictional space and narrative events can impact attention and comprehension. This informs two further directions of future studies. On one hand, it is necessary to establish a VR movie's capacities for controlling a viewer's attention (see Mateer, 2017; Slater & Wilbur, 1997), on the other, we must analyze the relationship between presence and recollection accuracy further. In the latter case, we would like to know if lower recollection accuracy is a result of merely turning away and thereby missing details, or if immersion or fascination would lead to reduced attention to details.
4.1 Limitations
This study was conducted in natural settings; each participant watched the movie on their own device and in an environment of their personal choice. Such a natural experiment leads to results with high ecological validity as participants followed their general routines for movie watching and virtual reality experiences. Our detailed instructions regarding participation allow for replicability; however, we were unable to control eventual extraneous variables, such as viewing environments and distractions. In addition, although using one's own personal devices and settings would likely lead to a comfortable and effective viewing experience, we cannot rule out the bias of individual devices (e.g., differences in field of view or resolution). Therefore, a laboratory study to confirm our results would be an informative next step. Such a laboratory study could also manipulate the independent variable of viewing condition rather than selecting it, which gives a better opportunity to make conclusions of the causality of the viewing condition. It would also allow for using a participant pool of both experienced and inexperienced users irrespective of access to virtual reality headsets.
In this study, we used self-reports and performance tests. This combination of measurements can capture viewer behavior in different viewing conditions and compensate for social and cultural biases or mental abilities. Yet, we cannot rule out that some of our results cannot be generalized beyond the stimuli used here. Previous research has found correlations between viewers’ responses and the types of movie sequences: for instance, suspenseful sequences can affect the sensation of presence and attention to narrative information (Bezdek & Gerrig, 2017; Finucane, 2011) and immersion correlates with the perception of genres (Visch, Tan, & Molenaar, 2010). Little evidence is provided for the case of animated dramas like the one used in this study, even though research into animated film experiences has shown that viewers engage with animated narratives as they would with live-action ones and comprehend narrative information through similar schematic structures (Grodal, 2018). It has also been demonstrated that interactions with animated characters in virtual reality are similar to that with real humans even if their fidelity is low (Freeman et al., 2008; Slater, Antley, et al., 2006; Slater, Pertaub, Barker, & Clark, 2006). These conclusions notwithstanding, the used sequence may have affected our results. To determine the effects of film form and content, additional studies are necessary.
5 Conclusion
The present study measured how experiencing cinematic virtual reality affects emotional engagement, the sensation of presence, memory characteristics, and recollection accuracy. We compared watching a movie using virtual reality headsets with viewing it on a stationary screen by a survey. VR viewers rated their sensation of presence in the displayed fictional environment higher than screen viewers and experienced more fascination and a loss of sense of their physical surroundings. However, VR viewers were more likely to experience discomfort, too. We also found that memories of the movie are more vivid, evoke stronger emotions, and are more likely to be recalled from a first-person visual perspective (i.e., from a vantage point inside of the fictional space) following virtual reality viewing than after screen viewing. Yet, VR participants could remember fewer details than screen viewers. Overall, the comparisons between virtual reality and screen viewing suggest that cinematic virtual reality involves more immersive and intense experiences, but poorer attention to details of a movie.