Volumetric video recordings of storytellers, when experienced in immersive virtual reality, can elicit a sense of copresence between the user and the storyteller. Combining a volumetric storyteller with an appropriate virtual environment presents a compelling experience that can convey the story with a depth that is hard to achieve with traditional forms of media. Volumetric video production remains difficult, time-consuming, and expensive, often excluding cultural groups who would benefit most. The difficulty is partly due to ever-increasing levels of visual detail in computer graphics, and resulting hardware and software requirements. A high level of detail is not a requirement for convincing immersive experiences, and by reducing the level of detail, experiences can be produced and delivered using readily available, nonspecialized equipment. By reducing computational requirements in this way, storytelling scenes can be created ad hoc and experienced immediately—this is what we are addressing with our approach. We present our portable real-time volumetric capture system, and our framework for using it to produce immersive storytelling experiences. The real-time capability of the system, and the low data rates resulting from lower levels of visual detail, allow us to stream volumetric video in real time to enrich experiences with embodiment (seeing oneself) and with copresence (seeing others). Our system has supported collaborative research with Māori partners with the aim of reconnecting the dispersed Māori population in Aotearoa, New Zealand to their ancestral land through immersive storytelling. We present our system in the context of this collaborative work.

Virtual and augmented reality, and other forms of mixed reality (MR), have become a focus of attention for companies and researchers. Before they can become successful in the market and in society, those MR systems must be able to deliver a convincing, novel experience for the users. By definition, the experience of mixed reality relies on the perceptually successful blending of reality and virtuality. Any MR system has to provide a sensory, in particular visually coherent, set of stimuli. Therefore, issues with visual coherence, that is, a discontinued experience of a MR environment, must be avoided. While it is very easy for a user to detect issues with visual coherence, it is very difficult to design and implement a system for coherence. This article presents a framework and exemplary implementation of a systematic enquiry into issues with visual coherence and possible solutions to address those issues. The focus is set on head-mounted display-based systems, notwithstanding its applicability to other types of MR systems. Our framework, together with a systematic discussion of tangible issues and solutions for visual coherence, aims at guiding developers of mixed reality systems for better and more effective user experiences.

It has long been argued that the possibility to interact in and with a virtual environment (VE) enhances the sense of presence. On the basis of a three-component model of presence, we specify this hypothesis and argue that the mental representation of possible actions should especially enhance spatial presence, and to a lesser extent the involvement and realness of a VE. We support this hypothesis in three studies. A correlative study showed that self-reported interaction possibilities correlated significantly with spatial presence, but not with the other two factors. A first experimental study showed that possible self-movement significantly increased spatial presence and realness. A second experimental study showed that even the illusion of interaction, with no actual interaction taking place, significantly increased spatial presence.

Within an embodied cognition framework, it is argued that presence in a virtual environment (VE) develops from the construction of a spatial-functional mental model of the VE. Two cognitive processes lead to this model: the representation of bodily actions as possible actions in the VE, and the suppression of incompatible sensory input. It is hypothesized that the conscious sense of presence reflects these two components as spatial presence and involvement. This prediction was confirmed in two studies ( N = 246 and N = 296) assessing self-reports of presence and immersion experiences. Additionally, judgments of “realness” were observed as a third presence component. A second-order factor analysis showed a distinction between presence, immersion, and interaction factors. Building on these results, a thirteen-item presence scale consisting of three independent components was developed and verified using confirmatory factor analyses across the two studies.