This article presents a detailed design, development, and implementation of a Mixed Reality Art-Science collaboration project which was exhibited during Darwin’s bicentenary exhibition at Shrewsbury, England. As an artist-led project the concerns of the artist were paramount, and this article presents Shift-Life as part of an ongoing exploration into the parallels between the nonlinear human thinking process and computation using semantic association to link items into ideas, and ideas into holistic concepts. Our art explores perceptions and states of mind as we move our attention between the simulated world of the computer and the real world we inhabit, which means that any viewer engagement is participatory rather than passive. From a Mixed Reality point of view, the lead author intends to explore the convergence of the physical and virtual, therefore the formalization of the Mixed Reality system, focusing on the integration of artificial life, ecology, physical sensors, and participant interaction through an interface of physical props. It is common for digital media artists to allow viewers to activate a work either through a computer screen via direct keyboard or mouse manipulation, or through immersive means. For “Shift-Life” the artist was concerned with a direct “relational” approach where viewers would intuitively engage with the installation’s everyday objects, and with each other, to fully experience the piece. The Mixed Reality system is mediated via physical environmental sensors, which affect the virtual environment and autonomous agents, which in turn reacts and is expressed as virtual pixels projected onto a physical surface. The tangible hands-on interface proved to be instinctive, attractive, and informative on many levels, delivering a good example of collaboration between the arts and science.

This article discusses the challenges in creating Mutator VR: Vortex , a virtual reality experience based on interaction with semi-autonomous, organically inspired agents. The work allows the immersant to morph between a vast number of procedurallygenerated microworlds, each with its own visual elements, sounds, agent dynamics, and user interactions. We outline two methods used for procedural generation that are based fundamentally on integration of different modalities. Curve-based synthesis is used for simultaneous generation of entity sounds and shape and flow grains are employed to determine both agent dynamics and user interaction with the agents.

Presence is a prominent component of our conscious human experience. Many researchers and philosophers have described presence simply as a state of being there, or in improvisation, the here and now. We explore presence by examining performers’ experiences in an improvisational, telematic performance, Presence in a Box: Crossing Liminal Spaces (see Figure 1). Through our analysis of this performance we define the concept of “Threaded Presence” which we bridge to existing literature in “Situated” and “Extended Presence.” We discuss the role of technology in creating threaded presence by acting as an additional performer, adding a layer of glitch and unpredictability to a performance that requires negotiation from the entire team. We suggest that technology does not need to be transparent in order to experience presence in a performance space and focus on three main components of experiencing presence: creating a container of presence, blending boundaries of time and space, and implementing restraints. It is the collision of these elements that brings a new sense of agency into performance and contributes to one’s experience of threaded presence.

What role will ubiquitous sensing play in our understanding and experience of ecology in the future? What opportunities are created by weaving a continuously sampling, geographically dense web of sensors into the natural environment, from the ground up? In this article, we explore these questions holistically, and present our work on an environmental sensor network designed to support a diverse array of applications, interpretations, and artistic expressions, from primary ecological research to musical composition. Over the past four years, we have been incorporating our ubiquitous sensing framework into the design and implementation of a large-scale wetland restoration, creating a broad canvas for creative exploration at the landscape scale. The projects we present here span the development and wide deployment of custom sensor node hardware, novel web services for providing real-time sensor data to end user applications, public-facing user interfaces for open-ended exploration of the data, as well as more radical UI modalities, through unmanned aerial vehicles, virtual and augmented reality, and wearable devices for sensory augmentation. From this work, we distill the Networked Sensory Landscape, a vision for the intersection of ubiquitous computing and environmental restoration. Sensor network technologies and novel approaches to interaction promise to reshape presence, opening up sensorial connections to ecological processes across spatial and temporal scales.

This article approaches binaural interactive environments from an artistic research perspective. Beyond content production, an aesthetic reflection of binaural media requires pervasive access to digital processing means and ways to employ them in composition. However, most conventional workflows separate media-specific rendering algorithms from object-based scene authoring. Such a delimitation between binaural engineering and its application restricts transdisciplinary creation that crosses both areas. This article assumes that the full potential of immersive media cannot be explored without investigating technology in the context of aesthetic experience. A case study is presented in which artistic references are regarded together with its technical realization. Contemporary user experience evaluation methods are adopted and refined with reference to the aims of the artist. A subsequent revision of the work is discussed along with implementation adjustments and conceptual alterations. The presented project shall exemplify how artistic research may bridge scholarly investigation and the creative acquirement of media technology beyond its mere application. A point of departure shall be provided for further cross-fertilization between engineering and the arts by identifying mutual implications.