Transformation of Buddhist Mandalas into a Virtual Reality Installation

Virtual reality (VR) adaptations of art pieces are a modern cornerstone of museums of this time. One can take a VR experience of the Mona Lisa at the Louvre [1]. The Dalí Museum created a surrealistic immersion into Salvador Dalí’s world in Dreams of Dalí [2]. In essence, these are novel ways to experience exhibits that heighten intimacy with the artworks. What is rarer is a transformation of the exhibit that blends the extant with the generative—to create a new piece of VR art that is born of the original works yet is distinct. By switching media, we find emergent properties in the artistic progeny. The Asian Art Museum of San Francisco embraced the challenge of utilizing VR to enhance the exhibit Awaken: A Tibetan Buddhist Journey toward Enlightenment. Our team created a VR installation (Mandala Flow State) inspired by the mandalas of the exhibit and integrated elements of the exhibit’s story that were not possible in a traditional experience. Below, we show how this installation affected the experience of museumgoers.

Experiencing art objects in a museum setting has, until relatively recently, by definition been a dualistic experience wherein a viewing subject confronts a viewed object [3]. The geometric meditation diagrams called mandalas, on the other hand, are a kind of artistic form that are explicitly created in Tibetan Buddhist artistic and meditative traditions to undermine subject-object perception through conscious visualization [4]. By visualizing a mandala in full detail and replicating it within the mind’s eye, meditators experience a vision that appears in multiple spatial dimensions but is nonetheless devoid of substance. To have such a perceptual experience—one in which the percept is recognized as both subject and object—amounts to the perfection of wisdom itself. The Prajnaparamita-hrdaya-sutra and many related Mahayana texts describe the nondual summum bonum, which transcends subject and object, in terms of the unity of emptiness (shunyata) and form (rupa) [5].

It can take decades to reach the levels of meditative skill that make such perceptions possible. Virtual reality, however, has the potential to catalyze a subjective, multidimensional perception that is hypothetically a functional equivalent to meditative realization of the type described above. In 2010, the VR company EON Reality created a 3D Kalachakra mandala that could be virtually “entered” through an Oculus headset. According to University of Miami officials, “It was very well received by His Holiness” [6]. This platform does not have a tangible, physical form, which made for an excellent opportunity to explore the phenomenological potential of a VR transformation of the mandala theme within the Awaken exhibit in a secular but contemplative setting. The Asian Art Museum has experimented with creating physically constituted mandala environments [7]. Thus, while the nested squares and circles characteristic of mandala geometries define the visual architectonics of Mandala Flow State, the highly coded iconography of sacred mandalas, some of which appear in the exhibit, is not reproduced.

The other side of this story is the challenge for the scientist to reach the public. For researchers, the default is to remain insular within one’s discipline. However, in many arms of science, such as neuroscience, communicating the philosophy and facts to the broader community can be a central goal. The science may inform values and attitudes that drive personal or social decision-making and acceptance of potential medical or behavioral interventions. There are clear examples of this two-way engagement model [8] in recent collaborations between artists and scientists. A-me adapted augmented reality (AR) neurosurgical tools to communicate the relevance of localization in the brain in the context of memory [9]. The installation Mental Work probed the question of separation of human and machine by using extracted signals from electroencephalograms (EEGs) to control geared machines inspired by the Victorian era [10]. My Virtual Dream pulled together the social nature of being by taking collective EEGs from 20 people to generate a live musical performance and dynamic dome projection [11]. The latter two works took advantage of the principles of neurofeedback, which trains a user to modify their brain activity. The work we present here also uses neurofeedback. We link neurofeedback to meditative practices by using brain signals to modulate the VR experience. By making this link, we can show how neurofeedback is used therapeutically in a manner similar to meditation.

Out of these motivations, Mandala Flow State (MFS) was born. The museum’s and researcher’s needs were both met through the mind meld of the neurofeedback VR installation. The sections below describe the inspiration, design and outcomes of the installation. We found that neurofeedback VR provided an opportunity for contemplative experience in which the boundary between subject and object, mind and matter, is called radically into question. Furthermore, insofar as it provides a platform for common experiential discussion across knowledge systems, we believe it will have therapeutic and dialogic applications.

In MFS, the user is directed to relax while concentrating on the mandala as it is revealed from its center outward and the music that progresses deliberately through the experience. The mandala and the soundscape are intended to promote focused relaxation. The scene is covered in a thick fog filter. The clearing of the fog is linked to the EEG signals, which correlate with focused relaxation. This element acts as the neurofeedback informing the user of the change in brain activity. In conjunction, the feedforward and feedback elements are analogous to the visualization and shifting mental state of meditation, respectively.

MFS demonstrates the principles of neurofeedback. In brief, neurofeedback is the externalization of brain signals into a perceptible form for the purposes of modulating thought and behavior [12]. Neurofeedback is a closed-loop system in which the user is the central point. The measured brain activity signal is transmitted to a computer, which then processes the signal to convert it to a control parameter for an output, such as a visual graded cue. The user attempts to change the stimulus representing the specific brain activity in a directed manner. The user improves over many sessions by maintaining the prescribed level of brain activity for longer durations in each session. While various ways of measuring brain activity are possible, we chose to use EEG for our system for its accessibility and physical compatibility with VR headsets.

Neurofeedback therapy is based on an analysis of the frequency spectrum of brain activity at specific EEG channels [13]. Cognitive neuroscience research has shown that particular profiles of relative frequency band contribution to the total brain activity are reliably associated with stereotyped mental states. In our application, we made use of the EEG pattern for focused relaxation. This mental state is associated with low autonomic arousal and strong attentiveness to internal thought processes or external stimuli [14]. Focused relaxation is marked by elevated alpha power in multiple brain regions, including anterior frontal cortices. In studies of mindfulness-based meditation, increased alpha power is found during the meditative state [15]. The empirical evidence indicating a reliable connection between anterior frontal alpha and spontaneous and induced states of mental relaxation led to our selection of this metric for our neurofeedback system. MFS is designed to promote focused relaxation through the sensory experience and to model the modulation of alpha power via the neurofeedback signals. In theory, the learning that occurs in the neurofeedback sessions transfers to real-life circumstances such that a person is better able to maintain or induce a calmer disposition and clearer thinking when needed [16]. As an enhancement to the museumgoers’ experience of Awaken, our system aimed to externalize the guided imagery of meditative visualization of Tibetan mandalas.

In designing MFS, we adopted the abstract and architectural elements of the traditional mandalas but did not attempt to appropriate overt religious symbolism and iconography. The 3D mandala had seven layers of circles and squares built from repeated design elements (Fig. 1). The design elements were created procedurally in Substance for novel geometric patterns. Traditional Tibetan motifs were selected from original sources with Creative Commons licenses. The environment and UV mapping were modeled in Maya. Unity was used as the hub of the animation, geometry and signal processing; it also drove all of the interaction and timing. Technical details of the VR design are reported in the supplemental materials.

The features of the experience and mandala design are shown in Fig. 2. The mandala was gradually revealed through a growing circular aperture. As the mandala progressed, the central point was pulled to greater depths to create flow and distance and to make space in the scene for the additional layers. The full mandala filled the visual scene by 180° and required looking up, down, right and left to see the edges, giving the illusion of being inside the mandala universe. These VR features contributed to the immersivity, which could not have been otherwise achieved. The closure of the experience was a disintegration of the mandala scene by a speckling filter and spiral convolution to mimic the appearance of the sweeping away of sand mandalas.

The music was written by Nicole Jacobus with the intention of guiding the viewer into a meditative state. The music also drew upon proven methods of music meditation and healing. While there are many inspirations for this piece, the most significant inspirations were Tibetan music and modern minimalism. It was essential to create something that could repeat and shift between Tibetan and Western instrumentation while not overpowering the visuals. The musical time course aligned with the VR time course such that there was a three-stage progression (Fig. 2): (1) opening/groundwork, (2) piano/focus, and (3) piano and traditional instruments/ ending. The music helped to promote immersion into the VR environment by masking ambient noise and involving more senses in the experience [17]. The description of the composition is in the supplemental materials.

The neurofeedback system is diagrammed in Fig. 3 and detailed in the supplemental materials. First, the Muse EEG transmitted absolute alpha (8–13 Hz) power from AF 7/8 channels via Muse Direct to Unity extOSC. Next, the data is processed to smoothly control the neurofeedback parameter (density of the fog shader) within Unity in real time. Consequently, an increase in alpha signal proportionately decreased the appearance of the fog.

An overview of the experience is charted in Fig. 4. The experience starts with the interior column of a Buddhist temple, where users had the opportunity to acclimate and the calibration of the EEG signal was conducted. A singing bowl chime signaled the start of the experience. From the center of the field of view, the mandala emerged and retreated farther in depth as the progressive layers of mandala were revealed. During the core of the experience, the user attempted to relax and the appearance of the fog indicated the level of relaxation. When the mandala reached its fullness after 6 minutes, the fog cleared and the mandala was swept away.

For the purposes of visualization in the installation, the raw EEG and the relative alpha power for AF 7/8 were displayed using Muse Lab. The graphs were displayed alongside the participant’s VR experience for museum visitors to see and so in essence peer into the experience and mind of the participant. The simulcast engaged museum visitors widely, creating a more communal experience.

Our observations on the session activity were from our general impressions of and user feedback from the sessions. Our team noticed that novice users of VR did not attend to the fog modulation; rather, they were in an exploratory state in the novel experience. Consequently, the fog fluctuated without a pattern and stayed relatively thick, as they were not in a state of focused relaxation. Other users stated that they could not detect a connection between their efforts and the fog. In response, we explained that the ability to do so is reliant upon greater practice, just like meditation. This gave us the opportunity to explain how neurofeedback works. Those who came into the experience with a strong ability to induce a state of focused relaxation, like meditation practitioners, did reduce the level of fog for noticeable periods (30–60 seconds). This anecdotal observation points in the direction of the technical accuracy of the system design, that is, focused relaxation is correlated with greater alpha power and the derived standardized score changes the fog density to a perceptible level. The collective insights from the diverse array of visitors elicited discussions with designers Julia Scott and Max Sims, which is the essence of the two-way model of engagement. The guide for facilitators is provided in the supplemental materials.

We collected survey results from 44 participants on four dates in February 2020 (Fig. 5). For more than half the participants, this was their introductory experience with VR. This showed the museum installation was an effective means of introducing emerging technologies to the public. Of those who viewed the exhibit ahead of our installation, 89% found a moderate to strong connection between the two. This synergy was a primary goal for the museum and was largely achieved by this evidence.

While participants’ relaxation states mostly stayed the same (51%), a large segment (39%) reported an increased feeling of relaxation afterward. Given the first-time exposure to the experience and the public setting, we find this outcome encouraging. In examining the design and user experience, we look to the survey of VR features. Overwhelmingly, participants liked the music as well as the motifs and geometric designs. The pacing, immersivity and color features were met with splits in preference, which gave direction for redesign for future versions that would mute the color scheme and increase the pace. User comments are listed in the supplemental materials.

Our goals were to (1) attract a more diverse pool of visitors to the Asian Art Museum, (2) enhance the museumgoers’ experience of the Awaken exhibit and (3) educate the public on the topic of neurofeedback. We achieved all of these marks. Many people came expressly to participate in MFS, as they were interested in the interactive VR experience. This brought more people through the museum doors who would then explore the museum widely. Specifically, more people came to Awaken and found more meaning in their walkthrough of the gallery. People who watched or participated in the installation reported a genuine connection with the theme of the exhibit. Through this connection, MFS provided many people a first-time exposure to VR technology and neurofeedback applications. The partnership supported the needs of both parties and created an experience that could not have been realized individually.

We thank the Asian Art Museum of San Francisco for the opportunity to showcase MFS. This project was funded by the BioInnovation and Design Lab and Center for Arts and Humanities of Santa Clara University, and the Asian Art Museum of San Francisco.