Transdisciplinarity, Composition, Expression: Reflections on a Spherical Way of Thinking

New mental structures arise from the connection of dissimilar concepts. The Theory of Conceptual Blending, which stems from cognitive linguistics, seeks to describe how this works in everyday thought and language: Two or more conceptual scenarios are projected to a separate mental space, which inherits a part of their structure and has an emergent structure of its own [1]. We can say that art involves conceptual blending. Yet the term “spherical thinking” seems preferable when we speak of how things exist beyond conceptualization.

Making art is a multidirectional process, which should not be confused with the execution of an idea. To create involves risk, and accepting the provisory transforms the unexpected and the “failure” into an impulse to discover, permeate and crystallize the paradoxical. The term “sublime” seems to have gone out of fashion, but it describes an extraordinary experience that will always be crucial to human beings: an amazement stemming from the discovery that we can apprehend a wider reality than we are able to intellectualize.

These ideas do not follow the dominant paradigm in our current societies, where all is supposed to be quantifiable, and the reality on a mobile phone screen is seemingly truer than direct experience. In addition, we are used to handling computers as magic black boxes that save us labor. When they work, their origins are forgotten. Yet software is necessarily biased: it mediates our action through code, and code embeds theories informed by specific purposes and criteria. The problem is that those theories are too often taken for granted. The more science and technology succeed, the more opaque and obscure they become, and the more distant we become from computation as creative material. If we want to grow humanly, we need to expand continuously beyond presuppositions, and that must be a personal process. This article will describe my own.

The desire to connect sound, light, and space drove my first approach to electronics and computing. Later, the desire to combine an ancient stringed instrument with audiovisual software based on its input led me to question and modify the interaction principles that govern 3D video game technologies. I wanted the image to expand the performative arena beyond the physical space, but not to distract the audience from the music. This is a challenge, because vision usually dominates over auditory stimuli. Research in neuroscience and psychology enabled me to clarify insights and identify gray areas, which became further creative material.

In the next section I will introduce an understanding of “spherical thinking” by looking at how science and philosophy embrace ambivalence in logics. Subsequently, the article will describe motivations to bridge music, visual arts, instrument design and perception science, while discussing composition and expression.

A spherical way of thinking implies a desire to accommodate all doubts, concerns and questions, while overcoming the dichotomy between “objectivity” and “subjectivity.” This might look like a grandiloquent idea, but it is certainly not exclusive to art.

For example, traditional physics tells us that something true cannot be false, and something false cannot be true. Yet quantum mechanics has shown that those propositions do not apply to the subatomic world. Both systems of knowledge are valid and demonstrable, but the laws of one domain do not apply to the other. This paradox led Stéphane Lupascu to formulate the Theory of the Included Middle [2]. It consists of a logical structure divided in three parts, corresponding to three possible positions—to affirm something, to deny that affirmation, and a third position: to affirm or deny the previous two positions.

Otto Roessler, a scientist dedicated to the study of turbulence, provides perspective on the third position. Evoking the sixteenth-century philosopher and mathematician Descartes, he argues that there is a universe independent of human perception [3]. He describes knowledge as a set of windows into reality; our observations depend on the windows we look through.

Quantum scientist and philosopher Basarab Nicolescu uses a cosmic image to illustrate the idea of transdisciplinarity [4]: While knowledge paradigms are like planets, art and the sacred exist in the interplanetary vacuum. This metaphor seems a good stepping stone for a discussion about creation and knowledge.

As one navigates through the “vacuum,” one might discover that the “planets” are not solid and tune themselves to that which philosopher Henri Bergson called “prime reality” [5]: the “perpetual happening,” an evolving dynamic flux that proceeds along a definite but unpredictable course. Bergson noted that the human mind is shielded from the perpetual happening by the intellect, which imposes “patterned immobility” on prime reality, separating it into discrete objects, events, and processes. He saw intuition as a way to attain direct contact with a prime reality ordinarily masked from human knowledge, stressing that the intellect can freely interact with intuition to develop an enriched personal perspective.

Art can be seen as a kind of practical philosophy; both exist between cultural paradigms. But philosophy systematizes the information to advance knowledge, while art fragments cultural paradigms and reassembles those fragments, activating memories that interact freely with each other in inconclusive ways. That is possibly the most precious function of art: to vitalize the mind.

I have always thought that music involves vision and space, beyond sound. During my six years of classical piano training as a child, I was advised to imagine different characters and situations for each part of the music, and develop the expression of my playing accordingly. During my visual arts education, my mentor Pedro Morais, an artist and Zen practitioner, taught us that “making for the sake of making” is a fundamental artistic principle, regardless of the medium. On the one hand, we need to discover the reality beyond our usual ways of perceiving the world. On the other, we need to permeate that wider reality through our works, so that the audience can have their own individually significant experiences. Armed with this understanding, my thinking about painting started to fluidly embrace volume, space, sound, and performance. At some point I realized that my thinking was actually about music.

In the 1990s I started using sensors and digital technologies to explore relations between sound and light, space, meteorology, architecture, and movement. The inspiration emerged when I met Sensorband, the trio formed by Atau Tanaka, Edwin Van der Heide, and Zbigniew Karkowski. They created powerful music performances using sensors to capture the electrical activity produced by muscles (EMG), ultrasound sensors, and infrared sensors. I had no technical skills or means but obtained orientation and support at STEIM (Amsterdam) and Metronom Studio (Barcelona), as well as from Michel Waisvisz, Frank Baldé, and Bert Bongers. That enabled me to create a “sounding light instrument” that could be configured to behave in many different ways.

I explored its versatility for more than 10 years, using natural and artificial light, body shadow, or video to modulate recorded sounds, live-input sounds, and synthesized audio. The sensor cables usually hung from the ceiling (see Fig. 1). That made the space reactive, and a compelling context for creative collaborations. One day, a musician stretched a metal wire from my gallery space to the surrounding forest; the wire was amplified with a piezo microphone, so that we could hear the falling snow. I started playing that wire; it felt similar to piano playing, but freed from musical traditions.

I multiplied the wires and developed a “bodiless instrument,” which acquired different timbres in every performance/installation. As I moved in space to play the wires, my shadow upon light sensors also affected the processing of digital sounds. Neither the digital nor the analogue components of my setup were fully controllable. I found this stimulating in performance, because the interaction with the system felt reciprocal, and I needed to reach a particular state of concentration to extract musical expression from the unexpected.

Unpredictability and the control of thresholding are fundamental aspects of my musical work. I make a graphic score to sketch the general structure of a composition, but real-time decisions and timings are primarily visceral. I find it useful to rely on some pre-set decisions, but every situation is unique, thus one also needs openness to making decisions in real time. My notation system, which I have used for more than twenty years, consists of a drawing and vertical lines that map its different parts to musical sections, playing techniques and instrumental combinations over time; examples can be seen in Figs 2 and 3. The drawing suggests densities, textures, and transitions between sections, leaving open clock-time and musical phrasing, as well as many other aspects of the sonic construction.

By inventing a graphic notation system, one can decide which aspects of the work are prescribed—as rules of the game, so to speak—and which aspects are left open. Similarly, an instrument can be configured to allow for any number of real-time decisions, while digital constraints rule out undesired outcomes. As Thor Magnusson observes, a musical instrument is also a score [6].

But there are some differences. While a graphic score leaves space for human interpretation, an instrument can convey bidirectional interaction; the responsibility for shaping musical direction continually shifts between musician and instrument. Andrew Johnston calls it “conversational interaction” [7].

One can forge this kind of reciprocity by configuring the instrument to behave in determined, but not fully determinable, ways. There are simple ways to do this with computers. For example, a single input signal can be processed through parallel digital mappings, so as to modulate multiple variables and obfuscate the base cause-and-effect relationships. Also, software can respond to physical events that one is not perceptually aware of. In fact, the disparities between human perception and digital analysis are unavoidable. Software cuts the audio signal in time sections and analyzes each section independently, whereas perception operates based on expectations derived from previous events. And the resilience of a physical input will always exceed its codified terms: The input can be continuously converted into voltage, but then the voltage flux is measured periodically so as to be represented through binary code. The measuring rate can be very high but it is not continuous.

When a digital musical instrument exhibits chaotic or entropic behavior, this is usually seen as a failure in design, a bug in the code or loose wiring in the hardware [8]. But those behaviors can also be regarded as creative material; unpredictability was being incorporated in music long before computers were available [9], and it also played an important role in early video art [10]. There is a wealth of related research that embraces the digital domain as well.

For example, Kim Cascone talked about an aesthetics of failure in computer music before 2000 [11]. Fifteen years later, Tom Mudd argued that nonlinear behaviors foster engagement with musical expression, as the musician focuses on the material properties of the interface [12]. Drawing from Chaos Theory, he created such behaviors by implementing algorithms extracted from the mathematics of fractals. Furthermore, Agostino Di Scipio stresses how today’s sound-making practices are forms of liberation and “construction of self” [13]. He suggests that what we hear in sound, in this context, is the physical and cultural interplay of the processes by which sound events are enacted. This seems applicable to acoustic, analogue, digital, and hybrid mediums. And it becomes particularly significant once we consider how computers are tabulae rasae.

Nevertheless, an artistic work conveys multiple layers of interpretation and experience, and one may find that the unexpected must be restricted so as to rule out undesired outcomes. As Joel Ryan points out, “each link between performer and computer has to be invented before anything can be played. . . . These ‘handles’ are just as useful for the development or discovery of the piece as for the performance itself” [14].

The development of an instrument brings the discovery of new compositional strategies, and those strategies originate new changes in the instrument; the cycle repeats, again and again, ad aeternum. From this point of view, artistic creation involves engineering. But I am certainly not an engineer.

Digital platform designs are governed by theories that are embedded and concealed in code, and circumventing those theories seems to entail a political dimension. This struck me when I decided to repurpose video game technologies and develop an instrument that combines acoustic sound, digital sound, and digital image. It includes a zither (an ancient stringed instrument) with less bright strings and custom (de)tuning, as well as 3D software that processes sound and graphics based on frequency analysis from the zither sound (Fig. 4). My husband John Klima wrote the code; beyond being an artist, musician, and sound engineer, he has been a video game programmer for many years.

The fact that I had never appreciated video games gave me a particular perspective on the theories that govern digital 3D platforms and everyday life technologies. Typically, their behaviors are consistent and clearly understandable. In this way, a person can focus on a purpose without being distracted by the interface itself—the medium becomes “transparent” because the interaction is seemingly immediate [15]. This is of course desirable when we want to win a game or write a text on a computer, but a musical instrument can be conceived with very different criteria.

The combination of acoustic and digital components raised questions about interaction, and I wanted to extract musical meaning from those questions by taking advantage of disparities between the direct acoustic output of the zither and the indirect outputs of digital sound and image. The first version of the instrument emphasized a few “chaotic” nonlinear features in the zither, exploring digital configurations and mappings that caused the software to behave inconsistently [16]. Recent versions implement compositional strategies at a low level in the digital architecture rather than solely at a high level, in parameterization [17,18].

When the software detects the zither sound, it plays back an audio sample mapped to the closest tone or half-tone. Then the sample plays back a second time, transformed according to the difference between the detected pitch and the closest tone/half-tone. In computer science, that mathematical difference is called “error.” Applied to the processing of digital sounds, it becomes creative material, exposing the black box of code.

The audio mappings combine acoustic and digital sounds based on tonality, timbre, semantics, and morphology; the zither intertwines with sounds of nature, urban sounds, musical instruments, and synthesized sounds. The expressive details of the sonic constructions emerge with intuitive precision and cannot be scored. They bring life to the music.

Instrument designs can reflect different understandings of expression, and the amount of cognitive effort, both conscious and unconscious, involved in the interaction is a distinguishing factor [19]. My personal understanding grounds a creative principle: to balance my control over the instrument and its unpredictability, so as to create sonic complexity.

An unexpected event might feel “wrong” within an existing musical logic; yet that logic can shift so that the event becomes gloriously “right.” Relying on skills to address the unexpected entails a level of risk, which makes the interaction somewhat effortful.

Effort directs attention to the skills of the performer/composer, conveying the expressive dimension of music. My sonic dramaturgies seek this expressive dimension to interlace with an environmental dimension, i.e. a focus on space. And they are playful, with recognizable sounds that have an informational load.

Articulating these three dimensions in a sonic construction involves a particular way of listening. In acoustic ecology, Murray Schafer invites us to discern a musical sense in any acoustic environment [20]. This requires one to dispense with treating sound as a vehicle to pursue other objects. Pierre Schaeffer, the father of acousmatic music, coined the terms “reduced listening” and “musical object” to describe how we can focus on the sound itself, independent from causes and meanings [21]. The word acousmatic recalls the Pythagorean lessons of ancient Greece, wherein the teacher hid behind a screen to enhance the abstract thinking of the students—who were called akousmatikoi (hearers). Today, many composers use the term to express how they are driven to draw attention to the relation between the sounds themselves. Interestingly, in film theory, Chion states that hiding the source of sounds “intensifies causal listening in taking away the aid of sight” [22]. The contradiction is only apparent: By shifting the usual relation between a sound and its source, music can playfully create perceptual shifts. Not coincidentally, acousmatic music often uses recordings taken from the world around us.

Schaeffer´s distinction between “reduced” and “causal” listening seems particularly relevant when we recall his use of recognizable sounds such as train sounds. His creative strategies subordinate their informational load, i.e. the meaning of train sounds, to a human-scaled manipulation, clearly informed by human gestures and timings. But suppressing the environmental meaning of sounds is not a requirement for emancipating the sonic experience.

To treat environmental sounds as “musical objects” and simultaneously evoke the original soundscape is a compelling motivation in some of my recent works with Klima. For example, an installation at a convent cloister overlapped a set of compositions made with recordings from the surrounding environment [23]. Natural light was a kind of “maestro,” as the loudness of each sound source depended on the light captured by sensors connected to the loudspeakers. That same light-sound system, given its potential for multiple interpretations and experiences, was adapted to an outdoor installation in Lisbon [24], which evoked our “off-grid” place in the countryside (Fig. 5).

The sonic identity of that place, rich in animal sounds and distant machinery, also played a central role in several collaborative performances [25]; I configured my 3D software to process audio samples recorded there. In performance, sometimes I allow them to emerge sparsely, and other times I (re)create the original soundscape through continuously overlapping samples, intertwined with the zither. In this way, sounds loaded with information about a site convey both the environmental and the expressive dimensions of music.

As Ryan brilliantly observed, music is the construction of experienced time [26]. We can add that attention informs the experience of time, drawing from all our physical senses.

Developing an audiovisual instrument made me ask if, and how, the audience could be driven to focus on the subtle relations between the sounds. The image should function as a reactive stage scene that extends the physical space into a digital 3D world that morphs with sound. That which Michel Chion calls “added value”—a surplus of meaning that is not contained in the sound or the image alone—can emerge through synchronization [27]. Yet I had the strong impression that vision tends to subordinate auditory input.

From Schaeffer [28] to recent acousmatic composers, many people have argued that sounds must be detached from their originating cause to be fully experienced. Jeff Pressing also investigated the audiovisual relationship in digital 3D environments, noting that perception operates from vision to sound whenever a direction of causation is discernible [29]. Meghan Stevens assessed that the music remains dominant in audiovisual performance when the audiovisual relationship is partially congruent, yet she admits that her theories rely on limited evidence [30].

Indeed, visual dominance is a challenging problem. In neuroscience, Sinnett et al. found that attention can be manipulated so that vision does not dominate over sound [31], but they did not explain how. To clarify the problem, I extrapolated from research in audiovisual theory, Gestalt psychology, and multisensory integration [32].

It became clear that perception is a process of multisensory synthesis and that the competition of stimuli to reach conscious awareness is usually driven by the primary goal of the brain: to detect, perceive, and respond to the world in a timely manner. We might prioritize vision due to its poor alerting capabilities compared to hearing. But we can also intuit a wider sensory complexity when we focus on perceptual dynamics itself, beyond any purpose.

The investigation led to the conclusion that sound can be in command under two conditions, and those conditions were adopted as creative principles for the visual dynamics and the audiovisual relationship.

Clearly perceivable cause-effect relations drive us to simplify the sensory information according to previous concepts and ignore diverging information [33]. Hence, one of my creative principles is to create a fungible audiovisual relationship: one that produces a sense of causation while obfuscating the base cause-effect relationships. A way to do this is to combine synchronized audiovisual components and unrelated components, with complexity enough to confound how the system works.

Using methods from experimental psychology, I conducted a study to demonstrate the effects of the fungible relationship [34]. It does produce a sense of causation, and as a person cannot segregate the fitting sounds and images from the nonfitting ones, cause-effect concepts remain inconclusive. Therefore, perception does not simplify the sensory information according to presuppositions, and the sense of causation extends to the whole.

Another creative principle is to dispense with disruptive visual changes. So far, the artistic debate about visual dominance has not considered the visual dynamics per se. Yet an abrupt discontinuity prompts automatic attention, monopolizing conscious awareness [35]—a survival instinct. We can say that continuity exists whenever expectations are fulfilled, and discontinuity whenever they are counteracted.

Attention also increases perceptual resolution, whether it is automatically driven through stimuli or under individual control [36]. This means that continuity becomes more discontinuous if one pays attention. That is an ambivalent type of discontinuity, corresponding to the fuzzy threshold between automatic and deliberate attention.

By ruling out sudden visual discontinuities, I found myself inspired to overlap multiple images with transparencies in ways that accentuate the impression of an organic, moving painting. Gestalt psychology is useful to understand perceptual simplification, but importantly, my creative decisions are not driven by demonstrative aims. In fact, the first version of the audiovisual instrument assured visual simplification, but the subsequent versions expose gray areas with respect to how the audience’s attention might change over time. The “gray” also reflects how volatile the frontiers of knowledge might be.

In any case, the relation between my artistic and scientific work also resulted in a parametric visualization model [37–39]. This model can facilitate the analysis of any time-based work, be it sonic, visual, or audiovisual. It can be used in instrument design, composition, and performance. It is applicable to any technical platform and aesthetical approach, revealing aspects relevant to sonic expression, sensory dominance, and spatial presence—the cognitive feeling of being physically located in a mediated space.

This text hopes to contribute to an understanding of how “making for the sake of making” is crucial in art: It enables creative practice to cross and connect multiple domains of experience without subordinating itself to established paradigms. I have narrated my personal path not to serve as an example but precisely because it is only an individual example, inimitable. As we navigate between disciplinary fields, our position stays single and relative to everything else. In fact, this relativity becomes more evident.

I am a specialist in being a nonspecialist, but that does not mean that my perspective is truer than others, nor does it mean that I can dispense with knowledge from specialists. My way of facing life changed when I dedicated myself to scientific research and learned to clearly communicate my artistic motivations, so that others could understand what I intended to say. Initially, I feared that explaining my work in this way would harm its reason-to-be. Then I realized that the opposite is actually the case: My praxis continues to explore the incommensurable, and my words seek to highlight the importance of that search.

Contextualizing our work with the work of others before us brings humility and pride, because we are addressing something that exceeds our personal interest and lifetime. The same happens when our methods become useful to other people. We need to consider our individual scale within the universe; overcoming the ego does not diminish us. On the contrary: it enables us to grasp reality beyond assumptions and existing fields of knowledge. Ultimately, nurturing a spherical kind of thinking brings a feeling of peace, a sense of belonging, a respectful acknowledgment of the infinite.