Ubiquitous music (“ubimus”) is a relatively new area of research, having developed from its embryonic stage in the mid 2000s to a multifaceted field of investigation. It arose out of the work of the Ubiquitous Music Group (g-ubimus), a research network encompassing information-technology and music practitioners from several Brazilian universities and partners in Ireland, Italy, Portugal, Sweden, England, and Australia. The group fostered the initial activities in ubimus by establishing a yearly international event, featuring the participation of multiple researchers from both peripheral and central countries—the Ubiquitous Music Workshops (UbiMus), held in Florianópolis, Brazil (2010), Vitória, Brazil (2011 and 2013), São Paulo, Brazil (2012), Porto Alegre, Brazil (2014), Växjö, Sweden (2015), São João del Rei, Brazil (2018), Marseille, France (2019), and Porto Seguro, Brazil (2020). A horizontal dialogue has encouraged an intense exchange of ideas and expertise among all the participants. The lack of prejudices regarding the geographical and cultural origins of the proposals and a careful alignment between the research targets and the organizational decisions have become hallmarks of the encounters of the ubimus community.

The ubimus literature has been featured in the online proceedings of the Ubiquitous Music Workshops, as well as in various other publications including papers and special issues of the Journal of New Music Research (2011 and 2019), Sonic Ideas (2013), Cadernos de Informática (2014), Journal of Cases on Information Technology (2015), Vortex (2018), Journal of Digital Media and Interaction (2020). Three reference volumes have also been edited: Ubiquitous Music (Keller, Lazzarini, and Pimenta 2014a), Applications of Ubiquitous Music (Keller and Lima 2018), and Ubiquitous Music Ecologies (Lazzarini et al. 2020a).

An early definition of ubimus was given in 2014: “In practice, Ubiquitous Music is music (or musical activities) supported by ubiquitous computing (or ubicomp) concepts and technology” (Keller, Lazzarini, and Pimenta 2014b, p. xiii). From this quote we find that ubimus was originally developed out of the concept of ubicomp (Weiser 1991), which describes computing as an all-pervasive and invisible presence in our everyday lives (Lazzarini et al. 2020b). This seminal idea was extended into a research approach specifically directed at music-making. Ubimus is concerned with all the ways in which people can participate in creative musical activities. This attitude contrasts with a more conservative view of music studies (particularly those involving electronics) circumscribed to the practices of the specialists (formally or informally trained) and the professional musicians. As we entered the 21st century, the environments for musical activities were significantly expanded, blurring the lines that separate players from spectators. The paradigm of ubimus reduces the dependency on specialized training and entertains a change of attitude toward the potential venues, providing new spaces for creative performances, which include both formal venues and everyday settings. Ubimus practices also take note of the changes brought about by the global computing network in musical knowledge production, distribution, and sharing. But not all changes are positive. For instance, Lazzarini et al. state that “[t]hese changes are not limited to the application of technology or to the sonic results. They affect the whole chain of social and material relationships that permeate the musical endeavors. Therefore, these emerging cultural practices demand flexible frameworks that support rapid technical reconfigurations, while enforcing a cautious and responsible attitude towards the potential environmental and social consequences of these changes” (Lazzarini et al. 2020b, p. 1). Such concerns are at the core of several ubimus research initiatives.

In its first decade of development, the field of ubimus grew from the need to give access to creative experiences to people who were previously treated as passive consumers of musical products. Another motivation has been the creative exploration of sites and contexts that were previously inaccessible to musical practices. These two factors—stakeholders and locations—played an important role in shaping the first wave of ubimus initiatives, pushing the focus of the deployments toward two goals: erasing the barriers to music-making by newcomers and casual participants, and expanding the limits of what it means to make music.

Sonification, musification, and auditory display furnish a variety of strategies to transform data sets into sounds, ranging from utilitarian usages (e.g., listening to the data to gain insights into abstract patterns and relationships) to a variety of artistic applications involving aesthetic explorations of data acquired through diverse techniques. Interestingly, the latter approach predates the former. There were several composers and computer musicians who contributed to the early explorations of the musical possibilities of sonification techniques. Two notable examples are Charles Dodge (1970) and Jeff Pressing (1997).

Despite the the field of sonification's wide range of approaches and fast expansion (Kramer 1994; Walker and Nees 2011; Worrall 2019), there are still pending issues related to the application of data sets for creative purposes targeting nonspecialists. Anything can yield data—from the behaviors of the largest objects of the universe (exemplified by the gravitational waves emitted by the black holes located at the centers of galaxies, which surprisingly can be cleanly mapped to pitches within the hearing range (Abbott et al. 2016) to the data describing the behaviors of some of the smallest and toughest living creatures on Earth (such as the bacteria that live in the volcanic vents, at the ocean floor, or in the upper mantle, close to the ocean floor). The musification of these behaviors and phenomena may provide a window to aesthetic experiences and knowledge that until recently were beyond the reach of both specialists and nonspecialists.

Ubimus and Auditory Display

This special issue is the first attempt to explore the scientific and artistic issues raised by the intersection of two practices that target the development and deployment of sound-making technologies: ubimus and auditory display. As an exploratory initiative, the featured topics present excellent potential for expansion highlighting the utilitarian possibilities of ubimus-powered sonification and the creative potential of auditory display for ubimus endeavors. The four papers that appear herein are the result of a careful selection from dozens of proposals to the Ubiquitous Music Workshop held in November 2020 at Porto Seguro, Bahia, Brazil.

The two goals previously mentioned—the expanded strategies for participation and the diversified forms of musical engagement—sometimes foster methodological synergies and other times play against each other. For instance, consider the emerging subfield of lay–musician interaction, which investigates the interface between nonspecialists and technology. Various studies indicate that the motivations to join in a musical activity may vary depending on the individuals' level of domain-specific training (Ferreira, Keller, and Lima 2016; Lima et al. 2012). Preconceptions of what is or is not a musical activity impact the attitude of the stakeholders, sometimes enticing them to be part of the activity (especially when laypeople see trained musicians playing) and on other occasions dissuading the potential participants from engaging in group activities, as a by-product of what is thought to be socially acceptable in public settings (cf. Löwgren 2007; Montero et al. 2010). Consequently, the design of support strategies does not only need to consider the material demands—usually related to the deployment of technological infrastructure—it also needs to account for the cognitive and social factors at play. These contrasting factors have motivated the notions of ubimus ecologies and ecosystems that form the core of a research thread of the second wave of ubimus initiatives: ecologically grounded creative practice (Keller and Lazzarini 2017; Lazzarini et al. 2020a).

The expansion of the available places for artistic endeavors envisaged by the ubimus initiatives not only fosters new forms of creative manifestations (Keller and Lima 2018), it also demands that one design support for these new settings. A case in point is domestic ubimus (Keller et al. 2013; Ylirisku et al. 2013). Given the mobility restrictions imposed by the COVID-19 pandemic, domestic environments have suddenly become the ideal places for physically distant socialization. Colocated acoustic instrumental music-making is currently a life-threatening activity. Consequently, creative initiatives done at home provide better alternatives for group-targeted deployments. On the one hand, domestic ubimus may involve the implementation of asynchronous forms of interaction featuring resources from multiple modalities. On the other hand, it may entail an intensive and synchronous usage of network-based resources. Both modalities are actively explored by various ubimus projects involving the implementation of synchronization protocols, the deployment of the Internet of Musical Things (IoMusT), and the conceptualization of cross-modal ecologies for creative action, such as the sound sphere metaphor (Keller and Lazzarini 2017; Camporez et al. 2018; Turchet et al. 2018; Keller et al. 2020c).

Lay–musician interaction, everyday musical creativity, and domestic ubimus usually deal with resources that are readily available to the average urban dweller. But what happens when ubimus activities envisage resources that lie beyond the familiar and easily accessible city environments? This challenge was previously encountered by some forms of ecologically grounded creative practice (Keller 2004; Burtner 2011; Connors 2015). Thus, it may be possible to apply the knowledge gained through these artistic experiences to the deployment of ubimus ecosystems. As proposed by Opie and Brown (2006) and later developed by Gomes et al. (2014), extracting local data sets and massaging these data through ecologically consistent strategies may provide a path to the artistic deployment of physically unreachable realities. For example, two of Teresa Connors's recent audiovisual installations—“From the Edge” and “Currents”—are based on field recordings done on the east coast of Newfoundland in Canada. These installations utilize live-streamed environmental data to apply diverse audio-processing techniques to the collected materials. The data sets function as co-creative devices, allowing the composer to stretch and modify the concept of agency through ecological performative techniques (Connors 2015). All these projects' artistic practices rely on the extraction of data from environmental phenomena. But rather than treat the data sets as culturally neutral objects, they reinterpret the abstract patterns through situated sonification strategies. Consequently, these proposals differ from the historical musical usage of sonification, as found in the early computer-music pieces (Dodge 1970).

The situated perspectives on auditory display highlight new issues raised by the second-wave approaches to ubimus. The limitations of the note-based, hierarchical, and centralized way of thinking enforced by the acoustic-instrumental genres have been unveiled by the evidence gathered through several ubimus deployments (see also the critical reviews by Bown, Eldridge, and McCormack 2009; Bhagwati 2013; Keller, Lazzarini, and Pimenta 2014c; Keller and Lazzarini 2017). Instruments, notes, rhythm, melody, and harmony are all concepts that stem from an acoustic-instrumental music tradition strongly biased toward metric and pitch-centered forms of sonic organization. Despite their applicability to genres of music made with acoustic instruments, these concepts imply choices that restrict the creative palette to a narrow set of options. Some may argue that restricting the aesthetic choices is a valid strategy to boost creativity. We agree. But this assertion should be qualified. It is one thing to reduce the available processes or products as a consequence of the active participation in a creative activity. It is another, very different thing to propose conceptual and technological frameworks that are restrictive by design. Ubimus research defines the latter approach as early domain restriction (Lima et al. 2012; Keller, Lazzarini, and Pimenta 2014c).

The acoustic-instrument perspectives present serious limitations when applied to music made of volatile, distributed, ametric, or pitch-free resources. According to Keller, Messina, and de Oliveira (2020), various lines of investigation converge in music-making as a key element of the evolution of socially motivated human cognition. Current creative practices indicate that Varèse's notion of music as organized sound is partially aligned with the genre-free approaches to creative music-making fostered by the ubimus conceptual frameworks. Furthermore, if musicality is grounded in evolutionary processes constituted by exchanges among human and nonhuman agents, then effective ubimus mechanisms for knowledge transfer cannot be restricted to a single instance of a cultural manifestation within a narrow historical time-frame—much less if the chosen genre carries multiple racial, geographical, and economic biases!

Would it be possible to develop and adopt genre-free design perspectives in ubimus? Would these frameworks be deployable, transferable, or generally applicable to diverse cultural contexts? The answers to these questions might shape the path of the second wave of ubimus initiatives. Some musical genres enforce a narrow focus aligned with the tradition of instrumental music training. They adopt well-trodden procedures that sometimes are formalized as sets of instructions used by specialists to communicate the expected musical results. But they present a serious danger to creative endeavors because they usually carry the weight of authoritative knowledge. This last aspect acts in tandem with a general tendency to continue the traditional procedures: What worked well yesterday, should also be effective today and tomorrow. This weight of the cultural status quo is further reinforced by the scientific demand of replicability. An argument to support this view is that adopting widely used procedures is easier than developing innovative or original techniques from scratch.

Given this conceptual weight or hysteresis of the pool of established musical genres, it is not surprising that genre-neutral design frameworks are fairly rare. There are at least two factors to consider. (1) Genre-free frameworks need to be general enough to be applicable to fast changes in material conditions. Consider, for instance, the pace of change of the interaction techniques on mobile platforms. Within a period of only a few years, touch-based support moved from single-point touch to multitouch. Current health concerns push the industry to adopt proximity-based and touchless modalities as standards for interaction. But how many design frameworks are available that can handle touch, proximity, and touchless musical interaction as a consistent set of techniques? Are we prepared to deal with a future generation of portable devices that are aware of our actions, intentions, and emotions? What are the ethical and social implications of the pervasive presence of these technologies in our everyday life? (2) Genre-free design concepts need to be specific enough to be musically useful. It may be tempting to sweep the history of compositional techniques to look for approaches that were previously successful in generating new materials for creative usage in instrumental and electroacoustic composition. Why not propose design metaphors based on ubiquitous serialism (to revisit Babbitt) or why not refloat a Schaefferian phenomenology through an Internet of musical objects? Although there is nothing wrong in recycling a technique for usage in a different material or cultural context, the relevance of a compositional technique depends on a tightly knit set of factors that encompass politics, economics, and individual and collective aesthetic needs (Haworth 2015). For example, current DJs apply techniques that date from 1948, and some of them are happy with their musical results. Our argumentation targets the sustainability of the technological deployments, their social implications, and their potential to support diverse cultural manifestations. To probe the opportunities and limitations of embedded, networked, and mobile technologies for various musical contexts featuring both symbolic data and audio representations (Keller, Lazzarini, and Pimenta 2014c), the first wave of ubimus research utilized exploratory, small-scale projects. No claims of generality or cross-cultural validity were made. After settling on sufficiently general and stable conceptual approaches to ubimus, personal preferences and individual aesthetic choices may be handled during a second stage of experimental work. As the field of ubimus accumulates experimental evidence and tackles cross-cultural initiatives, there is a tendency to coalesce and aggregate elements and procedures around conceptual systems. Ecologically grounded creative practice (mentioned above) and ubimus computational thinking are two examples of this tendency.

Questions of Design

Computational thinking (CT) is a relatively recent area of research that has arisen out of the interrelated notions of critical thinking and task-oriented problem solving, both approaches being useful to deal with today's societal challenges (Otero et al. 2020). Such skills can be developed through computational techniques but are not limited to the realm of computer programming. One of CT's core characteristics is the application of methods that involve abstraction, generalization, and algorithm design to problem finding, problem solving, and exploratory thinking strategies. The competences acquired through CT facilitate a strategic take on complex issues and highlight the common aspects of different scenarios, leading to (for example) the reuse of past experiences to resolve new challenges. Within the ubimus context, in particular, CT helps to provide a framework for developing responses to some key methodological questions—for instance, how to “encourage people to explore the conceptual issues behind music-making and facilitate understanding about the on-going processes that influence/shape [this] activity” (Otero et al. 2020, p. 129). This question is tackled by Kramann's article in this issue, through the development of a mathematically oriented compositional approach. Aspects of ubimus CT permeate this work, connecting very well with recent ubimus findings.

Both Kramann's proposal and that of Groß-Vogt et al. raise issues that have been treated within the realm of human–computer interaction but whose potentials for music-making remain unexplored. Kramann probes the feasibility of using tangible objects to furnish playful ways to deal with musical generative techniques. Groß-Vogt et al. tackle the incorporation of audio processing as a strategy to convey information on electrical usage in domestic settings. While Groß-Vogt et al. focus on the sonic display of information, Kramann proposes access to algorithmic processes through material resources. The study by Groß-Vogt et al. is aligned with a well-established perspective on calm technologies (Weiser and Brown 1997; Oleksik et al. 2008). Kramann's proposal finds a context in tangible computing (Fitzmaurice, Ishii, and Buxton 1995). These two studies provide an excellent opportunity to explore the limits of the current ubimus approaches to handle deployments that at first sight seem to have nothing in common. So how are these two technologically diverse initiatives connected to the ongoing ubimus research threads?

Weiser et al. (1997) suggested that technologies that do not demand focused attention have better chances to be incorporated into the flow of everyday activities than those that require full engagement. This either/or vision of human–computer interaction may be relevant for utilitarian tasks but it is somewhat limited when applied to creative actions that—as is usually the case in music creation—target cognitive flexibility allied with the abstraction of multiple sources of information. Interestingly, this defense of a calm technology was inspired by an installation designed and executed by the artist Natalie Jeremijenko, “Live Wire” (1995). “Live Wire” used data of network traffic to control the movements of a dangling wire. The spectators got a sense of network usage just by watching the wire while listening to the sounds produced by its movement. This idea was further developed by the musician Chris Chafe and the visual artist Greg Niemeyer in their piece “Ping” (Chafe and Niemayer 2001), an installation that mapped the delays of the network to the pitch of physically modeled plucked strings. These artistic proposals rely on the synchronous display of abstract information by sonic means, hence they qualify as musification. But Groß-Vogt et al. propose yet another twist that brings the auditory display techniques closer to an ecological perspective on ubimus.

Rather than choosing an arbitrary sonic parameter—through a standard sonification procedure—to convey the extant level of electrical consumption, Groß-Vogt et al. home in on reverberation as a carrier of information of physical size. The adoption of a sonic cue correlated to a physical dimension was shown to be effective when trying to represent an abstract behavioral quality such as the usage of electricity. In line with previous ubimus initiatives (Keller et al. 2010), the study by Groß-Vogt et al. explores the manipulation of local resources—in this case the reverberant properties of the kitchen where the activities take place. Thus, the domestic space becomes simultaneously a target of deployment and a provider of the sonic resources for the situated auditory display.

Kramann extends the ubimus what you do is what you hear (WYDIWYHE) design strategy to the realm of tangible computing. This approach features a close connection between actions and sonic byproducts, inviting playful and exploratory behaviors as effective ways to enhance the creative possibilities of casual or untrained participants. An example of the application of the WYDIWYHE proposal is provided by the implementation and deployment of the Sound Sphere Ecology (abbreviated as SFS, for ecologia da eSFera Sonora) (Bessa et al. 2015, 2020). This initiative of the Núcleo Amazônico de Pesquisa Musical (NAP, Amazon Center for Musical Research) aims to support asynchronous, networked creative activities. The Sound Sphere Ecology is a set of web-based tools, loosely organized around audio mixing and processing tasks. The underlying creative-action metaphor situates a user at the center of a sphere or globe while the sonic events are projected onto the surface of the sphere. The vertical axis of the interface corresponds to the mixing tracks—aligned events are rendered simultaneously—and the horizontal axis corresponds to time. Movements along the temporal axis toward the left uncover pasts events, while movements to the right unveil future sonic content (more details in Keller et al. 2020c). All the sonic content is furnished by the user.

Keller et al. (2020c) report the results of a study involving twenty individuals doing musical activities in transitional settings, supported by the Sound Sphere Ecology. To understand how the stakeholders influence and are influenced by this creative-action metaphor, the experimenters proposed various open-ended creative musical activities to assess the participants' behaviors and creative products. The results suggest that the sound sphere metaphor provides effective support for casual interaction, with an emphasis on the participants' level of engagement. In line with the use of other ubimus support metaphors—time tagging, spatial tagging, creative surrogates, graphic-procedural tagging (Keller et al. 2014c)—activities with SFS foster engagement and collaboration. Both the timings of the exploratory activities and the assessments of the creative processes and products corroborate the idea that subjects with no special training can handle unfamiliar resources in small-sized mixing and audio-processing projects. Previous studies in ubimus research indicate that transitional settings tend to exert a positive impact on originality (Keller et al. 2013). However, the effect on relevance may not be uniform. As a downside, the assessments point to ease of use as the lowest and least consistent score among the rated creative factors. The authors mention that this caveat may be due to the heterogeneous profiles of the casual stakeholders.

Regarding Kramann's The Flippin' Pompoms (TFP) interaction proposal, the compositional activity consists of filling a transparent ball with colored objects. The ball is placed on a rotating platform. Hence it can be described as a tangible sound sphere metaphor. A mounted camera detects the colors of the objects. Blue, red, green, and yellow are directly mapped to prime numbers, and the sizes of the objects are used to determine the transitions among the events and among the generative operations. According to Kramann, the nearest neighbors are always taken from the largest areas chosen among the four colors. Therefore, the paths may overlap. This process tends to produce closely related sequences of pitches, but each iteration remains unique.

The TFP multimodal interface provides two strategies for creative support: One involves the ornamentation of the transparent balls and the other targets the modification of the generative algorithms (tailored for technologically savvy users). The TFP system and the underlying generative models are strongly tilted toward the organization of pitch sequences. But future prototypes could modify this design choice to encompass more subtle strategies of sonic organization. The author stresses that this creative-action metaphor is open to the participation of advanced users to encourage exploratory and flexible deployments.

Summing up, both proposals—the situated auditory display of Groß-Vogt et al. using local domestic resources and proposing an information-sharing strategy that leaves room for multiple parallel activities, and Kramann's expansion of the WYDIWYHE approach to the realm of tangible computing—foster the incorporation of everyday settings for sonic purposes and the empowerment of nonspecialists through playful, open-ended, and flexible support for creative action. Both proposals use strategies based on auditory display, despite fostering different perspectives on the application of the ubimus frameworks. The TFP design incorporates materiality as a means to extend a novice-oriented approach to musical interaction. It also applies the knowledge gained through computationally oriented creative approaches to allow for a flexible reconfiguration of the system, targeting advanced usage. The proposal of Groß-Vogt et al. on peripheral sonification expands the limits of calm technology by utilizing an ubimus-oriented design strategy that rests on situated and sustainable repurposing of local resources. This study not only confirms the feasibility of a situated approach to auditory display, it also opens new perspectives for the second wave of ubimus initiatives. Not only can domestic ubimus be approached as a setting for creative endeavors, but also it furnishes resources that can boost the potential of the ubimus design ecologies.

Questions of Metaphor

According to Barth et al., in this issue, scientific sonification provides both raw materials and methods for creative usage of pattern designs for ubimus. Barth et al. propose that the facilitated access to environmental data enhances the opportunities for data-based music creation targeting artistic, educational, and scientific activities. Their object of study is a geyser at Yellowstone National Park (USA) whose behavior has been observed and documented for a long period of time. This wealth of asynchronous data, encompassing seismic and infrared data sets, is used for visualization through animation techniques and for auditory display through wavetable and granular synthesis techniques. Given that the data sets are known in advance, the audification strategies can be freely explored. As a proposal for future studies, Barth et al. point out that volcanic, seismic, or weather conditions entailing potential hazards are interesting targets for synchronous implementations. The problem with these scenarios is to establish viable strategies of anticipation to identify patterns while adjusting the sonification strategies to the targeted categorizations.

Converging with these observations, a recent proposal for ubimus research considers the realm of future events as a useful ground for creative action. Keller, Simurra, and Messina (2020) discuss the initial efforts of a research program using not only past and present events but also involving the future as a valid object of musical study. This idea may seem unintuitive for an activity such as music-making that is firmly rooted in pre-existing practices and that deals with a product—sound—that is usually handled as it unrolls in time. But music-making, and particularly creative music-making, may utilize various forms of anticipation when the future is targeted as an element of decision-making. The ability to imagine sonic results—particularly if these results are obtained through algorithmic procedures—demands long periods of effortful and intensive training. So, would it be possible to anticipate the impact of the aesthetic decisions on the musical products as the musical activity unfolds? Would it be possible to use the expectations of multiple stakeholders as tools to inform the current individual aesthetic decisions? We believe that auditory display may be applicable in future deployments of anticipatory ubiquitous music. We hope that the examples of ubimus applications provided in this issue will provide a stepping-stone for new technological, conceptual, and artistic developments in this field.

Some areas of ubimus—despite the intrinsic caveats of jitter and delay of any network-based activity (Barbosa 2010)—strive to deploy computational resources for distributed synchronous activities. For a small number of time-sensitive applications (mentioned by Barth et al.), emerging fast protocols such as the Tactile Internet could eventually provide partial solutions to synchronization and speed of transmission. Nevertheless, a point to be stressed in this regard is that there is no free lunch. Homo sapiens groups have been doing music possibly since the beginning of their biological history (Mithen 2007). There is mounting evidence that socialization is an important factor for the healthy development of human cognition. Hence, technologies that ignore or highjack our dependence on community exchanges for commercial or sectarian purposes sever the implicit bond of trust that fosters the development of musicality and socialization. It is very dangerous to replace physical presence, human contact, and other processes of musically based social bonding with prosthetics. These issues need to be at the forefront of the initiatives to develop human–computer interaction techniques, auditory-display tools, and the hybrid human–machine ecosystems that will emerge to enable music-making in times of pandemics.

A promising line of ubimus developments targets the incorporation of food-related resources and activities for creative purposes. Provisionally named gastrosonics, this area stems from the utilitarian deployment of synthetic nutrients through computational means (Wei et al. 2011). As a picturesque detail, these technologies materialize the food replicators prominently featured in Gene Roddenberry's Star Trek series. Nowadays, replicators are possible. But rather than just preparing food, an exciting potential of these technologies is the possibility of integrating multiple forms of social interaction within a single ecology of tools. So far, gastrosonics remains in the realm of prototyping and conceptual discussion. A preliminary study using drinks as triggers for sonic choices was done by the NAP team in collaboration with researchers at Universidad Nacional de Tres de Febrero (Buenos Aires, Argentina). The sonic materials included sounds that were previously classified as related to various tastes (e.g., bitter, sweet, sour, or neutral). Two aspects of the results were promising. The participants were able to use the sound sphere metaphor to do joint creative projects while sipping the drinks. Furthermore, the choices indicated an alignment between the tastes of the drinks and the simulated tastes of the sound samples.

The Biota Beats project reported by Kim, Guo, and coauthors in this issue may be aligned to the gastrosonics initiative. Biota Beats proposes the use of bacterial colonies for sonification purposes. The authors suggest that these data may provide signatures that are specific to each person, to the body part chosen for bacterial sampling, and to various environmental factors. The methods entail taking a snapshot of the bacterial growth on a custom-designed Petri dish and mapping the spatial distribution of the colonies to a sequence of pitches. This melodic material is then freely edited and sequenced using commercial audio tools. The authors chose to focus on one musical genre, thus a metric and tonal aesthetic was adopted.

As is the case with most artistic applications of auditory display, bacterial-colony data afford various aesthetic treatments and are not necessarily tied to any preconceived musical genre. Given a standard format and an alignment of the sonification strategies, it would be possible to incorporate the bacterial data sets as resources for ubimus ecosystems. Bacterial cultures and data extraction demand laboratory settings and are fairly awkward to deploy. But as demonstrated by Kim, Guo, and coauthors, visualization and usage of these data are feasible with browser-compatible tools. Thus, this project is potentially compatible with other web-based ubimus proposals that use distributed resources for group collaborations (see Keller et al. 2011; Miletto et al. 2011; Stolfi, Milo, and Barthet 2019; Yi and Letz 2020). Careful manipulation of the bacterial culture environment may yield different growth patterns. This mirrors, at a microscopic scale, the strategy applied in gastrosonics at a human scale.

Conclusions: Challenges for the Second-Wave Ubimus Frameworks

As an introduction to the contents of this special issue, we have reviewed some of the initial motivations of the field of ubimus and have summarized key concepts grounding auditory display. We identified two ubimus frameworks that have been used by the sonification proposals featured in this issue: ecologically grounded creative practice and computational thinking. The application of musification and audification methods in ubimus contexts confirms the synergies between the two frameworks. This issue's four articles provide specific contributions toward the development of second-wave ubimus. Kramann furnishes an example of the expansion of a creative-action metaphor through tangible-computing techniques and presents a strong argument toward the incorporation of computational thinking in ubimus design. Groß-Vogt et al. present and exemplify the concept of peripheral sonification as an ecologically grounded sonification technique targeting the deployment of domestic ubimus technologies. Kim, Guo, and coauthors propose the audification of bacterial colonies as a source for creative musical activities, one that we suggest could be incorporated as browser-based resources for gastrosonic activities. Finally, Barth et al. use data extracted from geyser eruptions as an example of sonification and visualization, establishing a bridge to resources that have remained inaccessible to urban ubimus stakeholders. They also point to potential deployments for synchronous notifications of hazards, indicating the need to apply pattern-tracking techniques. This observation reinforces the research initiative on anticipatory ubimus.

This body of contributions is very positive. But this collection of papers could not be called an ubimus volume if it did not point to methodological caveats while highlighting the potential limitations of the proposed concepts. One question that emerges from the confluence of auditory display methods with ubimus research is how to handle (or how to avoid) musical genres. Genre-neutral design approaches—such as ecological and computational thinking—seem to be suitable for the majority of everyday scenarios. But more research is needed to determine, for instance, how sonification could enhance the palette of ubimus “comprovisational” techniques. Another problematic issue is how to deal with musification within the context of distributed group-based creative activities that involve casual participants.

Auditory display furnishes musical depictions of—on the one hand—the smallest and largest phenomena in the universe and—on the other hand—the slowest and fastest sequences of physical events. For many years, acoustic-instrumental music has constrained the creative activity to actions exerted by humans on resonant objects designed exclusively for humans. The incorporation of multiple time scales and multiple spatial scales within hybrid music-making scenarios that feature both human and nonhuman agents breaks down the expectations created by the acoustic-instrumental paradigm. We need new conceptual frameworks to deal with a musical presence that has become widely distributed, increasingly ephemeral, increasingly mundane, less centralized, less manageable, and as a consequence, less obvious. In short, we need new frameworks for ubiquitous music.

Acknowledgments

We extend our thanks to Stephen Barrass, this special issue's reviewers, and the Ubimus 2020 program committee for their help during the selection and evaluation of proposals for this special issue.

REFERENCES

Abbott
,
B. P.
, et al.
2016
. “
Observation of Gravitational Waves from a Binary Black Hole Merger.
Physical Review Letters
116
(
6
):Art.
[PubMed]
. Available online at link.aps.org/doi/10.1103/PhysRevLett.116.061102.
Barbosa
,
Á.
2010
. “
Performance musical em rede
” [Network music performance]. In
Criação musical e tecnologias: teoria e prática interdisciplinar
[Musical creation and technologies: Interdisciplinary theory and practice]. Goiânia, Brazil: ANPPOM, pp.
180
200
. Available online at http://repositorio.ucp.pt/handle/10400.14/5008.
Bessa
,
W. R. B.
, et al.
2015
. “
SoundSphere v. 1.0: Análise e Experimentos
” [SoundSphere v. 1.0: Analysis and experiments]. In
J. T.
de Souza Mendes da Silva
et al., eds.
Anais do Simpósio Internacional de Música na Amazônia
[Proceedings of the Amazon International Symposium on Music], pp.
519
529
. Available online at revista.ufrr.br/sima/article/view/4716/2382.
Bessa
,
W. R. B.
, et al.
2020
. “
A metáfora da esfera sonora desde a perspectiva WYDIWYHE
” [The Sound Sphere metaphor from a WYDIWYHE perspective].
Journal of Digital Media and Interaction
3
(
5
):
60
88
. With abstract in English.
Bhagwati
,
S.
2013
. “Notational Perspective and Comprovisation.” In
P.
de Assis
,
W.
Brooks
, and
K.
Coessens
, eds.
Sound and Score: Essays on Sound, Score and Notation
.
Brussels
:
Leuven University Press
, pp.
165
177
.
Bown
,
O.
,
A.
Eldridge
, and
J.
McCormack
.
2009
. “
Understanding Interaction in Contemporary Digital Music: From Instruments to Behavioural Objects.
Organised Sound
14
(
2
):
188
196
.
Burtner
,
M.
2011
. “
EcoSono: Adventures in Interactive Ecoacoustics in the World.
Organised Sound
16
(
03
):
234
244
.
Camporez
,
H. A. F.
, et al.
2018
. “
RoboMus: Uma plataforma para performances musicais robóticas
” [RoboMus: A platform for robotic musical performances]. In
Keller
and
Lima
2018
, pp.
58
93
.
Chafe
,
C.
, and
G.
Niemayer
.
2001
. “
Ping.
” Available online at Chrischafe.net/portfolio/ping. Last accessed 1 October
2020
.
Connors
,
T. M.
2015
. “
Audiovisual Installation as Ecological Performativity.
” In
Proceedings of the International Symposium on Electronic Art
. Available online at isea2015.org/proceeding/submissions/ISEA2015_submission_149.pdf.
Dodge
,
C.
1970
.
Earth's Magnetic Field
.
New York
:
Nonesuch H-71250
, LP.
Ferreira
,
E.
,
D.
Keller
, and
M. H.
Lima
.
2016
. “
Sonic Sketches in Ubiquitous Music: Educational Perspectives.
Sonic Ideas
8
(
15
):Art.
[PubMed]
.
Fitzmaurice
,
G.
,
H.
Ishii
, and
W.
Buxton
.
1995
. “
Bricks: Laying the Foundations for Graspable User Interfaces.
” In
Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems
, pp.
442
449
.
Gomes
,
J.
, et al
2014
. “
Capture and Transformation of Urban Soundscape Data for Artistic Creation
.”
Journal of Science and Technology of the Arts
6
(
1
):
97
109
.
Haworth
,
C.
2015
. “
Sound Synthesis Procedures As Texts: An Ontological Politics in Electroacoustic and Computer Music.
Computer Music Journal
39
(
1
):
41
58
.
Keller
,
D.
2004
. “
Paititi: A Multimodal Journey to El Dorado.
” PhD dissertation,
Stanford University
, Department of Music.
Keller
,
D.
, and
V.
Lazzarini
.
2017
. “
Ecologically Grounded Creative Practices in Ubiquitous Music.
Organised Sound
22
(
1
):
61
72
.
Keller
,
D.
,
V.
Lazzarini
, and
M.
Pimenta
, eds.
2014a
.
Ubiquitous Music
.
Berlin
:
Springer
.
Keller
,
D.
,
V.
Lazzarini
, and
M.
Pimenta
.
2014b
. “
Prologue—Ubiquitous Music: A Manifesto.
” In Keller, Lazzarini, and Pimenta
2014a
, pp.
xi
xxiii
.
Keller
,
D.
,
V.
Lazzarini
, and
M.
Pimenta
.
2014c
. “
Ubimus through the Lens of Creativity Theories.
” In Keller, Lazzarini, and Pimenta 2014a, pp.
3
24
.
Keller
,
D.
, and
M. H.
Lima
, eds.,
2018
.
Aplicações em Música Ubíqua
[Applications in Ubiquitous Music]. São Paulo
:
ANPPOM
.
Keller
,
D.
,
M.
Messina
, and
F. Z. N.
de Oliveira
.
2020
. “
Second-Wave Ubiquitous Music: Collaboration, Automation and Knowledge Transfer for Creativity (Editorial).
Journal of Digital Media and Interaction
3
(
5
):
5
20
.
Keller
,
D.
,
I.
Simurra
, and
M.
Messina
.
2020
. “
Toward Anticipatory Ubimus.
EAI Endorsed Transactions on Creative Technologies: Online First
7
(
24
):
e4
. Available online at eudl.eu/pdf/10.4108/eai.13-7-2018.164664.
Keller
,
D.
, et al
2010
. “Anchoring in Ubiquitous Musical Activities.” In
Proceedings of the International Computer Music Conference
, pp.
319
326
.
Keller
,
D.
, et al.
2011
. “
Marcação espacial: Estudo exploratório
” [Spatial tagging: An exploratory study]. In
L.
Costalonga
et al., eds.
Anais do Simpósio em Computação Musical
[Proceedings of the Brazilian Symposium on Computer Music]. Available online at compmus.ime.usp.br/sbcm/2011, with an abstract in English.
Keller
,
D.
, et al.
2013
. “
Criatividade musical cotidiana: Um estudo exploratório com sons vocais percussivos
” [Everyday musical creativity: An exploratory study with vocal percussion sounds]. In
Congresso da Associação Nacional de Pesquisa e Pós-Graduação em Música
[Proceedings of the Congress of the National Association of Research and Graduate Studies in Music]. Available online at www.anppom.com.br/congressos/index.php/ANPPOM2013/Escritos2013/paper/view/2098/420.
Keller
,
D.
, et al
2020c
. “
Semantic Strategies in Ubiquitous Music: Deploying the Sound Sphere Ecology in Transitional Settings
.”
Heliyon
6
(
9
): e04843.
Kramer
,
G.
1994
.
Auditory Display: Sonification, Audification, and Auditory Interfaces
.
Boca Raton, Florida
:
CRC
.
Lazzarini
,
V.
, et al., eds.
2020a
.
Ubiquitous Music Ecologies
.
London
:
Routledge
.
Lazzarini
,
V.
, et al.
2020b
. “
The Ecologies of Ubiquitous Music.
” In Lazzarini et al.
2020a
, pp.
1
22
.
Lima
,
M. H. d.
, et al
2012
. “
Creativity-centred Design for Ubiquitous Musical Activities: Two Case Studies
.”
Journal of Music, Technology, and Education
5
(
2
):
195
222
.
wgren
,
J.
2007
. “
Pliability as an Experiential Quality: Exploring the Aesthetics of Interaction Design.
Artifact
1
(
2
):
85
95
.
Miletto
,
E. M.
, et al
2011
. “
Principles for Music Creation by Novices in Networked Music Environments
.”
Journal of New Music Research
40
(
3
):
205
216
.
Mithen
,
S.
2007
.
The Singing Neanderthals
.
Cambridge, Massachusetts
:
Harvard University Press
.
Montero
,
C. S.
, et al
2010
. “Would You Do That? Understanding Social Acceptance of Gestural Interfaces.” In
Proceedings of the International Conference on Human Computer Interaction with Mobile Devices and Services
, pp.
275
278
.
Oleksik
,
G.
, et al
2008
. “Sonic Interventions: Understanding and Extending the Domestic Soundscape.” In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
, pp.
1419
1428
.
Opie
,
T.
, and
A.
Brown
.
2006
. “
An Introduction to Eco-Structuralism.
” In
Proceedings of the International Computer Music Conference
, pp.
9
12
.
Otero
,
N.
, et al.
2020
. “
Computational Thinking in Ubiquitous Music Ecologies.
” In
Lazzarini
et al.
2020a
, pp.
129
153
.
Pressing
,
J.
1997
. “
Some Perspectives on Performed Sound and Music in Virtual Environments.
Presence
6:482
503
.
Stolfi
,
A. S.
,
A.
Milo
, and
M.
Barthet
.
2019
. “
Playsound.space: Improvising in the Browser with Semantic Sound Objects.
Journal of New Music Research
48
(
4
):
366
384
.
Turchet
,
L.
, et al.
2018
. “
Internet of Musical Things: Vision and Challenges.
IEEE Access
6:61994
62017
.
Walker
,
B. N.
, and
M. A.
Nees
.
2011
. “Theory of Sonification.” In
T.
Hermann
,
A.
Hunt
, and
J. G.
Neuhoff
, eds.
The Sonification Handbook
.
Berlin
:
Logos
, pp.
9
39
.
Wei
,
J.
, et al
2011
. “Food Media: Exploring Interactive Entertainment over Telepresent Dinner.” In
Proceedings of the International Conference on Advances in Computer Entertainment Technology
, pp.
1
8
.
Weiser
,
M.
1991
. “
The Computer for the 21st Century.
Scientific American
265
(
3
):
94
105
.
Weiser
,
M.
, and
J. S.
Brown
.
1997
. “The Coming Age of Calm Technology.” In
Beyond Calculation: The Next Fifty Years of Computing
.
Berlin
:
Springer
, pp.
75
85
.
Worrall
,
D.
2019
.
Sonification Design
.
Berlin
:
Springer
.
Yi
,
S.
, and
S.
Letz
.
2020
. “
The Browser as a Platform for Ubiquitous Music.
” In
Lazzarini
et al. 2020a, pp.
170
189
.
Ylirisku
,
S.
, et al
2013
. “Designing Web-connected Physical Artefacts for the ‘Aesthetic’ of the Home.” In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
, pp.
909
918
.