From Fiction to Function: Imagining New Instruments through Design Workshops

Digital musical instrument (DMI) design is a broad and interdisciplinary field (Miranda and Wanderley 2006) in which designers engage in the development of new instruments and explore novel approaches to musical performance, as well as composition and production, for a wide variety of reasons. Even where DMI design is fundamentally research-based, approaches and methods can vary widely, ranging from rigorous scientific experimentation to artistically motivated creative practice (Gurevich 2016). Fittingly, the field, and more generally the broad domain of music technology in which it lies, contributes a wide range of research outcomes both within and beyond specifically musical applications, such as the development of new technologies for interactive systems (Malloch, Sinclair, and Wanderley 2018) and the advancement of knowledge and theories on technology-mediated artistic performance (Tahlroǧlu et al. 2020).

Researchers have also shown, however, that many DMIs are neither intended nor suitable for use in real-world musical practice (Morreale and McPherson 2017) and may be limited in their musical potential by the scope of the research for which they were created. Jack, Harrison, and McPherson (2020, p. 451) suggest that designers and researchers will benefit from approaches to design and evaluation that can account for instruments as “situated, ecologically valid artefacts” in music-making contexts.

With this work, we were interested in exploring a novel method for the design of instruments expressly intended for real-world musical practice. Motivated by previous studies that examined key factors for user engagement (O'Brien and Toms 2008) and long-term use of DMIs in performance (Wallis et al. 2013; Sullivan, Guastavino, and Wanderley 2021), we conducted a workshop with expert musicians that led to a set of tangible design specifications for new performance DMIs. The workshop applied a user-driven approach to the early ideation stages of instrument development through the use of design fiction (Blythe 2014) and nonfunctional prototyping (Pigrem and McPherson 2018) to inspire creative concepts for new instruments.

We will discuss two contributions from our work. First, we present a DMI design workshop methodology that extends previous approaches found in the literature of human–computer interaction (HCI), applying output from creative ideation sessions held with active musicians to the generation of tangible design specifications that can be directly applied in the development of new instruments. A second practical contribution is made from our workshop results, in the form of empirically generated, high-level specifications that can be of practical use to other DMI designers.

In the following section, we review motivations and methods for designing novel DMIs, focusing on existing theories and approaches drawn from the literature of HCI and human-centered design (HCD). We then introduce our methodology and present results from the thematic analysis of the design workshop sessions we ran. Finally, we discuss the potential of a design-fiction approach to developing instruments that are viable for uptake and long-term use by expert musicians in real-world performance, and we summarize our continuing work that applies our workshop results to the development of new performance-ready DMIs.

An important aspect of the progression from acoustic to electronic and digital instruments has been the decoupling of the user input from sound production, leaving the designer free to choose any type of input control and mapping paradigm to control sound production. This freedom may be welcome, as evidenced by the quantity, complexity, and diversity of new DMIs that have been developed over the last few decades. This may also present a challenge for designers, however, when “any bodily gesture can be mapped to any sound and there is no natural paradigm at play [to which] we can relate” (Magnusson 2019, p. 34). Although the issue of mapping is a deep area of research and scholarship in its own right (for in-depth reports, see Wanderley 2002; Wanderley and Malloch 2014), here we focus on two more basic inquiries: First, if few technical and conceptual limitations exist, what guides the design of new DMIs? And second, what methods can be applied to the DMI design process that will make them appealing for use in artistic practice?

There is a wide range of factors that motivate the design of DMIs. In research settings, it may be useful to design DMIs that are specific to a particular experimental context (Marquez-Borbon et al. 2011). A survey by Morreale and McPherson (2017) of DMI designers presenting instruments at the International Conference on New Interfaces for Musical Expression (NIME) found this to be a common approach: Out of the 97 instruments included in the study, 38 (39 percent) were reported to have been designed as research probes. In these circumstances, the instruments' actual use in real-world performance may be of secondary importance to more immediate research objectives. When considering performance with new instruments in more-widespread contexts—particularly those outside of research—social, cultural, and economic factors come into play, in particular market and consumer-driven behaviors that influence trends, popularity, and visibility of commercial, off-the-shelf instruments (Théberge 1997). The diversity of design approaches and objectives, in particular between commercial and research-based designs, has been highlighted by McPherson, Morreale, and Harrison (2019) in a comparison of instruments emerging from different domains, including NIME, HCI, and crowdfunding campaigns.

Emerson and Egermann (2020) focus specifically on the design of DMIs expressly intended for artistic practice, identifying four primary categories of motivation: facilitating greater embodiment in performance, improving audience experience, developing new sounds, and building responsive systems for improvisation. They also highlight different motivations based on the context of participants' practices: Those more active in academic settings exhibit more interest in new sounds and responsive systems for improvisation, whereas others who perform in club settings are motivated to improve embodiment and audience experience. These different motivations might also be associated with the types of music the participants perform with their instruments, although that possibility is not explored in this current work.

The diverse motivations for designing new instruments are also reflected in various design approaches and methodologies used. Part of the discourse on DMI design comes from the field of HCI, which over time has evolved from quantitative and rigid strategies (Bødker 2015) to paradigms that are better suited to accommodate the idiosyncratic approaches frequently found in contexts of music interaction (Wanderley and Orio 2002). Second-wave HCI emerged to include perspectives on technology within social, cultural, and organizational contexts (Kaptelinin et al. 2003), with an emphasis on user-centered, qualitative methods theoretically grounded in situated action, distributed cognition, and activity theory (Bødker 2006). The third wave has continued to expand the purview of HCI to accommodate the ubiquitous nature of technology, prioritizing experiences, meaning making, and emergent use (Bødker 2015). This third paradigm is phenomenologically situated with a focus on embodied interaction to embrace multiple interpretations and yield rich understandings (Harrison, Tatar, and Sengers 2007), supported by ethnographic and practice-based research approaches (Bødker 2015).

User-centered design (UCD) requires the designer to “ask what the goals and needs of the users are, what tools they need, what kinds of tasks they wish to perform, and what methods they prefer to use” (Norman 1988, as cited in El-Shimy 2014, p. 44). Human-centered design has emerged as a subtle but important variation (Norman 2013), allowing for a broader consideration of people with regards to design, instead of “a narrower focus on peoples' roles as users” (Steen 2011, p. 45). This corresponds with trends in HCI:

In DMI design, the concept of “musician-as-user” has been challenged more recently by Rodger et al. (2020), who argue that there is no prototypical “user” for a given instrument, and characterizing musical activity is highly dependent on the context in which it occurs.

Participatory design (PD) is an HCD approach that became well-established with second-wave HCI (Bødker 2015) and has continued to be relevant in third-wave HCI as well (Muller and Druin 2012). It is predicated on the full participation of end users through all stages of the design process (Steen 2011) and is primarily concerned with the tacit knowledge of the involved participants. This knowledge, according to Spinuzzi (2005), is hard to formalize and had been missing from early HCI. In DMI research, PD methods have been applied in the design and evaluation of Theremin-based controllers (Geiger et al. 2008), audio–haptic interfaces for the visually impaired (Metatla et al. 2016), and more generally to investigate music interaction design based on conceptual metaphor theory (Wilkie, Holland, and Mulholland 2013) and digital musical interaction ecologies (Fyans et al. 2012).

It is important to note that PD is rooted in social activism and was originally envisioned as a way “to rebalance power and agency among managers and workers” (Bannon, Bardzell, and Bødker 2018, p. 1). Some current PD practices have been critiqued as merely UCD with a different name, lacking the original political and activist contexts (Bannon, Bardzell, and Bødker 2018). Our work presented here uses methods that fall under the PD umbrella but without any specific political motivation; therefore we identify our approach as HCD in deference to these objections.

Many different frameworks have been developed for the design and evaluation of new DMIs. Examples include models based on categories of musical interaction (Bongers 2000); structural components of DMIs (Wanderley and Depalle 2004); diverse needs of different performers (Jordà 2004a, b); interdisciplinary approaches combining music performance, HCI, and digital technology (Overholt 2009); and tracking of user experience (Morreale, Angeli, and O'Modhrain 2014). Evaluation frameworks include those based on user tasks (Wanderley and Orio 2002); the roles of various stakeholders (O'Modhrain 2011); and formal HCI techniques for assessing functionality, usability, and user experience (Young and Murphy 2015; Reimer and Wanderley 2021).

Although high-level design frameworks may help formulate a conceptual approach, they are generally not oriented toward specific design tools and methods or their use in artistic contexts. For our own work, we wish to develop effective strategies for developing new instruments that will be appealing for musicians to incorporate into their real-world creative practice. In particular, we look at two user-driven approaches to generating ideas for new designs: a physical DMI prototyping toolkit and a design workshop methodology.

Another compelling approach to idea generation and prototyping for DMI design comes with the concept of “Magic Machine” workshops, developed by Kristina Andersen (2017). The workshops “make use of the notion of technology as a ‘magical unknown’ as the starting point for a range of workshop techniques that begin with material exploration” (Blythe et al. 2016, p. 4971). In them, participants are prompted to make nonfunctional, low-fidelity prototypes out of generic crafting materials like cardboard, wood, string, and glue. Once finished, they present their creations, demonstrating their use in imagined scenarios.

The Magic Machine workshops have some basis in design fiction, where concepts and problems can be explored through the development of imaginary scenarios and “fantasy prototypes” (Sterling 2009). Importantly, the generated artifacts (the nonfunctional prototypes) are not overly meaningful in and of themselves, and the ultimate aim is not to solve any given problem. Rather, the processes of creating and engaging with the “magical unknown” serve “to give temporary body to concerns and questions” and “to consider the potential reality of a world in which such a thing might exist” (Blythe et al. 2016, p. 4971). Andersen has run the workshops in a variety of contexts for both adults and children, including workshops for the design of new, imagined musical instruments. The workshop has been used by other DMI researchers as well, including a study by Lepri and McPherson (2019) that explored diverse values and priorities of different music cultures, backgrounds, and contexts.

The Probatio toolkit and Magic Machine workshops present dynamic methods for early-stage ideation of DMI design that are closely related to our own work. The workshops offer a creative approach to generating design ideas free from technological or practical constraints. In Andersen's workshops, however, the physical prototypes created remain works of fiction, with no intent to develop and refine them or their constituent parts into tangible, functional designs. The Probatio toolkit, on the other hand, provides a clearly defined set of tools to quickly assemble and evaluate functional prototypes from preconstructed modules. This situates the ideation process in a functional prototyping activity (from which real, playable instruments are created), but the activity is creatively constrained by the limited number and fixed properties of modules contained in the kit.

Thus we find potential from each approach to benefit the other, where ideas generated from the unconstrained design space of the workshops could be used as input into applied instrument development, and where the quick functional prototyping model of Probatio could capitalize on creative and unconventional ideas coming out of design fiction activities. To explore this further, we developed and ran a workshop to bridge a perceived gap between fictional design and the development and evaluation of functional DMI prototypes aimed for use in real-world music performance contexts. In the following section, we present an account of the workshop and a discussion of the results, including our continuing work to apply outcomes from the workshop to functional and ecologically valid instrument designs.

The Design for Performance workshop was developed to involve expert musicians in the early-stage ideation of new instruments. The workshop draws from a variety of general methods from HCD and PD that have been mentioned so far: design workshops, nonfunctional and low-fidelity prototyping, and design fiction. The structure builds upon Andersen's Magic Machine workshops, with the outcomes applied to the generation of design specifications for use in the development of new performance-ready DMIs.

The context of the Design for Performance workshops marks an important shift from Andersen's Magic Machine workshops, which are specifically oriented towards building diverse design knowledge and complex understandings “about technology, rather than of technology” (Andersen and Wakkary 2019, p. 1). Our approach seeks to find a middle ground between theoretical knowledge and tangible design, connecting the diversity and creative freedom fostered by the Magic Machine activities with a holistic design ecology from ideation to finished product. In this way, the Design for Performance workshop is envisioned as a design tool that can elicit preliminary ideas from a group of expert practitioners and translate them into tangible elements that designers can work with. This may be especially valuable in the DMI design space, where idiosyncratic approaches and highly personalized designs are common, and in which widespread adoption of new DMIs is limited.

The workshop was structured as an in-person session for participants to engage in design activities and develop their designs and ideas together as a group. The workshop was composed of five activities:

1. Prompt: “Draw the music,” a short opening design activity to engage the participants in a creative space.

2. Crafting nonfunctional prototypes: the main activity where participants crafted imaginary musical instruments from provided crafting materials.

3. Presentations and identification of key elements: participants presented their instrument, and key design elements were identified and added to a whiteboard.

4. Dot voting: participants voted for the key elements they found most favorable.

5. Group discussion: an open discussion with the participants and facilitator.

These activities will be detailed in the Workshop Activities section.

To test and fine-tune the workshop design, we conducted a pilot with six graduate students enrolled in a graduate seminar on musical interface design. The first author acted as the facilitator and was assisted by another graduate student. Three observers (the second author, a visiting professor, and a graduate student) also attended the session to provide feedback on the workshop design. At the conclusion, an informal discussion was held to evaluate how the session had run and to make suggestions for improvements. All generally agreed on the activities and format, and details for minor changes were noted and incorporated into the final structure.

As the workshop was focused on the design of DMIs for use in live performance and intended to be held with practicing musicians, we identified three main criteria for prospective participants: (1) they should use, or at least be familiar with, DMIs; (2) they should maintain an active performance practice (performing publicly at least five times per year); and (3) their performance practice should be related to styles in which DMIs are typically used, such as electroacoustic and experimental styles.

A call for participation was circulated through local academic and performance-community mailing lists that were likely to reach individuals matching our criteria. Interested parties were invited to complete an online screening questionnaire. Recruitment lasted for two weeks and 25 responses were received. Fifteen individuals met the criteria and were invited to participate, of which 10 accepted. To accommodate schedules, the workshop was divided into two sessions, with three participants in session A and seven in session B.

The workshop activities and procedures were reviewed and approved by the Research Ethics Board Office of McGill University. All participants read and signed a form of informed consent to participate before the workshop started. The form also requested their permission for us to take photos and video recordings of the sessions, and for those photos in which they appeared to be used in future publications. These requests were all granted.

A schedule for the workshop had been developed based on Andersen's recommendations to run the workshop at a quick pace and keep a tight timeline. This was found to be an effective strategy to alleviate any potential anxieties or fears of failure that participants could experience during the creative and open-ended design activities (Andersen 2017). The sessions were held on consecutive days in a spacious and well-lit conference room at the Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT) at McGill University. Tables were arranged together so that participants sat around the outside facing each other. A variety of crafting materials was arranged on a separate table and covered with a cloth when the participants arrived. A video camera was set up to record the sessions.

The first author acted as the facilitator and was aided by the same assistant from the pilot session. Each session began with a welcome and introduction by the facilitator, then participants were asked to briefly introduce themselves and to give a summary of their musical practice and experience with DMIs.

The workshop activities began with a prompt. Participants were asked to think of the music that they currently make or would like to make, and then instructed to “draw the music” on an index card in front of them with a permanent marker. They were given two minutes to complete the activity, after which the workshop moved directly on to the next activity.

Andersen stipulates two important functions that this activity serves. First, it provides the specific context for the workshop focus. Drawing the participants' attention to making music in novel and unexpected ways (as suggested by creating abstract visual representations of the music) oriented their focus toward thinking creatively about instruments and performance. Second, the short activity serves as a preliminary task to complete “an initial goal . . . that tests competence and establishes confidence, acting as an on-ramp to an experience” (Andersen and Wakkary 2019, p. 5). This eases the transition to the more substantial design activity that follows, as one creative task has already been completed.

In this main activity of the workshop, “the content of the prompt must be translated into an imagination of the device that produces it” (Andersen and Wakkary 2019, p. 5). The use of rudimentary crafting materials moves the focus away from producing high-resolution or even technically feasible designs. Instead, the participants are asked to envision and craft a purely fictional instrument that they would want to use, and the materials (and especially their unsuitability for functional instrument design) allow participants to operate freely and instinctively without concern for implementation or technical constraints.

To assist the participants in developing their ideas into tangible designs, we introduced an informal list of eight considerations to which the participants could refer while building their instruments. These considerations were written on a large whiteboard. The considerations are divided into two categories: (1) high-level operational qualities and general characteristics that could describe the instrument's intended use (functionality, playability, musicality, and performance context), and (2) low-level essential features and fundamental components of the design (physical form and ergonomics, interaction methods, sound production, and feedback). Although some of the considerations can be classified as functional or nonfunctional requirements, as defined by requirements engineering (Glinz 2007) and commonly used in systems design, it is important to point out that these elements were empirically chosen based on our own prior knowledge and experience in DMI design, and left open-ended to provide helpful points of reference through the activity.

Following the construction, participants each gave a three-minute presentation of their instrument. Participants were encouraged to explain the links between the music on their index cards and the instruments, which helped to orient the presentations on the imagined outcomes rather than the technical details of the fabricated designs. A two-minute group discussion followed each presentation.

While the participants described their instruments the facilitator noted key elements of their designs, such as essential features, attributes, and characteristics that could help to define the instrument. These elements were written on sticky notes and posted to the whiteboard, clustered around the eight considerations that had been given in the previous activity. The presenter and other participants were encouraged to suggest elements to add to the board as well.

The workshop concluded with an open discussion with the facilitator and participants about the key elements identified and prospects for utilizing those elements in the development of new DMIs that the participants would want to use in their own practice. The facilitator also provided information about future steps for the project, including the eventual design of functional instrument prototypes based on the workshop outcomes. The workshop length for session A with three participants was 90 minutes. Session B, with seven participants, was completed in just under two hours.

Workshop results are reported in two parts. First, we present the direct outcomes of the workshop activities. Then we present results from our thematic analysis of the workshop sessions, which had been recorded on video.

In addition to the basic background information collected in the screening questionnaire, the self-introductions at the beginning of the workshop provided additional insights about the participants' own musical and DMI practices. Profiles of the ten participants are shown in Table 1.

During the self-introductions, participants were also asked if they had previous experience with designing DMIs. Although this was not a criterion for participation, six of the participants reported at least some previous experience, including two (P04 and P05) who came from engineering backgrounds and had significant technical knowledge and background in this area. This amount of experience is consistent with DMI literature that highlights the overlap between DMI design and practice (e.g., Magnusson and Hurtado 2008; Morreale and McPherson 2017; Morreale, McPherson, and Wanderley 2018). Although not a specific focus for this study, we can envision a future workshop variant that explicitly investigates the differences between designers and nondesigners.

During the workshop, each participant produced two physical design artifacts: the “draw the music” index card, created in Activity 1, and the instrument prototype. The outputs are shown in Table 2, with instruments categorized according to a classification of gestural controllers proposed by Miranda and Wanderley (2006): augmented, instrument-like, instrument-inspired, and alternate. As the drawing of the cards was done as a prompt for the main activity, they are not examined here.

Four of the remaining five instruments can be identified as either instrument-like or instrument-inspired, taking various elements from existing instruments and repurposing them in different ways. A noticeable trend among this group was to combine the functionalities of several instruments into a single instrument, either to be able to play multiple parts simultaneously (like P06's multifunction performance workstation, Figure 5b) or to mix them together in creative ways (like P02's instrument that would mix and cross-modulate vocals and guitar, Figure 5c).

There was one augmented instrument (designed by P09, Figure 5d). This participant is an expert instrumentalist with an advanced degree in performance on her instrument, the concert harp. She has been performing electroacoustic music with her harp and various external controllers including custom interfaces designed by the first author (Sullivan et al. 2018) and was clear in her needs and priorities as a performer, which was reflected in the pragmatic approach and practical utility of her design.

As described previously, key elements of the participants' prototypes were identified and posted to the whiteboard during the presentations. Afterward, the participants were asked to dot vote for the elements that they would most want to be incorporated into a new instrument design. In the planning of the workshop, we imagined these activities could serve two purposes. First, they could provide direct, quantifiable data that could be applied toward the eventual development of design specifications. Second, they could also serve as a catalyst for exploring the designs in the final group discussions.

Ultimately, we found that the data collected through these exercises contained limited insights, therefore we forego a detailed discussion of these results here. One reason for this was the rapid pace of the presentations, which made it difficult to thoroughly and consistently identify the most important aspects of each prototype. Furthermore, these activities marked a departure from Andersen's Magic Machine workshops, where the physical objects are intended to evoke inspirations for discussion and conversation within the workshop group and “serve as simple vessels for notions and ideas, which are somewhat or completely beyond what is represented in the model” (Andersen 2017, p. 63). Our attempt to quantify the designs during the workshop runs counter to this intention. On the other hand, we found that postworkshop analysis of the video-recorded presentations provided a much richer understanding of the workshop activities, as presented in the Results section.

The group discussions that concluded the workshop varied between the two sessions. The discussion with the smaller Session A group (only three participants) was free-flowing, and an ongoing conversation developed between the participants through the presentations, dot-voting exercise, and into the closing discussion. There was a general consensus around desirable instrument features and qualities despite each individual instrument being highly distinctive. In particular, the participants valued modular instrument designs that could facilitate the mixing and rerouting of signals and would allow the instruments to be flexible for use in a variety of ways. Additionally, the concept of “playful unreliability” was popular. In this concept, an instrument might behave in indeterministic or unexpected ways, leading to new sounds and unexpected ways of performing. This brings to mind the description of “interactive composing” given by Joel Chadabe (1984, p. 23), in which the performer “shares control of the music with the information that is automatically generated by the computer, and that information contains unpredictable elements to which the performer reacts while performing.”

In contrast, the discussion for the larger Session B (with seven participants) was short. The session had approached the two-hour mark and there was a sense that the participants were ready to leave. In addition, as so many disparate elements had been identified and voted on, it was difficult to facilitate a conversation around the distinct elements or collective design ideas. This was elucidated in a comment by P07:

The preceding quote elucidates the challenge of moving from the individual and idiosyncratic ideas of the participants to tangible design elements that can drive instrument designs. Our intent was for the sessions to serve as a space to freely generate ideas. As seen in the creativity of the designs, we feel this was successful. The in situ dot-voting and discussion activities provided valuable insights into the participants' prototypes and ideas. Yet these activities by themselves could not provide a systematic interpretation of the workshop outcomes that could be directly applied in the development of functional DMI prototypes. To facilitate this step, we conducted a thematic analysis of the participant presentations based on the methods presented by Braun and Clarke (2006), using an inductive approach similar to grounded theory (cf. Strauss and Corbin 1994).

The analysis was conducted using the qualitative data analysis software Nvivo by QSR International and entailed four steps:

1. Presentations of the ten participants were transcribed from the video recordings;

2. A round of open coding was performed, and a list of preliminary codes was developed;

3. A second round of coding was performed, in which incidents were compared to one another to identify similarities and relationships between them, yielding the final set of codes;

4. Finally, the codes were sorted into themes, which were in turn reviewed and refined, then defined and named.

In all, 152 incidents were coded across the ten presentations, yielding 56 individual codes categorized across eleven themes. The analysis codebook, containing the full list of codes and themes along with their mentions by participant, is included in the Appendix. In comparison to the in-workshop element identification and dot-voting activities, the bottom–up analysis captured more detail and nuance in the presentations and yielded clearer areas of consensus that were not apparent during the workshop, especially with the larger Session B.

To move from open exploration in the workshops to tangible design implementations, we examined the five most common themes (which were mentioned by at least half of the participants) and formulated them as a set of design specifications. In Table 3, we list each theme with a description, an exemplary quote, and the resulting design specification.

Considering the five themes, we make two general observations. First, Themes 1 and 4 offer a middle ground between novel and conventional design elements. In DMI research, both ends of the spectrum are well supported. The desire for novelty is a continuous driving force, illustrated, for example, by the “New Interfaces” in the title of NIME. On the other hand, utilizing existing instrument elements in new designs is also highly valued for many reasons, such as the transferral of learned technique to a new instrument. The workshop results suggest a preference for a balance between the two.

Second, Themes 2, 3, and 5 all reflect a desire for flexibility and versatility across a variety of different aspects: control mapping, signal mixing and routing, and modular designs that can be easily configured and changed by the performer. This trend is recognizable in computer and electronic music performance, highlighting a strength of digital instruments that have the capacity to dynamically change behavior (e.g., mappings or synthesis algorithms) through code.

The balance between novel and conventional design elements and the desire for flexible and versatile instruments support some of the findings from our previous survey on DMI use in performance (Sullivan, Guastavino, and Wanderley 2021). To the first point, survey respondents expressed motivation to experiment with new instruments, sounds, and performance techniques, but they were also highly committed to familiar instruments and interactions that are characterized by ownership, embodied performer--instrument connections, and “muscle memory.” Respondents also highly regarded qualities of flexibility and versatility, which allow for instruments to be customized and combined into elaborate and specific assemblages for a wide range of performance and musical contexts.

From a methodological perspective, the Design for Performance workshops were developed as a strategy to generate creative ideas for new DMIs using strategies drawn from HCI and HCD that prioritize qualitative and situated approaches to design and evaluation. The choice to use design fiction was made for two reasons. First, by removing technological constraints and considerations, the participants were allowed to freely build nonfunctional prototypes with a focus on their musical practice without worrying about the technical feasibility. Second, the design activities situated the participants in a fictional narrative of their own imagining, and the playful aspect urged the participants to be creative and unconventional in their endeavors. For designers, this approach to capturing ideas generated by musicians, especially those who are highly creative and not bound by technical limitations, can help to stave off potential creative paralysis, bringing in fresh ideas and a better understanding of priorities for performance.

Regarding the prospect of Magic Machine workshops to be used by other researchers, Andersen and Wakkary (2019, p. 12) propose that

Our work here aims to apply the unique and imaginative approach of design fiction to collaborate with expert musicians in order to generate creative new ideas and elements for the design of new instruments.

The path we envision from idea generation to the creation of functional instruments is similar to Probatio by Calegario et al. (2017), in which an entire design cycle is formed. With Probatio, this is achieved in a rapid succession in which ideas are generated and directly explored in hardware, allowing for instant testing and evaluation, and for rapid iteration. With our approach, the Design for Performance workshop is intended to be one element of a larger design ecosystem. We envision an iterative design sequence in which multiple workshops can be held to evaluate and refine the resulting instrument designs, similar to the method used by Absar and Guastavino (2015), where a sequence of three workshops iterated on the development of auditory feedback to assist navigation of a visual information system. An iterative process like this could also utilize the Probatio toolkit as a step in the design cycle: Ideas generated from nonfunctional prototyping can be explored in low-fidelity functional models with the Probatio hardware before moving toward the design of high-fidelity prototypes that would be viable for real-world use.

Finally, the comment by P07 about the risk of building “Frankenstein instruments” brings to mind the idea of specificity in design. Each participant created an instrument that was personalized for that participant's own needs and practice, and by combining elements of several different instruments into a single design the essence of any single one may be lost. On one hand, we are motivated to identify common areas of agreement among DMI performers. Our results revealed many design elements that were valued by several of the participants. This suggests the possibility of designing instruments that could be used by different performers across different contexts, possibly improving an instrument's chance for long-term and widespread adoption. On the other hand, P07's comment speaks to the idiosyncrasy that characterizes the field of DMI design, especially where design and performance roles commonly overlap. Although this issue is not covered in depth here, our continued work explores both sides, first through the instruments developed from the workshop design specifications and intended for nonspecific DMI performers, and second through focused collaborations with individual artists and groups, such as our work with P09.

Here we have reported on a novel approach for design workshops to generate creative ideas that can be applied to the development of new DMIs. Our approach is founded in HCI literature, using human-centered and participatory design methods by running workshops with musicians who maintain an active musical practice and by orienting the activities towards the development of new instruments and ideas that those musicians would use in their own practice. By combining a workshop methodology based on design fiction with formal approaches to analysis, iterative design, and evaluation, we aim to narrow a perceived gap in the literature between the generation of theoretical design knowledge and situated, ecologically valid musical instrument design.

We also present the design specifications that came from our workshop as an empirically generated set of considerations toward the design of instruments intended for use in real-world musical practice. Our analysis showed areas of common agreement and preference among the participants that they would want to see in new instruments they would be likely to use in their practice. Overall we found that participants appreciate a balance between novel and conventional design elements and between physical and embodied interaction styles, and they prioritize instruments that are flexible and versatile in their capabilities. In addition to informing our own new designs, these considerations may be useful to other DMI designers as well.

The Design for Performance workshop was initially intended to be followed by instrument design cycles and follow-up sessions, as described in the previous section. The COVID-19 pandemic caused prolonged and intermittent disruptions to in-person participant research, however, necessitating a flexible approach to continuing our work.

Following the conclusion of the workshops, we applied the design specifications to a parallel project to redesign an existing DMI. This resulted in three different prototypes, each intended to embody aspects that emerged from the workshop participants' designs (Sullivan et al. 2020). In lieu of running a follow-up workshop to evaluate the instrument prototypes, we selected one version for further iteration on the design, which yielded a stable instrument that was favorably evaluated and has been extensively used by professional musicians (Boettcher, Sullivan, and Wanderley 2022). There has also been continuing work that supports our original aims for situated and ecologically valid instrument design through longitudinal studies to evaluate DMIs and investigate how musicians integrate them into their performance practice (Reimer 2023; Yanaky et al. Forthcoming).

In closing, the work presented here proposes a creative model for the early-stage design of high-quality, functional, and finished DMIs that will be desirable for real-world use by active musicians. Although our workshop focused on the development of musical instruments, the theories and methods invoked come from the broad interdisciplinary fields of HCI and design. As such, they are appropriate for a wide range of applications within and beyond music and the arts.

We would like to extend our thanks to Collin Wang, who served as the assistant for the workshops. The work presented here was carried out at McGill University and the Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT), where the first author was centered at the time. We thank CIRMMT for the generous funding and facilities to carry out this research. This work was supported by research grants from the Natural Sciences and Engineering Research Council of Canada to the second and third authors.

This appendix presents in Table 4,⁰⁰ the full codebook of the thematic analysis presented in the results. “Cases” indicates the number of participants with incidents attributed to a specific theme or code, whereas “Refs” indicates the total number of incidents (references) coded at each node.