Although extensive research has been conducted over the past two decades on the development and implementation of sensors using inertial movement units for violin bows, most of these new interfaces share a common drawback—bulky and user-unfriendly design. Despite the advancements in sensor data processing for composition, performance, and pedagogy, interface design remains a critical bottleneck for the real-world implementation of these sensor-embedded devices. Our study introduces MetaBow, a low-cost, nonintrusive frog design for violin bows that can accommodate either a standard sensor kit or a custom-designed board. This interface eliminates the need for additional physical training and maintains the integrity of traditional violin performance mechanisms. We thus view MetaBow as heralding a new era in digital music interface design for the violin family with the potential for seamless human–computer and human–machine collaboration in music practice and performance.

MetaBow addresses the often-neglected significance of interface design in determining a device's real-world implementation and its enduring influence on performance and pedagogy, representing an attempt at reimagining instrumental interface design. This article opens with a discussion of our user-guided design methodology, an introductory examination of existing literature, and a detailed analysis of the MetaBow interface, which has now undergone 53 design iterations through five prototype updates in different versions for violin, viola, cello, and double bass (https://vimeo.com/579033792, see also Figure 1). Focusing on the violin model, we explore how MetaBow's design addresses issues found in previous research and outline our strategy for measuring bow and right-hand motion relative to the linked software environment currently under development. In doing so, we place the MetaBow within the broader framework of a design space for musical interface technology, a “conceptual framework for describing, analyzing, designing and extending the interfaces, mappings, synthesis algorithms and performance techniques for advanced musical instruments” (Overholt 2009, p. 220).
Figure 1.

Frog structure (a) and traditional bow frog (b) versus metabow prototype (c). [Note: Color versions of all figures are available in the Supplementary Materials to this article.]

Figure 1.

Frog structure (a) and traditional bow frog (b) versus metabow prototype (c). [Note: Color versions of all figures are available in the Supplementary Materials to this article.]

Close modal

MetaBow's design drew inspiration from current research on the effect of instrumental modifications on learned performative sensorimotor mechanisms and their resultant impact on performance quality (e.g., Morreale, Armitage, and McPherson 2018). This relates both to the notions of embodied music cognition (e.g., Leman 2007) and to the vision of the instrument as a mediation technology that, through the automation of its relationship to the body, becomes an integral part of the performer's physicality. Our design sought to avoid any alteration in the traditional frog structure. Following Krakauer and Shadmehr (2006), we attempted to minimize any disruption to generalized, prelearned, existing gestural frameworks in standard violin practice, avoiding the disruption of feedforward patterns. The MetaBow stems from the vision of the instrument not only as a “mechanical object but as a set of performance practices” (Morreale, Armitage, and McPherson 2018). By taking into account the centrality of the action–sound relationships, MetaBow does not challenge the instrument's recession to the performer's unconscious space, through what has been described as functional transparency (Rabardel 1995). Thus, the interface can be considered as what Paul Dourish (2001) defines, following Heidegger, as a ready-to-hand tool.

Our design is player-oriented. Considering retrospective taxonomies for the evaluation of digital music interface (DMIs) (e.g., Drummond 2009; O'Modhrain 2011), MetaBow exemplifies newer interface design methodologies, as introduced by Morreale, De Angeli, and O'Modhrain (2014) in the MINUET framework (see Figure 2). A preliminary discussion of the people (who), activities (what), and context (where and when) led to the specification of the technological and design frameworks. The MetaBow targets violin students and performers of all levels. It serves as both a pedagogical tool offering live feedback and a performative tool enabling compositional mapping, opening doors to new forms of collaboration (e.g., student–teacher, performer–composer). Users may integrate the interface into their solo practice, performances, group teaching sessions, or ensemble performances. It aims to enhance the performer's learning curve, working as an undisruptive facilitator of traditional learning and performative practices.
Figure 2.

The MINUET framework diagram, a visual representation of the framework as conceptualized by Morreale, De Angeli, and O'Modhrain (2014), which guided the user-centered design process of MetaBow.

Figure 2.

The MINUET framework diagram, a visual representation of the framework as conceptualized by Morreale, De Angeli, and O'Modhrain (2014), which guided the user-centered design process of MetaBow.

Close modal

A theoretical and practical examination of previous bow interfaces ignited our initial research on MetaBow, which was both practice-based (emerging from our performative explorations) and practice-led (as it aimed to generate new knowledge that could transform the analysis and understanding of violin performance). A review of available bow-tracking technologies revealed that most devices were either too costly (Maestre et al. 2007); nonportable (Bevilacqua et al. 2009; Schoonderwaldt, Rasaminana, and Bevilacqua 2006); a complete redesign of the violin bow, requiring relearning of the performing mechanisms (McMillen 2008); or had limited accuracy (Rasamimanana, Fléty, and Bevilacqua 2005). In this study, we have chosen to explore, for the sake of brevity, a selection of representative models designed between 2000 and 2008 that have had significant academic impact (Trueman and Cook 2000; Young 2002a; Guettler, Wilmers, and Johnson 2008; McMillen 2008; Bevilacqua et al. 2006). Table 1 provides a schematic overview of sensors, designs (either as an add-on to an existing bow, a bulky redesign of the frog structure, or an attempt to integrate the circuits into a modified version of a traditional bow frog), battery types, output formats, latency, connectivity protocols, sampling rates, and a list of the modifications that each device has made to the standard frog architecture with regards to overall design and weight, balance point (i.e., the distance from the frog at which the weight on each side of the bow is equal), and overall dimensions. In contrast, MetaBow introduces a simple integrated design that resembles a traditional bow frog. Its inertial measurement unit (IMU) embeds a 3-D axis accelerometer, gyroscope, magnetometer, pressure sensor, and microphone (used to infer bow force data) by using Bluetooth Low Energy (BLE) connectivity. It is powered by a small 100-mAh lithium-ion polymer (LiPo) battery and operates through an Open Sound Control (OSC) protocol.

Table 1.

Previous Violin Bow Interfaces

YearSensorsDesignBatteryOutput FormatLatencyConnectivitySampling Rate [Hz]Modified ElementsReferences
R-Bow (Bowed-sensor speaker array; BoSSA) 2000 Force-sensing resistors; Dual-axis accelerometer; 8-bit microcontroller Add-ons N/A MIDI 10 msec Wired N/A Overall design; Weight (N/A); Balance point (N/A); Usability Trueman and Cook 2000  
Hyperbow 2002 Resistive strip; Foil strain gauges; Violin-mounted antenna; Circuit board with dual-axis accelerometer Add-ons & redesign Lithium MIDI N/A Electrode antenna Position: 142.86; Acceleration and strain subsystems: 41.67 Overall design; Weight (60.93–89.75 g); Balance point (25.5–22); Dimension Young 2002a, 2002b  
IRCAM Bow 2006 3-D accelerometer; Gyroscope; Extras (e.g., optical pickup) Add-ons Lithium OSC N/A Subminiature radio transmitter 200 (at ten-bit resolution) Overall design; Weight (+17 g); Balance point (N/A); Dimension Rasamimanana, Fléty, and Bevilacqua 2005; Bevilacqua et al. 2006; Leroy, Fléty, and Bevilacqua 2006; Schoonderwaldt, Rasamimanana, and Bevilacqua 2006  
NOTAM Bow 2008 ADXL330 3-D accelerometer; Pressure sensor; IDG300 Gyroscope; Microphone signal to derive bow-force data; C8051F530 microcontroller Add-ons 1.5 V AAA CREATE USB Interface N/A Bluetooth 3200 (but data are averaged every 16 samples, for an effective sampling rate of 200) Overall design; Weight (+25 g); Balance point (N/A); Dimension Guettler, Wilmers, and Johnson 2008  
K-Bow 2008 Accelerometer; Grip pressure sensor; Antenna; Hair-pressure sensor; Bow-length sensor; Gyroscope Integrated redesign Lithium polymer (Li-Po) MIDI N/A Bluetooth Accelerometer: 606.06 Frog design McMillen 2008  
YearSensorsDesignBatteryOutput FormatLatencyConnectivitySampling Rate [Hz]Modified ElementsReferences
R-Bow (Bowed-sensor speaker array; BoSSA) 2000 Force-sensing resistors; Dual-axis accelerometer; 8-bit microcontroller Add-ons N/A MIDI 10 msec Wired N/A Overall design; Weight (N/A); Balance point (N/A); Usability Trueman and Cook 2000  
Hyperbow 2002 Resistive strip; Foil strain gauges; Violin-mounted antenna; Circuit board with dual-axis accelerometer Add-ons & redesign Lithium MIDI N/A Electrode antenna Position: 142.86; Acceleration and strain subsystems: 41.67 Overall design; Weight (60.93–89.75 g); Balance point (25.5–22); Dimension Young 2002a, 2002b  
IRCAM Bow 2006 3-D accelerometer; Gyroscope; Extras (e.g., optical pickup) Add-ons Lithium OSC N/A Subminiature radio transmitter 200 (at ten-bit resolution) Overall design; Weight (+17 g); Balance point (N/A); Dimension Rasamimanana, Fléty, and Bevilacqua 2005; Bevilacqua et al. 2006; Leroy, Fléty, and Bevilacqua 2006; Schoonderwaldt, Rasamimanana, and Bevilacqua 2006  
NOTAM Bow 2008 ADXL330 3-D accelerometer; Pressure sensor; IDG300 Gyroscope; Microphone signal to derive bow-force data; C8051F530 microcontroller Add-ons 1.5 V AAA CREATE USB Interface N/A Bluetooth 3200 (but data are averaged every 16 samples, for an effective sampling rate of 200) Overall design; Weight (+25 g); Balance point (N/A); Dimension Guettler, Wilmers, and Johnson 2008  
K-Bow 2008 Accelerometer; Grip pressure sensor; Antenna; Hair-pressure sensor; Bow-length sensor; Gyroscope Integrated redesign Lithium polymer (Li-Po) MIDI N/A Bluetooth Accelerometer: 606.06 Frog design McMillen 2008  

MetaBow

MetaBow seeks to address the design shortcomings identified in the aforementioned interfaces. Moreover, we propose that our interface, built upon the framework of border ecosystemic factors as defined by Marquez-Borbon and Martinez Avila (2018), presents a solution to the durability problems discussed in previous studies. It is important to acknowledge that the traditional design of the violin bow has remained unchanged since the late 18th to early 19th centuries (Stowell 1999, pp. 24–26). This historical adherence to conventional bow and violin models may explain the resistance to previous modifications within the established global community of classically trained professionals, including teaching institutions, orchestras, soloists, and chamber musicians, who have been hesitant to embrace design modifications. The MetaBow, although aligning with the technological paradigms of the hyperbow (Young 2002b), hyperinstrument, and augmented violin or bow (McMillen 2008), maintains the appearance of a traditional bow. Its shape remains unchanged, and it upholds identical weight balance, responsiveness, and tactility (a formal experiment that evaluates these somewhat subjective elements will be detailed in the Preliminary User Testing section). By incorporating the technology within a seemingly standard design, it epitomizes a nonintrusive form of transparent human–machine collaboration. As a result, MetaBow does not disrupt traditional violin performance or require any form of physical retraining. It can be seamlessly integrated into the practice and performance routines of any player, teacher, or student, demonstrating its potential to transform linked pedagogical and performative practices. By augmenting the technical capabilities of the instrument in a noninvasive manner, we contend that MetaBow can enhance our capacity to track and record gestural data from traditional violin performances, making a substantial contribution to the fields of musical pedagogy and embodied music interaction. In doing so, it bridges the divide between academic research and real-world implementation.

Compared with current alternatives—such as motion capture, machine-learning analysis, and video tracking—we argue that the MetaBow's implementation, given its feasibility as a low-cost and user-friendly device, would have the greatest short- and midterm impact on the development of string pedagogy, performance, and composition. Our initial research explored the implementation of BITalino's R-IoT sensor kit (see https://ismm.ircam.fr/riot). The MetaBow 1.1 prototype incorporated the R-IoT board and a 3.7-V, 240-mAh LiPo battery into a custom-designed hollowed 3-D printed bow frog. The R-IoT is the seventh generation of Institut de Recherche et Coordination Acoustique/Musique's (IRCAM's) wireless-sensor digitizing units and embeds a 9-axis digital IMU sensor (LSM9DS1) featuring a 3-axis accelerometer, 3-axis gyroscope, and 3-axis magnetometer. The sensor is attached to the Serial Peripheral Interface port to sample the 16-bit motion data at a high speed. In addition, there are two 12-bit ADC inputs that are compatible with BITalino sensor modules. The R-IoT operates via Wi-Fi connectivity with an OSC communication protocol. We eventually discarded BITalino in view of its large size and weight and because of the complexity of the Wi-Fi connection mechanism for a nonspecialist.

MetaBow's 1.2 prototype integrated a combination of two SensorTile units developed by STMicroelectronics: a STEVAL-STLCS02V1 soldered to a modified version of the basic cradle included in the STEVAL-STLKT01V1 kit (see https://www.st.com/en/evaluation-tools/steval-stlkt01v1.html). The cradle houses a mini-USB battery charger and port, humidity and temperature sensors, and an SD memory card slot and breakaway SWD connector that were removed from the unit. The basic sensor is a small, square-shaped Internet-of-Things (IoT) module that shares features with the BITalino but embeds an 80-MHz STM32L476JGY microcontroller, BLE connectivity, and a nano digital microphone. We utilized an OSC communication protocol for data sharing, and the BLE simplified the connectivity and significantly reduced battery size and consumption (see Table 2 for details).

Table 2.

MetaBow

MetaBow 1.1MetaBow 1.2
Sensors BITalino R-IoT Sensor; STEVAL-STLCS02V1; 
 3-axis accelerometer; 3-axis accelerometer; 
 3-axis gyroscope; 3-axis gyroscope; 
 3-axis magnetometer; 3-axis magnetometer; 
 etc. MP34DT04 digital microphone; 
  etc. 
Data calibration Includes data fusion for quaternion and Euler angle computing (Madgwick 2010Includes data fusion for quaternion and Euler angle computing (Madgwick 2010
Design Integrated, transparent HMC Integrated, transparent HMC 
Battery 700 mAh LiPo 100 mAh LiPo 
 2–5 V 3.7 V 
Output format OSC OSC 
Connectivity 2.4-GHz Wi-Fi BLE 
Latency 1–10 msec N/A 
Sampling rate 200 Hz with a resolution of 16 bits per IMU channel N/A 
Modified elements Weight Weight 
 Balance point Balance point 
 Dimension Dimension 
MetaBow 1.1MetaBow 1.2
Sensors BITalino R-IoT Sensor; STEVAL-STLCS02V1; 
 3-axis accelerometer; 3-axis accelerometer; 
 3-axis gyroscope; 3-axis gyroscope; 
 3-axis magnetometer; 3-axis magnetometer; 
 etc. MP34DT04 digital microphone; 
  etc. 
Data calibration Includes data fusion for quaternion and Euler angle computing (Madgwick 2010Includes data fusion for quaternion and Euler angle computing (Madgwick 2010
Design Integrated, transparent HMC Integrated, transparent HMC 
Battery 700 mAh LiPo 100 mAh LiPo 
 2–5 V 3.7 V 
Output format OSC OSC 
Connectivity 2.4-GHz Wi-Fi BLE 
Latency 1–10 msec N/A 
Sampling rate 200 Hz with a resolution of 16 bits per IMU channel N/A 
Modified elements Weight Weight 
 Balance point Balance point 
 Dimension Dimension 

BLE = Bluetooth Low Energy; IMU = inertial measurement unit; OSC = Open Sound Control.

Although the unit size improved with the ST models, the resulting MetaBow 1.2 interface remained somewhat larger than a traditional wooden frog. For the MetaBow 1.3 prototype, we designed a smaller sensor board, the MetaBoard (https://www.kickstarter.com/projects/metaboard/metaboard?ref=user_menu), which contains all the essential components, measures 14 × 27.5 mm, and can be embedded into a standard-size frog housing design (see Figure 3). The board incorporates a three-axis accelerometer, gyroscope, and magnetometer; a nano microphone; a microcontroller; a port for a force-sensing resistor; and a mini USB port used to charge the battery and for debugging and software-related purposes. The board has a four-layered structure with a minimum hole size of 0.15 mm.
Figure 3.

The MetaBoard design. This is a detailed view of the board, an integral technology embedded in the MetaBow, demonstrating its compact, innovative design, which is tailored for string instruments.

Figure 3.

The MetaBoard design. This is a detailed view of the board, an integral technology embedded in the MetaBow, demonstrating its compact, innovative design, which is tailored for string instruments.

Close modal
Although the MetaBoard played a crucial role in the development of the MetaBow, its key innovation had more to do with the housing design than with the embedded technology. All iterations of MetaBow's housing unit (57 iterations in total) were first developed as 3-D models under the guidance of an experienced bow maker, then printed and collaboratively tested. For the MetaBow 1.1 prototype, we needed to create a hollow frog that could house an R-IoT sensor kit. We had initially developed a rough model (MetaBow 1.0) with the correct upper dimensions to be attached to any violin bow and then focused on the design of the inner housing (see Figure 4). The next phase involved a detailed revision aimed at reducing the total volume of the units. The final MetaBow 1.1 R-IoT housing unit's print combined an SLS polyamide, black-dye, matte-polished print of the case, with a minimum wall thickness of 0.8 mm to guarantee durability, and a yellow gold-plated brass print of the lid (see Figure 5, bottom component). The substantial weight and size of the R-IoT led us to redesign the housing, abandoning the implementation of the R-IoT in favor of the ST Microelectronics's smaller SensorTile IMU.
Figure 4.

MetaBow 1.0, initial version of the 3-D design, illustrating the initial design concept and structural dimensions for the violin bow interface.

Figure 4.

MetaBow 1.0, initial version of the 3-D design, illustrating the initial design concept and structural dimensions for the violin bow interface.

Close modal
Figure 5.

MetaBow 1.1, 3-D design of the R-IoT version. This was a refined design of MetaBow 1.1, integrating the R-IoT sensor kit, which enhanced data capture capabilities for performance analytics.

Figure 5.

MetaBow 1.1, 3-D design of the R-IoT version. This was a refined design of MetaBow 1.1, integrating the R-IoT sensor kit, which enhanced data capture capabilities for performance analytics.

Close modal
Shifting from the R-IoT to the SensorTile allowed us to substantially reduce the MetaBow's size and weight (see Figure 6 for a visual comparison and Table 2 for measurements). The MetaBow 1.2 prototype housed the power button and mini-USB charging and debugging port on the rear part of the lid, and we left a hollow space on the side to improve sound capture through the nano microphone. The top part of the lid included the hole and additional support material for the insertion of the bow's eyelet, which can be easily replaced without damaging the interface. Finally, the areas for the insertion of the mother-of-pearl slide, ferrule, and horsehair mortise were designed for a standard bow rehair. We included an optional, specially designed widget to replace the wooden insertion in the mortise as an alternative to the traditional approach for securing horsehair. Figure 7 shows the 3-D rendering of MetaBow 1.2, and a comparison of the 1.2 and previous prototypes was given in Table 2.
Figure 6.

MetaBow prototype comparison: MetaBow 1.0 (a), MetaBow 1.1 (b), and MetaBow 1.2 (c), highlighting the design evolution.

Figure 6.

MetaBow prototype comparison: MetaBow 1.0 (a), MetaBow 1.1 (b), and MetaBow 1.2 (c), highlighting the design evolution.

Close modal
Figure 7.

MetaBow 1.2, 3-D design of the ST version. This design features ST Microelectronics's technology, showcasing a streamlined and efficient integration of advanced sensors.

Figure 7.

MetaBow 1.2, 3-D design of the ST version. This design features ST Microelectronics's technology, showcasing a streamlined and efficient integration of advanced sensors.

Close modal
Our current research focuses on the improvement of several elements of MetaBow's 1.2 (ST version) and 1.3 (MetaBoard modification) prototypes for a new 1.4 prototype, working closely with players and bowmakers to optimize the housing design (see Figure 8). We addressed weight issues by redesigning the lid, removing unnecessary metallic content to make the unit as light as possible, inserting anchoring points to guarantee robustness, and revising the horsehair rehairing channel (see Table 3). We are also advancing the software environment for MetaBow, positioning it as both a performative and pedagogical interface. But before delving into this critical aspect of our current research, let us discuss our strategy concerning the intertwined challenges of bow motion measurement and data processing.
Figure 8.

MetaBow 1.4 prototype. This is the latest prototype of MetaBow, version 1.4, displaying significant design improvements for enhanced performance and ease of use in string instrument play.

Figure 8.

MetaBow 1.4 prototype. This is the latest prototype of MetaBow, version 1.4, displaying significant design improvements for enhanced performance and ease of use in string instrument play.

Close modal
Table 3.

Weight Comparison of MetaBow Prototypes

Standard
Frog WeightMetaBow 1.2
9–11 g Housing Lid Circuit and Battery 
 3.5 g 9 g 5 g 
 17–19 g 
  
 MetaBow 1.3 
 Housing Lid Circuit and Battery 
 3.5 g 7.5 g 3 g 
 14 g 
Standard
Frog WeightMetaBow 1.2
9–11 g Housing Lid Circuit and Battery 
 3.5 g 9 g 5 g 
 17–19 g 
  
 MetaBow 1.3 
 Housing Lid Circuit and Battery 
 3.5 g 7.5 g 3 g 
 14 g 

Our review of existing research led us to reconsider the analytically relevant aspects of the violin's bow and right-hand movements that had to be inferred from MetaBow's embedded technology. The elements explored in previous interfaces (e.g., Askenfelt 1986, 1989) include: (1) transversal velocity (i.e., bow speed, using an accelerometer to measure the speed at which the bow moves across the strings); (2) transverse position (i.e., bow tilt, using a gyroscope to measure the bow's deviation from a vertical line to ground level); (3) bow location (using external antennas, resistance wires inserted into the horsehair or an electromagnetic field sensing system such as Polhemus); (4) bow force (the amount of force being applied to the bow at any given time, measured by a sensor located under the horsehair or by multiple strain gauges, or inferred from audio capture); (5) bow inclination (i.e., the bow's angle relative to the ground level); (6) bow–bridge distance (the distance between bow and violin bridge); and (7) finger pressure (measured with resistive strips placed along the bow-holding areas—the amount of force exerted by fingers when holding the bow). Although MetaBow's design seeks to avoid the disruption to traditional performing techniques, its sensors aim to generate a range of data similar to that of previous interfaces.

Bow Force and Finger Pressure

We infer the bow force from the sound captured through the microphone following the model used by Guettler, Wilmers, and Johnson (2008). Previous interfaces obtained this data using strain gauges mounted along the horsehair (Askenfelt 1989), a bracket insertion into the D-shaped ferrule (Demoucron, Askenfelt, and Caussé 2006), strain gauges located at different deflecting bow positions (Young and Deshmane 2007), or the inference of bow force from the pressure of the right-hand index finger (Paradiso and Gershenfeld 1997). We used instead the spectral envelope as a function of bow force and bow speed, independent of the contact point (see Guettler, Schoonderwaldt, and Askenfelt 2003). A fast algorithm that determines the approximate energy ratio between band-passed (5.0–7.5 kHz) and low-passed frequencies (2.5 kHz) provides an overall sense of bow-force variations.

To measure finger pressure, we inserted a modified version of the Interlink 402 round force-sensitive resistor (FSR) into the finger placement area of the frog's inner structure, under the lateral mother-of-pearl eye. This FSR sends finger pressure data through the board, allowing the system to infer finger force and pressure.

Calibration

Whereas we have inferred bow speed and tilting from the current OSC data stream, including gyroscope, accelerometer, and magnetometer values as floating-point numbers, we have had to propose alternative approaches to the acquisition and visualization of the remaining gestural elements. We are working on an initial calibration mechanism, which remains a work-in-progress, that does not rely on an electromagnetic motion capture system (Maestre et al. 2007) or the mounting of near-field optical sensors (Pardue, Harte, and McPherson 2015). The calibration necessary for the visualization of pedagogically relevant data requires the installation of a second adhesive MetaBoard unit under the instrument's tailpiece. Such a process would allow us to gather and visualize data related to bow location, inclination, and bow-bridge distance by using a detailed, 3-D framework.

Data Acquisition and Processing

At present, MetaBow's toolkit exports metadata—including timestamp, audio reference, score reference, and all MetaBoard sensor data—into an external tool that logs it to a cloud-based database. Although the MetaBow favors quantitative biomechanical movement analysis, we plan to explore the use of machine learning as a way to improve gesture-tracking accuracy, developing a qualitative analytical approach that combines elements from Laban movement analysis (Laban and Lawrence 1947) and Bartenieff fundamentals (Hackney 2000). We aim to introduce a new approach for the acquisition of instrumental gesture parameters in a live setting. The future development of the MetaBow interface and its associated software is expected to lead to the creation of an annotated database similar to the proposed Gesture Descriptor Interchange Format (GDIF), combining audio and parametric metadata. The GDIF was initially an OSC address space designed to “store all sorts of data from various commercial and custom-made controllers, motion capture, and computer-vision systems, as well as results from different types of gesture analysis, in a coherent and consistent way” (Jensenius, Kvifte, and Godøy 2006, p. 176), allowing for cross-platform and institutional data sharing.

Audio-to-Score Alignment and Score Following

The MetaBow includes an MP34DT05 ultracompact, low-power, omnidirectional, digital, microelectromechanical-system (MEMS) microphone built with a capacitive sensing element, and an IC interface. It uses a 64-dB signal-to-noise ratio and sensitivity of –26 dBFS ±3 dB with an acoustic overload point (AOP) of 122.5 dBSPL, and a pulse-density modulation (PDM) output. The microphone enables a simple implementation of an audio-to-score alignment module, providing real-time alignment between sounds and a given score, into the performative and pedagogical Max and Touch Designer patches that we are currently developing.

Over the past two decades, the vast amount of research on this topic (see Orio, Lemouton, and Schwarz 2003) has led to the creation of systems capable of detecting minor errors, skips, and mistakes (see Nakamura et al. 2014). Most of these combine the exploration of hidden Markov models and MIDI or musicXML scores. One of the most salient projects in the field, IRCAM's score follower, became the open-access Max patch Antescofo, a modular, polyphonic app-like score-following system (see https://www.antescofo.com). Further research in this area has been conducted as part of the TELMI project through the ViolinRT real-time play-along prototype (http://telmi.upf.edu) and the i-Maestro project (Ng and Nesi 2008).

Sound variability is a critical issue in analyzing a real performance setting, a complex dimension that relates to elements such as acoustic variations (e.g., spectral variations, background noise), temporal fluctuations, performance errors, and, during practice sessions, the repetition or skipping of specific sections. On this basis, we have implemented a modified and updated version of IRCAM's Antescofo into our Max MetaBow Toolkit as a starting point to explore future modifications that may improve the mapping and data visualization modules that will be part of the interface's software environment at a pedagogical and performative level.

IoT Hubbing

A further significant element of our current work is the implementation of an IoT hub-based system for distributed data logging, which is currently under development. We have considered some of the key providers of IoT cloud solutions (Microsoft's Azure IoT Hub, Amazon's AWS, and Google's Cloud IoT). Our midterm aim for MetaBow's commercialization is to use IoT hubbing both as a means to explore different approaches to networked performance and, more importantly, to generate a global database of right-hand objective gestural data. Such a database would allow us to gain new insights, both conditioned or unconditioned (organized by level, age-range, location, etc.), into the current gestural practices of violinists at the worldwide level by using an approach based on machine learning.

We are currently developing a software environment at the aforementioned two key levels: performance and pedagogy. We have developed a Max package, the MetaBow Toolkit, which, although based on existing research (e.g., Bevilacqua, Müller, and Schnell 2005; Schnell and Schwarz 2005; Fasciani and Wyse 2012), provides a clean and intuitive graphical user interface appropriate not only for experienced Max experts but also for novices (see Figure 9). The MetaBow Toolkit comprises a scalable, modular approach to continuous and selective multimodal gestural mapping and an initial calibration routine. We are also working on different approaches to data visualization aimed at using MetaBow as a tool for live pedagogical feedback that may be easily integrated into standard practice and teaching routines. Our present research explores both Max (including the MSP and Jitter components) and the use of TouchDesigner as core software frameworks.
Figure 9.

MetaBow Max Toolkit's main page. This is the main interface of the Toolkit, illustrating its user-friendly design and comprehensive functionality for both pedagogical and performative applications.

Figure 9.

MetaBow Max Toolkit's main page. This is the main interface of the Toolkit, illustrating its user-friendly design and comprehensive functionality for both pedagogical and performative applications.

Close modal

In September 2022, we conducted a preliminary session of user testing in Madrid with a professional string quartet. We invited the quartet to rehearse and perform Dmitri Shostakovich's String Quartet No. 3 using MetaBows and a customized, reactive, gesture-based, visual mapping model developed in TouchDesigner. The session aimed to evaluate the device's responsiveness, ease of use, and potential usefulness from the performers’ viewpoint, rather than assess the aesthetic quality of the visual framework specifically created for the event. Following the rehearsal, we presented the quartet with an anonymous questionnaire, adopting a modified version of the expanded technology acceptance model (TAM2, see Venkatesh and Davis 2000) with a five-point Likert scale (see Appendix).

Fred Davis (1989) outlined a three-stage process in TAM for technology acceptance, starting from external factors, moving through cognitive and affective responses, and leading to actual use behavior. It is based on elements such as perceived ease of use, perceived usefulness, and behavioral intention, with the first two directly impacting use behavior. In TAM2, an extension to TAM, additional elements are introduced, such as subjective norm (perceived social pressure influencing technology use), image (enhanced status from technology use), job relevance (technology's applicability to a specific job), output quality (i.e., the quality of technology's results), result demonstrability (tangible benefits of using the technology), experience (familiarity with the technology), and “voluntariness” (freedom in choosing to use the technology). Each plays a significant role in influencing technology acceptance and perceptions of usefulness. For instance, job relevance indicates how applicable technology is to one's job, directly affecting perceived usefulness and moderated by output quality.

In this particular study, elements such as subjective norm (Question 8), job relevance (Questions 7 and 12), result demonstrability (Question 15), and experience (Questions 5 and 6) have been specifically included to evaluate technology acceptance, helping in comprehensively understanding the factors influencing judgments about technology usefulness. Table 4 introduces an average of the results across the quartet members, followed by the average per area of evaluation using the proposed five-point Likert scale.

Table 4.

MetaBow User Testing: Mean and Consolidated Results

Perceived Ease of UsePerceived Usefulness
Weight Comparison 4.5 4.2 13 Usefulness 4.0 3.9 
Balance Point Impact 4.0 14 Market Need 3.75 
Responsiveness Comparison 3.75 15 Enjoyability 4.0 
Tactile Experience 4.25 Confirmation 
Disruption Level 4.0 16 Experience 4.5 4.1 
Ease of use 4.75 17 Expectations 3.75 
Behavioral Intention Free Comment Section 
Market Viability 3.75 3.8 It is an extremely comfortable, light, and easy-to-handle device.
Thanks to its revolutionary motion detection technology, we obtain greater creative ease as well as data from other musicians when using it.
I find it a very interesting device and recommendable for all levels, from beginner students to professionals. Additionally, it is a low-cost technology, designed to facilitate its purchase and make it accessible to all budgets.
MetaBow focuses on the future, introducing technology that eases teaching and learning. 
Purchase Likelihood 3.75 
Pedagogical Potential 4.25 
10 Performance Job Relevance 3.5 
11 Teaching Job Relevance 3.75 
12 Result Demonstrability 4.0 
Perceived Ease of UsePerceived Usefulness
Weight Comparison 4.5 4.2 13 Usefulness 4.0 3.9 
Balance Point Impact 4.0 14 Market Need 3.75 
Responsiveness Comparison 3.75 15 Enjoyability 4.0 
Tactile Experience 4.25 Confirmation 
Disruption Level 4.0 16 Experience 4.5 4.1 
Ease of use 4.75 17 Expectations 3.75 
Behavioral Intention Free Comment Section 
Market Viability 3.75 3.8 It is an extremely comfortable, light, and easy-to-handle device.
Thanks to its revolutionary motion detection technology, we obtain greater creative ease as well as data from other musicians when using it.
I find it a very interesting device and recommendable for all levels, from beginner students to professionals. Additionally, it is a low-cost technology, designed to facilitate its purchase and make it accessible to all budgets.
MetaBow focuses on the future, introducing technology that eases teaching and learning. 
Purchase Likelihood 3.75 
Pedagogical Potential 4.25 
10 Performance Job Relevance 3.5 
11 Teaching Job Relevance 3.75 
12 Result Demonstrability 4.0 

Overall, the performers praised the MetaBow for its ease of use and broad utility and described the device as comfortable, lightweight, and user friendly. Its motion-detection technology was highlighted as innovative, enhancing both creativity and insights among musicians. The MetaBow, as underscored by its high rating in pedagogical potential and its relevance to teaching, seems particularly advantageous for educational settings. Although there were mild reservations concerning its fit for professional performances and overall market necessity, the feedback largely tilts towards optimism, indicating that, with minor refinements, the device could effect a significant shift in its domain.

This article outlined our innovative approach to designing a frog interface for violin bows aimed at overcoming the usability constraints observed in earlier, bulkier, and less-ergonomic models. Beyond the violin prototype, MetaBow has expanded the range to include bows for viola, cello, and double bass. While recognizing that its design is a minor part of MetaBow's ongoing development, we assert that it is a crucial foundation for forthcoming related research. Metabow's success depends on our capacity to establish a broad community of practice through an interface ecology that intertwines its performative and pedagogical dimensions and thus ensures its seamless use as both a learning and playing tool. The short- to midterm commercial launch of this product will serve as a practical test for this community framework, sparking further research into its applications and potential to revolutionarily influence string pedagogy and performance.

This article introduces the preliminary research that led to the filing of an international Patent Cooperation Treaty application (MBow Ltd 2023). The team involved in the preliminary research presented here included Roberto Alonso Trillo (Hong Kong Baptist University, general interface conceptualization and design), Peter Nelson (Hong Kong Baptist University, general interface conceptualization and design), Tychonas Michailidis (Birmingham City University, interface design consultancy), James Bradbury (University of Huddersfield, Max development), Paulo Chiliguano (Max development), and Alexis Mailles (software development).

Askenfelt
,
A.
1986
. “
Measurement of Bow Motion and Bow Force in Violin Playing
.”
Journal of the Acoustical Society of America
80
(
4
):
1007
1015
. doi:.
Askenfelt
,
A.
1989
. “
Measurement of the Bowing Parameters in Violin Playing II: Bow–Bridge Distance, Dynamic Range, and Limits of Bow Force
.”
Journal of the Acoustical Society of America
86
(
2
):
503
516
. doi:.
Bevilacqua
,
F.
,
R.
Müller
, and
N.
Schnell
.
2005
. “
MnM: A Max/MSP Mapping Toolbox
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
85
88
.
Bevilacqua
,
F.
,
B.
Zamborlin
,
A.
Sypniewski
,
N.
Schnell
,
F.
Guédy
, and
N.
Rasamimanana
.
2009
. “
Continuous Realtime Gesture Following and Recognition
.” In
Proceedings of the International Gesture Workshop
, pp.
73
84
.
Bevilacqua
,
F.
, et al.
2006
. “
The Augmented Violin Project: Research, Composition and Performance Report
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
402
406
.
Davis
,
F. D.
1989
. “
Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology
.”
MIS Quarterly
13
(
3
):
319
340
. doi:.
Demoucron
,
M.
,
A.
Askenfelt
, and
R.
Caussé
.
2006
. “
Mesure de la ‘pression d'archet’ des instruments à cordes frottées: Application à la synthèse sonore
.” In
Actes du Congrès Français d'Acoustique
, pp.
475
478
.
Dourish
,
P.
2001
.
Where the Action Is: The Foundations of Embodied Interaction
.
Cambridge, Massachusetts
:
MIT Press
.
Drummond
,
J.
2009
. “
Understanding Interactive Systems
.”
Organised Sound
14
(
2
):
124
133
. doi:.
Fasciani
,
S.
, and
L.
Wyse
.
2012
. “
A Voice Interface for Sound Generators: Adaptive and Automatic Mapping of Gestures to Sound
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
,
Paper 57
. doi:.
Guettler
,
K.
,
E.
Schoonderwaldt
, and
A.
Askenfelt
.
2003
. “
Bow Speed or Bowing Position: Which One Influences the Spectrum the Most?
” In
Proceedings of the Stockholm Music Acoustics Conference
, pp.
67
70
.
Guettler
,
K.
,
H.
Wilmers
, and
V.
Johnson
.
2008
. “
VictoriaCounts: A Case Study with Electronic Violin Bow
.” In
Proceedings of the International Computer Music Conference
, pp.
569
662
.
Hackney
,
P.
2000
.
Making Connections: Total Body Integration Through Bartenieff Fundamentals
.
New York
:
Routledge
.
Jensenius
,
A. R.
,
T.
Kvifte
, and
R. I.
Godøy
.
2006
. “
Towards a Gesture Description Interchange Format
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
176
179
.
Krakauer
,
J. W.
, and
R.
Shadmehr
.
2006
. “
Consolidation of Motor Memory
.”
Trends in Neurosciences
29
(
1
):
58
64
. doi:.
Laban
,
R.
, and
F.
Lawrence
.
1947
.
Effort
.
London
:
Macdonald and Evans
.
Leman
,
M.
2007
.
Embodied Music Cognition and Mediation Technology
.
Cambridge, Massachusetts
:
MIT Press
.
Madgwick
,
S.
2010
. “
An Efficient Orientation Filter for Inertial and Inertial/Magnetic Sensor Arrays
.”
Available online at
x-io.co.uk/downloads/madgwick_internal_report.pdf.
Accessed May 2024
.
Maestre
,
E.
, et al.
2007
. “
Acquisition of Violin Instrumental Gestures Using a Commercial EMF Tracking Device
.” In
Proceedings of the International Computer Music Conference
, pp.
171
176
.
Marquez-Borbon
,
A.
, and
J.
Martinez Avila
.
2018
. “
The Problem of DMI Adoption and Longevity: Envisioning a NIME Performance Pedagogy
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
190
195
.
MBow Ltd
.
2023
.
A System for Interacting with a String Instrument, an Attachment Device, and a Communication Method
.
WIPO, PCT Patent Application No. PCT/CN2023/114502, filed 23 Aug 2023 (pending)
.
McMillen
,
K. A.
2008
. “
Stage-Worthy Sensor Bows for Stringed Instruments
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
347
348
.
Morreale
,
F.
,
A.
De Angeli
, and
S.
O'Modhrain
.
2014
. “
Musical Interface Design: An Experience-Oriented Framework
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
467
472
.
Morreale
,
F.
,
J.
Armitage
, and
A.
McPherson
.
2018
. “
Effect of Instrument Structure Alterations on Violin Performance
.”
Frontiers in Psychology
9
:
Art.
2436
. doi:.
Nakamura
,
E.
, et al.
2014
. “
Merged-Output Hidden Markov Model for Score Following of MIDI Performance with Ornaments, Desynchronized Voices, Repeats and Skips
.” In
Proceedings of the Joint International Computer Music Conference and the Sound and Music Computing Conference
, pp.
1185
1192
.
Ng
,
K.
, and
P.
Nesi
.
2008
. “
i-Maestro: Technology-Enhanced Learning and Teaching for Music
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
225
228
.
O'Modhrain
,
S.
2011
. “
A Framework for the Evaluation of Digital Musical Instruments
.”
Computer Music Journal
35
(
1
):
28
42
. doi:.
Orio
,
N.
,
S.
Lemouton
, and
D.
Schwarz
.
2003
. “
Score Following: State of the Art and New Developments
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
36
41
.
Overholt
,
D.
2009
. “
The Musical Interface Technology Design Space
.”
Organised Sound
14
(
2
):
217
226
. doi:.
Paradiso
,
J. A.
, and
N.
Gershenfeld
.
1997
. “
Musical Applications of Electric Field Sensing
.”
Computer Music Journal
21
(
2
):
69
89
. doi:.
Pardue
,
L. S.
,
C.
Harte
, and
A. P.
McPherson
.
2015
. “
A Low-Cost Real-Time Tracking System for Violin
.”
Journal of New Music Research
44
(
4
):
305
323
. doi:.
Rabardel
,
P.
1995
.
Les Hommes et les technologies; Approche cognitive des instruments contemporains
.
Paris
:
Armand Colin
.
Rasamimanana
,
N.
,
E.
Fléty
, and
F.
Bevilacqua
.
2005
. “
Gesture Analysis of Violin Bow Strokes
.” In
Gesture in Human–Computer Interaction and Simulation, 6th International Gesture Workshop
, pp.
145
155
.
Schnell
,
N.
, and
D.
Schwarz
.
2005
. “
Gabor, Multi-Representation Real-Time Analysis/Synthesis
.” In
Proceedings of the International Conference on Digital Audio Effects
, pp.
122
126
.
Schoonderwaldt
,
E.
,
N.
Rasamimanana
, and
F.
Bevilacqua
.
2006
. “
Combining Accelerometer and Video Camera: Reconstruction of Bow Velocity Profiles
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
200
203
.
Stowell
,
R.
1999
.
The Cambridge Companion to the Violin
.
Cambridge, UK
:
Cambridge University Press
.
Trueman
,
D.
, and
P. R.
Cook
.
2000
. “
Bossa: The Deconstructed Violin Reconstructed
.”
Journal of New Music Research
29
(
2
):
121
130
.
Venkatesh
,
V.
, and
F. D.
Davis
.
2000
. “
A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies
.”
Management Science
46
(
2
):
186
204
. doi:.
Young
,
D.
2002a
. “
The Hyperbow: A Precision Violin Interface
.” In
Proceedings of the International Computer Music Conference
, pp.
489
492
.
Young
,
D.
2002b
. “
The Hyperbow Controller: Real-Time Dynamics Measurement of Violin Performance
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
65
70
.
Young
,
D.
, and
A.
Deshmane
.
2007
. “
Bowstroke Database: A Web-Accessible Archive of Violin Bowing Data
.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
352
357
.

We adopt a modified version of the expanded technology acceptance model (TAM2) used to evaluate the acceptance of information systems. Our model includes a selection of the elements discussed in the Preliminary User Testing section: subjective norm (Question 8), job relevance (Questions 7 and 12), result demonstrability (Question 15), and experience (Questions 5 and 6).

Perceived Ease of Use

  1. Design Evaluation: Please provide your overall evaluation of the interface's design.

  2. Weight Comparison: How similar is the weight of the MetaBow to that of your traditional violin bow?

  3. Balance Point Impact: To what extent does the interface affect the weight balance of the bow?

  4. Responsiveness Comparison: How does the interface's response compare to that of your traditional violin bow?

  5. Tactile Experience: How closely does the interface replicate the tactile feel of a traditional violin bowing?

  6. Disruption Level: To what extent does the interface disrupt your standard practice and performance routines?

  7. Ease of use: How easy to use was the interface?

Perceived Usefulness

  1. Usefulness: How useful would it be to you as a performer to track and be able to consult after a practice session or performance elements such as bow speed, inclination, bow trajectory, bridge-to-tasto distance, right-hand finger pressure?

  2. Market Need: Do you believe there is a significant market need for an interface such as MetaBow?

  3. Enjoyability: To what extent was using the interface an enjoyable experience?

Confirmation

  1. Experience: Rate your experience using MetaBow.

  2. Expectations: To what extent were your expectations using MetaBow confirmed?

Behavioral Intention

  1. Market Viability: How viable would the market for this interface be?

  2. Purchase Likelihood: How likely would you be to buy this interface at a market price of US$ 300?

  3. Taking Question 8 into Account: How would you rate the transformative pedagogical potential of MetaBow?

  4. Job Relevance 1: How relevant would the interface be to your role as a performer?

  5. Job Relevance 2: How relevant would the interface be to your role as a teacher?

  6. Result Demonstrability: How simple would it be for you to communicate with your colleagues the benefits of using the technology?

Free Comment Section

Please provide a review where you can share your user experience, highlight both the strengths and weaknesses of the device, and provide any additional comments you may have.

Supplementary data