We present the second iteration of a Touch-Responsive Augmented Violin Interface System, called TRAVIS II, and two compositions that demonstrate its expressivity. TRAVIS II is an augmented acoustic violin with touch sensors integrated into its 3-D printed fingerboard that track left-hand finger gestures in real time. The fingerboard has four strips of conductive PLA filament that produce an electric signal when fingers press down on each string. Although these sensors are physically robust, they are mechanically assembled and thus easy to replace if damaged. The performer can also trigger presets via four sensors attached to the body of the violin. The instrument is completely wireless, giving the performer the freedom to move throughout the performance space. Although the sensing fingerboard is installed in place of the traditional fingerboard, all other electronics can be removed from the augmented instrument, maintaining the aesthetics of a traditional violin. Our design allows violinists to naturally create music for interactive performance and improvisation without requiring new instrumental techniques. The first author composed two compositions to highlight TRAVIS II: “Dream State” and “Kindred Dichotomy.” Both of these compositions involve improvisation in their creative process and include interactive visuals. In this article we describe the design of the instrument, experiments leading to the sensing fingerboard, performative applications of the instrument, and compositional considerations for the resultant pieces.

The violin's design continues to develop into the digital age, with the introduction of commercially available electric and MIDI violins. Augmented violins combine traditional instruments with computer programming, and thus offer new expressive possibilities for performers and composers, as well as new ways for violinists to improve their technique. The Touch-Responsive Augmented Violin Interface System (TRAVIS) II presents a new method of designing an augmented violin in which conductive strips on the fingerboard detect contact with the strings (see Figure 1).
Figure 1

TRAVIS II: with the removable electronics setup (a) and with the electronic components removed (b).

Figure 1

TRAVIS II: with the removable electronics setup (a) and with the electronic components removed (b).

Close modal

Both authors are classically trained violinists, and the first author has an extensive background in electroacoustic and interactive music composition and performance. As such, the TRAVIS project follows an autobiographical design approach in which the instrument was built to serve the first author's creative needs (Neustaedter and Sengers 2012).

TRAVIS II is an iteration on a prior project, TRAVIS I (Ko 2018). The earlier iteration is an acoustic violin with two linear SoftPot potentiometers placed under the G and E strings, and two force-sensitive resistors (FSRs) clamped to the right upper bout. The SoftPots in TRAVIS I do not cover the areas on the fingerboard below high second position; to sense in all positions, the nut, and therefore the string height, would need to be raised to prevent the SoftPots from contacting the vibrating strings. Also, the player can only use the SoftPots on the G and E strings; there is no sensing on the D and A strings. Furthermore, the aesthetic of TRAVIS I makes it difficult to use in traditional contexts. Its SoftPots and wiring are permanently attached and cannot be removed from the violin when stored in its case. Finally, TRAVIS I has only two FSRs to trigger presets, limiting its compositional potential.

The TRAVIS II design seeks to address these limitations in TRAVIS I and increase the expressive flexibility of TRAVIS as an augmented instrument. TRAVIS II uses four strips of conductive polylactic acid (PLA) filament in a custom 3-D printed fingerboard, which can sense the violinist's left-hand gestures on all four strings and for the full length of the strings. TRAVIS II includes a set of four FSRs that can select programmed presets for both sound and visuals. We also designed the violin to be continually useable in a traditional context such as an orchestra ensemble, rehearsal, or sight-reading session. We added all additional technology via removeable 3-D-printed clamps to preserve the original instrument's geometry (shown in Figure 1), and the sensing fingerboard does not alter the instrument's original dimensions (e.g., fingerboard thickness or string height). The instrument is physically robust and has easily replaceable sensing elements. In this article, we describe the instrument design, experiments in 3-D print filament, printer settings, string selection, and musical applications of the final instrument. The compositions for TRAVIS II highlight the instrument's affordances and draw a clear connection between the sensor data and the resultant sound.

Miranda and Wanderley (2006) proposed four categories of new digital instruments: alternative gestural controllers, instrument-inspired gestural controllers, instrument-like gestural controllers, and augmented musical instruments. We position our work as a new augmented musical instrument. Augmented instruments are broken down into two groups (Overholt 2011). The first group are instruments that track traditional playing gestures and techniques. As Kimura (2003) mentions, foot pedals are unnatural for violinists because they require a gesture not typically used with the instrument. Augmented instruments that only track the performer's natural playing gestures are beneficial for minimizing distractions and unnatural body movements for the performer. An example of one such instrument is Institut de Recherche et Coordination Acoustique/Musique (IRCAM)'s Augmented Violin Glove (Bevilacqua et al. 2006; Bevilacqua, Baschet, and Lemoutonn 2012; Kimura et al. 2012; Baschet 2013). The second group are instruments that require the invention of new extended gestures, such as Dan Overholt's (2005) Overtone Violin. These instruments expand both the sonic possibilities of the instrument and the virtuosic possibilities because the instrumentalist is expected to develop new playing gestures.

Many augmented violins and cellos have focused on tracking the bow or the right hand: IRCAM's Augmented Violin Glove, the Electronic Violin Bow (Guettler, Wilmers, and Johnson 2008), the K-Bow (McMillen 2008), and the Hyperbow (Young 2002). Moreover, Todd Machover's Hypercello (Machover and Chung 1989; Machover 1992) and the Tracking System for Violin (Pardue, Harte, and McPherson 2015) contain components for tracking both left and right hands.

There are a few other methods of capturing violin gestures. One is through optical tracking, such as Schoonderwaldt and Demoucron's (2009) research in tracking both the left and right hands. Jensenius and Johnson (2012) took a different approach by using video cameras to track a violinist's position on stage instead of tracking the specific finger or bowing gestures. Another method is with electromyographic (EMG) sensors. Dalmazzo and Ramirez's (2017) Air Violin also tracked the left hand with optical tracking and an EMG sensor. Seth Thorn's (2019) Transference also used an EMG and gyroscope to track the left-hand finger and arm gestures, as well as an inertial measurement unit and flex sensors to track the right hand.

Augmenting the fingerboard of string instruments is an attractive approach. This can present many physical and technical challenges, however. In working on the Hypercello, Machover found that the cello strings were too abrasive for the thermoplastic attached to the fingerboard, so the instrument's calibration changed over time. In the Augmented Cello project, Freed et al. (2006, 2013) tried adding touch sensors to the fingerboard but found that the resulting data were nonlinear and that the data varied between strings, so they abandoned it. There was one augmented bass project, and another two augmented violin projects, that successfully added touch sensors to the fingerboard. Bahn and Trueman (2001) added a mouse touch pad to the fingerboard of the SBass, Grosshauser and Troester's sensor fingerboard had embedded FSRs (Grosshauser 2008; Grosshauser, Großekathoefer, and Hermann 2010; Grosshauser, and Troester 2013, 2014). The instrument by Grosshauser and coworkers was used to study how violinists play and when they were pressing too hard with their fingers. The copper colored sensors aesthetically stood out, however, and we speculate that violinists would not be able to continue to use the instrument in a traditional context. Pardue, Harte, and McPherson (2015) sought to solve this problem with a low-cost real-time tracking system for violin. The system intended to help students improve their intonation by providing feedback based on the sensors that track left-hand finger position, combined with pitch tracking. The touch-sensitive component of the system was made with Velostat on top of the fingerboard and a voltage running down the strings. Although the technology could be attached and removed from any violin, the string height had to be raised to ensure that the strings were clear of the Velostat when playing open strings. It is important to note that in a later iteration of the project, Pardue and colleagues (2019) made a new violin for beginner violinists that no longer relied on touch sensors to detect intonation errors. Instead, it used digitally automated pitch and tone correction, so the violin always played in tune.

Beyond improving upon the TRAVIS I design, our approach to augmentation is inspired both by Pardue and coworkers and by Grosshauser and Troester. We seek to address design challenges presented by the violin's geometry with a novel solution. We use strips of conductive 3-D print filament embedded into a 3-D-printed fingerboard to create touch sensors. These conductive strips are more resistant to damage than thermoplastic or Velostat, and individually replaceable if damaged. The instrument is aesthetically and geometrically close to a classical violin such that it can be played in a traditional context. TRAVIS II is primarily used to explore new creative, expressive, improvisational, and compositional possibilities in interactive music.

TRAVIS II is a design iteration of the TRAVIS I concept. In this section, we discuss the technical implementation of TRAVIS II and how it improves upon the TRAVIS I design. In addition to starting from a better violin model than TRAVIS I, the augmentation includes full-length sensors on all four strings and four FSRs instead of two.

The Fingerboard and Sensors

The primary feature of TRAVIS II is its sensing fingerboard. The main fingerboard is 3-Dprinted with black PLA; its geometry includes four slots underneath each of the strings. Because of the build volume of the 3-D printer, the fingerboard was printed in two parts. Four sensor strips made from ProtoPasta's black conductive PLA slide into the slots (https://www.proto-pasta.com/pages/conductive-pla). The strips are designed to slide in and are secured with a press fit. If they become damaged, they can slide out to be replaced. The strips are flush with the surface of the fingerboard, so the strings do not need to be raised. Within close proximity the differences in texture between the strips and the PLA fingerboard are visible (see Figure 2). From the audience's more distant perspective this difference is not noticeable, however.
Figure 2

Side (a) and front (b) views of the “bridge end” of the fingerboard and support; without the Arduino and FSRs.

Figure 2

Side (a) and front (b) views of the “bridge end” of the fingerboard and support; without the Arduino and FSRs.

Close modal
For testing, we attached different PLA fingerboards to the neck of the violin with double-sided tape. In the first test, the PLA fingerboard was too flexible and, when played in high positions, it bent from regular finger pressure. Open-source designs for 3-D-printed violins, such as the ones by Formlabs and Openfab PDX, have addressed this problem with support rods (see the manufacturers' websites for further details: https://formlabs.com/blog/sdesigning-a-3d-printed-acoustic-violin and https://openfabpdx.com/fffiddle). The slots for the conductive strips leave very little room to include a support rod, however, so we worked with a luthier who placed a thin, flat piece of ebony onto the neck of the violin to support the PLA surface. After a few weeks, although the fingerboard continued to support the weight of the finger pressure, we found the natural placement of the fingerboard started to sag. In response, the luthier also placed a small piece of wood underneath the fingerboard to prop it up into the correct position (see Figure 3). A copy of the fingerboard was made from polyethylene terephthalate glycol (PETG)-modified filament. It was not noticeably stronger than the PLA version. Aesthetically, the PLA version is preferred because it is not as shiny as the PETG.
Figure 3

Drawing of the cross section between the face of the “A string strip” (dark outline) and the face of its slot (faded outline). The drawing is from the “nut end” of the fingerboard. Measurements are in millimeters.

Figure 3

Drawing of the cross section between the face of the “A string strip” (dark outline) and the face of its slot (faded outline). The drawing is from the “nut end” of the fingerboard. Measurements are in millimeters.

Close modal

The aesthetic of the sensing fingerboard more closely resembles the aesthetic of a traditional fingerboard; both the regular and conductive PLA are black. Although some layer lines from fused deposition modeling printing are visible on the PLA surface, the texture of these lines against the fingertips does not distract from playing.

As documented by Leigh et al. (2012) and McGhee et al. (2018), conductive 3-D-print filament works well as the resistive component of touch sensors. Similar to the design by Pardue and coworkers (2015), TRAVIS II runs a voltage of 3.3 V down the strings; when a metal string comes in contact with a conductive strip, the string's voltage changes.

We also placed four small FSRs to a 3-D-printed clamp and mounted it on the right upper bout of the violin (shown in Figure 1 a). Here they are easily accessible to the performer; violinists rest their left hand on the right upper bout of the violin when not playing, and the FSRs are directly below the left hand when playing in high positions.

TRAVIS II uses an Arduino MKR1000 to send data to the software Max via OSC (Open Sound Control) messaging. TRAVIS II's Arduino is multiplexed for eight sensors and sits inside a custom 3-Dprinted case mounted on the left upper bout of the violin. This placement is based on the TRAVIS I design. We considered alternative placements for the Arduino. The Arduino could clamp onto the left lower bout, where the weight would be less noticeable on the shoulder. But this placement would obstruct the performer from bowing behind the bridge, an extended bowing technique. Alternatively, the Arduino could attach underneath the shoulder rest, as in Grosshauser and Troester's violin. Shoulder rests are not very secure, however, and many accidentally fall off midperformance. Placement near the shoulder rest would also limit how the shoulder rest can be adjusted; some performers would not be able to use their preferred shoulder rest height. By placing our Arduino and battery on the left upper bout of the violin, our added electronics do add some weight to the violin. We have found that this added weight is not uncomfortable, however, and the design keeps all components safely out of the way while playing.

The wires, the Arduino case, and the FSR clamp easily disassemble from the violin, both for storage in the case and when playing the violin in a traditional context. The wires connect to the Arduino via an insulation-displacement contact connector, to the 3-D conductive strips with Swiss machine headers, and to the strings with a JST (Japan Solderless Terminal) connector. The JST connector sits underneath the chin rest and the wires are soldered onto the balls of the strings. Strings are easily replaceable by cutting off the wires, then resoldering the wires to the new strings.

Our initial tests ensured that TRAVIS II could work sufficiently as an augmented instrument, by producing a distinct range of values from the 3-D-printed touch sensors. We then tested the 3-D print settings, resistors, and strings to ensure that the instrument could produce reliable and stable data for the Arduino with the largest range of values between 0 and 1,023. We placed higher priority on the physical design, however. This included the sensor strips having dimensions that fit into the fingerboard, being durable so they do not break when assembling them, and having a smooth, high-quality finish on the surface.

We printed different dimensions of the conductive strips and used different settings for 3-D printing to see how these variables affect the strip's resistance. We tested the resistance with a multimeter at its 20-kΩ setting. We recorded the resistance at three points along the strips. First, we measured the resistance across 1 cm of the strip at one end. Then, we measured the resistance from one end to the approximate center of the strip (about 13.2 cm). Finally, we measured the resistance at either end of the strip (about 26.4 cm). After we finalized the dimensions and the settings for the 3-D printer, we tested each of the final strips for the fingerboard one last time before integrating them with the violin.

Afterwards, we tested the range of analog data values read into the Arduino with various resistors in the circuit (1, 3.3, 4.7, 10, 22, and 47 kΩ). We set up the circuit with the test strips, the strings, and resistors; then we recorded sensor values resulting from pressing the string at each end of the conductive stip. We selected final resistors based on which provided the largest range of sensor data.

The Conductive 3-D Print Filament

The dimensions of the strips are tightly constrained, particularly when fitting four into the geometry of a standard fingerboard. If strips are too wide, the slots are too large to fit into the fingerboard. If strips are too narrow, they could break while sliding them into the fingerboard, or the string may miss the strip surface when pressed.

We initially tested a conductive strip that was 265-mm long, 2.5-mm tall, 5-mm wide at the end closer to the bridge, and 4-mm wide at the end closer to the nut. However, when measuring its resistance, the resultant readings were not stable, and we could not record approximate measurements. The next test strip was narrower; consistently 3-mm wide at both ends, and produced a stable, measurable resistance value.

We tested different cross-sectional geometries to identify a series of four strips that both fit within the physical constraints of the fingerboard and had a relatively consistently varying resistance across the strip. We tested the resistance with a multimeter (20-kΩ setting) at three distances from the nut: 10 mm, about 134 mm (middle), and approximately 268 mm (end to end). In our final strips (cross-sectional geometry in Figure 3), the resistance changed proportionally across the length of the strip (see Table 1). We tested the resistance again after ten months and found values close to our readings from our initial testing; thus, the conductive materials in the filament had not degraded over this time span.

Table 1:

Approximate Resistance of Final Conductive Strips

String1 cm (Close)13.2 cm (Middle)26.4 cm (Ends)
2.19 kΩ 6.67 kΩ 10.89 kΩ 
1.97 kΩ 8.25 kΩ 16.7 kΩ 
1.64 kΩ 6.63 kΩ 12.3 kΩ 
3.07 kΩ 7.55 kΩ 10.49 kΩ 
String1 cm (Close)13.2 cm (Middle)26.4 cm (Ends)
2.19 kΩ 6.67 kΩ 10.89 kΩ 
1.97 kΩ 8.25 kΩ 16.7 kΩ 
1.64 kΩ 6.63 kΩ 12.3 kΩ 
3.07 kΩ 7.55 kΩ 10.49 kΩ 

Measurements taken at the 20kΩ setting.

We also fine-tuned the length of the strips to ensure that they could easily be installed; if the end of the strip fit too firmly with the nut end of the slot, it was difficult to later remove. The final length was 268.5 mm; we are able to slip a sewing needle behind the strips at the nut end of the slot to push them out.

Settings for 3-D Printing

Once we had finalized the physical dimensions, we tested various slicing settings in Cura to see whether printing parameters affected resistance (https://ultimaker.com/software/ultimaker-cura). Cura is one of many programs that slices 3-D models and prepares them into a G-code file that can be read by a 3-D printer. We used the same process as above, measuring the resistance at three points along each strip.

First, we varied the layer heights from 0.1 mm to 0.5 mm. As the layer height of each test strip increased, the approximate resistance decreased (see Table 2). Also, the taller the layer height, the lower the resultant quality of the print. We decided to keep the layer height at 0.1 mm so that the header holes of the strips had the best print quality possible (shown in Figure 2), and so that the surface of the strips could be as smooth as possible.

Table 2:

Test Strip Resistance Relative to Print Layer Height

Layer Height1 cm (Close)13.2 (Middle)26.4 cm (Ends)
0.1 mm 2.55 kΩ 6.1 kΩ 11.6 kΩ 
0.2 mm 3.38 kΩ 6.5 kΩ 8.66 kΩ 
0.3 mm 1.27 kΩ 5.8 kΩ 7.7 kΩ 
0.5 mm 1.64 kΩ 4.06 kΩ 6.82 kΩ 
Layer Height1 cm (Close)13.2 (Middle)26.4 cm (Ends)
0.1 mm 2.55 kΩ 6.1 kΩ 11.6 kΩ 
0.2 mm 3.38 kΩ 6.5 kΩ 8.66 kΩ 
0.3 mm 1.27 kΩ 5.8 kΩ 7.7 kΩ 
0.5 mm 1.64 kΩ 4.06 kΩ 6.82 kΩ 

We tested three different infill patterns—triangles, lines, and cubic—with a 70 percent infill density. The resistances of these patterns were not significantly different from one another. We did not test different infill densities because of strength concerns; the strips needed to slide in and out of the fingerboard without breaking.

By trial and error, we discovered that sanding the top surface of the conductive filament ruined its performance. In an effort to minimize the layer lines from 3-D printing, we had placed the strips into their slots in the fingerboard and then sanded the entire surface together. This damaged the conductivity and resistivity of the sanded surfaces of the strips, and there were barely any electrical readings from the multimeter. We reprinted the fingerboard and all strips after this discovery, and no longer attempted postprocessing the surface finish.

Testing Violin String Sensors

Each of the four strings on violins are made of different metals, and different brands of strings vary in the materials and manufacturing of the cores and windings. We had full sets of Dominant and Evah Pirazzi strings and a Pirastro Gold E. The metals from which they were made are listed in Table 3.

Table 3:

Summary of Tested Strings and Materials

String BrandString NameCoreWinding
Pirastro Gold (N/A) Tin-plated carbon 
   steel 
Dominant (135) E 130 Steel Aluminum 
 A 131 Synthetic Aluminum 
 D 132 Synthetic Aluminum 
 G 133 Synthetic Silver 
Evah Pirazzi (N/A) Silvery steel 
 Synthetic Aluminum 
 Synthetic Silver 
 Synthetic Silver 
String BrandString NameCoreWinding
Pirastro Gold (N/A) Tin-plated carbon 
   steel 
Dominant (135) E 130 Steel Aluminum 
 A 131 Synthetic Aluminum 
 D 132 Synthetic Aluminum 
 G 133 Synthetic Silver 
Evah Pirazzi (N/A) Silvery steel 
 Synthetic Aluminum 
 Synthetic Silver 
 Synthetic Silver 

(N/A) = not applicable.

We set up these strings in the sensor ci rcuit to test the range of data received from them with different resistor values (1 kΩ, 3.3 kΩ, 4.7 kΩ, 10 kΩ, 22 kΩ, and 47 kΩ). The Evah Pirazzi's range of data was sufficient, but we concluded that the Dominant pack, with the Pirastro Gold E, had the largest range of data (see Table 4). In its final configuration, the E string uses a 4.7 kΩ resistor, and the rest of the strings have a 3.3 kΩ resistor. None of the tested completed sensors reached their full range of 1,023 values. This is not necessarily a limitation, because the values are rescaled in Max.

Table 4:

Range of Values Received from Sensors

Dominants with Pirastro Gold EEvah Pirazzi
StringResistorValuesRangeValuesRange
3.3 kΩ 260–720 460 255–685 430 
3.3 kΩ 255–645 390 265–640 375 
3.3 kΩ 245–630 385 235–490 255 
4.7 kΩ 315–765 450 320–660 340 
Dominants with Pirastro Gold EEvah Pirazzi
StringResistorValuesRangeValuesRange
3.3 kΩ 260–720 460 255–685 430 
3.3 kΩ 255–645 390 265–640 375 
3.3 kΩ 245–630 385 235–490 255 
4.7 kΩ 315–765 450 320–660 340 

TRAVIS II is currently used as a controller for interactive compositions made with Max and is designed so that there is no need for an additional person to control the computer.

At the time of writing, the first author has composed and performed one solo work with TRAVIS II, “Dream State,” and one duet with both TRAVIS I and II, “Kindred Dichotomy.” The duet was performed by both authors at the Conference of the Alliance of Women in Media Arts and Sciences (AWMAS) held at the University of California at Santa Barbara on 7 February 2020.

The first author also composed and performed five pieces with the initial TRAVIS I prototype; because TRAVIS I did not have sensors under the D and A strings, these compositions can still be performed on TRAVIS II by ignoring the D and A string sensors in the Max patch. In this article we will discuss in detail “Dream State” and “Kindred Dichotomy” (viewable online at https://youtu.be/FxaaYOO3Knw and https://youtu.be/RuqMSzlXKJs, respectively).

Composers are able to generate a range of new expressive effects by utilizing the features of TRAVIS II. The four FSRs are primarily used to trigger presets for different sections of the composition. Because they give a range of data based on pressure, however, it is possible to use the FSRs to scrub through audio samples as well.

In addition to using finger position on the fingerboard to control sound processing parameters, different processes can be assigned to each individual string sensor; when a string is pressed, it routes the sound to its assigned process. This method of mapping different processes to each string was used in all current pieces for TRAVIS II.

TRAVIS II works particularly well with several specific violin techniques, such as fast passages and double stops. If the different effects are assigned to individual strings, double stops would simultaneously route the sound to two effects, or scrub through two prerecorded samples. This was used in the cadenza of “Dream State” and at the end of “Kindred Dichotomy.” To achieve combined effects, the violinist also does not necessarily have to play both strings with the bow; it is enough to keep the fingers on multiple strings while playing a melody with the bow on only one string. This single technique opens a wide range of possibilities, and compositionally this method is more interesting than simply assigning two sound processes to one string.

The software can also recognize techniques where fingers are either changing quickly or playing fast gestures, such as trills, vibrato, glissandos, and shifts (see Figure 4). These fast techniques are recognized by setting a threshold on the timer object in Max. The program is unable to differentiate these techniques from one another, however; the composer or performer would need to take this limitation into consideration during the creative process. The sensitivity of this kind of gesture tracking also must be preadjusted to the speed of the violinist's technique. This method of tracking will be used in future compositions.
Figure 4

Example Max patch implementing recognition of finger changes.

Figure 4

Example Max patch implementing recognition of finger changes.

Close modal

The Music of “Dream State”

“Dream State” is the first piece made for TRAVIS II. The majority of the piece is improvised, and the ending cadenza is precomposed. The improvisations are a slow legato and reverberation is added to achieve a dream-like effect. To link the piece together, some of the motifs played during the improvised sections are drawn from musical material written in the cadenza.

The piece begins with a pizzicato arpeggio that is recorded in real time and played back for the remainder of the improvised sections. The recording is turned off for the cadenza section, but the performer plays this arpeggio motif again at the very end, this time arco (with the bow) instead of pizzicato.

As mentioned earlier, different sound processes are mapped to each string sensor: The G-string sensor routes the sound through a flanger; the D-string sensor is mapped to a pitchshift module; the A-string sensor sound goes through a chorus; the E-string sensor to two consecutive delays that each range from 500 to 2,000 msec. The second delay is always delayed an additional 500 msec from the first. As the first TRAVIS II composition, its compositional goal is to draw attention to the changes in timbre on each string from these different processes. Therefore, before the cadenza, there are four improvised sections. Each of these sections are predominantly played on an individual string, starting on the G string, and with each consecutive section moving to the next string. At the beginning of each section, the performer presses an FSR to transpose the recorded pizzicato arpeggio and change the geometric shape in the visuals. Many glissandos are also used to show how the sound processes change along the length of the string sensors. Although the improvised sections show the individual timbres of each string, the cadenza exemplifies the unified sounds from when string sensors are activated together with double stops.

The Music of “Kindred Dichotomy”

“Kindred Dichotomy” is the first composition written for both TRAVIS I and TRAVIS II. As such, the piece demonstrates how well the two instruments could be used simultaneously. Aesthetically, “Kindred Dichotomy” explores the concept of two entities establishing their opposite identities, as well as discovering core similarities between them. There are motifs that characterize each violin, and in the middle of the piece the violins start to blend and quote each other's style. In the end, they return to their original motifs and style.

When composing “Kindred Dichotomy,” the hardware limitations and the resultant limitations on possible violin techniques were considered first. For TRAVIS I, the SoftPots are only under the G and E strings, and the lowest finger placements that activate the sensors are B-natural on the G string, and G-sharp on the E string. This limits the main melodic passages for TRAVIS I to be written only for the G or E strings with considerable shifting on these strings. TRAVIS II has fewer limitations for violin techniques because the full lengths of all four strings are touch sensors.

Given these limitations, the compositional goals were to support the abstract themes of “opposites and similarities” between two entities:

  1. The piece should be idiomatic to the violin.

  2. The piece should not be dependent on the electronics to sound cohesive. Instead, the electronics should augment and add to the expressivity.

  3. Even though TRAVIS II's hardware is superior to TRAVIS I's, each violin should have equal musical importance. Therefore, they should each have their own melodic sections to establish the “dichotomy” between them. Each of these melodic sections has its own distinctive musical characteristic.

  4. Countermelodies are used as much as possible. This is so each violin can maintain some of its unique characteristics, while the other has the melody.

  5. The underlining similarity between entities is reflected in how the music does not modulate from its home key.

  6. The musical form should be reflected in the visuals.

Even though “Kindred Dichotomy” has a precomposed score, improvisation still plays a key role in the compositional process. The first author internalized the limitations and goals, then improvised the melodies. The improvisations were transcribed and edited, then used as base material to write the accompaniment part above or underneath. The composition uses the pitches of the D harmonic minor scale, however, like a jazz mode, the pitches of the scale are rotated so that A is the tonic instead of D.

Unlike previous interactive pieces for solo TRAVIS I or solo TRAVIS II, the score to “Kindred Dichotomy” was written before programming the Max patches. This method allowed the electronics to augment the existing musical material instead of the composition relying on the electronic sound. It also made the interplay between the two violins sound more natural and musically complex.

As previously mentioned, depending on the section of the composition and which instrument has the melody, a different sound process is assigned to each string. Therefore, it is sonically interesting when the performer plays a fast passage with many changes between strings (see Figure 5). There is a one-to-one mapping between the violinists' left-finger gesture data and the resultant computer processes in real time. Each violin controls parameters of its own processes, and do not affect each other's sound or visuals. To conserve CPU, the audio patch and the visuals patch are divided onto two computers that are networked with OSC messaging.
Figure 5

Measures 38–44 in “Kindred Dichotomy.”

Figure 5

Measures 38–44 in “Kindred Dichotomy.”

Close modal

Audio Visual Considerations

Both “Dream State” and “Kindred Dichotomy” include interactive visuals projected on screen. As elaborated by John Coulter (2010), aesthetically, there are two viewpoints when working with both music and visuals. One is that visual art and music can complement each other. The other is the acousmatic approach, in which visuals are distracting, and the ideal listening conditions are only met when sound is isolated from image. To prevent the visuals from being too distracting, the compositions with TRAVIS II avoid overly fast movement in the visuals.

The main goal in both compositions is to have the music and visuals synchronously linked. Momeni and Henry (2006) classify audio visual work into three categories based on how the sound and visuals are controlled: sound-to-image, image-to-sound, or concurrent generation of sound and image. Both “Dream State” and “Kindred Dichotomy” fall under the third category. The data from the touch sensors are used to control the sound and visuals concurrently; the sound does not directly affect the images and the data from the images are not mapped to control the sound.

Unlike “Kindred Dichotomy,” the visuals used in “Dream State” do not have an abstract meaning that relates to the piece's theme; they are only meant to mirror what is happening in the music and physical gestures. The visuals (see Figure 6) consist of a cluster of shapes that spin around each other. Each improvised section of “Dream State” has a different geometric shape: open cube, open cylinder, sphere, and torus. Each string sensor is mapped to a different parameter of the visuals. The G-string sensor controls the scale and xyz rotation of the objects. The D-string sensor increases and decreases the space between the multiple objects on screen. The A-string sensor increases and decreases the number of objects on screen. The E-string sensor adjusts the z position of the camera angle, or “closeness” to the objects.
Figure 6

Performance of “Dream State” with the visuals.

Figure 6

Performance of “Dream State” with the visuals.

Close modal
Throughout “Kindred Dichotomy,” both the music and visuals aim to illustrate the abstract idea of opposites that have some underlying similarities. For example, the kaleidoscope-like image (see Figure 7) was created and used as the main visual theme because of how the design appears unified in the center. This relates to the abstract theme of “kindred.” By changing half of the image to a green hue, and the other half to a blue hue, there is a clear divide between each half. This ties into the theme of “dichotomy.” Throughout the piece, the TRAVIS I sensor data is assigned to affect the green half of the image and TRAVIS II affects the blue half. The image transforms with each section of the composition and likewise the one-to-one mappings of the visuals change with each section as well.
Figure 7

Main image used for the visuals in “Kindred Dichotomy.”

Figure 7

Main image used for the visuals in “Kindred Dichotomy.”

Close modal

Rehearsing “Kindred Dichotomy”

“Kindred Dichotomy” is the first piece performed on TRAVIS I by someone other than the first author. We devised a strategy to gradually introduce Oehlberg to TRAVIS I and the composition.

The first rehearsal served the dual purpose of proofreading the composition and allowing us to understand how the instrumental parts feel when played together. As such, we used our own violins without any augmentation. In the second rehearsal we practiced with TRAVIS I and II with all of the electronic hardware clamped to the bodies of the instruments. The instruments were not turned on, however, and the computer, loudspeakers, and projector were not set up. This rehearsal was aimed to practice the newly revised version of “Kindred Dichotomy” while Oehlberg was also being introduced to the aural and kinesthetic differences of TRAVIS I from her personal violin. The TRAVIS I violin itself is a low-quality instrument; it takes time for the ear to adjust to the different acoustic timbre and it requires more effort to create a clean sound. The SoftPots also feel slippery in comparison to the typical ebony fingerboard. Although the SoftPots' different texture does not affect playing technique, it is another unusual sensation that the performer is aware of while also becoming accustomed to everything else about the instrument that is different. In this rehearsal, excluding the computer and loudspeakers allowed us to continue focusing on the newly written music without feeling distracted or overwhelmed by the electronic sounds. In the third rehearsal all of the electronics and computer was set up.

TRAVIS II achieved its goal in augmenting a violin and improved upon TRAVIS I's design. In comparison to TRAVIS I, the violin used to make TRAVIS II is a much higher-quality violin model. Before augmentation took place, based on the first author's subjective standards, the TRAVIS I violin would only be used by a beginner student. In comparison, the TRAVIS II violin would have been considered suitable for intermediate to lower-advanced levels. It would have been suitable for playing in an orchestra or middle-level examination. It would not have previously been used for situations such as a professional solo recital or high-level examination. TRAVIS II has more FSRs available to change settings and all four strings can be tracked. It expands the composer's available palette of timbres, effects, and recorded samples.

One of the main limitations of augmenting a cello with touch sensors is that the strings are large and thick, therefore any resistive film placed on top of the cello fingerboard is more receptive of damage than on a violin. We wonder if this method of augmentation with conductive 3-D print filament can be successfully applied to a cello, and other instruments of the violin family. A much larger 3-D printer would be needed.

There are some limitations to the sensor results and TRAVIS II's design. Only packs of Dominant and Evah Pirrazi strings, plus a Pirastro Gold E string, were tested. Also, only one brand of conductive PLA, by ProtoPasta, was in stock at the time of purchase. A more thorough study would include testing multiple brands of strings and conductive filament. Other brands of conductive filament have different magnitudes of resistivity, and therefore could provide different sensor data (McGhee et al. 2018).

Printing only the top half of the fingerboard and sitting it on top of an ebony piece greatly improves the overall strength. As a filament for fused deposition modeling printers, PETG is stronger than acrylonitrile butadiene styrene or PLA (for a comparison of the materials, cf. Kondo 2019). We found the PETG version was not noticeably stronger on the violin, it did not hide the layer lines more than the PLA, and it was much shinier. Therefore, we continue using the PLA model. Layer lines are visible, and we speculate that a resin filament would make a cleaner-looking surface. Unfortunately, we did not have a resin 3-D printer available.

The objective of the project was to construct an instrument that serves the first author's needs as a violinist. These include being able to remove all of the technology, with exception of the 3-D-printed fingerboard and hidden JST connector for the strings, to be able to play the violin in a traditional context. There are still a few subtle variations that differentiate TRAVIS II from a traditional violin geometry. For example, the traditional fingerboard has an undercurve. The small PLA header piece does have an undercurve (seen in Figure 2), but the rest of the ebony piece is flat. Visually, this difference is not noticeable from the audience's perspective, so the violin can continue to be played as a conventional acoustic violin. The flat ebony piece does slightly affect the tone of the violin, making it sound tinnier. The tinny quality contributes to why we would not recommend playing this violin as a solo acoustic instrument, although it may still be suitable in an orchestral setting.

Although the Max patch shown in Figure 4 can measure the speed of data to recognize when fingers are playing fast gestures, it cannot differentiate vibrato from other fast gestures (e.g., shifts, glissandos, trills). It also only recognizes where it was pressed, and not which finger is pressing down. Finger recognition could be achieved with EMG sensors (Dalmazzo and Ramirez 2017; Thorn 2019), or with a fingertip-free glove with flex sensors on each finger. Vibrato recognition may also be more accurately achieved with an inertial measurement unit on the hand (Thorn 2019), or flex sensors on the wrist, and elbow. These design concepts may be considered for future research.

Reflection: Compositions

“Dream State” is a composition meant to premiere TRAVIS II. The slow, reverberant nature of the piece, with only scant compositional instructions, made it easy to improvise and perform with few rehearsals beforehand. It is simple in both its musical form and mappings to effectively exemplify TRAVIS II's technical affordances, but still allows space for performers to showcase their virtuosity.

Compositionally, “Kindred Dichotomy” posed a challenge because it involved two violins sharing similar sonic qualities and timbres. Therefore, the processed sound coming out of the loudspeakers blends together seamlessly, making the sound of the two instruments less distinct from each other. Even though each violin has a direct one-to-one mapping of its sensor data to sound processes, this mapping was not as perceptible as in other solo works for either TRAVIS I or II. If it were two instruments of different orchestral families, the resultant processed sound from each instrument would likely be more distinctive. This led to panning each violin's sound to different loudspeakers to accentuate distinction between them. In the future, any duets for TRAVIS I and II, or of instruments of the same musical family, should involve more interplay between instruments and solos with sparse accompaniment. In the middle of “Kindred Dichotomy,” instead of both violins affecting different parameters of a single shape, each violin was assigned to control its own half of the kaleidoscope-like image. This led to a more perceptible understanding for how the violins were mapped to the visuals.

Interactive computer music is commonly performed via improvisation with the computer, because computers are capable not only of analyzing gestures but also of generating new music and responding to performers' gestures. As Bresson and Chadabe (2017, p. 1) note,

In the context of computer music, the question becomes one of how a computer might be programmed to generate audio as well as compositional information that is flexible enough in its reaction to a performer to qualify as interactive. Interaction, at whatever level, refers to a mutually influential process.

Improvisation is not commonly taught to classically trained violinists, or to many other classically trained musicians, however. Through the rehearsal process of “Kindred Dichotomy,” we speculate that slowly introducing a performer to a new digital musical instrument with a score is a good pedagogical approach before asking the performer to improvise. Understanding how instrumentalists adapt to interactive performance was not the intent of “Kindred Dichotomy,” however, and for conclusive results in this particular area, a proper study with multiple participants should be conducted.

Overall, TRAVIS II achieved its design goals, and at the time of writing this article, it has been used in two successful interactive compositions. It is an example of a new method to track left-hand finger gestures through conductive 3-D print filament and encourages speculation on how this method could be applied to other string instruments.

We would like to recognize Aaron Pratte, the luthier who detached the original fingerboard of TRAVIS II and made the ebony piece of the modified fingerboard. The wireless version of TRAVIS I was a collaboration of the first author with Robert Pritchard and the engineering team of WiRED: Jin Han, Esther Mutinda, Carol Fu, and Lily Shao.

Bahn
,
C.
, and
D.
Trueman
.
2001
. “
Interface: Electronic Chamber Ensemble.
” In
Proceedings of the CHI Workshop on New Interfaces for Musical Expression
, pp.
19
23
.
Baschet
,
F.
2013
. “
Instrumental Gesture in StreicherKreis.
Contemporary Music Review
32
(
1
):
17
28
.
Bevilacqua
,
F.
,
F.
Baschet
, and
S.
Lemouton
.
2012
. “
The Augmented String Quartet: Experiments and Gesture Following
.”
Journal of New Music Research
41
(
1
):
103
119
.
Bevilacqua
,
F.
, et al
2006
. “The Augmented Violin Project: Research, Composition and Performance Report.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
402
406
.
Bresson
,
J.
, and
J.
Chadabe
.
2017
. “
Interactive Composition: New Steps in Computer Music Research.
Journal of New Music Research
46
(
1
):
1
2
.
Coulter
,
J.
2010
. “
Electroacoustic Music with Moving Images: The Art of Media Pairing
,”
Organised Sound
15
(
1
):
26
34
.
Dalmazzo
,
D.
, and
R.
Ramirez
.
2017
. “
Air Violin: A Machine Learning Approach to Fingering Gesture Recognition.
” In
Proceedings of ACM SIGCHI International Workshop on Multimodal Interaction for Education
, pp.
63
66
.
Freed
,
A.
, et al
2006
. “Augmenting the Cello.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
409
413
.
Freed
,
A.
, et al
2013
. “‘Old’ is the New ‘New’: A Fingerboard Case Study in Recrudescence as a NIME Development Strategy.” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
442
445
.
Grosshauser
,
T.
2008
. “
Low Force Pressure Measurement: Pressure Sensor Matrices for Gesture Analysis, Stiffness Recognition and Augmented Instruments.
” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
97
202
.
Grosshauser
,
T.
,
U.
Großekathoefer
, and
T.
Hermann
.
2010
. “
New Sensors and Pattern Recognition Techniques for String Instruments.
” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
271
276
.
Grosshauser
,
T.
, and
G.
Troester
.
2013
. “
Finger Position and Pressure Sensing Techniques for String and Keyboard Instruments.
” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
479
484
.
Grosshauser
,
T.
, and
G.
Troester
.
2014
. “
Musical Instrument Interaction: Development of a Sensor Fingerboard for String Instruments.
” In
Proceedings of the International Conference on Tangible, Embedded, and Embodied Interaction
, pp.
177
180
.
Guettler
,
K.
,
H.
Wilmers
, and
V.
Johnson
.
2008
. “
Victoria Counts: A Case Study with Electronic Violin Bow.
” In
Proceedings of the International Computer Music Conference
, pp.
569
662
.
Jensenius
,
A.
, and
V.
Johnson
.
2012
. “
Performing the Electric Violin in a Sonic Space.
Computer Music Journal
36
(
4
):
28
39
.
Kimura
,
M.
2003
. “
Creative Process and Performance Practice of Interactive Computer Music: A Performers Tale.
Organised Sound
8
(
3
):
289
296
.
Kimura
,
M.
, et al
2012
. “
Extracting Human Expression for Interactive Composition with the Augmented Violin.
” In
Proceedings of the International Conference on New Interfaces for Musical Expression
. Available online at vhosts.eecs.umich.edu/nime2012/Proceedings/papers/279_Final_Manuscript.pdf. Accessed February 2021.
Ko
,
C.
2018
. “
TRAVIS: Touch Responsive Augmented Violin Interface System.
” Paper presented at the Conference for Interactive Art Science and Technology, 25–27 October, University of Lethbridge, Alberta, Canada.
Kondo
,
H.
2019
. “
PETG vs ABS: The Differences, Simply Explained.
” Available online at all3dp.com/2/petg-vs-abs-3d-printing-filaments-compared. Accessed February 2021.
Leigh
S. J.
, et al
2012
. “
A Simple, Low-Cost Conductive Composite Material for 3D Printing of Electronic Sensors.
PLOS ONE
7
(
11
):
e49365
. Available online at doi.org/10.1371/journal.pone.0049365. Accessed January 2021.
Machover
,
T.
1992
. “
The Hypercello.
” In
Hyperinstruments: A Progress Report, 1987–1991
.
Cambridge, Massachusetts
:
MIT Media Laboratory
, pp.
53
78
.
Machover
,
T.
, and
J.
Chung
.
1989
. “
Hyperinstruments: Musically Intelligent and Interactive Performance and Creativity Systems.
” In
Proceedings of the International Computer Music Conference
, pp.
186
190
.
McGhee
,
J.
, et al
2018
. “
Strain Sensing Characteristics of 3D-Printed Conductive Plastics
.”
Electronics Letters
54
(
9
):
570
571
.
McMillen
,
K. A.
2008
. “
Stage-Worthy Sensor Bows for Stringed Instruments.
” In
Proceedings of the Conference on New Interfaces for Musical Expression
, pp.
347
348
.
Miranda
,
E. R.
, and
M.
Wanderley
.
2006
.
New Digital Musical Instruments: Control and Interaction Beyond the Keyboard
.
Middleton, Wisconsin
:
A-R
.
Momeni
,
A.
, and
C.
Henry
.
2006
. “
Dynamic Independent Mapping Layers for Concurrent Control of Audio and Video Synthesis.
Computer Music Journal
30
(
1
):
49
66
.
Neustaedter
,
C.
, and
P.
Sengers
.
2012
. “
Autobiographical Design in HCI Research: Designing and Learning Through Use-It-Yourself.
” In
Proceedings of the Designing Interactive Systems Conference
, pp.
514
523
.
Overholt
,
D.
2005
. “
The Overtone Violin.
” In
Proceedings of the International Conference on New Interfaces for Musical Expression
, pp.
34
37
.
Overholt
,
D.
2011
. “
Violin-Related HCI: A Taxonomy Elicited by the Musical Interface Technology Design Space.
” In
A. L.
Brooks
, ed.
Arts and Technology: Second International Conference.
Berlin
:
Springer
, pp.
80
89
.
Pardue
,
L. S.
,
C.
Harte
, and
A. P.
McPherson
.
2015
. “
A Low-Cost Real-Time Tracking System for Violin.
Journal of New Music Research
44
(
4
):
1
19
.
Pardue
,
L. S.
, et al
2019
. “Separating Sound from Source: Sonic Transformation of the Violin through Electrodynamic Pickups and Acoustic Actuation.” In
Proceedings of the Conference on New Interfaces for Musical Expression
, pp.
278
283
.
Schoonderwaldt
,
E.
, and
M.
Demoucron
.
2009
. “
Extraction of Bowing Parameters from Violin Performance Combining Motion Capture and Sensors.
Journal of the Acoustical Society of America
126
(
5
):
2695
2708
.
Thorn
,
S. D.
2019
. “
Transference: A Hybrid Computational System for Improvised Violin Performance.
” In
Proceedings of the International Conference on Tangible, Embedded and Embodied Interaction
, pp.
541
546
.
Young
,
D.
2002
. “
The Hyperbow: A Precision Violin Interface.
” In
Proceedings of the International Computer Music Conference
, pp.
489
492
.