Abstract
We present the second iteration of a Touch-Responsive Augmented Violin Interface System, called TRAVIS II, and two compositions that demonstrate its expressivity. TRAVIS II is an augmented acoustic violin with touch sensors integrated into its 3-D printed fingerboard that track left-hand finger gestures in real time. The fingerboard has four strips of conductive PLA filament that produce an electric signal when fingers press down on each string. Although these sensors are physically robust, they are mechanically assembled and thus easy to replace if damaged. The performer can also trigger presets via four sensors attached to the body of the violin. The instrument is completely wireless, giving the performer the freedom to move throughout the performance space. Although the sensing fingerboard is installed in place of the traditional fingerboard, all other electronics can be removed from the augmented instrument, maintaining the aesthetics of a traditional violin. Our design allows violinists to naturally create music for interactive performance and improvisation without requiring new instrumental techniques. The first author composed two compositions to highlight TRAVIS II: “Dream State” and “Kindred Dichotomy.” Both of these compositions involve improvisation in their creative process and include interactive visuals. In this article we describe the design of the instrument, experiments leading to the sensing fingerboard, performative applications of the instrument, and compositional considerations for the resultant pieces.
Both authors are classically trained violinists, and the first author has an extensive background in electroacoustic and interactive music composition and performance. As such, the TRAVIS project follows an autobiographical design approach in which the instrument was built to serve the first author's creative needs (Neustaedter and Sengers 2012).
TRAVIS II is an iteration on a prior project, TRAVIS I (Ko 2018). The earlier iteration is an acoustic violin with two linear SoftPot potentiometers placed under the G and E strings, and two force-sensitive resistors (FSRs) clamped to the right upper bout. The SoftPots in TRAVIS I do not cover the areas on the fingerboard below high second position; to sense in all positions, the nut, and therefore the string height, would need to be raised to prevent the SoftPots from contacting the vibrating strings. Also, the player can only use the SoftPots on the G and E strings; there is no sensing on the D and A strings. Furthermore, the aesthetic of TRAVIS I makes it difficult to use in traditional contexts. Its SoftPots and wiring are permanently attached and cannot be removed from the violin when stored in its case. Finally, TRAVIS I has only two FSRs to trigger presets, limiting its compositional potential.
The TRAVIS II design seeks to address these limitations in TRAVIS I and increase the expressive flexibility of TRAVIS as an augmented instrument. TRAVIS II uses four strips of conductive polylactic acid (PLA) filament in a custom 3-D printed fingerboard, which can sense the violinist's left-hand gestures on all four strings and for the full length of the strings. TRAVIS II includes a set of four FSRs that can select programmed presets for both sound and visuals. We also designed the violin to be continually useable in a traditional context such as an orchestra ensemble, rehearsal, or sight-reading session. We added all additional technology via removeable 3-D-printed clamps to preserve the original instrument's geometry (shown in Figure 1), and the sensing fingerboard does not alter the instrument's original dimensions (e.g., fingerboard thickness or string height). The instrument is physically robust and has easily replaceable sensing elements. In this article, we describe the instrument design, experiments in 3-D print filament, printer settings, string selection, and musical applications of the final instrument. The compositions for TRAVIS II highlight the instrument's affordances and draw a clear connection between the sensor data and the resultant sound.
Related Work
Miranda and Wanderley (2006) proposed four categories of new digital instruments: alternative gestural controllers, instrument-inspired gestural controllers, instrument-like gestural controllers, and augmented musical instruments. We position our work as a new augmented musical instrument. Augmented instruments are broken down into two groups (Overholt 2011). The first group are instruments that track traditional playing gestures and techniques. As Kimura (2003) mentions, foot pedals are unnatural for violinists because they require a gesture not typically used with the instrument. Augmented instruments that only track the performer's natural playing gestures are beneficial for minimizing distractions and unnatural body movements for the performer. An example of one such instrument is Institut de Recherche et Coordination Acoustique/Musique (IRCAM)'s Augmented Violin Glove (Bevilacqua et al. 2006; Bevilacqua, Baschet, and Lemoutonn 2012; Kimura et al. 2012; Baschet 2013). The second group are instruments that require the invention of new extended gestures, such as Dan Overholt's (2005) Overtone Violin. These instruments expand both the sonic possibilities of the instrument and the virtuosic possibilities because the instrumentalist is expected to develop new playing gestures.
Many augmented violins and cellos have focused on tracking the bow or the right hand: IRCAM's Augmented Violin Glove, the Electronic Violin Bow (Guettler, Wilmers, and Johnson 2008), the K-Bow (McMillen 2008), and the Hyperbow (Young 2002). Moreover, Todd Machover's Hypercello (Machover and Chung 1989; Machover 1992) and the Tracking System for Violin (Pardue, Harte, and McPherson 2015) contain components for tracking both left and right hands.
There are a few other methods of capturing violin gestures. One is through optical tracking, such as Schoonderwaldt and Demoucron's (2009) research in tracking both the left and right hands. Jensenius and Johnson (2012) took a different approach by using video cameras to track a violinist's position on stage instead of tracking the specific finger or bowing gestures. Another method is with electromyographic (EMG) sensors. Dalmazzo and Ramirez's (2017) Air Violin also tracked the left hand with optical tracking and an EMG sensor. Seth Thorn's (2019) Transference also used an EMG and gyroscope to track the left-hand finger and arm gestures, as well as an inertial measurement unit and flex sensors to track the right hand.
Augmenting the fingerboard of string instruments is an attractive approach. This can present many physical and technical challenges, however. In working on the Hypercello, Machover found that the cello strings were too abrasive for the thermoplastic attached to the fingerboard, so the instrument's calibration changed over time. In the Augmented Cello project, Freed et al. (2006, 2013) tried adding touch sensors to the fingerboard but found that the resulting data were nonlinear and that the data varied between strings, so they abandoned it. There was one augmented bass project, and another two augmented violin projects, that successfully added touch sensors to the fingerboard. Bahn and Trueman (2001) added a mouse touch pad to the fingerboard of the SBass, Grosshauser and Troester's sensor fingerboard had embedded FSRs (Grosshauser 2008; Grosshauser, Großekathoefer, and Hermann 2010; Grosshauser, and Troester 2013, 2014). The instrument by Grosshauser and coworkers was used to study how violinists play and when they were pressing too hard with their fingers. The copper colored sensors aesthetically stood out, however, and we speculate that violinists would not be able to continue to use the instrument in a traditional context. Pardue, Harte, and McPherson (2015) sought to solve this problem with a low-cost real-time tracking system for violin. The system intended to help students improve their intonation by providing feedback based on the sensors that track left-hand finger position, combined with pitch tracking. The touch-sensitive component of the system was made with Velostat on top of the fingerboard and a voltage running down the strings. Although the technology could be attached and removed from any violin, the string height had to be raised to ensure that the strings were clear of the Velostat when playing open strings. It is important to note that in a later iteration of the project, Pardue and colleagues (2019) made a new violin for beginner violinists that no longer relied on touch sensors to detect intonation errors. Instead, it used digitally automated pitch and tone correction, so the violin always played in tune.
Beyond improving upon the TRAVIS I design, our approach to augmentation is inspired both by Pardue and coworkers and by Grosshauser and Troester. We seek to address design challenges presented by the violin's geometry with a novel solution. We use strips of conductive 3-D print filament embedded into a 3-D-printed fingerboard to create touch sensors. These conductive strips are more resistant to damage than thermoplastic or Velostat, and individually replaceable if damaged. The instrument is aesthetically and geometrically close to a classical violin such that it can be played in a traditional context. TRAVIS II is primarily used to explore new creative, expressive, improvisational, and compositional possibilities in interactive music.
TRAVIS II Implementation
TRAVIS II is a design iteration of the TRAVIS I concept. In this section, we discuss the technical implementation of TRAVIS II and how it improves upon the TRAVIS I design. In addition to starting from a better violin model than TRAVIS I, the augmentation includes full-length sensors on all four strings and four FSRs instead of two.
The Fingerboard and Sensors
The aesthetic of the sensing fingerboard more closely resembles the aesthetic of a traditional fingerboard; both the regular and conductive PLA are black. Although some layer lines from fused deposition modeling printing are visible on the PLA surface, the texture of these lines against the fingertips does not distract from playing.
As documented by Leigh et al. (2012) and McGhee et al. (2018), conductive 3-D-print filament works well as the resistive component of touch sensors. Similar to the design by Pardue and coworkers (2015), TRAVIS II runs a voltage of 3.3 V down the strings; when a metal string comes in contact with a conductive strip, the string's voltage changes.
We also placed four small FSRs to a 3-D-printed clamp and mounted it on the right upper bout of the violin (shown in Figure 1 a). Here they are easily accessible to the performer; violinists rest their left hand on the right upper bout of the violin when not playing, and the FSRs are directly below the left hand when playing in high positions.
TRAVIS II uses an Arduino MKR1000 to send data to the software Max via OSC (Open Sound Control) messaging. TRAVIS II's Arduino is multiplexed for eight sensors and sits inside a custom 3-Dprinted case mounted on the left upper bout of the violin. This placement is based on the TRAVIS I design. We considered alternative placements for the Arduino. The Arduino could clamp onto the left lower bout, where the weight would be less noticeable on the shoulder. But this placement would obstruct the performer from bowing behind the bridge, an extended bowing technique. Alternatively, the Arduino could attach underneath the shoulder rest, as in Grosshauser and Troester's violin. Shoulder rests are not very secure, however, and many accidentally fall off midperformance. Placement near the shoulder rest would also limit how the shoulder rest can be adjusted; some performers would not be able to use their preferred shoulder rest height. By placing our Arduino and battery on the left upper bout of the violin, our added electronics do add some weight to the violin. We have found that this added weight is not uncomfortable, however, and the design keeps all components safely out of the way while playing.
The wires, the Arduino case, and the FSR clamp easily disassemble from the violin, both for storage in the case and when playing the violin in a traditional context. The wires connect to the Arduino via an insulation-displacement contact connector, to the 3-D conductive strips with Swiss machine headers, and to the strings with a JST (Japan Solderless Terminal) connector. The JST connector sits underneath the chin rest and the wires are soldered onto the balls of the strings. Strings are easily replaceable by cutting off the wires, then resoldering the wires to the new strings.
Results
Our initial tests ensured that TRAVIS II could work sufficiently as an augmented instrument, by producing a distinct range of values from the 3-D-printed touch sensors. We then tested the 3-D print settings, resistors, and strings to ensure that the instrument could produce reliable and stable data for the Arduino with the largest range of values between 0 and 1,023. We placed higher priority on the physical design, however. This included the sensor strips having dimensions that fit into the fingerboard, being durable so they do not break when assembling them, and having a smooth, high-quality finish on the surface.
We printed different dimensions of the conductive strips and used different settings for 3-D printing to see how these variables affect the strip's resistance. We tested the resistance with a multimeter at its 20-k setting. We recorded the resistance at three points along the strips. First, we measured the resistance across 1 cm of the strip at one end. Then, we measured the resistance from one end to the approximate center of the strip (about 13.2 cm). Finally, we measured the resistance at either end of the strip (about 26.4 cm). After we finalized the dimensions and the settings for the 3-D printer, we tested each of the final strips for the fingerboard one last time before integrating them with the violin.
Afterwards, we tested the range of analog data values read into the Arduino with various resistors in the circuit (1, 3.3, 4.7, 10, 22, and 47 k. We set up the circuit with the test strips, the strings, and resistors; then we recorded sensor values resulting from pressing the string at each end of the conductive stip. We selected final resistors based on which provided the largest range of sensor data.
The Conductive 3-D Print Filament
The dimensions of the strips are tightly constrained, particularly when fitting four into the geometry of a standard fingerboard. If strips are too wide, the slots are too large to fit into the fingerboard. If strips are too narrow, they could break while sliding them into the fingerboard, or the string may miss the strip surface when pressed.
We initially tested a conductive strip that was 265-mm long, 2.5-mm tall, 5-mm wide at the end closer to the bridge, and 4-mm wide at the end closer to the nut. However, when measuring its resistance, the resultant readings were not stable, and we could not record approximate measurements. The next test strip was narrower; consistently 3-mm wide at both ends, and produced a stable, measurable resistance value.
We tested different cross-sectional geometries to identify a series of four strips that both fit within the physical constraints of the fingerboard and had a relatively consistently varying resistance across the strip. We tested the resistance with a multimeter (20-k setting) at three distances from the nut: 10 mm, about 134 mm (middle), and approximately 268 mm (end to end). In our final strips (cross-sectional geometry in Figure 3), the resistance changed proportionally across the length of the strip (see Table 1). We tested the resistance again after ten months and found values close to our readings from our initial testing; thus, the conductive materials in the filament had not degraded over this time span.
String . | 1 cm (Close) . | 13.2 cm (Middle) . | 26.4 cm (Ends) . |
---|---|---|---|
G | 2.19 k | 6.67 k | 10.89 k |
D | 1.97 k | 8.25 k | 16.7 k |
A | 1.64 k | 6.63 k | 12.3 k |
E | 3.07 k | 7.55 k | 10.49 k |
String . | 1 cm (Close) . | 13.2 cm (Middle) . | 26.4 cm (Ends) . |
---|---|---|---|
G | 2.19 k | 6.67 k | 10.89 k |
D | 1.97 k | 8.25 k | 16.7 k |
A | 1.64 k | 6.63 k | 12.3 k |
E | 3.07 k | 7.55 k | 10.49 k |
Measurements taken at the 20k setting.
We also fine-tuned the length of the strips to ensure that they could easily be installed; if the end of the strip fit too firmly with the nut end of the slot, it was difficult to later remove. The final length was 268.5 mm; we are able to slip a sewing needle behind the strips at the nut end of the slot to push them out.
Settings for 3-D Printing
Once we had finalized the physical dimensions, we tested various slicing settings in Cura to see whether printing parameters affected resistance (https://ultimaker.com/software/ultimaker-cura). Cura is one of many programs that slices 3-D models and prepares them into a G-code file that can be read by a 3-D printer. We used the same process as above, measuring the resistance at three points along each strip.
First, we varied the layer heights from 0.1 mm to 0.5 mm. As the layer height of each test strip increased, the approximate resistance decreased (see Table 2). Also, the taller the layer height, the lower the resultant quality of the print. We decided to keep the layer height at 0.1 mm so that the header holes of the strips had the best print quality possible (shown in Figure 2), and so that the surface of the strips could be as smooth as possible.
Layer Height . | 1 cm (Close) . | 13.2 (Middle) . | 26.4 cm (Ends) . |
---|---|---|---|
0.1 mm | 2.55 k | 6.1 k | 11.6 k |
0.2 mm | 3.38 k | 6.5 k | 8.66 k |
0.3 mm | 1.27 k | 5.8 k | 7.7 k |
0.5 mm | 1.64 k | 4.06 k | 6.82 k |
Layer Height . | 1 cm (Close) . | 13.2 (Middle) . | 26.4 cm (Ends) . |
---|---|---|---|
0.1 mm | 2.55 k | 6.1 k | 11.6 k |
0.2 mm | 3.38 k | 6.5 k | 8.66 k |
0.3 mm | 1.27 k | 5.8 k | 7.7 k |
0.5 mm | 1.64 k | 4.06 k | 6.82 k |
We tested three different infill patterns—triangles, lines, and cubic—with a 70 percent infill density. The resistances of these patterns were not significantly different from one another. We did not test different infill densities because of strength concerns; the strips needed to slide in and out of the fingerboard without breaking.
By trial and error, we discovered that sanding the top surface of the conductive filament ruined its performance. In an effort to minimize the layer lines from 3-D printing, we had placed the strips into their slots in the fingerboard and then sanded the entire surface together. This damaged the conductivity and resistivity of the sanded surfaces of the strips, and there were barely any electrical readings from the multimeter. We reprinted the fingerboard and all strips after this discovery, and no longer attempted postprocessing the surface finish.
Testing Violin String Sensors
Each of the four strings on violins are made of different metals, and different brands of strings vary in the materials and manufacturing of the cores and windings. We had full sets of Dominant and Evah Pirazzi strings and a Pirastro Gold E. The metals from which they were made are listed in Table 3.
String Brand . | String Name . | Core . | Winding . |
---|---|---|---|
Pirastro Gold | E | (N/A) | Tin-plated carbon |
steel | |||
Dominant (135) | E 130 | Steel | Aluminum |
A 131 | Synthetic | Aluminum | |
D 132 | Synthetic | Aluminum | |
G 133 | Synthetic | Silver | |
Evah Pirazzi | E | (N/A) | Silvery steel |
A | Synthetic | Aluminum | |
D | Synthetic | Silver | |
G | Synthetic | Silver |
String Brand . | String Name . | Core . | Winding . |
---|---|---|---|
Pirastro Gold | E | (N/A) | Tin-plated carbon |
steel | |||
Dominant (135) | E 130 | Steel | Aluminum |
A 131 | Synthetic | Aluminum | |
D 132 | Synthetic | Aluminum | |
G 133 | Synthetic | Silver | |
Evah Pirazzi | E | (N/A) | Silvery steel |
A | Synthetic | Aluminum | |
D | Synthetic | Silver | |
G | Synthetic | Silver |
(N/A) not applicable.
We set up these strings in the sensor ci rcuit to test the range of data received from them with different resistor values (1 k, 3.3 k, 4.7 k, 10 k, 22 k, and 47 k. The Evah Pirazzi's range of data was sufficient, but we concluded that the Dominant pack, with the Pirastro Gold E, had the largest range of data (see Table 4). In its final configuration, the E string uses a 4.7 k resistor, and the rest of the strings have a 3.3 k resistor. None of the tested completed sensors reached their full range of 1,023 values. This is not necessarily a limitation, because the values are rescaled in Max.
. | . | Dominants with Pirastro Gold E . | Evah Pirazzi . | ||
---|---|---|---|---|---|
String . | Resistor . | Values . | Range . | Values . | Range . |
G | 3.3 k | 260–720 | 460 | 255–685 | 430 |
D | 3.3 k | 255–645 | 390 | 265–640 | 375 |
A | 3.3 k | 245–630 | 385 | 235–490 | 255 |
E | 4.7 k | 315–765 | 450 | 320–660 | 340 |
. | . | Dominants with Pirastro Gold E . | Evah Pirazzi . | ||
---|---|---|---|---|---|
String . | Resistor . | Values . | Range . | Values . | Range . |
G | 3.3 k | 260–720 | 460 | 255–685 | 430 |
D | 3.3 k | 255–645 | 390 | 265–640 | 375 |
A | 3.3 k | 245–630 | 385 | 235–490 | 255 |
E | 4.7 k | 315–765 | 450 | 320–660 | 340 |
Performative Applications
TRAVIS II is currently used as a controller for interactive compositions made with Max and is designed so that there is no need for an additional person to control the computer.
At the time of writing, the first author has composed and performed one solo work with TRAVIS II, “Dream State,” and one duet with both TRAVIS I and II, “Kindred Dichotomy.” The duet was performed by both authors at the Conference of the Alliance of Women in Media Arts and Sciences (AWMAS) held at the University of California at Santa Barbara on 7 February 2020.
The first author also composed and performed five pieces with the initial TRAVIS I prototype; because TRAVIS I did not have sensors under the D and A strings, these compositions can still be performed on TRAVIS II by ignoring the D and A string sensors in the Max patch. In this article we will discuss in detail “Dream State” and “Kindred Dichotomy” (viewable online at https://youtu.be/FxaaYOO3Knw and https://youtu.be/RuqMSzlXKJs, respectively).
Composers are able to generate a range of new expressive effects by utilizing the features of TRAVIS II. The four FSRs are primarily used to trigger presets for different sections of the composition. Because they give a range of data based on pressure, however, it is possible to use the FSRs to scrub through audio samples as well.
In addition to using finger position on the fingerboard to control sound processing parameters, different processes can be assigned to each individual string sensor; when a string is pressed, it routes the sound to its assigned process. This method of mapping different processes to each string was used in all current pieces for TRAVIS II.
TRAVIS II works particularly well with several specific violin techniques, such as fast passages and double stops. If the different effects are assigned to individual strings, double stops would simultaneously route the sound to two effects, or scrub through two prerecorded samples. This was used in the cadenza of “Dream State” and at the end of “Kindred Dichotomy.” To achieve combined effects, the violinist also does not necessarily have to play both strings with the bow; it is enough to keep the fingers on multiple strings while playing a melody with the bow on only one string. This single technique opens a wide range of possibilities, and compositionally this method is more interesting than simply assigning two sound processes to one string.
The Music of “Dream State”
“Dream State” is the first piece made for TRAVIS II. The majority of the piece is improvised, and the ending cadenza is precomposed. The improvisations are a slow legato and reverberation is added to achieve a dream-like effect. To link the piece together, some of the motifs played during the improvised sections are drawn from musical material written in the cadenza.
The piece begins with a pizzicato arpeggio that is recorded in real time and played back for the remainder of the improvised sections. The recording is turned off for the cadenza section, but the performer plays this arpeggio motif again at the very end, this time arco (with the bow) instead of pizzicato.
As mentioned earlier, different sound processes are mapped to each string sensor: The G-string sensor routes the sound through a flanger; the D-string sensor is mapped to a pitchshift module; the A-string sensor sound goes through a chorus; the E-string sensor to two consecutive delays that each range from 500 to 2,000 msec. The second delay is always delayed an additional 500 msec from the first. As the first TRAVIS II composition, its compositional goal is to draw attention to the changes in timbre on each string from these different processes. Therefore, before the cadenza, there are four improvised sections. Each of these sections are predominantly played on an individual string, starting on the G string, and with each consecutive section moving to the next string. At the beginning of each section, the performer presses an FSR to transpose the recorded pizzicato arpeggio and change the geometric shape in the visuals. Many glissandos are also used to show how the sound processes change along the length of the string sensors. Although the improvised sections show the individual timbres of each string, the cadenza exemplifies the unified sounds from when string sensors are activated together with double stops.
The Music of “Kindred Dichotomy”
“Kindred Dichotomy” is the first composition written for both TRAVIS I and TRAVIS II. As such, the piece demonstrates how well the two instruments could be used simultaneously. Aesthetically, “Kindred Dichotomy” explores the concept of two entities establishing their opposite identities, as well as discovering core similarities between them. There are motifs that characterize each violin, and in the middle of the piece the violins start to blend and quote each other's style. In the end, they return to their original motifs and style.
When composing “Kindred Dichotomy,” the hardware limitations and the resultant limitations on possible violin techniques were considered first. For TRAVIS I, the SoftPots are only under the G and E strings, and the lowest finger placements that activate the sensors are B-natural on the G string, and G-sharp on the E string. This limits the main melodic passages for TRAVIS I to be written only for the G or E strings with considerable shifting on these strings. TRAVIS II has fewer limitations for violin techniques because the full lengths of all four strings are touch sensors.
Given these limitations, the compositional goals were to support the abstract themes of “opposites and similarities” between two entities:
The piece should be idiomatic to the violin.
The piece should not be dependent on the electronics to sound cohesive. Instead, the electronics should augment and add to the expressivity.
Even though TRAVIS II's hardware is superior to TRAVIS I's, each violin should have equal musical importance. Therefore, they should each have their own melodic sections to establish the “dichotomy” between them. Each of these melodic sections has its own distinctive musical characteristic.
Countermelodies are used as much as possible. This is so each violin can maintain some of its unique characteristics, while the other has the melody.
The underlining similarity between entities is reflected in how the music does not modulate from its home key.
The musical form should be reflected in the visuals.
Even though “Kindred Dichotomy” has a precomposed score, improvisation still plays a key role in the compositional process. The first author internalized the limitations and goals, then improvised the melodies. The improvisations were transcribed and edited, then used as base material to write the accompaniment part above or underneath. The composition uses the pitches of the D harmonic minor scale, however, like a jazz mode, the pitches of the scale are rotated so that A is the tonic instead of D.
Unlike previous interactive pieces for solo TRAVIS I or solo TRAVIS II, the score to “Kindred Dichotomy” was written before programming the Max patches. This method allowed the electronics to augment the existing musical material instead of the composition relying on the electronic sound. It also made the interplay between the two violins sound more natural and musically complex.
Audio Visual Considerations
Both “Dream State” and “Kindred Dichotomy” include interactive visuals projected on screen. As elaborated by John Coulter (2010), aesthetically, there are two viewpoints when working with both music and visuals. One is that visual art and music can complement each other. The other is the acousmatic approach, in which visuals are distracting, and the ideal listening conditions are only met when sound is isolated from image. To prevent the visuals from being too distracting, the compositions with TRAVIS II avoid overly fast movement in the visuals.
The main goal in both compositions is to have the music and visuals synchronously linked. Momeni and Henry (2006) classify audio visual work into three categories based on how the sound and visuals are controlled: sound-to-image, image-to-sound, or concurrent generation of sound and image. Both “Dream State” and “Kindred Dichotomy” fall under the third category. The data from the touch sensors are used to control the sound and visuals concurrently; the sound does not directly affect the images and the data from the images are not mapped to control the sound.
Rehearsing “Kindred Dichotomy”
“Kindred Dichotomy” is the first piece performed on TRAVIS I by someone other than the first author. We devised a strategy to gradually introduce Oehlberg to TRAVIS I and the composition.
The first rehearsal served the dual purpose of proofreading the composition and allowing us to understand how the instrumental parts feel when played together. As such, we used our own violins without any augmentation. In the second rehearsal we practiced with TRAVIS I and II with all of the electronic hardware clamped to the bodies of the instruments. The instruments were not turned on, however, and the computer, loudspeakers, and projector were not set up. This rehearsal was aimed to practice the newly revised version of “Kindred Dichotomy” while Oehlberg was also being introduced to the aural and kinesthetic differences of TRAVIS I from her personal violin. The TRAVIS I violin itself is a low-quality instrument; it takes time for the ear to adjust to the different acoustic timbre and it requires more effort to create a clean sound. The SoftPots also feel slippery in comparison to the typical ebony fingerboard. Although the SoftPots' different texture does not affect playing technique, it is another unusual sensation that the performer is aware of while also becoming accustomed to everything else about the instrument that is different. In this rehearsal, excluding the computer and loudspeakers allowed us to continue focusing on the newly written music without feeling distracted or overwhelmed by the electronic sounds. In the third rehearsal all of the electronics and computer was set up.
Discussion: Electronics
TRAVIS II achieved its goal in augmenting a violin and improved upon TRAVIS I's design. In comparison to TRAVIS I, the violin used to make TRAVIS II is a much higher-quality violin model. Before augmentation took place, based on the first author's subjective standards, the TRAVIS I violin would only be used by a beginner student. In comparison, the TRAVIS II violin would have been considered suitable for intermediate to lower-advanced levels. It would have been suitable for playing in an orchestra or middle-level examination. It would not have previously been used for situations such as a professional solo recital or high-level examination. TRAVIS II has more FSRs available to change settings and all four strings can be tracked. It expands the composer's available palette of timbres, effects, and recorded samples.
One of the main limitations of augmenting a cello with touch sensors is that the strings are large and thick, therefore any resistive film placed on top of the cello fingerboard is more receptive of damage than on a violin. We wonder if this method of augmentation with conductive 3-D print filament can be successfully applied to a cello, and other instruments of the violin family. A much larger 3-D printer would be needed.
There are some limitations to the sensor results and TRAVIS II's design. Only packs of Dominant and Evah Pirrazi strings, plus a Pirastro Gold E string, were tested. Also, only one brand of conductive PLA, by ProtoPasta, was in stock at the time of purchase. A more thorough study would include testing multiple brands of strings and conductive filament. Other brands of conductive filament have different magnitudes of resistivity, and therefore could provide different sensor data (McGhee et al. 2018).
Printing only the top half of the fingerboard and sitting it on top of an ebony piece greatly improves the overall strength. As a filament for fused deposition modeling printers, PETG is stronger than acrylonitrile butadiene styrene or PLA (for a comparison of the materials, cf. Kondo 2019). We found the PETG version was not noticeably stronger on the violin, it did not hide the layer lines more than the PLA, and it was much shinier. Therefore, we continue using the PLA model. Layer lines are visible, and we speculate that a resin filament would make a cleaner-looking surface. Unfortunately, we did not have a resin 3-D printer available.
The objective of the project was to construct an instrument that serves the first author's needs as a violinist. These include being able to remove all of the technology, with exception of the 3-D-printed fingerboard and hidden JST connector for the strings, to be able to play the violin in a traditional context. There are still a few subtle variations that differentiate TRAVIS II from a traditional violin geometry. For example, the traditional fingerboard has an undercurve. The small PLA header piece does have an undercurve (seen in Figure 2), but the rest of the ebony piece is flat. Visually, this difference is not noticeable from the audience's perspective, so the violin can continue to be played as a conventional acoustic violin. The flat ebony piece does slightly affect the tone of the violin, making it sound tinnier. The tinny quality contributes to why we would not recommend playing this violin as a solo acoustic instrument, although it may still be suitable in an orchestral setting.
Although the Max patch shown in Figure 4 can measure the speed of data to recognize when fingers are playing fast gestures, it cannot differentiate vibrato from other fast gestures (e.g., shifts, glissandos, trills). It also only recognizes where it was pressed, and not which finger is pressing down. Finger recognition could be achieved with EMG sensors (Dalmazzo and Ramirez 2017; Thorn 2019), or with a fingertip-free glove with flex sensors on each finger. Vibrato recognition may also be more accurately achieved with an inertial measurement unit on the hand (Thorn 2019), or flex sensors on the wrist, and elbow. These design concepts may be considered for future research.
Reflection: Compositions
“Dream State” is a composition meant to premiere TRAVIS II. The slow, reverberant nature of the piece, with only scant compositional instructions, made it easy to improvise and perform with few rehearsals beforehand. It is simple in both its musical form and mappings to effectively exemplify TRAVIS II's technical affordances, but still allows space for performers to showcase their virtuosity.
Compositionally, “Kindred Dichotomy” posed a challenge because it involved two violins sharing similar sonic qualities and timbres. Therefore, the processed sound coming out of the loudspeakers blends together seamlessly, making the sound of the two instruments less distinct from each other. Even though each violin has a direct one-to-one mapping of its sensor data to sound processes, this mapping was not as perceptible as in other solo works for either TRAVIS I or II. If it were two instruments of different orchestral families, the resultant processed sound from each instrument would likely be more distinctive. This led to panning each violin's sound to different loudspeakers to accentuate distinction between them. In the future, any duets for TRAVIS I and II, or of instruments of the same musical family, should involve more interplay between instruments and solos with sparse accompaniment. In the middle of “Kindred Dichotomy,” instead of both violins affecting different parameters of a single shape, each violin was assigned to control its own half of the kaleidoscope-like image. This led to a more perceptible understanding for how the violins were mapped to the visuals.
Interactive computer music is commonly performed via improvisation with the computer, because computers are capable not only of analyzing gestures but also of generating new music and responding to performers' gestures. As Bresson and Chadabe (2017, p. 1) note,
In the context of computer music, the question becomes one of how a computer might be programmed to generate audio as well as compositional information that is flexible enough in its reaction to a performer to qualify as interactive. Interaction, at whatever level, refers to a mutually influential process.
Improvisation is not commonly taught to classically trained violinists, or to many other classically trained musicians, however. Through the rehearsal process of “Kindred Dichotomy,” we speculate that slowly introducing a performer to a new digital musical instrument with a score is a good pedagogical approach before asking the performer to improvise. Understanding how instrumentalists adapt to interactive performance was not the intent of “Kindred Dichotomy,” however, and for conclusive results in this particular area, a proper study with multiple participants should be conducted.
Overall, TRAVIS II achieved its design goals, and at the time of writing this article, it has been used in two successful interactive compositions. It is an example of a new method to track left-hand finger gestures through conductive 3-D print filament and encourages speculation on how this method could be applied to other string instruments.
Acknowledgments
We would like to recognize Aaron Pratte, the luthier who detached the original fingerboard of TRAVIS II and made the ebony piece of the modified fingerboard. The wireless version of TRAVIS I was a collaboration of the first author with Robert Pritchard and the engineering team of WiRED: Jin Han, Esther Mutinda, Carol Fu, and Lily Shao.