Innovation and tradition are two fundamental factors in the design of new digital musical instruments. Although apparently mutually exclusive, novelty does not imply a total disconnection from what we have inherited from hundreds of years of traditional design, and the balance of these two factors often determines the overall quality of an instrument. Inspired by this rationale, in this article we introduce the Hyper Drumhead, a novel augmented virtual instrument whose design is deeply rooted in traditional musical paradigms, yet aimed at the exploration of unprecedented sounds and control. In the first part of the article we analyze the concepts of designing an augmented virtual instrument, explaining their connection with the practice of augmenting traditional instruments. Then we describe the design of the Hyper Drumhead in detail, focusing on its innovative physical modeling implementation. The finite-difference time-domain solver that we use runs on the parallel cores of a commercially available graphics card and permits the simulation of real-time 2-D wave propagation in massively sized domains. Thanks to the modularity of this implementation, musicians can create several 2-D virtual percussive instruments that support realistic playing techniques but whose affordances can be enhanced beyond most of the limits of traditional augmentation.

In this article we describe a user-driven adaptive method to control the sonic response of digital musical instruments using information extracted from the timbre of the human voice. The mapping between heterogeneous attributes of the input and output timbres is determined from data collected through machine-listening techniques and then processed by unsupervised machine-learning algorithms. This approach is based on a minimum-loss mapping that hides any synthesizer-specific parameters and that maps the vocal interaction directly to perceptual characteristics of the generated sound. The mapping adapts to the dynamics detected in the voice and maximizes the timbral space covered by the sound synthesizer. The strategies for mapping vocal control to perceptual timbral features and for automating the customization of vocal interfaces for different users and synthesizers, in general, are evaluated through a variety of qualitative and quantitative methods.

TouchNoise is a multitouch interface for creative work with noise. It allows direct and indirect manipulation of sound particles, which are added together in a panning and frequency space. Based on the mechanics of a multiagent system and flocking algorithms, novel possibilities for the creation and modulation of noise and harmonic spectra are supported. TouchNoise underwent extensive revisions and extensions throughout a three-year, iterative development process. This article provides a comprehensive overview of the final TouchNoise concept and its approach to mapping and interaction, from which a variety of unique sonic capabilities derives. This article is based on our experiences with a fully functional prototype implementation, and focuses on the systematic exploration and discussion of these new sonic capabilities and corresponding playing techniques, which differ strongly from traditional synthesis interfaces.

Motion-sensing technologies enable musical interfaces where a performer controls sound by moving his or her body “in the air,” without touching a physical object. These interfaces work well when the movement and resulting sound are smooth and continuous, but it has proven difficult to design air instruments that trigger discrete sounds with precision that feels natural to performers and allows them to play rhythmically complex music. This article presents a study of “air drumming” gestures. Participants performed drumming-like gestures in time to simple recorded rhythms. These movements were recorded and examined to look for aspects of the movement that correspond to the timing of the sounds. The goal is to understand what we do with our bodies when we gesture in the air to trigger a sound. Two movement features of the hand are studied: Hits are the moment where the hand changes direction at the end of the striking gesture, and acceleration peaks are sharp peaks in magnitude acceleration as the hand decelerates. Hits and acceleration peaks are also detected for the movement of the wrist. It is found that the acceleration peaks are more useful than the hits because they occur earlier and with less variability, and their timing changes less with note speed. It is also shown that timing differences between hand and wrist features can be used to group performers into different movement styles.

The keyboard is one of the most popular and enduring musical interfaces ever created. Today, the keyboard is most closely associated with the acoustic piano and the electronic keyboards inspired by it, which share the essential feature of being discrete: Notes are defined temporally by their onset and release only, with little control over each note beyond velocity and timing. Many keyboard instruments have been invented, however, that let the player continuously shape each note. This article provides a review of keyboards whose keys allow continuous control, from early mechanical origins to the latest digital controllers and augmented instruments. Two of the author’s own contributions will be described in detail: a portable optical scanner that can measure continuous key angle on any acoustic piano, and the TouchKeys capacitive multi-touch sensors, which measure the position of fingers on the key surfaces. These two instrument technologies share the trait that they transform the keys of existing keyboards into fully continuous controllers. In addition to their ability to shape the sound of a sustaining note, both technologies also give the keyboardist new dimensions of articulation beyond key velocity. Even in an era of new and imaginative musical interfaces, the keyboard is likely to remain with us for the foreseeable future, and the incorporation of continuous control can bring new levels of richness and nuance to a performance.

The technology of depth cameras has made designing gesture-based augmentation for existing instruments inexpensive. We explored the use of this technology to augment keyboard performance with 3-D continuous gesture controls. In a user study, we compared the control of one or two continuous parameters using gestures versus the traditional control using pitch and modulation wheels. We found that the choice of mapping depends on the choice of synthesis parameter in use, and that the gesture control under suitable mappings can outperform pitch-wheel performance when two parameters are controlled simultaneously.

This article presents alternative ways to produce efficient custom force or pressure sensors using cellulose paper filled with carbon-black pigments. We have produced several prototypes of force sensors to observe and compare their response with applied forces with those of commercial sensors, as well as those of other conductive materials. Advantages of custom sensors are presented through the description of various examples of music controllers using several kinds of paper touch sensors with a large variety of size and shape. Our goal is to provide digital musical instrument designers with new strategies to circumvent the limited offering of commercial sensors, and to propose ways to exploit these concepts through a dedicated Web site.