This article presents design and performance practices for movement-based digital musical instruments. We develop the notion of borrowed gestures, which is a gesture-first approach that composes a gestural vocabulary of nonmusical body movements combined with nuanced instrumental gestures. These practices explore new affordances for physical interaction by transferring the expressive qualities and communicative aspects of body movements; these body movements and their qualities are borrowed from nonmusical domains. By merging musical and nonmusical domains through movement interaction, borrowed gestures offer shared performance spaces and cross-disciplinary practices. Our approach centers on use of the body and the design with body movement when developing digital musical instruments. The performer's body becomes an intermediate medium, physically connecting and uniting the performer and the instrument. This approach creates new ways of conceptualizing and designing movement-based musical interaction: (1) offering a design framework that transforms a broader range of expressive gestures (including nonmusical gestures) into sonic and musical interactions, and (2) creating a new dynamic between performer and instrument that reframes nonmusical gestures—such as dance movements or sign language gestures—into musical contexts. We aesthetically evaluate our design framework and performance practices based on three case studies: Bodyharp, Armtop, and Felt Sound. As part of this evaluation, we also present a set of design principles as a way of thinking about designing movement-based digital musical instruments.

ChucK is a programming language designed for computer music. It aims to be expressive and straightforward to read and write with respect to time and concurrency, and to provide a platform for precise audio synthesis and analysis and for rapid experimentation in computer music. In particular, ChucK defines the notion of a strongly timed audio programming language, comprising a versatile time-based programming model that allows programmers to flexibly and precisely control the flow of time in code and use the keyword now as a time-aware control construct, and gives programmers the ability to use the timing mechanism to realize sample-accurate concurrent programming. Several case studies are presented that illustrate the workings, properties, and personality of the language. We also discuss applications of ChucK in laptop orchestras, computer music pedagogy, and mobile music instruments. Properties and affordances of the language and its future directions are outlined.

Ocarina, created in 2008 for the iPhone, is one of the first musical artifacts in the age of pervasive, app-based mobile computing. It presents a flute-like physical interaction using microphone input, multi-touch, and accelerometers—and a social dimension that allows users to listen in to each other around the world. This article chronicles Smule's Ocarina as a mobile musical experiment for the masses, examining in depth its design, aesthetics, physical interaction, and social interaction, as well as documenting its inextricable relationship with the rise of mobile computing as catalyzed by mobile devices such as the iPhone.