Over the last five years, KIMA , an art and research project on sound and vision, has investigated visual properties of sound. Previous iterations of KIMA focused on digital representations of cymatics—physical sound patterns—as media for performance. The most recent development incorporated neural networks and machine learning strategies to explore visual expressions of sound in participatory music creation. The project, displayed on a 360-degree canvas at the London Roundhouse, prompted the audience to explore their own voice as intelligent, real-time visual representation. Machine learning algorithms played a key role in meaningful interpretation of sound as visual form. The resulting immersive performance turned the audience into cocreators of the piece.

ABSTRACT Transmission is both a telepresence performance and a research project. As a real-time visualization tool, Transmission creates alternate representations of neural activity through sound and vision, investigating the effect of interaction on human consciousness. As a sonification project, it creates an immersive experience for two users: a soundscape created by the human mind and the influence of kinetic interaction. An electroencephalographic (EEG) headset interprets a user’s neural activity. An Open Sound Control (OSC) script then translates this data into a real-time particle stream and sound environment at one end. A second user in a remote location modifies this stream in real time through body movement. Together they become a telematic musical interface-communicating through visual and sonic representation of their interactions.

ABSTRACT KIMA is a holographic surround-sound installation that visualizes telepresence as both a phonetic and a synaesthetic phenomenon. The performance piece is based on the physical conditions of cymatics-the study of physically visible sound wave patterns. Two environments, a quad surround and a holographic interface, build the framework of a telematic experience that illustrates communication as wave forms while focusing on the relationship between sound and matter.