We report here the processes involved in creating our entry in the 2020 AI Song Contest, “Beautiful the World”; the technical innovations from the project; and the decision-making that divided tasks between human and machine in a way that ensured that the final creation was AI-inspired but human-created, starting from generated melodies, lyrics, and timbres. Key innovations include the use of lyric stress patterns as queries to a stress-based melody index to a database of generated melodies, and the creation of a novel instrument timbre with differential digital signal processing, trained on Australian animal calls. We reflect on how human–AI cocreativity occurred during the process and how it may develop in the future.

This article introduces an open-source Java-based programming environment for creative coding of agglomerative systems using Internet-of-Things (IoT) technologies. Our software originally focused on digital signal processing of audio—including synthesis, sampling, granular sample playback, and a suite of basic effects—but composers now use it to interface with sensors and peripherals through general-purpose input/output and external networked systems. This article examines and addresses the strategies required to integrate novel embedded musical interfaces and creative coding paradigms through an IoT infrastructure. These include: the use of advanced tooling features of a professional integrated development environment as a composition or performance interface rather than just as a compiler; techniques to create media works using features such as autodetection of sensors; seamless and serverless communication among devices on the network; and uploading, updating, and running of new compositions to the device without interruption. Furthermore, we examined the difficulties many novice programmers experience when learning to write code, and we developed strategies to address these difficulties without restricting the potential available in the coding environment. We also examined and developed methods to monitor and debug devices over the network, allowing artists and programmers to set and retrieve current variable values to or from these devices during the performance and composition stages. Finally, we describe three types of art work that demonstrate how the software, called HappyBrackets, is being used in live-coding and dance performances, in interactive sound installations, and as an advanced composition and performance tool for multimedia works.

We present and discuss the Agent Designer, a system that enables users of digital audio workstations to generate novel high-level structures for their compositions based on previous examples. The system uses variable-order Markov models and rule induction to learn both temporal relations and structural relations between parts in a piece of music. As is usual in machine learning, however, the quality of the learning can be improved greatly by users specifying relevant features. The Agent Designer therefore points to important design and human–computer interaction problems, as well as algorithmic challenges. We present a number of studies that help to understand how effective the Agent Designer is and how we might design a user interface that best enables users to obtain quality results from the system. We show that the Agent Designer is effective for certain musical styles, such as loop-based electronic music, and that we as expert users can design agents that produce the most effective results. We also note that it remains a challenge to automate this process fully.