This article discusses the challenges in creating Mutator VR: Vortex , a virtual reality experience based on interaction with semi-autonomous, organically inspired agents. The work allows the immersant to morph between a vast number of procedurallygenerated microworlds, each with its own visual elements, sounds, agent dynamics, and user interactions. We outline two methods used for procedural generation that are based fundamentally on integration of different modalities. Curve-based synthesis is used for simultaneous generation of entity sounds and shape and flow grains are employed to determine both agent dynamics and user interaction with the agents.