This paper analyzes the integration of East Asian literati traditions with contemporary multimedia through two artworks. It proposes a multimodal transcoding framework converting video frames into multisensory outputs via data-driven visualization and sonification. Using a modified turntable, the system interprets an artificial marble disc, created from the artist’s daily creation, to depict mental space using sumi ink. This aligns with literati traditions of expressing inner worlds through landscape paintings as contemplative practice. The study explores how translating these embodied practices into multimodal manifestations offers new perspectives in new media art while broadening the representations of landscape in an intercultural context.

The authors present gOd, mOther and sOldier—Nowhere in Somewhere Series 2022 , a work that was conceptualized and created by artist Jinjoon Lee and his TX Creative Media Lab at KAIST, realized through the remote cooperation of eight local collaborators across Southeast Asia. The authors used artificial intelligence–based object detectors and sonification techniques in a work of media art to symbolize the voicelessness of those at the margins of society in Southeast Asia. These algorithms and concepts, and the work as a whole, artistically demonstrate how marginalized people are misrepresented and misunderstood when interpreted out of context.