Virtual and augmented reality technologies are increasingly utilized by public-facing galleries, libraries, archives, and museums (GLAM) institutions to convey both heritage and contemporary stories, information, and experiences. However, within these media, the visitors are often overwhelmed by the virtual environment, resulting in the cultural heritage content being pushed to the background. In this article, we present the alternating reality (AltR) narrative that supports the communication of cultural heritage as an interweaving experience that alternates between real and virtual environments. AltR allows both the cultural heritage content and digital technology to play an equal role in augmenting, highlighting, or explicating each other and conveying a shared narrative to the visitors of GLAM institutions. We introduce the AltR experience through an exhibition highlighting the physical replica and contemporary reimagining of one of Walter and Marion Griffin’s buildings. We conducted a study of the AltR experience during the exhibition using a low-cost virtual reality cardboard viewer that lends itself to an intuitive transition between the two environments. The findings showed that the interweaving experiences between the physical replica of cultural heritage and its virtual reimagining allows the visitors to draw the connection between the two sources of information. The narrative is created and controlled by the visitors, leading to an enjoyable experience. Finally, we reflect on the design implications of an AltR narrative for cultural heritage applications derived from the findings of the study.