The Pickrick Protests is a project exploring the use of Augmented Reality (AR) to recreate a sequence of Civil Rights--era incidents that occurred at the Pickrick restaurant, in Atlanta, over a period of 8 months in 1964–1965. This project gives a connected and chronological narrative of the historic events to form a cohesive experience, with AR projections of computer graphic depictions of the 1960s buildings, and events on the site where they happened. The objective of this project is to make the history of the Civil Rights Movement visible by privileging the perspective of the African Americans who fought for equality. This paper focuses on introducing the narrative design for this location-based AR application (Bowman et al., 1998). We will discuss our approach to creating a meaningful narrative within the technical affordances and environmental constraints of the AR medium and the specific location. The design challenges posed by this project were how to merge the virtual content with the existing, albeit changed, physical space; how to recenter the story on the activists rather than on the segregationist restaurant owner who went on to become a governor of Georgia; and how to direct attention in AR in order to allow interactors to follow a multiepisode narrative, and to focus on the nonviolent activists rather than the aggressive segregationists.

The Pickrick Protests project is an interactive Augmented Reality (AR) mobile experience located at Georgia Institute of Technology's (Georgia Tech's) EcoCommons site in Atlanta. This project aims to commemorate the African American activists who engaged in a series of desegregation actions from 1964 to 1965 that would eventually uphold the Civil Rights Act of 1964 (Zhao et al., 2021). Our team consists of a faculty director, a project manager, a historical researcher, three programmers, and a 3D modeler, and has benefitted from direction from two specialist historians, Dr. Todd Michney and Dr. Karcheik Sims-Alvarado. The project investigates AR storytelling as a means of making history visible.

AR is an emerging technology that combines real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects (Wu et al., 2013). In our project, we use these affordances with system and narration design strategies of AR to integrate virtual content into physical environment and highlight African American activists during the Pickrick Protests. The project represents multiple separate incidents in a concerted campaign to enforce the Civil Rights Act of 1964 by forcing the notoriously segregated Pickrick restaurant to serve African Americans. The project is aimed at highlighting the notable people involved in the legal case, whose names are often omitted from accounts of the events at the site. By naming the people involved, it also clarified the political landscape just prior to the actions and the role of the Atlanta historically Black colleges and universities (HBCUs) in supporting Civil Rights actions. One design challenge is to present these separate incidents in a connected and chronological order to provide a coherent experience, as opposed to a collection of disconnected stories. Another challenge is to marry the past events to the current, very different physical environment, to present site visitors with a compelling experience of “time travel” to witness the historical events in the place where they happened.

The significance of this project lies in its use of emerging media to create site specificity to the EcoCommons site that will inform students of the history that occurred at what is now part of Georgia Tech's campus. As a research project in narrative design, we are exploring how to present history in location-based installations drawing on archival elements from news sources of the past, allowing participants to gain a deeper understanding of the present-day world around them. Unlike AR applications in museums and historical buildings, this application requires additional considerations because the site is outside. In addition, although the site is memorialized as part of the EcoCommons, the theme of the area is in the landscaping, and the historical footprint, which is a small part of the larger park-like area, has been obscured. The EcoCommons space where the original actions took place is marked by a plaque that originally did not include the names of the Civil Rights activists and has three abstract pillars meant to symbolize the “three ministers,” who were on the central lawsuit. Our research revealed four ministers who were key actors in the demonstrations, as well as a network of other activists who provided support for their actions. In most printed accounts of the events and in the original Georgia Tech plaque, the focus is on the white segregationist future governor who owned the restaurant, and the activists are presented only generically as “ministers” or “negros.” In the surviving television coverage, the microphones are pointed at the segregationist, not the activists. Given this, a major part of our effort has been not just to make history visible, but to recenter it on the collective battle for Civil Rights rather than the career of a particular prominent segregationist.

The physical building where the Pickrick restaurant once stood no longer exists and has been replaced with the EcoCommons, which is a new location that is meant to serve multiple functions to support students’ learning, quiet reflection, and gathering in a nature setting. The educational aspect of the EcoCommons space is multipurpose. Much of the information is about sustainable plants and the intention to make the urban setting of campus greener. One of the tertiary functions of the EcoCommons space is to commemorate the protests at the Pickrick. This area includes the aforementioned symbolic pillars and signage that offer a small snippet of information about the significance of the location.

The layout of the AR experience is constrained and potentially disrupted by installed elements that serve the primary parklike function of the space: trees, benches, and narrow pathways. These elements restrict our design choices, challenging us to integrate the AR content in a coherent manner and to ensure the safety of users. Considerations of safety by way of preventing users from walking in the main road and walkways, as well as bumping into physical structures while concentrating on the AR content, became key when geotagging the location. While the old Pickrick was oriented towards the main road and easily accessed through a parking lot, the current site is enclosed by low barrier walls and plantings that restrict access to the sidewalk. Since one of the incidents we are capturing involves an attack on the activists’ car in the street, we made the decision to reproduce the street area which is external to the site within the barrier walls to facilitate a safer interactive experience. Videos that are available at the beginning of the AR experience remind users to be mindful of their safety and of other users when walking around the site. For research purposes, we are using temporary markings, but our design assumes that these could be transferred to permanent markers to enhance safety and to allow us to predict the point of view of the interactors.

This paper's primary contribution is a demonstration of our design ideas and process for historic narratives in AR, considering both virtual and real elements. Elements that are the most critical include how to choose and organize historic archival resources and exhibit them virtually, utilize the physical surroundings, and blend visual and auditory narration to emphasize the protagonists (activists) in the story over the antagonists (segregationists). More importantly, the events of 1964 at the restaurant itself are only part of the story. How does a site-specific installation tell visitors about events that happened off-site? How do we introduce the key people, especially the four ministers, Willis, Dunn, Wells, and Lewis, and the historic off-stage presence of Constance Baker Motley?  Here, as we discuss later, the site itself offered a solution, in the form of walls that enclose the area.

We also will analyze the physical characteristics and historical context of the location as they relate to theories and our specific research efforts. Then, we will go over our design considerations where we will describe the entire design process, challenges, and our solutions. In the concluding section, we will discuss how AR narratives can influence future digital heritage and historical teaching.

2.1 Related Works

Our design process started with analyzing projects that focus on race and history in AR/VR areas. I AM A MAN is an interactive VR experience that allows players to participate in the 1968 Memphis sanitation workers’ strike and the events before the assassination of Dr. Martin Luther King Jr. (Ham, 2018). Here, interactors take on the role of a Black sanitation worker and are transported between scenarios, such as a kitchen with a TV playing Dr. King's speech and a newspaper displaying the current date; a wall with archival audio, images, and short vignettes from that time; a street filled with protesting Black people; and near the motel where Dr. King was assassinated. This project reinforced our interest in incorporating archival materials and recreating scenarios. We also found that the use of original audio enhanced the feeling of immersion within the experience.

The AMP'UP SEATTLE application is a project to preserve Black Lives Matter artwork using AR (Melnick, 2020). The designers of the architectural and design firm GGLO created an AR display as an homage to the diverse collection of street art produced during the height of the Black Lives Matter movement. Users can use their smartphones to explore Seattle in search of new art, and according to their location, they will see a different BLM-themed virtual picture. This project inspired us to combine location and race as the centralizing theme to the AR experience.

The United States Holocaust Memorial Museum utilizes AR to bring history to life (Takahashi, 2019). Visitors to the museum can use their smartphones to scan the images on the wall, and additional information will appear in AR, such as a brief introduction to the experience, and the names of the people who were killed during the Holocaust, to honor the past and make it visible. This project is an example of installations that incorporate historic details alongside the AR content. These specific documentary details give more context to the larger events, and therefore enhance the AR storytelling experience.

We learned about additional AR guide applications in historic sites. In The Voices of Oakland, an XR experience in Oakland Cemetery (Dow et al., 2005), writers have demonstrated how the physical surroundings and media affordances shape the narrative's content and structure. They use voices of historical figures in conjunction with the cemetery setting to create an immersive XR experience. Dramatic narrative, which considers telling stories through characters and plot, is brought to life by voice actors who narrate stories about the lives of cemetery residents. In addition, Kleftodimos et al. (2023) have developed location-based, educational AR applications that utilize gamification and storytelling to provide cultural heritage knowledge about a prehistoric lake settlement.

For much of the work of our group, we were inspired by the immersive journalism work of Nonny de la Peña, including her original project, Hunger in LA, which combines computer-generated images with authentic sound to immerse the interactor in a scene. In addition, de la Peña's work is notable for its dramatic compression, its focus on a particular moment of engaging action, to which the viewer of the VR experience is given privileged access. Although de la Peña works in VR rather than AR, her combination of computer graphics and real audio and her focus on moments of dramatic action provide useful design guides.

2.2 Related Theories

Multiple theories influenced our narrative design and helped us comprehend the affordances of the AR medium, especially the theories of interactive digital narrative (IDN). Janet Murray's work (2011a, 2011b) identifies four key affordances of the digital medium and emphasizes dramatic agency, the matching of interaction patterns with narrative curiosity and reward, as the goal of interactive narrative design. Our AR experience exploits all of Murray's four affordances of digital media. It is procedural and participatory, creating the experience of agency by offering interactors control over which incident is displayed and what angle they view it from. It is immersive in its attempt to provide detailed “encyclopedic” layers of information about this sequence of events, and most of all it is spatial in its linking access to the history to navigation of the space, turning the present day actual EcoCommons into an “information repository and/or a virtual place” (Murray, 2011b). Based on this framework, certain design questions were proposed to aid in the design process, such as how should we organize historical materials to help the users comprehend the causality of Pickrick events? What is the best way to incorporate interactivity into the AR story? Where should different media be integrated into AR to enrich the narrative? How can physical space be utilized to generate a spatial narrative?1

We also examined the application of IDN to AR design by examining Koenitz's (2015) SPP (system, process, and product) model that identifies the components of interactive narrative design. “System” is the artifact containing multiple narratives; “process” is the interactive engagement; and “product” is the playthrough recording (Koenitz, 2015). This model helps us organize the narrative and determine how to develop interactions that engage the audience with the story. In contrast to the traditional media's narrative style, we are attempting to employ AR to construct a series of historical events, engage the audience as curious witnesses within the space of the events, and replay it with multiple perspectives, particularly from the perspective of African Americans.

Furthermore, because IDN has the potential to express complicated topics, Koenitz has created an approach for depicting complex histories in IDN: The first step is to consider assets and complications. Our historical research included biographical and historic books, archives from newspapers, archived videos, etc. We then attempted to discover the most dramatic events in history and recreate them in 3D, analyzing how to construct the narrative so that the causality between each event is evident. We chose Jack Googer (the first African American to enter the Pickrick restaurant peacefully) to close our story since his entry into the space represents the end of segregated eating at this site, and thus the concrete local result of the political activism. The second stage is to identify the complexity mechanisms. More specifically, in our project, we continue to investigate how to exploit AR's affordances to provide interactors more viewpoints, and particularly to look for ways to put interactors in the perspective of the nonviolent activists exhibiting discipline despite the provocations of the aggressive segregationist crowd. Third, we considered which type of IDN works best for our experience. According to Koenitz, there are seven types, including parallel perspectives, stakeholder, alternate or fictional realities, decision maker, data narrative, and behind-the-scenes (Koenitz, 2021). To show what happened in front of the Pickrick restaurant, we used the “parallel perspective.” We 3D-modeled the entire event and allowed the interactor to view it from various perspectives. Based on our design, the interactor can choose to sit with the three activists in the car to experience Lester Maddox's hitting the car with a bat, and at the end, the interactor can even sit alongside Jack Googer.

We also investigated Mads Haahr's (2017) thesis on how to apply locative game mechanisms to cultural heritage. The author distinguishes between standard locative game design and cultural heritage game design. First, it is essential to develop navigational elements, such as maps and radars; second, photorealistic visuals and audios assist in blurring the boundary between the actual and virtual worlds and enhance immersion; and third, it is best to use a coherently sequenced narrative structure. Based on these three ideas, we aligned the placement of the model with the existing street and the original footprint of the restaurant and provided historical images to help interactors map from the virtual to the historical restaurant. We also constructed realistic models and, wherever possible, we drew on authentic historical archived documents. And finally, we are making clear the historical sequence of events.

The EcoCommons is a physical site, one which no longer holds the presence of the old Pickrick location or any aura of any physical building that would connote time and space. Because of this, the idea of aura became important in the construction of the project. To that end, we drew on Jay Bolter's work on Presence and the Aura of Meaningful Places (MacIntyre et al., 2002) as a theoretical framework to guide how we would create aura at this location. The authors of the research use Benjamin's classic “aura” to differentiate between the AR and VR mediums (Benjamin, 1968). The aura of an object or place is the “combination of its cultural and personal significance for a user or group of users,” and “provides an important complement to presence and can enrich our understanding of users’ responses to a variety of different computer-mediated experiences, especially in cultural heritage sites.” The writers feel that virtual reality lacks aura and is therefore not as compelling an experience in the physical location. (See North of 41, 2018, for a discussion of the differences between AR, MR, VR, and XR.)

Aura can be used to improve a user's experience with a mixed reality (MR)2 to make it more engaging or educationally rewarding (MacIntyre et al., 2002). If designers wish to increase aura, they must consider presenting a variety of interrelated information, which necessitates contributing to the user's current network of knowledge by establishing new ties, such as connecting a specific event to its broader social or historical context. Blumenthal and Fisher (2021) offer some practical suggestions for enhancing the aura of a historical location. For instance, reconstructing a demolished or vanished structure using a photo or film might improve aura; storytelling can also be utilized as a memory construct to enhance aura (Blumenthal & Fisher, 2021). This also allows users to create meaning, and to become aware that all spaces have past existences. Incorporating aura as a design goal can theoretically make history less remote and more grounded in the reality of an actual site space.

The “Sense of Place Theory” (SOP) can then further describe the aura of the location. SOP is a concept that includes the feelings of attachment, dependence, concern, identity, and belonging that people develop towards a particular location (Chang et al., 2015, Turner & Turner, 2006). The SOP includes three subparts: place attachment (PA), place dependence (PD), and place identification (PI), “which describe, respectively, the affective, behavioral, and cognitive relationship that people acquire with places” (Chang et al., 2015). According to the AR guidance application of Chang et al. (2015), and the theories of Tilden (2009) and Beck et al. (2002), multimedia, including images, audio, video, text, and songs, have proven to be beneficial in enriching the SOP. In other words, according to the SOP hypothesis, giving visitors a variety of place-related information enhances their feeling of place.

Because our AR experience is rooted in highly situated events, we see our work as in conversation with “situated documentary storytelling” and “location-based MR/AR storytelling” as narrative design theories. The concept of “situated documentary” places a narrated multimedia documentary in the same physical setting as the events and locations it portrays (Hollerer et al., 1999), superimposed over a particular location, like the experience that we created in the EcoCommons. We see aspects of situated documentary storytelling instructive for narratives that are multisequential rather than unisequential (like traditional documentaries), immersive, first-person perspective, interactive with numerous modalities, more contextualized, and embedded within a larger context of events (Pavlik & Bridges, 2013). What was most important to our experience was being able to implement different modalities while maintaining the focus on the activist experience.

In the design process, we understood that the physical space could either be a distraction or could be used in the design. We found the three strategies utilized in location-based MR/AR storytelling—reinforcing, reskinning, and remembering—informative in thinking of the specificity of the site (Azuma, 2015). Given this, reinforcing the EcoCommons was inherently compelling by itself because it was the actual location of the restaurant. Then we needed to consider reskinning as a strategy to remake different versions of the Pickrick restaurant reflect its changes throughout time. Remembering as an AR storytelling strategy means drawing upon memories—in our case, archival footage and recordings—to retell those stories at the place where those memories and stories happened. These concepts provided the framework that guided our approach and practice in creating the AR experience.

The system design considerations of the AR experience were restricted to the physical layout of the EcoCommons site and were unable to add semipermanent objects. The historical context of this site, however, with the construction that includes both natural and structural symbolic elements, does provide potential for integrating storytelling with environmental aspects. In this section, we cover various aspects related to the design, including the design context which encompasses the site and its history, our selection of AR technology and device, merging physical surroundings with virtual content, navigating through the story, and designing the interface.

3.1 The EcoCommons Site Details

The EcoCommons site is where we set up our AR content. It is a newly constructed site at the corner of Hemphill Avenue and Ferst Drive on the Georgia Tech campus in Atlanta, Georgia (see Figure 1). Northeast of the EcoCommons site was the Fred W. Ajax building, formerly the Pickrick restaurant, which Georgia Tech purchased in 1965 and utilized for many years as the placement center before its demolition in May 2009 (Lester Maddox, 2012). This, of course, means that the physical structure that once held the Pickrick restaurant no longer exists.
Figure 1.

EcoCommons Map (Georgia Tech, 2019).

Figure 1.

EcoCommons Map (Georgia Tech, 2019).

Close modal
The site has an enclosure, and within this enclosed area there are three openings that allow people to enter the space. Three pillars (see Figure 2) inside of the space represent the three men who were included in the lawsuit against segregationist restaurant owner and future Georgia governor Lester Maddox for violating their Civil Rights: George Willis Jr., Albert L. Dunn, and Woodrow T. Lewis,3 who were all students at the Interdenominational Theological Center. Three large wooden benches are also at the site to inspire contemplation and discussion of the progress and future of racial equality (Brim, 2021) (see Figure 3).
Figure 2.

The walls and pillars. Photo by the authors.

Figure 2.

The walls and pillars. Photo by the authors.

Close modal
Figure 3.

The benches. Photo by the authors.

Figure 3.

The benches. Photo by the authors.

Close modal

3.2 Historic Details

Our AR experience is focused on the sequence of events that began the day after the passage of the Civil Rights Act on July 2, 1964, until the restaurant was integrated in February 1965. The restaurant “Pickrick” was owned by segregationist Lester Maddox, who notoriously refused to open his restaurant to Black patrons despite Atlanta naming itself an “Open City” (a phrase which expressed lawful compliance with integration though there remained widespread de facto segregation). The Civil Rights Act of 1964 outlawed racial segregation in public accommodations and mandated that businesses be integrated, accepting both Black and white customers, or be in violation of federal law.4 Once President Lyndon Johnson signed the Civil Rights Act, four students from the Interdenominational Theological Center (ITC), George Willis Jr., Albert Dunn, Charles Wells, and Woodrow T. Lewis, attempted to enter the Pickrick to insist that Maddox (who was a leading local segregationist, having been twice defeated in his bid to be mayor of Atlanta) obey the law.

The first attempt was unsuccessful and uneventful because the restaurant was not open. During the second incident, the returning group of three student ministers (Willis, Dunn, and Lewis) were met by Maddox and a group of men who confronted the nonviolent activists with axe handles, rifles, and baseball bats. A small contingent of media recorded the incident. The three students were met at their dorm by the FBI, and eventually would join a lawsuit that involved Robert Kennedy as the intervenor and had the assistance of a legal team that included William Alexander, Burke Marshall, and Constance Baker Motley of the NAACP. The three men won their lawsuit against Maddox for violating their civil rights. However, Maddox resisted the ruling by filing for appeals and engaging in various actions to subvert the rulings including renaming the business and repeatedly refusing admission (Stirgus, 2021), each time engaging with Atlanta media, and each time finding himself in contempt of court.

Finally, after exhausting his legal options and incurring several fines for being in contempt of court, Maddox decided to walk away from his business, subletting the building to his managerial staff who reopened the cafeteria with the name Gateway Café, where all were welcome to eat. On February 23, 1965, Jack Googer, not related to the core group of activists, was able to eat lunch peacefully in the renamed Gateway Cafeteria. Soon after, Georgia Tech purchased the restaurant, and after about 50 years of service as an administration facility and campus police station, it was demolished to make room for the EcoCommons site.

3.3 AR Technology and Device Choice

Understanding AR types can assist in selecting the most appropriate technology for a project. Marker-based AR and marker-less AR are the two most common types of AR (Digital Promise, 2022). To activate an augmentation, marker-based AR uses markers or a user-defined image/object recognition technique. Markers should be distinct patterns that can be recognized by cameras and scanned by users to show virtual content. Location-based AR, projection-based AR, superimposition-based AR, and user-defined marker-less AR are under the marker-less AR group. Location-based AR links virtual material to a specific location by reading data from a device's camera, GPS, compass, and accelerometer, like in Pokémon Go (Digital Promise, 2022). Our AR application utilizes both marker-based and location-based AR.

According to Koenitz (2021), in the context of interactive digital narratives, AR can be categorized by the following five aspirational conditions:

  1. Nearly everyone has access to the necessary technology.

  2. Augmentation via digital devices is widely accepted.

  3. Interactors are willing to provide user-generated content, thereby extending and modifying AR projects.

  4. AR has demonstrated the ability to alter behavior because it can make the familiar unfamiliar.

  5. Augmented real-world interaction offers the ubiquitous moment.

Due to its accessibility, we decide to use mobile AR in our project rather than glasses AR, and to use tablets rather than smartphones to provide a larger field of view. We chose to employ AR to reconstruct history because AR is already widely used to augment history in museums and historical places. Interactors in our project will view the events from a variety of perspectives, especially that of African Americans (which is always neglected in the history). We hope that through using AR, people will be able to feel and comprehend the situation of Black people in the 1960s.

3.4 Physical Environment + Virtual Content

It is crucial to explore how to construct narratives depending on the surroundings, as AR will add additional virtual layers to the real world. Our AR application is in the northwest corner of the EcoCommons site, where deployment space is limited due to the presence of numerous objects, such as walls, pillars, trees, and benches. These environmental constraints and affordances provide unique opportunities when designing the content. After carefully examining the environment layout, we decided to use the site's walls as a vertical space with the marker-based recognition of AR to display the project's introduction, where users can view images, text, and videos on the walls to learn about the Pickrick incidents' background (see Figure 4). In addition, we explore utilizing the narrow pathway as an entry point to guide users along the walls to the central location where the AR confrontations are deployed (see Figure 5). We are using markers that preview the content of the associated media and creating an introductory sequence. When an African American evaluator noted that given the political climate of the 2020s it felt immediately menacing to enter a simulated crowd of jeering, threatening, segregationist white men, we added a trigger warning to the introductory material.
Figure 4.

Virtual content on the wall.

Figure 4.

Virtual content on the wall.

Close modal
Figure 5.

Pathway design prototype.

Figure 5.

Pathway design prototype.

Close modal

3.5 Navigating the Story

It is important that interactors experience these events in sequence, although within each event it is equally important that they are free to move around. We at first addressed the sequencing with a compass and floating arrow but we found that visitors needed human guides to readjust the perspective and orient them to the virtual scene (see Figure 6). Our second and current strategy is to place footprints on the ground, and we are exploring strategies for moving from one footprint to another (see Figure 7).
Figure 6.

Arrow design.

Figure 7.

Footprint design.

Figure 7.

Footprint design.

Close modal

3.6 Interface Designs

Our objective is to provide a continuous and comprehensive historical narrative of the series of Civil Rights events as well as the broader historical context. We aim to display the most dramatic confrontations, but also the causal linkages between those confrontations and their corresponding historical events. All of the events will be presented in a connected and chronological order to provide a coherent experience. Using Dünser et al.’s (2007) general HCI principles for AR system design (including affordance, little cognitive overhead, low physical effort, high learnability, and high flexibility of usage), we opted to create a first version of our device-based interface using the tool “Timeline.” As a common element in mobile applications, Timeline serves multiple purposes, including: (1) creating a familiar interface for users to improve learnability; (2) integrating and weaving together multiple pieces of information and multiple media formats; and (3) increasing user freedom in its slide-ability between events (see Figure 8). Ideally the site itself should provide navigation, and we look to the symbolic pillars and the inside of the enclosing walls as offering additional opportunities for orienting the interactor to the progression of time, and providing context in the form of video, text, and image projections separate from the virtual restaurant and animated people. The limitation here is the size of the viewing device, and for now the surface of the mobile device may be more practical than a projected image seen through the device for the kinds of supplementary information contained in the Timeline.
Figure 8.

Timeline design.

Figure 8.

Timeline design.

Close modal

In the section on narrative design, Murray's (2011) and Koenitz's (2015) design principles and models, respectively, serve as an overarching framework that facilitates the formulation of issues and the identification of solutions. For instance, procedurally, how can interactors experience this story? How can interactors participate in the narrative and how much agency can we give them? How can we strike a balance between authorial control and interactor agency? How does narrative become participative? How do we insert several types of content into AR? How can we spatially integrate the physical environment and navigate the interactor in the AR environment?

4.1 Storyline Construction

The objective of the content design is to introduce the Pickrick events of July 1964–February 1965. In the following sections, we address several design considerations that must be made, including: what type of information should be displayed and in what format (text, image, 3D models, etc.); should it be nonlinear or linear, and what is the story's sequence? The design team also had to consider how to privilege the perspective of the activists in the AR experience.

According to the situated documentary theory (Hollerer et al., 1999), one of the characteristics of the location-based storytelling should be nonlinear. For our purposes, following Murray (2017) we substitute the term “multisequential” since nonlinear is a vague purely negative design term. For consistency we use “unisequential,” for what is usually called “linear.” Multisequential storytelling is intrinsic to AR installations in which users physically move through space. At the same time, we want them to be aware of the cause-and-effect sequence that makes for narrative meaning. For example, when Willis, Dunn, and Lewis and the legal team of the NAACP get a fine assessed against the Pickrick, Maddox changes the name of the restaurant to “The Lester Maddox Cafeteria,” and we want interactors to understand what caused that name change in September 1964 and why the protesters had to return and cause the court to invoke the same fines all over again against the newly named restaurant for once more refusing service on the basis of race.

The order of events in the key period is as follows:

  • - July 2, 1964. Civil Rights Act signed.

  • - July 3, 1964. Willis, Dunn, and Lewis in a car are threatened by Maddox.

  • - July 9, 1964. Willis, Dunn, and Lewis filed a lawsuit against Maddox.

  • - Aug. 11, 1964. Maddox refuses service to several groups of African Americans including Willis, Lewis, Dunn, and Wells, refusing to adhere to the court injunction.

  • - Sept. 26, 1964. Maddox renames the Pickrick to the Lester Maddox Cafeteria. Ministers Willis, Lewis, Dunn, and Wells return to the restaurant and are again refused service.

  • - Jan. 29, 1965. Willis, Dunn, Wells, and Lewis have another unsuccessful attempt.

  • - Feb. 5, 1965. Court rules against Maddox.

  • - Feb. 7, 1965. Maddox ultimately chooses to close his restaurant rather than integrate it.

  • - Feb. 23, 1965. The Lester Maddox Cafeteria is reopened under new management as the Gateway Cafeteria.

4.2 Scene Design

AR can contain a variety of content types, including text, image, video, audio, 3D models, and animation. In the design phase, we evaluated how each scene should be designed and what components should go in it. Even though the 3D animation format in immersive technology can increase immersion and presence (Zhao et al., 2018), we at first assumed that we would be unable to develop all the scenes in this format due to time constraints and as well as the anticipatory performance capacity of iPad. The following objects and scenes were prioritized to be rendered in 3D:

  • The Pickrick restaurant (see Figure 9). This 3D reproduction can help identify both the past and the current site.
    Figure 9.

    Pickrick restaurant 3D model.

    Figure 9.

    Pickrick restaurant 3D model.

    Close modal
  • The Pickrick signage (see Figure 10). Based on our study, we discovered that the Pickrick sign changed frequently because Maddox used this method to avoid complying with the ruling.
    Figure 10.

    The Pickrick signage frequently changed to reflect story progress.

    Figure 10.

    The Pickrick signage frequently changed to reflect story progress.

    Close modal
  • The three confrontations on July 3, 1964; August 11, 1964; and January 29, 1965, for which we have archival footage (eFootage, 2019a, 2019b; WSB-TV, 1965) which served as the primary materials for constructing the characters and animations (see Figure 11).
    Figure 11.

    Three confrontations made from archival video footage.

    Figure 11.

    Three confrontations made from archival video footage.

    Close modal
  • The final scene where Jack Googer peacefully integrates the restaurant (see Figure 12). We reproduced it in 3D by referring to newspaper and book accounts (Kruse, 2013).
    Figure 12.

    Jack Googer peacefully enters the Gateway Cafeteria, the first African American to do so.

    Figure 12.

    Jack Googer peacefully enters the Gateway Cafeteria, the first African American to do so.

    Close modal

To facilitate access to the original historical sources, all 3D scenes have been paired with images or videos from the era. In the initial application we made these available via the “More” button at the upper-left corner of the Timeline screen on the mobile device, but we found that users did not notice it. We are currently exploring other approaches.

4.3 Privileging the Perspective of the Activists

While conducting historical research on the Pickrick incidents, we discovered that the names of the protestors were frequently omitted from text materials, while Lester Maddox is the focus of the media attention at the time. In archival footage of the first incident between Maddox and the protestors, the camera and microphone focused on Lester Maddox and his menacing actions. In the archival clip called “Segregation at the Pickrick Restaurant” (eFootage, 2019c), the camera focuses on Maddox and amplifies his voice and picks up the jeers of the surrounding segregationist crowd; the viewer finds it hard to hear what the African Americans are saying, and in particular the words of Reverend Woodrow Lewis who tells Maddox, “You are distorting the gospel.”

We have employed several design tactics to diminish the visual and aural dominance of the more dramatically aggressive segregationists (Maddox and the mobs) and to visually and aurally amplify the presence of the activists who are responding without physical aggression but with dignity and discipline. To guide users towards paying more attention to the ITC student activists, we gave them name tags (see Figure 13) by floating their names above their heads so users will be able to identify the activists with the photographic introduction in the wall. The name tags face the users regardless of where they position themselves. Aural considerations led us to reduce the volume of Maddox's voice in the original audio while enhancing the volume of ministers' voices. Because Maddox orchestrated and was the focus of media attention, the audio in the archival footage that we found was often focused on him, and thus made the original audio of the ministers difficult to hear. Once we enhanced their audio to gain clarity of what the ministers were saying, we subtitled their statements with speech bubbles to add clarity to their statements. (see Figure 14).
Figure 13.

Name tags and transparent design prototype.

Figure 13.

Name tags and transparent design prototype.

Close modal
Figure 14.

Subtitle design prototype.

Figure 14.

Subtitle design prototype.

Close modal
Privileging the ministers meant that we needed the users to see different perspectives of them (see Figure 15). In the original footage, Maddox was the centralized character; in the AR experience, we shifted the viewpoint towards the ministers in the AR scene. For example, in the July 3 confrontation, users can adjust their perspective by sitting in the car with the ministers to experience the perspective of how it looks when Maddox threatens the ministers by hitting the car with a bat (see Figure 16). To do this, we are adapting an existing physical feature of the site, a low boundary wall, to situate the interactor and allow us to predict their sightline. To diminish Maddox, we adjusted the level of opacity. As the user approaches the confrontation, the mob's transparency will increase, rendering them unidentifiable (see Figure 13). This is done so users can see the ministers within the mob scene more clearly without indicating a diminished scope of danger that surrounded the activists.
Figure 15.

Model design.

Figure 16.

Perspective design prototype.

Figure 16.

Perspective design prototype.

Close modal

The Pickrick Protests is an AR project that recreates the confrontations that occurred at the Pickrick restaurant to memorialize African Americans who battled for equal rights in 1964–1965. Based on the affordances of AR, we have proposed design strategies for AR storytelling, such as how to integrate the virtual elements with the physical realities of a particular site, and how to make the hidden history visible through narration design.

AR can potentially deepen social involvement by creating immersive, complex stories that attempt to reveal hidden histories and points of view, according to experts who study AR activism (Silva et al., 2022). At the same time, such uses of AR raise complex ethical questions. Nassim Parvin (2018) has detailed the obligations that digital archivists are under in retelling the stories of others, and with recent histories and histories of oppressed peoples, these considerations are particularly strong. Digital instantiation carries an illusion of authority, and reconstructions linked to a particular place can seem particularly authentic. In addition, potentially spectacular AR experiences at dark tourism sites (sites featuring pain, misery, or death) are a particularly problematic possibility (Fisher & Bolter, 2018). Because of the potential of AR as an expressive medium, we believe there is value in continuing to explore the design space of site-specific historical applications. It is important that actual historians be involved in creating these immersive experiences and that more research be done into how effective they are in augmenting understanding of the past.

This project is supported by DILAC (Digital Integrative Liberal Arts Center) lab at Georgia Institute of Technology and was funded by a generous grant from the Andrew W. Mellon Foundation. We thank Dr. Todd Michney and Dr. Karcheik Sims-Alvarado for their suggestions on historical materials. We thank Georgia Institute of Technology and Preserve Black Atlanta. We also appreciate our reviewers’ suggestions.

1

These questions will be addressed in the Design section.

2

In MacIntyre et al.’s (2002) paper, they refer to all systems that combine physical and virtual worlds as Mixed Reality (MR), including AR. And according to Speicher et al. (2019), MR can be “many things and its understanding is always based on one's context.” In our paper, we would like to categorize our project into AR by referring to the classical “reality-virtuality continuum” (Milgram et al., 1994).

3

The three pillars represent the names on the lawsuit but are not representative of those who were constantly engaged in the actions at the Pickrick. A fourth man who was also consistently involved with the protests was Charles Wells. Reverend Wells approached Maddox earlier in the day but was not with the group at the time of the protest when Maddox confronted the group with the firearm. Therefore, he was not named on the lawsuit. While he was not in the formal case against Maddox, he was present for all the subsequent protests after that.

4

Heart of Atlanta Motel also was on the lawsuit for resisting integration. Both lost their cases, and Heart of Atlanta decided to follow the law. Maddox, however, decided to appeal.

Azuma
,
R.
(
2015
).
Location-based mixed and augmented reality storytelling
. In
W.
Barfield
(
Ed.
),
Fundamentals of wearable computers and augmented reality
(2nd
ed.
, pp. 
259
276
).
CRC Press
.
Beck
,
L.
,
Cable
,
T.
, &
Tilden
,
F.
(
2002
).
The meaning of interpretation
.
Journal of Interpretation Research
,
7
(
1
),
7
10
.
Benjamin
,
W.
(
1968
).
Art in the Age of Mechanical Reproduction
. In
Illuminations
(pp.
217
251
).
Schocken Books
,
New York
.
Blumenthal
,
H.
, &
Fisher
,
J. A.
(
2021
).
Augmented reality, aura, and the design of cultural spaces
. In
Augmented and mixed reality for communities
(pp.
199
212
).
CRC Press
.
Bowman
,
D.
,
Hodges
,
L.
, &
Bolter
,
J.
(
1998
).
The virtual venue: User-computer interaction in information-rich virtual environments
.
Presence: Teleoperators and Virtual Environments
,
7
(
5
),
478
493
.
Brim
,
C.
(
2021, April 14
).
The grand opening of the EcoCommons
.
Georgia Tech News Center
.
Retrieved from
https://news.gatech.edu/features/2021/07/grand-opening-ecocommons
Chang
,
Y. L.
,
Hou
,
H. T.
,
Pan
,
C. Y.
,
Sung
,
Y. T.
, &
Chang
,
K. E.
(
2015
).
Apply an augmented reality in a mobile guidance to increase sense of place for heritage places
.
Journal of Educational Technology & Society
,
18
(
2
),
166
178
.
Dow
,
S.
,
Lee
,
J.
,
Oezbek
,
C.
,
Maclntyre
,
B.
,
Bolter
,
J. D.
, &
Gandy
,
M.
(
2005, June
).
Exploring spatial narratives and mixed reality experiences in Oakland Cemetery
. In
Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology
, pp. 
51
60
.
Dünser
,
A.
,
Grasset
,
R.
,
Seichter
,
H.
, &
Billinghurst
,
M.
(
2007
).
Applying HCI principles to AR systems design
.
University of Canterbury. Human Interface Technology Laboratory
.
eFootage
. (
2019a, April 23
).
Atlanta restaurant serves Black customers, HD, Film & Video Stock [Video]
.
eFootage
.
Retrieved from
https://www.efootage.com/videos/86590/atlanta-restaurant-serves-black-customers
eFootage
. (
2019b, April 23
).
Maddox won't serve Blacks, HD, Film & Video Stock [Video]
.
eFootage
.
Retrieved from
https://www.efootage.com/videos/78420/maddox-wont-serve-blacks
eFootage
. (
2019c, April 23
).
Segregation at the Pickrick restaurant, HD, Film & Video Stock [Video]
.
eFootage
.
Retrieved from
https://www.efootage.com/videos/78492/segregation-pickrick-restaurant
Fisher
,
J. A.
, &
Bolter
,
J. D.
(
2018
,
October
).
Ethical considerations for AR experiences at dark tourism sites
. In
2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct
, pp.
365
369
.
Georgia
Tech
. (
2008
).
Pickrick restaurant/Ajax Building [Photograph]
.
Georgia Tech Archives
.
Retrieved from
https://history.library.gatech.edu/exhibits/show/bldg/item/3047
Georgia
Tech
. (
2019, October 4
).
Eco-Commons construction begins next week [Photograph]
.
Georgia Tech News Center
.
Retrieved from
https://news.gatech.edu/news/2019/10/04/eco-commons-construction-begins-next-week
Haahr
,
M.
(
2017, November
).
Creating location-based augmented-reality games for cultural heritage
. In
Joint International Conference on Serious Games
, pp.
313
318
.
Hollerer
,
T.
,
Feiner
,
S.
, &
Pavlik
,
J.
(
1999, October
).
Situated documentaries: Embedding multimedia presentations in the real world
. In
Digest of Papers. Third International Symposium on Wearable Computers
, pp.
79
86
.
Kleftodimos
,
A.
,
Moustaka
,
M.
, &
Evagelou
,
A.
(
2023
).
Location-based augmented reality for cultural heritage education: Creating educational, gamified location-based AR applications for the prehistoric lake settlement of Dispilio
.
Digital
,
3
(
1
),
18
45
.
Koenitz
,
H.
(
2015
).
Towards a specific theory of interactive digital narrative
. In
Interactive digital narrative: History, theory, and practice
.
Routledge
.
Koenitz
,
H.
(
2021
).
Reflecting in space on time: Augmented reality interactive digital narratives to explore complex histories
. In
Augmented and mixed reality for communities
, pp. 
183
198
.
CRC Press
.
Kruse
,
K. M.
(
2013
).
White flight
.
Amsterdam University Press
.
Lester
Maddox
. (
2012, February 28
). In
Wikipedia
. https://en.wikipedia.org/wiki/Lester_Maddox
MacIntyre
,
B.
,
Bolter
,
J. D.
, &
Gandy
,
M.
(
2004
).
Presence and the aura of meaningful places
.
Proceedings of the 7th Annual International Workshop on Presence
, pp. 36–43.
MacIntyre
,
B.
,
Lohse
,
M.
,
Bolter
,
J. D.
, &
Moreno
,
E.
(
2002
).
Integrating 2-D video actors into 3-D augmented-reality systems
.
Presence: Teleoperators and Virtual Environments
,
11
(
2
),
189
202
.
Melnick
,
K.
(
2020, August 15
).
Preserving the art of Black Lives Matter using AR
.
VRScout
. https://vrscout.com/news/preverving-the-art-of-blm-in-ar/
Milgram
,
P.
,
Takemura
,
H.
,
Utsumi
,
A.
, &
Kishino
,
F.
(
1994
).
Augmented reality: A class of displays on the reality-virtuality continuum
.
Telemanipulator and Telepresence Technologies
,
2351
.
Murray
,
J. H.
(
2011a
).
Four affordances
. https://inventingthemedium.com/four-affordances/
Murray
,
J. H.
(
2011b
).
Inventing the medium: Principles of interaction design as a cultural practice
.
MIT Press
.
Murray
,
J. H.
(
2017
).
Hamlet on the Holodeck: The future of narrative in cyberspace
. (Updated
ed.
)
MIT Press
.
North of 41
. (
2018, March 20
).
What really is the difference between AR /MR / VR / XR?
Medium
. https://medium.com/@northof41/what-really-is-the-difference-between-ar-mr-vr-xr-35bed1da1a4e
Parvin
,
N.
(
2018
).
Doing justice to stories: On ethics and politics of digital storytelling
.
Engaging Science, Technology, and Society
,
4
,
515
534
.
Pavlik
,
J. V.
, &
Bridges
,
F.
(
2013
).
The emergence of augmented reality (AR) as a storytelling medium in journalism
.
Journalism & Communication Monographs
,
15
(
1
),
4
59
.
Silva
,
R. M. L.
,
Principe
Cruz
,
E.
,
Rosner
,
D.
K.
,
Kelly
,
D.
,
Monroy-Hernández
,
A.
, &
Liu
,
F
. (
2022
).
Understanding AR activism: An interview study with creators of augmented reality experiences for social change
. In
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
,
1
15
.
Speicher
,
M.
,
Hall
,
B. D.
, &
Nebeling
,
M.
(
2019, May
).
What is mixed reality?
In
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
, pp.
1
15
.
Stirgus
,
E.
(
2021, April 21
).
Georgia Tech unveils civil rights memorial at old Pickrick Restaurant
.
The Atlanta Journal-Constitution
. https://www.ajc.com/education/georgia-tech-unveils-civil-rights-memorial-at-old-pickrick-restaurant/CR4MXYGEZ5HP5LIKLPI3KVKAJU/
Takahashi
,
D.
(
2019, November 15
).
Holocaust Memorial Museum uses augmented reality to make history visceral
.
VentureBeat
. https://venturebeat.com/2018/08/31/holocaust-memorial-museum-uses-augmented-reality-to-make-history-visceral/
Tilden
,
F.
(
2009
).
Interpreting our heritage
.
University of North Carolina Press
.
Turner
,
P.
, &
Turner
,
S.
(
2006
).
Place, sense of place, and presence
.
Presence: Teleoperators and Virtual Environments
,
15
(
2
),
204
217
.
WSB-TV
. (
1965, January 29
).
WSB-TV newsfilm clip of Lester Maddox pushing African Americans away from his cafeteria with an ax handle, Atlanta, Georgia, 1965 January 29 [Video]
.
Civil Rights Digital Library
. Retrieved from http://crdl.usg.edu/cgi/crdl?format=_video&query=id%3Augabma_wsbn_wsbn47697&_cc=1
Wu
,
H.-K.
,
Lee
,
S. W.-Y.
,
Chang
,
H.-Y.
, &
Liang
,
J.-C.
(
2013
).
Current status, opportunities and challenges of augmented reality in education
.
Computers & Education
,
62
,
41
49
.
Young
,
J.
(
2021
).
The EcoCommons at the Georgia Institute of Technology [Photograph]
.
THE FIELD
.
Retrieved from
https://thefield.asla.org/2021/04/29/the-ecocommons-at-the-georgia-institute-of-technology/
Zhao
,
Y.
,
Keehn
,
D. P.
,
Wang
,
A. Y.
,
Jiang
,
J.
, &
Murray
,
J.
(
2021, March
).
The Pickrick Protests: An interactive geo AR experience
. In
2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops
, pp.
675
676
.
Zhao
,
Y.
,
Forte
,
M.
, &
Kopper
,
R.
(
2018, March
).
VR touch museum
. In
2018 IEEE Conference on Virtual Reality and 3D User Interfaces
, pp.
741
742
.