Prior theoretical work on memory schemas, an influential concept of memory from the field of cognitive psychology, is presented for application to fidelity of computer graphics simulations. The basic assumption is that an individual's prior experience will influence how he or she perceives, comprehends, and remembers new information in a scene. Schemas are knowledge structures; a scene could incorporate objects that fit into a specific context or schema (e.g., an academic's office) referred to as consistent objects. A scene could also include objects that are not related to the schema in place referred to as inconsistent objects. In this paper, we describe ongoing development of a rendering framework related to scene perception based on schemas. An experiment was carried out to explore the effect of object type and rendering quality on object memory recognition in a room. The computer graphics simulation was displayed on a Head Mounted Display (HMD) utilizing stereo imagery and head tracking. Thirty-six participants across three conditions of varied rendering quality of the same space were exposed to the computer graphics environment and completed a memory recognition task. Results revealed that schema consistent elements of the scene were more likely to be recognized than inconsistent information. Overall higher confidence ratings were assigned for consistent objects compared to inconsistent ones. Total object recognition was better for the mid-quality condition compared to the low-quality one. The presence of shadow information, though, did not affect recognition of either consistent or inconsistent objects. Further explorations of the effect of schemas on spatial awareness in synthetic worlds could lead to identifying areas of a computer graphics scene that require better quality of rendering as well as areas for which lower fidelity could be sufficient. The ultimate goal of this work is to simulate a perceptual process rather than to simulate physics.

This paper describes a methodology based on human judgments of memory awareness states for assessing the simulation fidelity of a virtual environment (VE) in relation to its real scene counterpart. To demonstrate the distinction between task performance-based approaches and additional human evaluation of cognitive awareness states, a photorealistic VE was created. Resulting scenes displayed on a head-mounted display (HMD) with or without head tracking and desktop monitor were then compared to the real-world task situation they represented, investigating spatial memory after exposure. Participants described how they completed their spatial recollections by selecting one of four choices of awareness states after retrieval in an initial test and a retention test a week after exposure to the environment. These reflected the level of visual mental imagery involved during retrieval, the familiarity of the recollection and also included guesses, even if informed. Experimental results revealed variations in the distribution of participants' awareness states across conditions while, in certain cases, task performance failed to reveal any. Experimental conditions that incorporated head tracking were not associated with visually induced recollections. Generally, simulation of task performance does not necessarily lead to simulation of the awareness states involved when completing a memory task. The general premise of this research focuses on how tasks are achieved, rather than only on what is achieved. The extent to which judgments of human memory recall, memory awareness states, and presence in the physical and VE are similar provides a fidelity metric of the simulation in question.