We provide an overview of the early stages of three related research projects whose goals are to exploit augmented reality, virtual worlds, and artificial intelligence to explore relationships between perceived architectural space and the structural systems that support it. In one project, we use a see-through head-mounted display to overlay a graphic representation of a building's structural systems on the user's view of a room within the building. This overlaid virtual world shows the out-lines of the concrete joists, beams, and columns surrounding the room, as well as the reinforcing steel inside them, and includes displays from a commercially available structural analysis program. In a related project, the structural view is exposed by varying the opacity of room finishes and concrete in a 3D model of the room and surrounding structure rendered on a conventional CRT. We also describe a hypermedia database, currently under construction, depicting major, twentieth-century American buildings. The interactive, multidisciplinary elements of the database—including structural and thermal analyses, free body diagrams (which show how forces are resisted by portions of a structure under various loading conditions), facsimiles of construction documents, and critical essays—are bound together and made available over the World-Wide Web. Finally, we discuss the relationships among all these projects, and their potential applications to teaching architecture students and to construction, assembly, and repair of complex structures.

This research aims at the realization of a networked virtual environment for the design of three-dimensional (3-D) objects. Based on an analysis of an ordinary collaborative design, we illustrate that a collaborative work space consists of a dialog space and an object space . In the dialog space, a participant interacts with partners, and in the object space with an object. The participants enter the dialog space and the object space in turn, appropriately. In addition, collaborative design of 3-D objects is carried out with multimodal interactions: visual, auditory, and haptic. A networked virtual environment must support these interactions without contradiction in either time or space. In this paper, we propose a networked virtual environment for a pair of participants to satisfy the conditions described above. To implement the networked system, we take into account the necessity of visual, auditory, and haptic interactions, the need for participants to switch between the dialog space and the object space quickly and appropriately, and human ergonomics on the functional space of hands and eyes. An experiment on hand-over task was done to investigate the effect of the networked haptic device with the proposed system. Object layout tasks, such as toy block layout, office furniture layout, city building layout, etc., can be performed by using this environment.

The objective of this research was to study the capabilities of sensory substitution for force feedback through the tactile and auditory senses for teleoperation tasks, with and without time delay. The motivation and potential benefits of sensory substitution for force feedback with vibrotactile and auditory displays are discussed. Teleoperator experiments that examined the presentation of basic force information through object contact tasks indicated that operator performance was improved by using the vibrotactile and auditory displays to present force information. Further, the vibrotactile and auditory displays compared favorably to traditional bilateral force feedback. Common manipulation experiments with peg-in-hole tasks of varying complexity were also conducted and showed that when the subjects' view was fully obstructed, the subjects were able to successfully complete the task by using either of the sensory substitution displays. Sensory substitution was also tested in the presence of a 3 sec time delay and significantly improved performance without instabilities.

We track, in real-time, the position and posture of a human body, using a minimal number of six DOF sensors to capture full body standing postures. We use four sensors to create a good approximation of a human operator's position and posture, and map it on to our articulated computer graphics human model. The unsensed joints are positioned by a fast inverse kinematics algorithm. Our goal is to realistically recreate human postures while minimally encumbering the operator.