Augmented reality (AR) is a technology in which a user's view of the real world is enhanced or augmented with additional information generated from a computer model. To have a working AR system, the see-through display system must be calibrated so that the graphics are properly rendered. The optical see-through systems present an additional challenge because, unlike the video see-through systems, we do not have direct access to the image data to be used in various calibration procedures. This paper reports on a calibration method we developed for optical see-through headmounted displays. We first introduce a method for calibrating monocular optical seethrough displays (that is, a display for one eye only) and then extend it to stereo optical see-through displays in which the displays for both eyes are calibrated in a single procedure. The method integrates the measurements for the camera and a six-degrees-offreedom tracker that is attached to the camera to do the calibration. We have used both an off-the-shelf magnetic tracker as well as a vision-based infrared tracker we have built. In the monocular case, the calibration is based on the alignment of image points with a single 3D point in the world coordinate system from various viewpoints. In this method, the user interaction to perform the calibration is extremely easy compared to prior methods, and there is no requirement for keeping the head immobile while performing the calibration. In the stereo calibration case, the user aligns a stereoscopically fused 2D marker, which is perceived in depth, with a single target point in the world whose coordinates are known. As in the monocular case, there is no requirement that the user keep his or her head fixed.

Augmented reality (AR) is a technology in which a user's view of the real world is enhanced or augmented with additional information generated from a computer model. Using AR technology, users can interact with a combination of real and virtual objects in a natural way. This paradigm constitutes the core of a very promising new technology for many applications. However, before it can be applied successfully, AR has to fulfill very strong requirements including precise calibration, registration and tracking of sensors and objects in the scene, as well as a detailed overall understanding of the scene. We see computer-vision and image-processing technology playing an increasing role in acquiring appropriate sensor and scene models. To balance robustness with automation, we integrate automatic image analysis with both interactive user assistance and input from magnetic trackers and CAD models. Also, in order to meet the requirements of the emerging global information society, future human-computer interaction will be highly collaborative and distributed. We thus conduct research pertaining to distributed and collaborative use of AR technology. We have demonstrated our work in several prototype applications, such as collaborative interior design and collaborative mechanical repair. This paper describes our approach to AR with examples from applications, as well as describing the underlying technology.