This paper describes current work on a cooperative teleassistance system for semiautonomous control of mobile robots. This system combines a robot architecture for limited autonomous perceptual and motor control with a knowledge-based operator assistant that provides strategic selection and enhancement of relevant data. It extends recent developments in artificial intelligence in modeling the role of visual interactions in problem solving for application to an interface permitting the human and remote to cooperate in cognitively demanding tasks such as recovering from execution failures, mission planning, and learning. The design of the system is presented, together with a number of exception-handling scenarios that were constructed as a result of experiments with actual sensor data collected from two mobile robots.