We present a physically-based approach to grasping and manipulation of virtual objects that produces visually realistic results, addresses the problem of visual inter-penetration of hand and object models, and performs force rendering for force-feedback gloves, in a single framework. Our approach couples a simulation-controlled articulated hand model to tracked hand configuration using a system of linear and torsional virtual spring-dampers. We discuss an implementation of our approach that uses a widely available simulation tool for collision detection and response. We pictorially illustrate the resulting behavior of the virtual hand model and of grasped objects, discuss user behavior and difficulties encountered, and show that the simulation rate is sufficient for control of current force-feedback glove designs. We also present a prototype system for natural whole-hand interactions in a desktop-sized workspace.

We present a haptic feedback technique that combines feedback from a portable force-feedback glove with feedback from direct contact with rigid passive objects. This approach is a haptic analogue of visual mixed reality, since it can be used to haptically combine real and virtual elements in a single display. We discuss device limitations that motivated this combined approach and summarize technological challenges encountered. We present three experiments to evaluate the approach for interactions with buttons and sliders on a virtual control panel. In our first experiment, this approach resulted in better task performance and better subjective ratings than the use of only a force-feedback glove. In our second experiment, visual feedback was degraded and the combined approach resulted in better performance than the glove-only approach and in better ratings of slider interactions than both glove-only and passive-only approaches. A third experiment allowed subjective comparison of approaches and provided additional evidence that the combined approach provides the best experience.