Haptic augmented reality (AR) mixes a real environment with computer-generated virtual haptic stimuli, enabling the system to modulate the haptic attributes of a real object to desired values. This paper reports our second study on this functionality, with stiffness as a goal modulation property. Our first study explored the potential of haptic AR by presenting an effective stiffness modulation system for simple 1D interaction. This paper extends the system so that a user can interact with a real object in any 3D exploratory pattern while perceiving its augmented stiffness. We develop a complete set of algorithms for contact detection, deformation estimation, force rendering, and force control. The core part is the deformation estimation where the magnitude and direction of real object deformation are estimated using a contact dynamics model identified in a preprocessing step. All algorithms are designed in a way that maximizes the efficiency and usability of the system while maintaining convincing perceptual quality. In particular, the need for a large amount of preprocessing such as geometry modeling is avoided to improve the usability. The physical performance of each algorithm is thoroughly evaluated with real samples. Each algorithm is experimentally verified to satisfy the physical performance requirements that need to be satisfied to achieve convincing rendering quality. The final perceptual quality of stiffness rendering is assessed in a psychophysical experiment where the difference in the perceived stiffness between augmented and virtual objects is measured. The error is less than the human discriminability of stiffness, demonstrating that our system can provide accurate stiffness modulation with perceptually insignificant errors. The limitations of our AR system are also discussed along with a plan for future work.

Vibrotactile rendering is one of the most popular means for improving the user interface of a mobile device, but the availability of related perceptual data that can aid vibrotactile effect design is not currently sufficient. The present paper reports data from a series of psychophysical studies designed to fill this gap. In Experiment I, we measured the absolute detection thresholds of sinusoidal vibrotactile stimuli transmitted to the hand through a mobile phone. Stimuli were generated by a mechanical shaker system that can produce vibrations over a broad frequency and amplitude range. The detection thresholds reported here are a new addition to the literature, and can serve as a baseline for vibrotactile stimulus design. In Experiment II, we estimated the perceived intensities of mobile device vibrations for various frequencies and amplitudes using the same shaker system. We also determined a form of parametric nonlinear function based on Stevens' power law and fit the function to the measured data. This psychophysical magnitude function, which maps vibration frequency and amplitude to a resulting perceived intensity, can be used to predict the perceived intensity of a mobile device vibration from its physical parameter values. In Experiment III, we measured another set of perceived intensities using two commercial miniature vibration actuators (vibration motor and voice-coil actuator) in place of the mechanical shaker. The purpose of this experiment was to evaluate the utility of the psychophysical magnitude function obtained in Experiment II, as vibrotactile stimuli produced by miniature actuators may have different physical characteristics, such as vibration direction and ground condition. Comparison of the results of Experiments II and III confirmed that the psychophysical magnitude function can reliably predict changing trends in the perceived intensity of mobile device vibration. We also discuss further research issues encountered during the investigation. The results presented in this paper may be instrumental in the design of effective vibrotactile actuators and perceptually-salient rendering algorithms for mobile devices.

Haptic augmented reality (AR) enables the user to feel a real environment augmented with synthetic haptic stimuli. This article addresses two important topics in haptic AR. First, a new taxonomy for haptic AR is established based on a composite visuo-haptic reality-virtuality continuum extended from the conventional continuum for visual AR. Previous studies related to haptic AR are reviewed and classified using the composite continuum, and associated research issues are discussed. Second, the feasibility of haptically modulating the feel of a real object with the aid of virtual force feedback is investigated, with the stiffness as a goal haptic property. All required algorithms for contact detection, stiffness modulation, and force control are developed, and their individual performances are thoroughly evaluated. The resulting haptic AR system is also assessed in a psychophysical experiment, demonstrating its competent perceptual performance for stiffness modulation. To our knowledge, this work is among the first efforts in haptic AR for systematic augmentation of real object attributes with virtual forces, and it serves as an initial building block toward a general haptic AR system. Finally, several research issues identified during the feasibility study are introduced, with the aim of eliciting more research interest in this exciting yet unexplored area.

This study investigates the effect of update rate on the quality of haptic virtual textures, with the goal to develop a guideline for choosing an optimal update rate for haptic texture rendering. Two metrics, control stability and perceived quality of the virtual haptic texture, were used. For control stability, we examined the effect of update rate on the “buzzing” of virtual haptic textures. For perceived quality, we measured the discriminability of virtual haptic textures rendered at different update rates. Our study indicates that update rates much higher than the conventional 1 kHz are needed in order to achieve a stable rendering of “clean and hard” textured surfaces. We also found that our ability to distinguish textures rendered with different update rates depends on whether the virtual textures contain perceived instability. Based on these results, we provide a general guideline for selecting an optimal update rate for rendering virtual textured surfaces.

This article reports the second study in a series that investigates perceived instability—unrealistic sensations associated with virtual objects—of virtual haptic texture. Our first study quantified the maximum stiffness values under which virtual haptic textures were perceived to be stable (Choi & Tan, 2004). The present study investigated the effect of the collision-detection algorithm by removing the step changes in force magnitude that could have contributed to perceived instability in the first study. Our results demonstrate a significant increase in the maximum stiffness for stable haptic texture rendering. We also report a new type of perceived instability, aliveness, that is characterized by a pulsating sensation. We discuss the possible cause of aliveness and show that it is not always associated with control instability. Our results underscore the important roles played by environment modeling and human haptic perception, as well as control stability, in ensuring a perceptually stable virtual haptic environment.

This paper presents a quantitative characterization of the instability that a human user often experiences while interacting with a virtual textured surface rendered with a force-reflecting haptic interface. First, we quantified the degree of stability/ instability during haptic texture rendering through psychophysical experiments. The stiffness of the virtual textured surface upon detection of instability was measured under a variety of experimental conditions using two texture rendering methods, two exploration modes, and various texture model parameters. We found that the range of stiffness values for stable texture rendering was quite limited. Second, we investigated the attributes of the proximal stimuli experienced by a human hand while exploring the virtual textured surface in an attempt to identify the sources of perceived instability. Position, force, and acceleration were measured and then analyzed in the frequency domain. The results were characterized by sensation levels in terms of spectral intensity in dB relative to the human detection threshold at the same frequency. We found that the spectral bands responsible for texture and instability perception were well separated in frequency such that they excited different mechanoreceptors and were, therefore, perceptually distinctive. Furthermore, we identified the high-frequency dynamics of the device to be a likely source of perceived instability. Our work has implications for displaying textured surfaces through a force feedback device in a virtual environment.