HaptoFloater: Visuo-Haptic Augmented Reality by
Embedding Imperceptible Color Vibration Signals for
Tactile Display Control in a Mid-Air Image
Abstract
We propose HaptoFloater, a low-latency mid-air visuo-haptic augmented reality (VHAR) system that utilizes imperceptible color vibrations. When adding tactile stimuli to the visual information of a mid-air image, the user should not perceive the latency between the tactile and visual information. However, conventional tactile presentation methods for mid-air images, based on camera-detected fingertip positioning, introduce latency due to image processing and communication. To mitigate this latency, we use a color vibration technique; humans cannot perceive the vibration when the display alternates between two different color stimuli at a frequency of 25 Hz or higher. In our system, we embed this imperceptible color vibration into the mid-air image formed by a micromirror array plate, and a photodiode on the fingertip device directly detects this color vibration to provide tactile stimulation. Thus, our system allows for the tactile perception of multiple patterns on a mid-air image in 59.5 ms. In addition, we evaluate the visual-haptic delay tolerance on a mid-air display using our VHAR system and a tactile actuator with a single pattern and faster response time. The results of our user study indicate a visual-haptic delay tolerance of 110.6 ms, which is considerably larger than the latency associated with systems using multiple tactile patterns.
keywords:
Visuo-haptic displays, mid-air images, LCD displays, imperceptible color vibrationspacing
\onlineid1080
\vgtccategoryResearch
\authorfooter
Rina Nagano, Takahiro Kinoshita, Shingo Hattori, and Takefumi Hiraki are with University of Tsukuba. E-mail: [email protected], [email protected].
Takahiro Kinoshita, Shingo Hattori, Yuichi Hiroi, and Takefumi Hiraki are with Cluster Metaverse Lab. E-mail: {s.hattori, y.hiroi, t.hiraki}@cluster.mu.
Yuta Itoh is with the University of Tokyo. E-mail: [email protected].
\teaser
(a) Conceptual image of HaptoFloater. Haptic vibration control information is embedded in each pixel of the mid-air image using imperceptible color vibration. This light information controls a finger-worn haptic device. (b) Overview of the assembly of a mid-air display. It consists of a high-brightness LCD display and a micro-mirror array plate (MMAP). (c) Overview of the finger-worn haptic device. It has a light sensor for receiving color vibration at the tip and a tactile actuator to present vibration on the side of the fingernail. (d) HaptoFloater displays a visuo-haptic texture image as an application scenario of texture design. The user can not only see the texture of a mid-air image but can also touch it.