haptic technology that indicates shape, hardness …...haptic technology that indicates shape,...

1
REPORT 22 With advances in Augmented Reality (AR) /Virtual Real- ity (VR) technology, it is becoming possible to experience being immersed in a world of content by having characters appear to spring out from the television screen, or by wear- ing a head-mounted display device. At STRL, we are re- searching and developing haptic technology that enhances the immersive realism of AR and VR by providing suitably timed stimuli when the viewer attempts to touch objects that exist only in these virtual worlds, thereby allowing the viewer to feel the shape, hardness and movement of these virtual objects. Control technology for the representation of shape and size Since virtual objects in AR/VR spaces are displayed using computer graphics (CG) and do not exist in the real world, there is a need for a certain mechanism that can convey the shape and size of these objects when the viewer tries to touch them. We have therefore developed a device that expresses irregularities on the object surface by moving four stimulus points up and down on the pad of a finger. This device can also convey the size of an object by changing the overall size of the device. We have also developed a technique to control the positions of these tactile stimuli with suitable timing by using a high-speed camera to acquire the spatial position of the hand. Force control technology for the representation of hardness We have developed a technique to control the reaction force with which the surface of an object pushes back against the fingers in accordance with the hardness of the object when the viewer tries to grasp it. This technique involves dividing a CG virtual object into cubes (voxels) of arbitrary size and calculating the magnitude of the reaction force based on the hardness of each voxel. This makes it possible to assign different levels of hardness to different parts of the same object. We hope that these technologies will contribute to the spread of AR and VR and facilitate new ways of repre- senting content so that it can be enjoyed even by people with impaired vision. This technology can also be used to represent moving objects as well as static objects. For ex- ample, it could increase the enjoyment of sports programs by recreating the impact of a baseball being caught in a player’s glove. In the future, we will continue to research and develop technologies that can represent content in new ways. * Haptics: Technology that uses force vibrations or the like to convey the sensation of objects touching the skin Takuya Handa Haptic Technology That Indicates Shape, Hardness and Movement of Objects Tactile stimulus points (positional controls) on the fingers and palm Real space The device represents differences in the shape, size, and hardness of the touched part Synchronization of virtual CG and the state of the device in the real world CG indicating the state of contact between the haptic stimulus points and the object Hardness distribution data CG in virtual space Soft Hard Motor (force control) for the presentation of hardness The relationship between motion in real space and CG in virtual space Devices that represent the shape and hardness of objects

Upload: others

Post on 02-Jun-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Haptic Technology That Indicates Shape, Hardness …...Haptic Technology That Indicates Shape, Hardness and Movement of Objects Tactile stimulus points (positional controls) on the

REPORT

22

With advances in Augmented Reality (AR) /Virtual Real-ity (VR) technology, it is becoming possible to experience being immersed in a world of content by having characters appear to spring out from the television screen, or by wear-ing a head-mounted display device. At STRL, we are re-searching and developing haptic technology that enhances the immersive realism of AR and VR by providing suitably timed stimuli when the viewer attempts to touch objects that exist only in these virtual worlds, thereby allowing the viewer to feel the shape, hardness and movement of these virtual objects.

Control technology for the representation of shape and size

Since virtual objects in AR/VR spaces are displayed using computer graphics (CG) and do not exist in the real world, there is a need for a certain mechanism that can convey the shape and size of these objects when the viewer tries to touch them. We have therefore developed a device that expresses irregularities on the object surface by moving four stimulus points up and down on the pad of a finger. This device can also convey the size of an object by changing the overall size of the device. We have also developed a technique to control the positions of these tactile stimuli with suitable timing by using a high-speed camera to acquire the spatial position of the hand.

Force control technology for the representation of hardness

We have developed a technique to control the reaction force with which the surface of an object pushes back against the fingers in accordance with the hardness of the object when the viewer tries to grasp it. This technique involves dividing a CG virtual object into cubes (voxels) of arbitrary size and calculating the magnitude of the reaction force based on the hardness of each voxel. This makes it possible to assign different levels of hardness to different parts of the same object.

We hope that these technologies will contribute to the spread of AR and VR and facilitate new ways of repre-senting content so that it can be enjoyed even by people with impaired vision. This technology can also be used to represent moving objects as well as static objects. For ex-ample, it could increase the enjoyment of sports programs by recreating the impact of a baseball being caught in a player’s glove. In the future, we will continue to research and develop technologies that can represent content in new ways.

* Haptics: Technology that uses force vibrations or the like to convey the sensation of objects touching the skin

Takuya Handa

Haptic Technology That Indicates Shape, Hardness and Movement of Objects

Tactile stimulus points (positional controls)

on the fingers and palmReal space The device represents differences in the

shape, size, and hardness of the touched part

Synchronization of virtual CG and the state of the device in the real world

CG indicating the state of contact between the haptic stimulus points and the object

Hardness distribution data

CG in virtual space Soft Hard

Motor (force control) for the presentation of hardness

The relationship between motion in real space and CG in virtual space

Devices that represent the shape and hardness of objects