Survey of finger gesture sensing(working memo)

Yoshiharu Sato

Scopes

• Not • 3D gesture in air

• But• Gesture on a surface (palm, desk, etc)• Simple gestures

Optical approach

• Infrared Camera (Depth of Field)• Leap Motion, MSR, HP• Advantages

• Remote sense (no obstacle on fingers)• Technology is matured, Already in commercial product

• Disadvantages• Problem of occlusion• Need an artifact in environment (eg desktop)

• Finger nail sensor• “Photoplethysmograph Fingernail Sensors for Measuring Finger Forces

Without Haptic Obstruction”, MIT, 2001. • The force on finger is detected by an optical sensor mounted on the fingernail.

• Advantages• Can capture finger tactile information (pressure)

• Disadvantages• Artifact on nail

Acoustic approach

• “The Sound of One Hand: A Wrist-mounted Bio-acoustic Fingertip Gesture Interface”, ATT, CHI2002• It uses piezo microphone in wrist band. It can classify tap/rub/flick

gestures and distinguish them from everyday life motions.

• “Skinput: Appropriating the Body as an Input Surface”, MSR, 2010 • It uses a custom piezo film array sensor in arm band. It can classify

position on one hand by the other hand’s tap. It can classify which finger tap is made against a thump finger.

• Advantages• Remote sense (no obstacle on fingers)• Can classify simple gestures and their position on body wide surface

• Disadvantages• ?

Accelerometer approach

• “Body Coupled FingeRing: Wireless Wearable Keyboard", CHI 97• Sensors are attached at fingers, output is converted to a frequency spectrum which is

filtered by a band-pass.

• “Finger Ring Device for Tactile Sensing and Human Machine Interface” (2007), “Finger Ring Tactile Interface Based on Propagating Elastic Waves on Human Fingers”, U-Tokyo• A sensor is attached at its finger base. It can classify movement and tap of fingertip. It can

estimate a finger movement distance on a hard surface.

• https://www.jstshingi.jp/abst/p/11/1151/aizu05.pdf, Aizu Univ.• Shiseido https://www.shiseidogroup.com/releimg/1959-e.pdf , 2011• Ring http://logbar.jp/ring/ • Advantages

• Matured technology (though Ring development is in trouble now)• Can detect simple gestures• Artifact on finger base is not an obstacle of everyday life.

• Disadvantages• ?