gesture technology and recognition

GESTURE TECHNOLOGY

DESIGNED BYABHISHEK PAL

A little about me

My name is Abhishek Pal. I am an IT Engineer

I am from India and i love interacting with new people.

I love visiting new places.

Contact -abhishekpal57@gmail.com

CONTENTS

CONTENTS

What is Gesture

Types of Gestures

What is Gesture recognition

Basic working of Gesture Technology

Types of Gesture Recognition

Technology behind it

Uses of Gesture Recognition

Gesture Recognition Challenges

GESTURE, what?

A gesture is a form of non-verbal communication in which visible bodily actions communicate particular messages, either in place of speech or together and in parallel with words.

What Are GESTURES

Gestures are an important aspect of human interaction, both interpersonally and in the context of man-machine interfaces.

Gestures include movement of the hands, face, or other parts of the body.

Military air marshals use hand and body gestures to direct flight operations aboard aircraft carriers.

TYPES OF GESTURE

Types of GestureIconic

 Iconic gestures represent object attributes, spatial relationships, and actions. For example, think about how  you would describe

gesturally a tall person or a wide river as you were talking about each of those things.

Deictic, or Pointing These gestures are pretty basic, connecting speech to another idea, object of location. For instance, if you were talking about

someone across the room you might point them.

Metaphoric Metaphoric gestures put an abstract idea into a more literal,

concrete form. Making your fingers into a v shape and raising above indicates your victory or win

 Beat Gestures Basically, these just keep the rhythm of speech, and they

convey no semantic content whatsoever

GESTURE RECOGNITION

What is Gesture Recognition ?

Interface with computers using gestures of the human body, typically hand movements.

Gesture recognition is an important skill for robots that work closely with humans.

Gesture recognition is especially valuable in applications involving interaction human/robot for several reasons.

A smart T.V recognizing the hand movement to control the channels or to select a option from

the menu

A gesture recognition

software in a laptop that

recognizes the movement of

fingers used for performing various

tasks

BASIC WORKING

Basic working of Gesture Technology

TYPES OF GESTURE

Types of Gesture Recognition

Hand Gesture Recognition

Hand gesture recognition technology allows operations of complex machines using only a series of fingers and hand movements , eliminating the need of physical contact between operator and

machine

Facial Gesture Recognition

Facial Gesture Recognition creates an effective non-

contact interface between user and their machines

The goal of facial Gesture Recognition is to make

machine effectively understand human

emotions , regardless of the countless physical differences between

individuals

Even most cameras detects your smile to automatically

click a picture

Sign Language Recognition

Certain types of gesture recognition software can transcribe the symbols

represented through sign language into text

Effective sign language recognition can help the

deaf people to communicate with both other people as

well as the machine

Electrical Field Sensing Technology

Proximity of a human body or body part can be

measured by sensing electric fields .

These measurements can be used to measure the

distance of a human hand or other body part from

an object; this facilitates a vast range of applications

for a wide range of industries.

Types of Gesture Sensing Technology

Device Sensing Technology

Device-based techniques use a glove, stylus, or other position

tracker, whose movements send signals that the system uses to identify the gesture.

The glove is equipped with a variety of sensors to provide

information about hand position, orientation, and flex of fingers.

Vision Based Technologies

There are two approaches to vision based gesture

recognition:

Model based techniques:

They try to create a three dimensional

model of the users hand and use this for

recognition. Image based

methods:Image-based techniques

detect a gesture by capturing pictures of a user’s motions during

the course of a gesture.

TECHNOLOGY BEHIND IT

Technology Behind It

Wired gloves These can provide input to the

computer about the position and rotation of the hands using

magnetic or inertial tracking devices.

The first commercially available hand-tracking glove-type device

was the Data Glove , a glove-type device which could detect hand position, movement and finger

bending.

This uses fiber optic cables running down the back of the hand. Light pulses

are created and when the fingers are bent, light leaks through small cracks and the loss is registered, giving an

approximation of the hand pose.

Stereo cameras

A Stereo camera is a camera that has two lenses about the same distance apart as your eyes and takes two pictures at the same

time. This simulates the way we actually see and therefore creates

the 3D effect when viewed.

Using two cameras whose relations to one another are known, a 3D representation can be approximated by the

output of the cameras.

Depth-aware cameras

Using specialized cameras such as structured

light or time-of-flight cameras, one can generate a depth map of what is being seen through the camera at a short range, and use this data to approximate a 3d representation of what is

being seen.

These can be effective for detection of hand gestures due to

their short range capabilities.

Thermal camerasAn infrared camera is a device that

detects infrared radiation(temperature) from the

target object and converts it into an electronic signal to generate a

thermal picture on a monitor or to make temperature calculations on it.

The temperature which is captured by an infrared camera can be measured or quantified

exactly, so that not only the thermal behavior can be observed but also the relative magnitude of temperature related problems can

be recognized and noted.

Controller –based gesturesThese controllers act as an extension of the body so that

when gestures are performed, some of their motion can be conveniently captured by software. 

Single camera or a Proximity sensor

A normal camera or a proximity sensor can be used for gesture recognition where the resources/environment would not be

convenient for other forms of image-based recognition.

USES OF GESTURE RECOGNITION

Uses Of Gesture RecognitionSocially assistive robotics

By using proper sensors worn on the body of a patient and by reading the

values from those sensors, robots can assist in patient rehabilitation. The

best example can be stroke rehabilitation.

Sign language recognition

Just as speech recognition can transcribe speech to text, certain

types of gesture recognition software can transcribe the symbols

represented through sign language into text.

Virtual controllers

For systems where the act of finding or acquiring a physical controller

could require too much time, gestures can be used as an alternative control mechanism. Controlling secondary

devices in a car, or controlling a television set are examples of such

usage.

Remote control

Through the use of gesture recognition, remote control with the wave of a hand of various

devices is possible.

Aid to physically challenged

Machine wheel chairs are coming with gesture based system. The movement of

the hand will act as a controller of speed as well as

direction

Immersive gaming technology

In video game consoles such as Microsoft X-box with kinect sensors, often the user is the controller and has to perform all physical movements that they desire the character in

the game to do.

GESTURE RECOGNITION CHALLENGES

Lack of Gesture Language

Gesture Recognition Challenges

Different users make gestures differently, causing difficulty in identifying motions

Many gesture recognition systems do not read motions accurately or

optimally due to factors like insufficient background light, high

background noise etc

Robustness