sign language recognition using hmm
DESCRIPTION
TRANSCRIPT
![Page 1: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/1.jpg)
Sign Language Recognition
UsingHidden Markov
ModelPresented by:
Vipul Agarwal - 070905060
![Page 2: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/2.jpg)
Outline#INTRODUCTION #SIGN LANGUAGE#PRE-PROCESSING #SKIN AND HAND
DETECTION#OPTICAL FLOW ANALYSIS#FEATURE EXTRACTION FOR
TRAINING DATA#HIDDEN MARKOV MODEL &
ITS USE#PROGRESS REPORT#DEMONSTRATION
![Page 3: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/3.jpg)
Introduction#Interaction with computers may often not
be a comfortable experience.
#Computers should be able to communicate with people with body language.
#Hand gesture recognition becomes important …– Interactive human-machine interface and
virtual environment
![Page 4: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/4.jpg)
Introduction#Two common technologies for hand
gesture recognition
– GLOVE-BASED METHOD• Using special glove-based device to extract
hand posture
– VISION-BASED METHOD• 3D hand/arm modeling• Appearance modeling
![Page 5: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/5.jpg)
Introduction
#3D hand/arm modeling– Highly computational complexity – Using many approximation process
#Appearance modeling– Low computational complexity– Real-time processing
![Page 6: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/6.jpg)
Sign Language#Rely on the hearing society#Two main elements:
– Low and simple level signed alphabet, mimics the letters of the spoken language.
– Higher level signed language, using actions to mimic the meaning or description of the sign.
#The project aim is to make the computer recognize low and simple level American Sign Language.
![Page 7: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/7.jpg)
Sign Language
#American Sign Language
#26 signs to denote the alphabets.
#10 signs to denote numbers
![Page 8: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/8.jpg)
Pre - ProcessingThe video sequence used has a lot of noise due to:
#Low quality of the webcam
#Improper lighting conditions
#Background
![Page 9: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/9.jpg)
Pre - Processing
Pre-processing involves reducing the noise and illumination problems.The morphological operations used for reducing the noise involves:
#Dilation#Statistical Elimination
![Page 10: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/10.jpg)
Pre - ProcessingDILATION>#A disc shaped region is traversed over
every blob and the ones which do not fit the disc are removed completely.
![Page 11: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/11.jpg)
Pre - ProcessingSTATISTICAL ELIMINATION>
#For every region the area is computed. Since hand is the one with the largest area, all blobs having less than a specified area are removed.
![Page 12: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/12.jpg)
Hand Detection#First all the noise is removed in the
pre-processing stage.#Now we assume that the hand is the
largest skin blob in our video sequence.
#We calculate the area of every blob and take the one with the largest area.
#We also calculate the bounding box of the region containing the hand for further analysis
![Page 13: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/13.jpg)
Hand Detection
![Page 14: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/14.jpg)
Optical Flow Analysis
DEFINITION:#Optical flow is the pattern of
apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer (an eye or a camera) and the scene.
![Page 15: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/15.jpg)
Optical Flow Analysis
Why Optical Flow Analysis?#Till now the system is just
able to detect the hand and follow the bounding box as the hand moves.
#The problem now is that we need to define a way to take a snapshot of the hand when the hand is not moving.
![Page 16: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/16.jpg)
Optical Flow Analysis
Using this technique we find the motion in the hand. When the hand has stabilized, we assume that the gesture is ready. We then take a snapshot of the hand and perform the recognition on that image.
![Page 17: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/17.jpg)
Feature ExtractionFor training the network with test images we perform the following feature extraction technique:-#Thresholding of the test hand#Converting to a binary image#Finding the centroid of the hand and
orientation of the minor axis.#Making feature vectors using a predefined
number of features.
![Page 18: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/18.jpg)
Feature Extraction
#Extracting the intersection of the feature vectors with the boundary points.
#Finding the scalar length of the vectors from the centroid.
#Normalising the lengths in a scale of 1 to 100 to make it scaling invariant.
![Page 19: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/19.jpg)
Feature Extraction
![Page 20: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/20.jpg)
Hidden Markov Model (HMM)
• HMMs allow you to estimate probabilities of unobserved events
• Given plain text, which underlying parameters generated the surface
![Page 21: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/21.jpg)
HMMs and their Usage• HMMs are very
common in Computational Linguistics:
– GESTURE RECOGNITION (observed: image, hidden: alphabets)
![Page 22: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/22.jpg)
Progress ReportWORK COMPLETED:#Data Collection#Pre-processing #Skin And Hand
Detection#Optical Flow Analysis#Feature Extraction For
Training DataWORK REMAINING:#Training The Hidden
Markov Model
![Page 23: sign language recognition using HMM](https://reader033.vdocuments.site/reader033/viewer/2022061122/546ff0ecaf7959f5288b45b6/html5/thumbnails/23.jpg)
Any Questions …?