joint eye tracking and head pose estimation for gaze estimation
DESCRIPTION
Joint Eye Tracking and Head Pose Estimation for Gaze Estimation. CS365 Project Guide - Prof. Amitabha Mukerjee. Ankan Bansal Salman Mohammad. Motivation. Human Computer Interaction Information about interest of the subject, e.g. advertisement research Analyze driver attention - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/1.jpg)
Joint Eye Tracking and Head Pose Estimation for Gaze Estimation
Ankan BansalSalman Mohammad
CS365 ProjectGuide - Prof. Amitabha Mukerjee
![Page 2: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/2.jpg)
Motivation• Human Computer Interaction• Information about interest of the
subject, e.g. advertisement research• Analyze driver attention• Device control by disabled people
through eye position and head pose.
![Page 3: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/3.jpg)
Past Work•Eye locations and head pose have been separately used for gaze estimation• Eye location algorithms are sensitive to head pose - Allow limited motion of head• Head pose estimation often requires multiple cameras• Head pose estimation alone fails to finely locate the gaze
![Page 4: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/4.jpg)
Approach• Appearance based• Integration of head tracker and eye location estimation• Head pose defines the field of view• Eye locations adjust the gaze estimation
![Page 5: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/5.jpg)
AlgorithmFace Detection
• Haar-like features for face detection• Viola Jones object detection framework• Classifier cascades
![Page 6: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/6.jpg)
AlgorithmFace Detection
• Isophotes – Curves connecting points of equal intensity• Eyes are characterized by radially symmetric brightness patterns• Find centres of the curved isophotes in the image
Eye Location
![Page 7: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/7.jpg)
AlgorithmFace
Detection
• Cylindrical Head Model• Pose comprises of rotation parameters and translation parameters• Tracking is done using the Lucas Kanade Algorithm• Initial Eye locations used as reference points• An area is sampled around each reference point and eye detector is applied to these regions• These eye locations are used to validate the tracking process
Eye Location
Head Pose Estimation
![Page 8: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/8.jpg)
AlgorithmFace
DetectionEye
LocationHead Pose Estimation
Sampling Points on the Face for Tracking
![Page 9: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/9.jpg)
AlgorithmFace
DetectionEye
LocationHead Pose Estimation
Face Motion Tracking
![Page 10: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/10.jpg)
AlgorithmFace
Detection
• Assumptions - Visual Field of View is defined by the head pose only• Point of interest is defined by the eyes• Visual Field of View can be approximated by a pyramid
Eye Location
Head Pose Estimation
Gaze Estimation
Image from [1]
![Page 11: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/11.jpg)
Tools and Data• Live data from web camera• Eye API from [3]• Boston University Dataset [5]
![Page 12: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/12.jpg)
Work Done• Face Detection• Eye Locations• 2D - Face Tracking
Promised but Not Done• 3D Head Pose Estimation and Tracking• Visual Gaze Estimation
![Page 13: Joint Eye Tracking and Head Pose Estimation for Gaze Estimation](https://reader033.vdocuments.site/reader033/viewer/2022061522/568165be550346895dd8bc3c/html5/thumbnails/13.jpg)
References1. Roberto Valenti, Nicu Sebe, Theo Gevers: Combining Head Pose and Eye
Location Information for Gaze Estimation. IEEE Transactions on Image Processing 21(2): 802-815 (2012)
2. Jing Xiao, Takeo Kanade, Jeffrey F. Cohn: Robust Full-Motion Recovery of Head by Dynamic Templates and Re-Registration Techniques. FGR 2002: 163-169
3. Roberto Valenti, Theo Gevers: Accurate eye center location and tracking using isophote curvature. CVPR 2008
4. Roberto Valenti, Zeynep Yücel, Theo Gevers: Robustifying eye center localization by head pose cues. CVPR 2009: 612-618
5. http://csr.bu.edu/headtracking/uniform-light/6. Jun-Su Jang, Takeo Kanade: Robust 3D Head Tracking by Online Feature
Registration. IEEE International Conference on Automatic Face and Gesture Recognition, September, 2008.
7. Paul Viola, Michael Jones: Robust real-time object detection. IJCV, 2001