wave to me: user identification using body lengths and natural gestures, at chi 2014
DESCRIPTION
We introduce a body-based identification system that leverages individual differences in body segment lengths and hand waving gesture patterns. The system identifies users based on a two-second hand waving gesture captured by a Microsoft Kinect. To evaluate our system, we collected 8640 gesture measurements from 75 participants through two lab studies and a field study. In the first lab study, we evaluated the feasibility of our concept and basic properties of features to narrow down the design space. In the second lab study, our system achieved a 1% equal error rate in user identification among seven registered users after two weeks following initial registration. We also found that our system was robust even when lower body segments could not be measured because of occlusions. In the field study, our system achieved 0.5 to 1.6% equal error rates, demonstrating that the system also works well in ecologically valid situations. Lastly, throughout the studies, our participants were positive about the system.TRANSCRIPT
Wave to Me: User Identification Using Body Lengths and Natural Gestures
Eiji HayashiManuel Maas
Jason Hong
Human-Computer Interaction InstituteCarnegie Mellon University
Slick user identification
Slick user identificationwith reasonable security
Gesture
EachDifferent Gesture for user
AllSame Gesture for user
11
97% accurate
SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study
SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study
Body-basedUser Identification
Overview
Body-basedUser Identification
Registration
Body-basedUser Identification
Registration
Identification
Body-basedUser Identification
Registration
Identification
User IDReject
Kinect
Kinect
Kinect
JointPositions
KinectFeatureExtraction
JointPositions
KinectFeatureExtraction
Physiological
17 body segmentlengths
KinectFeatureExtraction
Physiological
17 body segmentlengths
Behavioral
26 movementproperties
KinectFeatureExtraction
43 Features
KinectFeatureExtraction
Feature Vector
KinectFeatureExtraction
SVM
Feature Vector
KinectFeatureExtraction
SVM Pre-RecordedData
Feature Vector
KinectFeatureExtraction
SVM Pre-RecordedData
User ID + Confidence
KinectFeatureExtraction
SVM
Threshold
Pre-RecordedData
User ID + Confidence
KinectFeatureExtraction
SVM
Threshold
Pre-RecordedData
User ID or Reject
Errors
False Acceptance Rate (FAR)Accept others as a registered user
False Rejection Rate (FRR)Reject a registered user as others
Errors
False Acceptance Rate (FAR)Accept others as a registered user
False Rejection Rate (FRR)Reject a registered user as others
Equal Error Rate (EER)FAR = FRR = EER
Errors
False Acceptance Rate (FAR)Accept others as a registered user
False Rejection Rate (FRR)Reject a registered user as others
Equal Error Rate (EER)FAR = FRR = EER
Accuracy = 1 – 2 x EER
Assumption
There are 7 registered users in our system
Assumption
There are 7 registered users in our system
Make comparison among studies easy
Assumption
There are 7 registered users in our system
Make comparison among studies easy
Be reasonable for home use
Assumption
There are 7 registered users in our system
Choose 10,000 combination of 7 participants
Calculate EER over them
SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study
Gestures
Hand Waving
Come-Over
One Hand Raised
Making a Phone Call
Data Collection
Hand WavingCome-OverOne Hand RaisedPhone Call
Gesture
Data Collection
Hand WavingCome-OverOne Hand RaisedPhone Call
StandingSitting
Gesture Posture
Data Collection
Hand WavingCome-OverOne Hand RaisedPhone Call
StandingSitting
1st Day3 days later
Gesture Posture Session
Data Collection
Hand WavingCome-OverOne Hand RaisedPhone Call
StandingSitting
1st Day3 days later
Gesture Posture Session
10
Data Collection
Hand WavingCome-OverOne Hand RaisedPhone Call
StandingSitting
1st Day3 days later
Gesture Posture Session
10
160 / participants
Participants
36 participants
14 males / 22 females
19 – 64 years old
168cm (SD=10.2)
78.0 kg (SD=22.0)
Gestures
Hand Waving
Come-Over
One Hand Raised
Making a Phone Call
Using Either Gesture or Lengths
Same day & posture
3 days later
Different Posture
Same day & posture
3 days later
Different Posture
Gesture Body Lengths
EER [%]
Using Either Gesture or Lengths
Same day & posture
3 days later
Different Posture
Same day & posture
3 days later
Different Posture
Gesture Body Lengths
EER [%]
2.1%
0.5%
Using Either Gesture or Lengths
Same day & posture
3 days later
Different Posture
Same day & posture
3 days later
Different Posture
Gesture Body Lengths
EER [%]
11.8%
19.8%
Using Either Gesture or Lengths
Same day & posture
3 days later
Different Posture
Same day & posture
3 days later
Different Posture
Gesture Body Lengths
EER [%]
10.0%
41.5%
Using Both Gesture and Lengths
3 days later
3 days later
Gesture Body Lengths
EER [%]
Both
3 days later (Standing)
3 days later (Sitting)
4.3%
6.2%
SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study
Data Collection
Hand WavingStandingSitting
1st Day3 days later1 week later2 weeks later
Gesture Posture Session
10
80 / participants
Participants
27 participants
20 males / 7 females
19 – 62 years old
173cm (SD=9.8)
75.1 kg (SD=21.1)
Long term StabilityE
ER
[%]
Days
Long term StabilityE
ER
[%]
Days
Sitting
Standing
Long term StabilityE
ER
[%]
Days
Stable after the 3rd session
Training with 2 sessionsE
ER
[%]
Days
Training with 2 sessionsE
ER
[%]
Days
Sitting
Standing
Training with 2 sessionsE
ER
[%]
Days
EER < 1%
SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study
Does it Work at Homes?
• Collected data at participants’ living rooms• Placed a Kinect on a TV• Asked participant to behave as usual
– Stand where you feel reasonable– Sit as you normally do in a living room
Participants
12 participants (5 house hold)
5 males / 7 females
18 – 42 years old
159.5cm (SD=11.4)
56.9 kg (SD=6.9)
It Worked!
EER [%]
Standing
Sitting
Standing
Sitting
Lab Study 2 Field Study
Implication
Recognizing a gesture
Recognizing a gesture AND a user’s identity
• Natural gestures + body lengths• 2 seconds of hand waving gesture
Conclusion
Gesture
Body Lengths
Proposed Scheme
EER [%]
Wave to Me: User Identification Using Body Lengths and Natural Gestures
Eiji [email protected]/~ehayashi/
Human-Computer Interaction Institute
Carnegie Mellon University
Backup slides
# of Registered UsersE
ER
[%]
# of Registered Users
# of Registered UsersE
ER
[%]
# of Registered Users
2.8%
2.3%
N=25
Open Questions
Getting worse constantly?
Training with two sessions?
Data Collection
Hand WavingStandingSitting
1st Session2nd Session3rd Session
Gesture Posture Session
10
60 / participants
Yet Another Open Question
Does it actually work at home?