attentional cascade improvements research by jeffrey a. edlund and greg s. griffin learning systems...
TRANSCRIPT
Attentional CascadeImprovements
Research by Jeffrey A. Edlund and Greg S. GriffinLearning Systems CS 156b, Mar 9 2006
Simple Training Sets
Suggested Improvements
1. Dual Cascade– When: data is balanced– Why: faster
2. Fade Cascade– When: data is unbalanced– Why: fewer training examples
required
Dual Cascade: What Is It
NHU=4 NHU=8 NHU=12
€
⇒
€
⇒
€
⇒
€
⇓
€
⇓
€
⇓
Early Rejection
Dual Cascade: What Is It
NHU=4 NHU=8 NHU=12
€
⇒
€
⇒
€
⇒
€
⇓
€
⇓
€
⇓€
⇑
€
⇑
€
⇑
Early Rejection
Early Acceptance
Unbalanced Data
Balanced Data
Dual Cascade: Pros & Cons
• Major Advantage: speed– ~ 2 - 4 times faster on equally balanced data sets– Even faster if you have more positive examples– Little or no increase if positive examples are rare
• e.g. Viola and Jones
• Minor Disadvantage: accuracy– Small increase in out-of-sample error
Dual Cascade: Applications
Fade Cascade AddressesThe Big Problem:
ntest =16,000
x ~ 2
x ~ 10
Fade Cascade
• Instead of throwing out points that are clearly positive or clearly negative, we reduce the weights for those points and renormalize.
Balanced Data
Unbalanced Data
Fade Cascade: Pros & Cons
• Major Advantage: fewer training examples– ~ 2 - 4 times less training data required, for the
same out-of-sample error
• Minor Disadvantage: less efficient training– All data points are now used for training, at all
levels of the cascade (we’re weighting them instead of dropping them)
– But: we now require less training data
Suggested Improvements
1. Dual Cascade– When: data is balanced– Why: faster
2. Fade Cascade– When: data is unbalanced– Why: fewer training examples
required
Unbalanced Data
Balanced Data
Unbalanced Data
Preliminary!
Balanced Data
Preliminary!
“Smart” Cascade
• Improved performance on both balanced and unbalanced datasets?
• We don’t know yet!