learning on the fly: rapid adaptation to the image
DESCRIPTION
Learning on the Fly: Rapid Adaptation to the Image. Erik Learned-Miller with Vidit Jain, Gary Huang, Laura Sevilla Lara, Manju Narayana , Ben Mears . “Traditional” machine learning. Learning happens from large data sets With labels: supervised learning - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/1.jpg)
Computer Science Department
Learning on the Fly:Rapid Adaptation to the Image
Erik Learned-Millerwith Vidit Jain, Gary Huang,
Laura Sevilla Lara, Manju Narayana, Ben Mears
![Page 2: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/2.jpg)
2Learning on the Fly
“Traditional” machine learning Learning happens from large data sets
• With labels: supervised learning• Without labels: unsupervised learning• Mixed labels: semi-supervised learning,
transfer learning,learning from one (labeled) example,self-taught learning,domain adaptation
![Page 3: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/3.jpg)
3Learning on the Fly
Learning on the Fly Given:
• A learning machine trained with traditional methods• a single test image (no labels)
Learn from the test image!
![Page 4: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/4.jpg)
4Learning on the Fly
Learning on the Fly Given:
• A learning machine trained with traditional methods• a single test image (no labels)
Learn from the test image!• Domain adaptation where the “domain” is the new
image• No covariate shift assumption.• No new labels
![Page 5: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/5.jpg)
5Learning on the Fly
An Example in Computer Vision Parsing Images of Architectural Scenes
Berg, Grabler, and Malik ICCV 2007.• Detect easy or “canonical” stuff.• Use easily detected stuff to bootstrap models of harder
stuff.
![Page 6: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/6.jpg)
6Learning on the Fly
Claim This is so easy and routine for humans that it’s
hard to realize we’re doing it.• Another example…
![Page 7: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/7.jpg)
7Learning on the Fly
Learning on the fly…
![Page 8: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/8.jpg)
8Learning on the Fly
Learning on the fly…
![Page 9: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/9.jpg)
9Learning on the Fly
Learning on the fly…
![Page 10: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/10.jpg)
10Learning on the Fly
What about traditional methods… Hidden Markov Model for text recognition:
• Appearance model for characters• Language model for labels• Use Viterbi to do joint inference
![Page 11: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/11.jpg)
11Learning on the Fly
What about traditional methods… Hidden Markov Model for text recognition:
• Appearance model for characters• Language model for labels• Use Viterbi to do joint inference
DOESN’T WORK!
Prob( |Label=A) cannot be well estimated, fouling up the whole process.
![Page 12: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/12.jpg)
12Learning on the Fly
Lessons We must assess when our models are broken,
and use other methods to proceed….• Current methods of inference assume probabilities are
correct!• “In vision, probabilities are often junk.”• Related to similarity becoming meaningless beyond
a certain distance.
![Page 13: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/13.jpg)
13Learning on the Fly
2 Examples Face detection (CVPR 2011) OCR (CVPR 2010)
![Page 14: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/14.jpg)
14Learning on the Fly
Preview of results: Finding false negatives
Viola-Jones Learning on the Fly
![Page 15: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/15.jpg)
15Learning on the Fly
Eliminating false positives
Viola-Jones Learning on the Fly
![Page 16: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/16.jpg)
16Learning on the Fly
Eliminating false positives
Viola-Jones Learning on the Fly
![Page 17: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/17.jpg)
17Learning on the Fly
Run a pre-existing detector...
![Page 18: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/18.jpg)
18Learning on the Fly
Run a pre-existing detector...
Key
Face
Non-face
Close to boundary
![Page 19: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/19.jpg)
19Learning on the Fly
Gaussian Process Regression
negative positive
learn smooth mappingfrom appearance to score
apply mapping to borderline patches
![Page 20: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/20.jpg)
20Learning on the Fly
Major Performance Gains
![Page 21: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/21.jpg)
21Learning on the Fly
Comments No need to retrain original detector
• It wouldn’t change anyway! No need to access original training data Still runs in real-time GP regression is done for every new image.
![Page 22: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/22.jpg)
22Learning on the Fly
Noisy Document
We fine herefore tlinearly rolatcd to thewhen this is calculated equilibriurn. In short,on the null-hypothesis:
Initial Transcription
![Page 23: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/23.jpg)
23Learning on the Fly
Premise We would like to fine confident words
to build a document-specific model,but it is difficult to estimate Prob(error).
However, we can bound Prob(error). Now, select words with
• Prob(error)<epsilon.
![Page 24: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/24.jpg)
24Learning on the Fly
“Clean Sets”
![Page 25: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/25.jpg)
25Learning on the Fly
Document specific OCR Extract clean sets (error bounded sets) Build document-specific models from clean set
characters Reclassify other characters in document
• 30% error reduction on 56 documents.
![Page 26: Learning on the Fly: Rapid Adaptation to the Image](https://reader035.vdocuments.site/reader035/viewer/2022062502/568164d8550346895dd71df5/html5/thumbnails/26.jpg)
26Learning on the Fly
Summary Many applications of learning on the fly. Adaptation and bootstrapping new models is
more common in human learning than is generally believed.
Starting to answer the question: “How can we do domain adaptation from a single image?”