teaching machines to learn by metaphors
DESCRIPTION
Teaching Machines to Learn by Metaphors. Omer Levy & Shaul Markovitch Technion – Israel Institute of Technology. Concept Learning by Induction. Few Examples. Transfer Learning. Target (New). Source (Original). Define: Related Concept. Transfer Learning Approaches. - PowerPoint PPT PresentationTRANSCRIPT
Teaching Machines to Learn by Metaphors
Omer Levy & Shaul MarkovitchTechnion – Israel Institute of Technology
Concept Learning by Induction
Few Examples
Transfer Learning
Target (New)
Source (Original)
Define: Related Concept
Transfer Learning Approaches
• Common Inductive Bias
• Common Instances
• Common Features
Different Feature Space
Example
0 2 3-3 -2
Example
0 2 3-3 -2
0 4 9
Example
0 2 3-3 -2
0 4 9
𝑥𝑠=𝑥𝑡2
Common Inductive Bias
0 2 3-3 -2
0 4 9
Common Inductive Bias
0 2 3-3 -2
0 4 9
Common Instances
0 2 3-3 -2
0 4 9
Common Features
23
-3-2
4 9
𝑥𝑠=𝑦𝑡2
New Approach to Transfer Learning
Our Solution: Metaphors
Metaphors
Target (New)
Source (Original)
Concept Learner
Metaphor Learner
𝜇 h𝑠
Source
Target
+/-𝑥𝑡 𝑥𝑠
h𝑡 (𝑥𝑡 )=h𝑠 (𝜇 (𝑥𝑡 ))
is a perfect metaphor if:
1. is label preserving
2. is distribution preserving
Theorem
If is a perfect metaphor- and -
is a source hypothesis with error- then -
is a target hypothesis with error
The Metaphor Theorem
If is an -perfect metaphor- and -
is a source hypothesis with error- then -
is a target hypothesis with error
Redefine Transfer Learning
Given source and target datasets, find a target hypothesis such that is as small as possible.
Redefine Transfer Learning
Given source and target datasets, find an -perfect metaphor such that is as small as possible.
Metaphor Learning Framework
h
Concept Learning Framework
Search Algorithm
Hypothesis Space
Evaluation Function
Data
𝜇
Source
Target
Metaphor Learning Framework
Search Algorithm
Metaphor Space
Evaluation Function
Metaphor Evaluation
Metaphor Evaluation
1. is label preserving
2. is distribution preserving
Metaphor Evaluation
1. is label preservingEmpirical error over target dataset
2. is distribution preservingStatistical distance between and
Metaphor Evaluation
𝑆𝐷 (𝜇 (𝑥𝑡 ) ,𝑥𝑠 )
Metaphor Evaluation
𝑆𝐷 ¿
Metaphor Evaluation
𝑆𝐷 (𝜇 (𝑥𝑡− ) , 𝑥𝑠−)
Metaphor Evaluation
𝑆𝐷 ¿
Metaphor Spaces
Metaphor Spaces
• General
• Few Degrees of Freedom
• Representation-Specific Bias
Geometric Transformations
Я R
Dictionary-Based Metaphors
cheese queso
Linear Transformations
Which metaphor space should I use?
Automatic Selection of Metaphor Spaces
Which metaphor space should I use?
Occam’s Razor
Automatic Selection of Metaphor Spaces
Which metaphor space should I use?
Structural Risk Minimization
Occam’s Razor
Automatic Selection of Metaphor Spaces
Which metaphor space should I use?
Automatic Selection of Metaphor Spaces
ℳ1
ℳ2
ℳ3
ℳ 4
Automatic Selection of Metaphor Spaces
ℳ1
ℳ2
ℳ3
ℳ 4
𝜇1
𝜇2
𝜇3
𝜇4
Automatic Selection of Metaphor Spaces
ℳ1
ℳ2
ℳ3
ℳ 4
𝜇1
𝜇2
𝜇3
𝜇4
60 %
9 0 %
91 %
7 0 %
Empirical Evaluation
Reference MethodsBaseline• Target Only• Identity Metaphor• Merge
State-of-the-Art• Frustratingly Easy Domain Adaptation
– Daumé, 2007
• MultiTask Learning– Caruana, 1997; Silver et al, 2010
• TrAdaBoost– Dai et al, 2007
Digits: Negative Image
Digits: Negative Image
𝜇 (𝑥𝑡 )=1−𝑥𝑡
Digits: Negative Image
Digits: Higher Resolution
Digits: Higher Resolution
→
Digits: Higher Resolution
Wine
Wine
Qualitative ResultsTransfer Learning
TaskTarget
InstanceTarget Sample Size
1 2 5 10
Digits: Negative Image
Digits: Higher Resolution
Discussion
Recap
• Problem: Concept learning with few examples• Solution: Metaphors
Recap
• Problem: Concept learning with few examples• Solution: Metaphors
• Target Source
Recap
• Problem: Concept learning with few examples• Solution: Metaphors
• Target Source• Generic framework
Recap
• Problem: Concept learning with few examples• Solution: Metaphors
• Target Source• Generic framework• Wide range of relations
Recap
• Problem: Concept learning with few examples• Solution: Metaphors
• Target Source• Generic framework• Wide range of relations• Learn the difference
What if the concepts are not related?
What if the concepts are not related?
Metaphors are not a measure of relatedness
Metaphors are not a measure of relatedness
Metaphors explain how concepts are related
Vision
Explaining how concepts are related since 2012.M E T A P H O R S