translated learning
TRANSCRIPT
![Page 1: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/1.jpg)
Translated Learning:Transfer learning across different feature spacesWenyuan Dai, Yuqiang Chen, Gui-Rong Xue, Qiang Yang, and Yong Yu. In Proceedings of Twenty-Second Annual Conference on Neural Infor-mation Processing Systems (NIPS 2008)
![Page 2: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/2.jpg)
Definition Transfer Learning across different feature spaces When labeled data are more insufficient in target feature space than in other feature spaces.
E.g. Web data(text document > images), cross-language classification(English > Bangla, or other languages..)
![Page 3: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/3.jpg)
Human Learning Example Task: tyrannosaurus vs stegosaurus - tyrannosaurus: bipedal carnivore with a mas-
sive skull balanced by long, heavy tail. Its fore-limbs were small and retained only two digits.
- stegosaurus: quadruped ornithischian dinosaur of four long bony spikes on a flexible tail and two rows of upright triangular bony plates running along the back.
![Page 4: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/4.jpg)
Model-level Translation
LearningInput Output
LearningInput Output
Elephants are big mammals on earth...massivehoofedmammalof Africa... translating learning
models
make the best use of available data that have both features of the source and target domains
![Page 5: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/5.jpg)
Model-level Translation A language model to link the class labels to the features in the source spaces, which is translated to the features in the target spaces. It is com-pleted by tracing back to the instances in the tar-get spaces
๐ ๐ฆ ๐ ๐ฆ ๐ก ๐ฅ๐กFeature-level translator
Features in source space
Features in target space
๐ ๐ฆ ๐ ๐ฅ๐ ๐ ๐ฆ ๐ก ๐ฅ๐ก
![Page 6: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/6.jpg)
Translated Learning Classify the instances as accurately as possible using the labeled training data and the translator .
Elephants are big mammals on earth...
massivehoofedmammalof Africa...
source spacelabeled unlabeledlabeled
target space
๐ (๐ฆ๐ก|๐ฆ ๐ )โ๐ (๐ฆ ๐ก , ๐ฆ ๐ )
๐ฅ๐ =(๐ฆ๐ 1 ,โฆ, ๐ฆ ๐
๐๐ ) ๐ฅ๐ก ๐ฅ๐ขโ๐ โ๐ก ๐ฐ
feature translator
![Page 7: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/7.jpg)
Risk Minimization Frame-work Risk function measure the risk for classifying to the category .
Loss function loss with respect to the event of and being rele-
vant.
the label of is Models with respect to and involving all the possible models
Distance function between two models and
![Page 8: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/8.jpg)
Model Estimation Approximate the risk function as
Assume there is no prior difference among all the classes
This prior balances the influence of different classes in the class-imbalance case
![Page 9: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/9.jpg)
Translated Learning via Risk Min-imization
Training phase For each
Test phase For each
Predict the label ,
Source space labeled data Target space labeled data
๐ ๐ฆ ๐ ๐ฆ ๐ก ๐ฅ๐ก๐๐ ๐๐ฅ๐ก
![Page 10: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/10.jpg)
Feature Translator ๐ฅ๐ ๐ฅ๐ก
๐ฆ ๐ ๐ฆ ๐ก
Source(text)Target(images)
Instance
Feature social annotations on images
Search engine results in response to queries
Web pages including text and pictures
If we use instance-level co-occurrence data,
![Page 11: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/11.jpg)
EvaluationText-aided Image Classification
Images from Caltech-256, Documents from ODP(Open Directory Project) Auxiliary data: binary labeled text documents Objective: Image classification when co-occurrence data is insuffi-cient Evaluated under three dissimilarity functions
Kullback-Leibler divergence(KL), Negative of cosine function(NCOS), Negative of the Pear-sonโs correlation coefficient(NPCC)
Co-occurrence data collected from a image search engine,
> >
![Page 12: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/12.jpg)
EvaluationText-aided Image Classification
(the size of )
![Page 13: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/13.jpg)
EvaluationText-aided Image Classification
the classification model relies more on the auxiliary text training data
![Page 14: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/14.jpg)
EvaluationCross-language Classification
Dataset from ODP English/German pages English documents are used to help classify German documents only 16 German labeled documents are available in each category co-occurrence data: English-German dictionary, NCOS is used for dissimilarity function Assume that machine translation is unavailable and they rely on dictionary only
![Page 15: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/15.jpg)
EvaluationCross-language Classification
when is small, the performance of TLRisk is better and sta-ble (
![Page 16: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/16.jpg)
Conclusions A translated learning framework for classifying tar-get data using data from another feature space. We can find a bridge to link the two spaces with only a little labeled data in the target space. They formulated translated learning framework us-ing risk minimization and an approximation method for model estimation . Showed effectiveness through two applications, the text-aided image classification and the cross-lan-guage classification.
![Page 17: Translated learning](https://reader035.vdocuments.site/reader035/viewer/2022062401/58eff4941a28ab74298b457f/html5/thumbnails/17.jpg)
Thank you !
Q&A