online manifold regularization: a new learning setting and empirical study

Online Manifold Regularization: A New Learning Setting and Empirical Study

Andrew B. Goldberg, Ming Li, and Xiaojin Zhu.(ECML PKDD, 2008).

Hu EnLiang Friday, April 17, 2009

Standard online learning VS. Online Manifold Regularization Both of them are long-life learning and learn

non-iid sequentially;

Standard online learning: traditionally assumes that every input point is fully labeled, it cannot take advantage of unlabeled data;

Online MR: it learns even when the input point is unlabeled.

Online MR VS. batch MR (advantages) Online MR scales better than batch MR in time and

space;

Online MR achieves comparable performance to batch MR;

Online MR can handle concept drift;

Online MR is an “anytime classifier”, which continuously is being improved and its training is cheap.

The principle of online MR

The relationship of batch risk, instantaneous regularized risk and average instantaneous risk

How to accelerate online MR

Continue !!!

A Brief Introduction to CBIR(Content-based Image

Retrieval)

Hu en liang

Tuesday, April 08, 2008

Background:Content-based Image Retrieval

Properties: Querying image according to user’s semantic-co

ncepts. Querying images according to image’s contents,

such as: color, texture, shape, etc.

Hypothesis——similar contents means semantic affinity；

‘Semantic gap’——semantic affinity doesn't means similar contents.

A prototype of feedback-based CBIR

Background: The Difficulty of ‘Semantic Gap’ Key problems:

1. How to extract user’s semantic-concept (intention)?2. How to bridge between content and semantic ?

Main methods:

1. Machine learning based RF (Relevance-Feedback); 2. The prior knowledge such as the historical logs.

How to Connect CBIR to ML?

(Semi-)supervised Metric Learning;

Manifold Learning, Dimension Reduction…

(Semi-)supervised Classification;

Active Learning; Co-training;

Assembling Classifier;

Ranking; …

Some Individual Characteristics for feedback-based CBIR In contrast to typical ML, there are some special

characteristics for RF-CBIR :

The problem of the small size sample;

The problem of asymmetrical training sample;

The online algorithm with real-time requirement;

Manifold Regularization (MR)

Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Mikhail Belkin, Partha Niyogi, Vikas Sindhwani. Journal of machine Learning Research 7, pp 2399-2434, 2006

To Modify MR for CBIR

There are some intrinsic characteristics for CBIR :

The problem of the small size sample; The problem of asymmetrical training sample; The online algorithm with real-time requirement;

The (1+x)-manifolds hypothesis

There only single submanifold for positive class, but multi-submanifolds for negative class!

Negative manifold

positive manifold

The Problem of MR for the Multi-Submanifolds Case

The Bias-MR Focusing on Single-Submanifold

A review of LapSVM

A review of LapSVM

O(l3) O(n3)O(n3)

A higher efficiency in BLapSVM

O(q3

)

The BLapSVM Algorithm for CBIR

The ‘BEP’ Performance Chart

The ‘Efficiency’ Performance Chart

Thanks for Your

Attention !