3 Small Comments

Alex BergStony Brook University

I work on recognition:

features – action recognition – alignment – detection – attributes – hierarchical image classification & retrieval

+ machine learning for large scale recognition

Collaborations: words & pictures – ImageNet – human visual search – neural coding of visual memory

0. Good computer vision tells us something about the structure of the visual world

or about our descriptions of the visual world.

1. We should know what we are recognizing, and be able to prove

it!

2. Large datasets can be gallant but doomed attempts to avoid hard

representation problems. (Small datasets are worse.)

3. We should try to understand uncertainty in computer vision.

Alex Berg – Stony Brook University

3 Small Comments

The structure of the visual world is complex. So are the ways we

describe it.

Recognition experiments probe mappings from samples of the

visual world to descriptions.

Alex Berg – Stony Brook University

0. Good computer vision tells us something about the structure of the visual world.

This is a PR problem!

Advertise our successes and abilities, but avoid over-selling.

Example Given training and evaluation of an X detector on my own

dataset:

Ideal If your dataset has Xs, then my algorithm can

detect them.

Great If people can detect Xs in your dataset, then

my algorithm will detect them.

Good I can predict performance on your dataset

based on a statistical characterization.

Okay I can very roughly predict performance on your

dataset based on a characterization.

Now If I were to run my algorithm on your dataset,

we could determine performance.

Bad I don’t know how it works on my dataset!

Alex Berg – Stony Brook University

1. We should know what we are recognizing, and be able to prove it!

Describable Visual Attributes for Face Verification and Image Search

N. Kumar, A.C. Berg, P.N. Belhumeur, S.K. Nayar (T.PAMI 2011)

Verification

classifier

6

LFW Results

In 2009In 2009

Describable Visual Attributes for Face Verification and Image Search

N. Kumar, A.C. Berg, P.N. Belhumeur, S.K. Nayar

Verification

classifier

2. Large datasets can be gallant but doomed attempts to avoid hard representation problems.

Alex Berg – Stony Brook University

w/ Li Fei-Fei & Jia Deng @ Stanford

Efficient Additive Models forDetection & Classificationw/ Subhransu Maji @ UCB −> TTI-C

Test Image

Large Dataset

There comes a point when it is necessary to go into more detail – this is the regime of mid-level vision.

More data provides better joint statistics, but is enough only sometimes.

There is some boost from looking at large data, but, to do well we still need to address hard (mid-level) representation problems.

Might hope that matching/classifying a whole image / pattern / patch @ large scale “just works” for recognition.

Large Scale Recognition Challenge going on now (part of PASCAL VOC)Image Classification + Object Detection!

(Small datasets are worse!)

3. We should try to understand uncertainty in computer vision.

Alex Berg – Stony Brook University

1. Need an explicit idea of what is possible/likely given observations.2. We need this for low, mid, and high level vision.3. This is a difficult representational challenge – mainly because of complex structure.

What does classifying more than 10,000 image categories tell us?

J. Deng, A.C. Berg, K. Li, L. Fei-Fei (ECCV 2010)

Correlation between CV classifier confusions and WordNet!

0. Good computer vision tells us something about the structure of the visual world

or about our descriptions of the visual world.

1. We should know what we are recognizing, and be able to prove

it!

2. Large datasets can be gallant but doomed attempts to avoid hard

representation problems. (Small datasets are worse.)

3. We should try to understand uncertainty in computer vision.

Alex Berg – Stony Brook University

3 Small Comments