The Truth About Cats And Dogs Omkar M Parkhi1,2 Andea Vedaldi1 C. V. Jawahar2 Andrew Zisserman1

Objective

1Department of Engineering Science, University of Oxford 2 International Institute of Information Technology, Hyderabad, India E-mail : {omkar,vedaldi,az}@robots.ox.ac.uk, jawahar@iiit.ac.in

This research is funded by UKIERI, ONR MURI N00014-07-1-0182 and ERC grant VisRec no. 228180.

Quantitative Results

Cat detection on PASCAL VOC 2010 Validation data.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Recall

Prec

isio

n

PR Curve: Cat detection results

gc−basic AP: 0.37dpm−ber−rr AP: 0.41dpm−rr AP: 0.46dpm AP: 0.48ub AP: 0.67

Method AP Basic GrabCut 0.37 Adding Global Posteriors 0.41 Adding Berkeley Edges 0.46 Re ranking the detections 0.48

Effect of the system components on validation data

Qualitative Results

Overview

Distinctive Part Detection

Image

Object Detection

Grab-Cut Segmentation

To detect the whole object: 1)  Detect the distinctive part (head). 2)  Segment the animal based on distinctive part. 3)  Obtain the animal bounding box from the segmentation and the distinctive part.

Motivation

Cats and dogs are very varied in their imaged shape

•  Key observation: DefPMs are •  very poor at detecting the whole cat. •  very good at detecting a distinctive part such as the head.

3) Post Processing •  Alternate two steps [Rother et al. 04]

•  Estimate appearance model •  Minimize standard graph-cut energy function to assign a correct label yi to each pixel xi.

Appearance Model •  Data term

•  from GMM estimated using head region and from global posterior probabilities.

•  Pairwise term from edge detector output •  Model initialization

•  Foreground From head detection. •  Background From predicted bounding box. (Boundary region).

•  Model update re-estimating GMMs using the output from the previous step.

2-a) Grab-Cut Segmentation

! !,! = !− !!!!!log!(!!! !!! + ! !(!! ,!,!!!!Neighbours

!!|!!)!Energy Data Term Pairwise Term

Seeds for modeling GMMs Segmentation Output: Effect of poor data and edge terms.

Method Cat Dog Felzenswalb et. al 2010 31.8 21.5 Our Method 45.3 36.8

Detection results on PASCAL VOC 2010 Test data

Dataset and Annotations

Object Part Trimap

•  PASCAL VOC 2010 annotations: cats and dogs •  Bounding boxes •  Segmentations (trimaps)

•  Additional head annotations •  Bounding boxes

2-c) Global Model

•  Head seed insufficient to model object foreground. •  Posterior probabilities given by histograms learnt from

ground truth segmentations over all training images •  Used as data term for the first iteration of segmentation.

Posteriors using head seed only

Posteriors after introduction of global model

Improved segmentation due to global posteriors.

•  Clean up: erode, dilate, and select component connected to head.

•  Adjust predicted bounding box to be consistent with head detection.

•  Rerank detections based on the head size.

Problem Overview Deformable Part Models (DefPMs) such as [Felzenszwalb et al. 09] are not deformable enough for very flexible objects such as cats and dogs. We extend DefPMs by introducing Distinctive Parts Models (DisPMs), which combine DefPMs with segmentation and successfully detect highly deformable animals.

The distinctive part is modeled by a DefPM. •  Parts connected by springs •  HOG + LBP features •  Fast inference with dynamic programming •  Discriminatively trained by Latent variable SVM

•  DefPMs are much better at detecting the head than the whole animal.

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

Recall

Prec

isio

n

PR Curve: Object Detection and Part Detection

Cat Detection AP: 0.28Cat Head Detection AP: 0.49

1) Part Model

Deformable Parts Model for Cat Head

2-b) Modeling Edges •  Berkeley PB edge detector [Martin et. al. 04] •  Edge detector response used as pairwise term.

•  Cut encouraged at higher edge potential. •  Significant improvement over gradients edge terms.

! !! ,!! !) = !!!!!(!!(!)/!) !

Edge Detector Response Improvement segmentation output due to Berkeley edge model

Incorrect Segmentation

Incorrect Segmentation

Correct Segmentation

Correct Segmentation

Foreground Seed

Background Seed