Slide 1

Quantifying and Transferring Contextual Information in Object Detection Professor: S. J. Wang Student : Y. S. Wang 1

Slide 2

Outline Background Goal Difficulties in Usage of Contextual Information Provided solutions Another method: TAS Experimental Results and Discussion Conclusion and Future Direction 2

Slide 3

Background (I) Only the properties of target object used in the detection task in the past. Problem: Intolerable number of false positive 3

Slide 4

Background (I) Only the properties of target object used in the detection task in the past. Problem: Intolerable number of false positive 4

Slide 5

Background (II) What else??? Contextual information! 5

Slide 6

Goal Establish a model to efficiently utilize the contextual information to boost the performance of detection accuracy. 6

Slide 7

Difficulties (I) Diversity of Contextual Information There are may different types of context often co-existing with different degrees of relevance to the detection for the target object(s) in different images. Terminology: Things (e.g. cars and people) Stuffs (e.g. roads and sky) Scene (e.g. what happen in the image) Thing-Thing, Thing-Stuff, Stuff-Stuff and Scene-Thing 7

Slide 8

Difficulties (II) Ambiguity of Contextual Information Contextual information can be ambiguous and unreliable, thus may not always have a positive effect on object detection. Ex: Crowded Scene with constant movement and occlusion among multiple objects. 8

Slide 9

Difficulties (III) Lack of Data for Context Learning Not enough training data : Over-fitting problem Wrong degree of relevance Ex: The contextual information of people on top of sofa can be more useful than people on top of grass. 9

Slide 10

Training Data Preparation & Notation Representation 10 Base Detector (HOG) Training Image Candidate windows Positive sample: Red Negative sample: Green

Slide 11

Provided Solutions A polar geometric descriptor for contextual representation. A maximum margin context model (MMC) for quantifying context. A context transfer learning model for context learning with limited data. 11

Slide 12

Polar Geometric Descriptor Instead of traditional annotation based descriptor, here we use polar geometric descriptor to describe two kind of contextual information (Thing-Thing, Thing-Stuff). 12 r :orientation b+1 :radial bins r*b+1 :patches 0.5, and 2 :bin length Feature :HOG Patch representation: Bag of Words method using K-means with K = 100

Slide 13

Provided Solutions A polar geometric descriptor for contextual representation. A maximum margin context model (MMC) for quantifying context. A context transfer learning model for context learning with limited data. 13

Slide 14

Quantifying Context (I) Quantifying Context (I) 14 Risk function:

Slide 15

Quantifying Context (II) Quantifying Context (II) Goal = Minimize the Risk function 15 Minimize L equal to fulfill the following constraint Hard to be solved, could be replaced by

Slide 16

Quantifying Context (III) Maximum Margin Context Model 16 Add some extra variables and constraints

Slide 17

Provided Solutions A polar geometric descriptor for contextual representation. A maximum margin context model (MMC) for quantifying context. A context transfer learning model for context learning with limited data. 17

Slide 18

Context Transfer Learning Context Transfer Learning Two Cases: Similar contextual information Ex: Cars and motorbikes Little in common in both appearance and context, but similar level of assistance provided by contextual information. Ex: People and bikes 18

Slide 19

TMMC-I: Transferring Discriminant Contextual Information TMMC-I: Transferring Discriminant Contextual Information Similar context provide the assistance on the learning of w. 19

Slide 20

TMMC-I: Transferring Discriminant Contextual Information TMMC-I: Transferring Discriminant Contextual Information New Constraint: 20 Modified optimization function:

Slide 21

TMMC-II: Transferring the Weight of Prior Detection Score Similar level of assistance, same weight 21

Slide 22

TMMC-II: Transferring the Weight of Prior Detection Score 22 New Constraint: Modified optimization function:

Slide 23

Another Method: TAS 23

Slide 24

Another Method: TAS (I) 24 Steps: 1.Segmenting image into regions. 2.Use base-detector to get the candidate patches. 3.Establish the relationships between candidate patches and regions. 4.Use the relationships to judge there is a target object in the patch or not.

Slide 25

Another Method: TAS (II) Region clusters: 25

Slide 26

Another Method: TAS (III) Examples of experiment: 26

Slide 27

Experimental Result and Discussion Use four data sets for testing: VOC 2005 VOC 2007 I-LIDS FORECOURT 27

Slide 28

Experimental Result and Discussion 28

Slide 29

Experimental Result and Discussion 29

Slide 30

Experimental Result and Discussion Context Transfer Learning Models: 30

Slide 31

Experimental Result and Discussion Context Transfer Learning Models: 31

Slide 32

Conclusion and Future Direction In this paper, the author proposes a contextual information model to quantify and select useful context information to boost the detection performance. What can we do next? HOG feature not suits for stuff (e.g. sky, road) Automatic selection between TMMC-I, TMMC-II Automatic selection between target object category and source category 32

Slide 33

Reference Wei-Shi Zheng, Member, IEEE, Shaogang Gong, and Tao Xiang, Quantifying and Transferring Contextual Information in Object Detection , PAMI accepted. Geremy Heitz, Daphne Koller, Learning Spatial Context: Using Stuff to Find Things, ECCV 2008. Youtube Search Hard-Margin SVM 33