Slide 1

Generic Framework for Context-Dependent Fusion with Application to Landmine Detection Ahmed Chamseddine Ben Abdallah Multimedia Research Lab CECS Department University of Louisville June 2009 1

Slide 2

Outline Motivational Example Related Work Global Fusion Local Fusion Contributions Context-Extraction for Local Fusion (CELF) CELF with Feature Discrimination (CELF-FD) Application to Landmine detection Conclusions and Future Work 2

Slide 3

MOTIVATIONAL EXAMPLE 3

Slide 4

Global Fusion: Ask the audience or the wisdom of crowds Majority Voting 4

Slide 5

Example 1 5

Slide 6

Example 2 6

Slide 7

Example 3 (1) 7

Slide 8

Expert selection: Call a friend Doctor News Music Movies EconomistMusician Literature Artist CECS Phd Student Nurse Fashion Music Fashion Politics Basketball player Cooking 8

Slide 9

Medical question What is the scientific name of the swine Flu virus? A: H1N1B: H3N2 C: H2N2D: H5N1 A: H1N1 A A A A C B B B C C D C 9 Doctor News Music Nurse

Slide 10

Medical question What is best-selling music album worldwide? A: Come on OverB: Thriller C: Falling into YouD: Daydream B: Thriller B A A A C B B B C C D C 10 Doctor News Music Musician Fashion Music

Slide 11

Medical question Which of the following is not a prime number? A: 19B: 29 C: 39D: 59C: 39 A A A A A B B B C C D C 11 Economist CECS Phd Student

Slide 12

Hey I know about fashion! Hey I know about Music! 12

Slide 13

Conclusions Multiple sources better than single source need for fusion. Local fusion is better than global fusion. Grouping experts. Identifying the context/domain. Combining the experts decision. 13

Slide 14

RELATED WORK Global Fusion 14

Slide 15

Related work Global fusion When combining multiple independent and diverse decisions each of which is at least more accurate than random guessing, random errors cancel each other out, correct decisions are reinforced. 15

Slide 16

Related work Classifier fusion architecture 16 Combiner Decision Level Feature Level Data Level Raw Data 1 Feature Extraction 1 Classifier 1 Raw Data 2 Feature Extraction 2 Classifier 2 Raw Data K Feature Extraction K Classifier K

Slide 17

Related work Global Fusion approaches Bayesian Fusion ANN Fusion Borda Count Fusion Dempster-Shafer Fusion Decision Template Fusion Fuzzy Integral 17

Slide 18

RELATED WORK Local Fusion 18

Slide 19

Related work Local fusion Divide and Conquer approach 19 Context extraction Context extraction Decision fusion Decision fusion Feature set Global decision Classifier output Two independent tasks

Slide 20

Related work Local Fusion approaches Category 1: find the neighborhood of the testing sample and create a fusion model in the testing phase Dynamic classifier by local accuracy, time consuming Category 2: cluster and create fusion models in the training phase Clustering and selection, Context-Dependent Fusion, . Treats the context extraction and the decision fusion components independently. 20

Slide 21

CONTRIBUTIONS 21

Slide 22

Contributions 1. A local fusion approach (CELF) based on a novel objective function that combines Context identification Multi-algorithm fusion 2. CELF with adaptive feature weight assignment (CELF-FD) 3. Application to landmine detection 22

Slide 23

CONTRIBUTIONS 1- Context Extraction for Local Fusion (CELF) 23

Slide 24

ContributionsNotations 24 Feature Extraction 1 Classifier 1 Feature Extraction 2 Classifier 2 Feature Extraction K Classifier K Data sample j Ground truth t j (xj)1(xj)1 (xj)2(xj)2 (xj)K(xj)K y j1 y j2 y jK (x j, y j, t j ) Feature vector Decision vector Ground truth

Slide 25

ContributionsClustering FCM Algorithm Update c i Update u ij where 25 x c1c1 x c2c2 x c3c3 x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 x9x9 x 10 x 11 x 13 x 12 x 14 x 15 (u 1,5,u 2,5,u 3,5 )

Slide 26

ContributionsClassification 26 Conf 1 Conf 2 w1w1 w2w2

Slide 27

Contributions Confidence space Proposed approach 27 Decision space

Slide 28

Contributions Context-Extraction for Local Fusion (CELF) CELF Combines: Context identification Multi-algorithm fusion 28 Context extraction Context extraction Decision fusion Decision fusion Train Samples (Features, Algorithm outputs) Fusion output

Slide 29

Contributions Objective Function 29 Clustering Component: FCM type Clustering Component: FCM type Classification component

Slide 30

Contributions Update Equations (1) 30 Optimizing J w.r.t the centers yields

Slide 31

Contributions Update Equations (2) 31 Optimizing J w.r.t the membership yields Deviation from desired output Clusters with consistent fusion weights Similarity in the feature space Compact clusters

Slide 32

Contributions Update Equation (3) 32 Optimizing J w.r.t the aggregation weights yields

Slide 33

Contributions CELF Algorithm Initialize U and W. repeat Update cluster centers. Update W. Update U. until stopping condition satisfied return Centers, U, W 33

Slide 34

Contributions Illustrative example (1) 34 Classifier 1 Classifier 2 Decision space Feature space

Slide 35

Contributions Illustrative Example (2) 35

Slide 36

Contributions Illustrative Example (3) 36 Classifier 1 Classifier 2 CELF Y P(Y)

Slide 37

CONTRIBUTIONS 2- Context Extraction for Local Fusion with feature discrimination (CELF) 37

Slide 38

Contributions Context Extraction for Local Fusion with feature discrimination (CELF-FD) For high dimensional spaces, standard clustering algorithms cannot generate a meaningful partition. To alleviate this drawback, we introduce feature weighting aspect. CELF-FD combines: Clustering Feature Discrimination Selection of local expert classifiers 38

Slide 39

ContributionsNotations V set of features weights. 39 Cluster i Features from classifier l

Slide 40

Contributions Objective Function 40 Feature discrimination Optimized in CELF-FD

Slide 41

Contributions CELF-FD Algorithm Initialize U, V and W. repeat Update cluster centers. Update W. Update U. Update V. until stopping condition satisfied return Centers, U, V, W 41

Slide 42

Contributions Toy data (1) 42

Slide 43

Contributions Toy data (2) Classifier 1 Accuracy 69% Classifier 2 Accuracy 81% 43

Slide 44

Contributions Experimental results using CELF (1) 44

Slide 45

Contributions Experimental results using CELF (2) 45 Y P(Y)

Slide 46

CONTRIBUTIONS 3- Application to landmine detection 46

Slide 47

Contributions Landmine problem Objective: analyze data collected by multiple sensors and make a decision if there is buried mine. Different mine types Soil properties: Asphalt, gravel, sand Varying density Water held by vegetation roots Rain, snow Various minerals 47

Slide 48

Contributions Ground Penetration component 48 Autonomous Mine Detection System Vehicle

Slide 49

Contributions WEMI Data taken at 21 frequencies (logarithmically spaced from 330 Hz to 90.03 KHz). 49 Magnitude Frequency Blank NMC LM mineHMC

Slide 50

Contributions Landmine detectors Landmine detectors using GPR EHD SCF HMM Landmine detector using WEMI 50

Slide 51

Contributions GPR Algorithm 1: Edge Histogram Descriptor 51 Based on the EHD used in the MPEG-7 standard. Encodes spatial distribution of edges in a 3-D volume (down-track & cross-track) Edges are categorized into 4 types: {V, H, 45 o, -45 o }. Uses a possibilistic K-NN rule for confidence assignment.

Slide 52

Contributions GPR Algorithm 2: HMM-based Treats the down-track dimension as the time variable. A sequence of 15 observation vectors is produced for each alarm. Each observation has 4 features: (+/- Diag. edges & +/- Anti-diag edges) 52. MINE STATE 1 MINE STATE 2 MINE STATE 3 STATES X1 X2 X3... X14 X15 MINE STATE 1 MINE STATE 2 MINE STATE 3

Slide 53

Contributions GPR Algorithm 3: Spectral Detector Captures the characteristics of a target in the frequency domain. Extracts the alarm Spectral Correlation Feature (SCF). Assigns a confidence value based on similarity to prototypes that characterize mine objects. 53

Slide 54

Contributions WEMI Detection Algorithm Extracts 4 features: One model fitting parameter ( ) Fitting error 2 spread features (Q spread, T spread ) A Neural Network classifier was trained using the above 4 features to assign a confidence value to each alarm. 54

Slide 55

Contributions CELF Architecture 55

Slide 56

Contributions Data collection Data Collected from 2 different sites 864 Alarms: 308 mines, classified into 4 categories : Anti-tank with high metal content (ATM) Anti-tank with low metal content (ATLM) Anti-personal with high metal content (APM) Anti-personal with low metal content (APLM) 556 False Alarms, classified into 3 categories: High metal clutter (HMC) Non-metal clutter (NMC) Blank Targets buried up to 5 inches deep. 56

Slide 57

Contributions Motivation for fusion 57

Slide 58

Contributions Fusion results (1) 58

Slide 59

Contributions Fusion results (2) 59

Slide 60

Contributions Fusion results (3) 60

Slide 61

Contributions CELF for a Vehicle Mounted GPR System 61

Slide 62

Contributions Fusion results 62

Slide 63

CONCLUSIONS & FUTURE WORK 63

Slide 64

Conclusions A new local fusion approach based on a novel objective function that combines: Context identification (clustering component). Multi-algorithm fusion (classification component). Two variants: CELF and CELF-FD. Promising results on synthetic data on landmine detection problem. 64

Slide 65

Future Work Investigate other clustering techniques Kernel clustering, mahalanobis distance, Dirchlet distribution Integrate other fusion techniques Fuzzy integral Optimize the number of clusters CA, AIC Generalize the algorithm to support data with more classes. Optimize . 65

Slide 66

Questions? A. C. Ben Abdallah, H. Frigui and P. Gader "Context Extraction for Local Fusion using Fuzzy Clustering and Feature Discrimination", Fuzz-ieee, Korea, April 2009. H. Frigui, A. C. Ben Abdallah, and P. Gader "Context-dependent fusion for landmine detection with multisensor systems", SPIE, Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XIV, Orlando, April 2009. H. Frigui, J. Caudill, and A. C. Ben Abdallah, "Fusion of Multi-Modal Features for Efficient Content-Based Image Retrieval", IEEE World Congress on Computational Intelligence (WCCI 2008), Hong Kong, June 2008. H. Frigui, P. Gader, and A. C. Ben Abdallah, "A Generic Framework for Context-Dependent Fusion with Application to Landmine Detection", SPIE Defense and Security Symposium, Orlando, March 2008. 66

Slide 67

Objective Function 67 Feature discrimination Cluster-dependent Cluster-dependent

Slide 68

Experimental results using other local fusion method 68