m.s. thesis defenseroadmap • literature review (5 min) • research objectives (5 min) •...

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

M.S. Thesis Defense________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Bryan David ThompsonMarch 29th, 2007

________________________________________________________________________________________________________________________________

Adviser:Dr. Mahmood R. Azimi-Sadjadi

Committee:Dr. Robert Liebler

Dr. Edwin K. P. Chong

Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)

– Implementation– Experimental Results

• Theoretical Contribution (10 min)– Development– Simulation Results

• Q&A (15 min)• Additional slides

Literature Review

Buried Underwater Target Detection and Classification• Sensor development & Data acquisition

[Schock] BOSS, AUVs, SAS Processing, Imagery[Harbaugh] Synthesized data sets

• Feature Extraction[Maussong] SAS Imagery (mean-standard deviation)[Sternlicht] Sediment volume imagery (clustering and ellipse)[Azimi] Canonical Correlation Analysis

• Detection & Classification[Dobeck] Detection & Classification Algorithm Fusion[Sternlicht] 2D & 1D Fusion (Gaussian-Bayesian classifiers)[Azimi] HMMs, NNs, D/F-level Fusion, CMAC,

Success of CCA as a coherence-based Feature extraction procedure

Research Objectives

• To implement a multi-channel extension of CCA as a feature extraction procedure

• To demonstrate its effectiveness via practical application

• To develop an iterative learning algorithm providing numerous advantages

Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)

– Implementation– Experimental Results

• Theoretical Contribution (10 min)– Development– Simulation Results

• Q&A (15 min)• Additional slides

Algorithm Review: Multi-channel Coherence Analysis

Assume n data channels

Composite data channel vector

Corresponding correlation matrix

MCA Review (2)

MCA Review (3)CCA: 1st coordinate pair and correlation

MCA: 1st coordinate set and correlations

MCA Review (4)Constraint (1) Unit Variance:

SUMCOR + Unit Variance = No unique solution (thesis)

Constraint (2) Unit Trace:

SUMCOR + Unit Trace = Generalized eigensystem

MCA Review (5)CCA: Higher order coordinates/correlations

MCA: Higher order coordinates/correlations

s.t.

Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)

– Implementation– Experimental Results

• Theoretical Contribution (10 min)– Development– Simulation Results

• Q&A (15 min)• Additional slides

Practical Application• Date: May of 2004 • Platform: Disk BOSS• Location: St. Andrews Bay, Panama City, FL • 11 objects

– 5 mine-like (ML)• Boundary markers• Bomb-shaped object• Large cylinders

– 6 non-mine-like (NML)• Artillery shells• Small cylinder• Bullet-shaped objects• Spheres

• Naturally occurring objects (rocks / clutter)

Implementation

Implementation (2)Experimentally determined ping separations

Experimental Results

• MCA-based single-ping classifier – Outperforms CCA-based single-ping classifier– Outperforms Decision-level Fusion for testing data

• Uses 67% more sonar returns per decision

– Highest generalization ability

Experimental Results

Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)

– Implementation– Experimental Results

• Theoretical Contribution (10 min)– Development– Simulation Results

• Q&A (15 min)• Additional slides

Theoretical Contribution

An Iterative Learning Algorithm for MCA• Data-driven• Real-time• Circumvented calculations:

– Calculating sample covariance matrices for all data channel pairs

– Solving generalized eigenvalue problems• Recent IJCNN 2007 acceptance

– Best Paper Award nomination by one reviewer

Development1st coordinate set

Iterative method of steepest descent (or ascent… maximizing)

Concisely

Development (2)1st correlation sum

Higher order (ith) coordinate sets / correlation sums

subject to:

Development (3)Lagrange multiplier method with penalty parameter

Iterative method of steepest descent (ascent)

Development (4)Conversion to Least-Mean-Square (LMS) based updating

For the composite coordinate mapping vector

For the correlation sum

Simulation Results

• Simulated three-channel data set– -

• Guided by the linear model

Simulation Results (2)

Squared estimation error plots vs. iteration

Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)

– Implementation– Experimental Results

• Theoretical Contribution (10 min)– Development– Simulation Results

• Q&A (15 min)• Additional slides

Q&A

• Supported by ONR– Contract #N00014-05-1-0014

• Special Thanks– Dr. Azimi, Dr. Liebler, and Dr. Chong– Gordon, Nicholas, Jered, Amanda, Neil, Ali,

Jaime, Makoto, Derek, and Michael.– Family and friends– Coworkers at Agilent Technologies, Inc.

• Incredible experience because of the people

Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)

– Implementation– Experimental Results

• Theoretical Contribution (10 min)– Development– Simulation Results

• Q&A (15 min)• Additional slides

Experimental Results

Additional: MCA

FONC: (interior case)

SOSC: Satisfied because A (the Hessian) is positive semi-definite

Additional: MCA (2)

Additional: Mutual Information and Coherence

Additional: BOSS Data set

Additional: Preprocessing

Additional: Concatenation

• For each channel, a preprocessed sonar return yields a a time series of 200x1

• Concatenated together to yield 9600x1• Partition into overlapping blocks 40x1

– 50% overlap– Statistically rich

Additional: Feature Extraction• To capture ML vs NML discriminatory factors in sonar

returns using coherence-based feature extraction• CCA

– Input two preprocessed sonar ping partitions (each of 40x1)– Output vector of canonical correlations 40x1

• MCA– Input three preprocessed sonar ping partitions (each of 40x1)– Output vector of correlation sums 40x1

• Use dominate 20 as feature vector– To simplify classifier complexity

Additional: Clarity• Preprocessing

– To remove undesirable portions of the signal– Matched filtering (requires incident signal)– Windowing– Inverse Matched Filtered

• Concatenation– Concatenate kth return for each channel into one time series– Form overlapping blocks with 50% overlap– Statistically rich

• Feature extraction– To capture ML and NML discriminatory factors in sonar

returns using coherence-based feature extraction– Canonical Correlation Analysis (two-channel)– Multi-channel Coherence Analysis (three-channel)

Additional: Clarity (2)

• Single-ping classification– Back propagation neural network (BPNN)– Training data set: ½ of Line 4– Validation data set: ½ of Line 4– Testing data set: Lines 2 and 8

• Decision-level fusion– Nonlinear (BPNN-based)

Additional: Single-ping Classifier• To capture ML vs. NML discriminatory factors in feature vectors• Batch process training using entire set of training patterns

– 20 dominant correlations (sums) extracted via CCA (MCA)– Equally split ML and NML features between training and validation

data• Back propagated error (between the desired and actual outputs)

through the system and the various weight layers are adjusted. – Mine-like output is [1 0]T– Non-mine-like output is [0 1]T

• Structure – Several different two- and three-layer BPNNs were tested– Two-layer implementation determined best– Tested ranged from 40 to 50 hidden layer neurons– Randomly initialized layer weights and biases (several times)– Optimal structure: 20 inputs, 50 hidden layer neurons, 2 output

neurons

Additional: Single-ping Classifier (2)

• Optimal classifier chosen based on training and validation data set classification performance– ML classification performance favored over NML

• Training– Fixed learning rate (heuristically determined)– Levenberg-Marquardt (LM) learning rule [54]– No momentum factor

• Line 2 and 8 testing data sets determined classifiers’ generalization ability

Additional: Decision-level Fusion

• To further exaggerate the captured ML vs. NML discriminatory factors of previously made decisions

• Similarly training as BPNN-based Single-ping classifiers

• Similarly obtained optimal structure– Optimal structure: 6 inputs, 6 hidden layer

neurons, 2 output neurons

Additional: Implementation

Additional: Implementation (2)

No more additional slides