![Page 1: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/1.jpg)
An introduction to automatic classification
Andy French February 2010
![Page 2: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/2.jpg)
Target recognition “at a glance”
“One of the most potent of human skills is the ability to rapidly recognize and classify environmental stimuli, often when such signals are severely corrupted. Of this toolkit of sensors and processing, the method of visual facial recognition is perhaps the most impressive. Typically, a successful recognition (i.e. a name attached) will occur in 120 ms, with cruder classifications (for example classification of a species group from a background) in as little as 50ms.”
Can a machine be built with this level of performance?
![Page 3: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/3.jpg)
This ‘introduction’ is but a glance at a large and active research area!
For a much more complete introduction see
Webb. A., Statistical Pattern Recognition. 2nd Edition. John Wiley & Sons Ltd. 2002.
Duda, R.O., Hart, P.E, Stork, D.G., Pattern Classification. 2nd Edition. John Wiley & Sons Inc. 2001.
Let us start in a similar fashion to Duda, with a practical example of a classification problem…
![Page 4: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/4.jpg)
A problem of cat classification….
There are many cats out there
How can I be sure to let the right one in…?
Dorset Big Cat
![Page 5: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/5.jpg)
Hairyness
Roar
Mechanized Entrance Test Of Cats
![Page 6: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/6.jpg)
)roarhairyness,(
)roarhairyness,(
lionlioni
catcati
hg
fg
Feature measurements
Classifier discriminant functionsClass label
![Page 7: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/7.jpg)
Hairyness
Roar
catilioni gg
lionicati gg
Decision boundary
catilioni gg
![Page 8: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/8.jpg)
NCTR with MESAR2
Start
Finish
Radar target classification
![Page 9: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/9.jpg)
Inbound Falcon jet aircraft
Length threshold
Feature extraction: Radar length
![Page 10: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/10.jpg)
Doppler processing: Jet Engine Modulation (JEM)
JEM lines
![Page 11: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/11.jpg)
(Aside) Doppler processing: Propeller modulation
Dash8 six blade propeller aircraftDoppler spectrum for 32 pulse,
32 frequency step waveform E 2.5kHz PRF
![Page 12: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/12.jpg)
Feature extraction: Doppler spectra
Inbound Boeing 777 jet aircraft
![Page 13: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/13.jpg)
Aim: Design a classifier based upon measured feature statistics
Example #1: a parametric (Gaussian) classifier
i.e. feature data is assumed to adopt a Gaussian distribution, characterized by mean and covariance parameters
![Page 14: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/14.jpg)
Gaussian distribution of feature vectors x, given class wi
meancovariance
Apply Bayes Theorem to determine the posterior probability, which will be proportional to our desired discriminant function
posterior likelihood prior
Voila! The Gaussian classifier. But how do we compute the mean and covariance from training data?
![Page 15: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/15.jpg)
M is the number of features
Sample mean
Sample covariance
![Page 16: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/16.jpg)
Tii
C
ii
B
i
C
ii
W
Z
ZZ
Z
))((1
1
mmmmS
ΣS
Bayesian FRD classifier (Bayesian) Friedman regularized discriminant function
Sample within class covariance
Sample between class covariance
Computed for TRAINING vectors t
ii
W
iii
i
ii
ii
Ziii
iiii
Tii
ZZ
Z
Z
ZZ
Zc
c
Z
Zg
Σ
SS
ΣS
SSΣ
Σ
IΣΣ
ΣmxΣmxx
1
1
Tr)(
)()1(
loglog)()()(
,
,211,
21
![Page 17: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/17.jpg)
Example #2: k-means non-parametric classifier
![Page 18: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/18.jpg)
k-means classification does not assume an a-priori feature distribution. Instead one uses the k-means clustering algorithm to automatically group training data into K clusters
Radii of cluster hyperspheres
Centre of cluster hyperspheres
Distance between training data and cluster centres
(binary) cluster membership matrix U. Start with random assignments!
Training data for class i
Update membership U based on nearest hypersphere centre for each training feature vector
![Page 19: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/19.jpg)
![Page 20: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/20.jpg)
![Page 21: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/21.jpg)
![Page 22: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/22.jpg)
![Page 23: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/23.jpg)
![Page 24: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/24.jpg)
Alternative “Fuzzy” membership matrix
K-means classifier discriminant function
![Page 25: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/25.jpg)
Radar example: Gaussian & Fuzzy logic classification methods employed
Define the Membership function
)feature(classg
![Page 26: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/26.jpg)
Radar target classification: truth assignments
![Page 27: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/27.jpg)
Radar Length feature based classification
![Page 28: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/28.jpg)
The Confusion matrix and its ‘off-diagonal-extent’
?
![Page 29: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/29.jpg)
Prop, JEM or No-Non-Skin-Doppler (NNSD) classification
“Doppler fraction” feature
Classes are visually separable
![Page 30: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/30.jpg)
Confusion matrix for Prop, JEM or No-Non-Skin-Doppler (NNSD) classification
![Page 31: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/31.jpg)
Classification performance vs length thresh & QFour lengths classes: VS, S, L, VL
![Page 32: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/32.jpg)
Classification performance vs dfrac thresh & P
![Page 33: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/33.jpg)
Classification based on combined length & dfrac features
![Page 34: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/34.jpg)
Classification performance vs frequency jitter
Maximum P and Q used for all waveforms
![Page 35: An introduction to automatic classification Andy French February 2010](https://reader035.vdocuments.site/reader035/viewer/2022062322/56814523550346895db1e84f/html5/thumbnails/35.jpg)
Any questions?