artificial intelligence laboratory & laboratory for information and decision systems
DESCRIPTION
A Unified Multiresolution Framework for Automatic Target Recognition. Eric Grimson, Alan Willsky, Paul Viola, Jeremy S. De Bonet, and John Fisher. Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems Massachusetts Institute of Technology. Outline. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/1.jpg)
MIT AI Lab / LIDS
Artificial Intelligence Laboratory &Laboratory for Information and Decision SystemsMassachusetts Institute of Technology
A Unified Multiresolution Framework for
Automatic Target Recognition
Eric Grimson, Alan Willsky, Paul Viola, Jeremy S. De Bonet, and John Fisher
![Page 2: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/2.jpg)
MIT AI Lab / LIDS
Outline
• Review Multiresolution Analysis Models– MAR (Multiresolution Auto-Regressive)
– MNP (Multi-scale Nonparametric)
• Applications of MNP Models– Classification/Recognition
– Segmentation and Multi-Look Registration
– Synthesis and Super-Resolution
• Continuing Efforts
![Page 3: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/3.jpg)
MIT AI Lab / LIDS
MAR Processes for SAR
Pyramid Residuals
Irving,Willsky &Novak
![Page 4: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/4.jpg)
MIT AI Lab / LIDS
Intuition: Construct a Model for the Scale-to-scale Dependency in SAR imagery
V(x,y)={ }
Parent Vector
),(,
2,
2,...,
2,
2,
2,
2, 01111 yxI
yxI
yxI
yxIyxV
NNNNNN
L0
L1
L2L3
L4
![Page 5: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/5.jpg)
MIT AI Lab / LIDS
Build a Model for Observed Distribution
yxVP ,
IWN: Conditionally Gaussian
![Page 6: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/6.jpg)
MIT AI Lab / LIDS
![Page 7: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/7.jpg)
MIT AI Lab / LIDS
Multi-scale Non-parametric Models
• Two key insights:– Alternative multi-scale representation
• Sub-band oriented representations (Wavelets, Gabor Filters)
– Non-parametric models of conditional dependence
De Bonet & Viola (1997)
![Page 8: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/8.jpg)
MIT AI Lab / LIDS
Freeman and Simoncelli
Steerable Pyramids
),( yxFl ),( yxFl
![Page 9: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/9.jpg)
MIT AI Lab / LIDS
…for a SAR image
![Page 10: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/10.jpg)
MIT AI Lab / LIDS
V(x,y)={ }
coarse fine
Parent Vector
Multiresolution parent vector
yxFyxFyxF
yxF
yxF
yxF
yxF
yxF
yxFyxV
MNNN
M
NNM
NNNNNNN
,,,,,,
,2
,2
,,2
,2
,2
,2
,2
,2
,,2
,2
,2
,2
,
10
11
10
1
10
![Page 11: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/11.jpg)
MIT AI Lab / LIDS
Build a Model for Observed Distribution
yxVP ,
DB: Non-parametricDistribution
![Page 12: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/12.jpg)
MIT AI Lab / LIDS
Probabilistic Model
)...,(),,(|),(
),(}{|),(
21 yxVyxVyxVP
yxVFeaturesyxVP
lll
ll
)...,(),,(|),(
),(}{|),(
21 yxVyxVyxVP
yxVFeaturesyxVP
lll
ll
...),,|(
),|(
)|()()()(
2103
102
010
VVVVP
VVVP
VVPVPfeaturesPIP
...),,|(
),|(
)|()()()(
2103
102
010
VVVVP
VVVP
VVPVPfeaturesPIP
yx
llll yxVyxVyxVPVP,
21 ...),(),,(|),( yx
llll yxVyxVyxVPVP,
21 ...),(),,(|),(
Markov
ConditionallyIndependent
SuccessiveConditioning
![Page 13: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/13.jpg)
MIT AI Lab / LIDS
Estimating Conditional Distributions
...),(),,(
...),(),,(),,(
...),(),,(
...),(),,(),,(
...),(),,(|),(
21*
21*
21
21
21
yxVyxVP
yxVyxVyxVP
yxVyxVP
yxVyxVyxVP
yxVyxVyxVP
ll
lll
ll
lll
lll
...),(),,(
...),(),,(),,(
...),(),,(
...),(),,(),,(
...),(),,(|),(
21*
21*
21
21
21
yxVyxVP
yxVyxVyxVP
yxVyxVP
yxVyxVyxVP
yxVyxVyxVP
ll
lll
ll
lll
lll
i
ixxRxP )()(* i
ixxRxP )()(*• Non-parametrically
![Page 14: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/14.jpg)
MIT AI Lab / LIDS
distribution Similarity
sample
Likelihood
distribution condition
exampleimage
synthesis
discrimination
registration
segmentation
denoising
super resolution
Outline
![Page 15: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/15.jpg)
MIT AI Lab / LIDS
Capturing Structure (Texture Perspective)
![Page 16: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/16.jpg)
MIT AI Lab / LIDS
Synthesis Results
![Page 17: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/17.jpg)
MIT AI Lab / LIDS
Synthesis Results
![Page 18: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/18.jpg)
MIT AI Lab / LIDS
![Page 19: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/19.jpg)
MIT AI Lab / LIDS
Alternative 1: Gaussian Distribution: GMRF
2
221
||
|)(|
),,()(
WWI
I
e
e
INIP
2
221
||
|)(|
),,()(
WWI
I
e
e
INIP
W
W
W
21
W
W
W
21
Chellappa and Chattergee
![Page 20: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/20.jpg)
MIT AI Lab / LIDS
![Page 21: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/21.jpg)
MIT AI Lab / LIDS
Alternative 2: Statistical Wavelet Models
yxlll
yxllyxl
yxFP
yxFPObsP
,,,,
,,,,,,
),(
),()(
yxlll
yxllyxl
yxFP
yxFPObsP
,,,,
,,,,,,
),(
),()(
(.):
)},({:
:
,
lPModel
yxFObserve
WGiven
l
(.):
)},({:
:
,
lPModel
yxFObserve
WGiven
l
Bergen and Heeger
Donoho
Simoncelli and Adelson
![Page 22: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/22.jpg)
MIT AI Lab / LIDS
Heeger and Bergen Texture Synthesis Model
![Page 23: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/23.jpg)
MIT AI Lab / LIDS
Heeger and Bergen Texture Synthesis Model
![Page 24: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/24.jpg)
MIT AI Lab / LIDS
orig
inal
text
ure
patc
h
synt
hesi
zed
text
ure
patc
h
Sam
plin
g P
roce
dure
Analysis Synthesis
![Page 25: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/25.jpg)
MIT AI Lab / LIDS
Not quite right...
Very similar to a Gaussian Model(i.e. no phase alignment)
![Page 26: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/26.jpg)
MIT AI Lab / LIDS
Wavelet Representation of Edges
WaveletTransform
![Page 27: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/27.jpg)
MIT AI Lab / LIDS
Pyramid Representation
![Page 28: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/28.jpg)
MIT AI Lab / LIDS
Conditional Distributions
WaveletTransform
![Page 29: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/29.jpg)
MIT AI Lab / LIDS
orig
inal
text
ure
patc
h
Sam
plin
g P
roce
dure
Analysis Synthesis
synt
hesi
zed
text
ure
patc
h
![Page 30: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/30.jpg)
MIT AI Lab / LIDS
Multiresolution progression
![Page 31: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/31.jpg)
MIT AI Lab / LIDS
![Page 32: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/32.jpg)
MIT AI Lab / LIDS
Joint feature occurrence across resolution
![Page 33: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/33.jpg)
MIT AI Lab / LIDS
Joint feature occurrence across resolution
![Page 34: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/34.jpg)
MIT AI Lab / LIDS
![Page 35: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/35.jpg)
MIT AI Lab / LIDS
![Page 36: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/36.jpg)
MIT AI Lab / LIDS
Texture Synthesis Results
![Page 37: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/37.jpg)
MIT AI Lab / LIDS
BMP2-C21 BTR70-C71 T72-132
Models
Models for target vehicles were generated from example images:
• generated from vehicles with different numbers from the target vehicles• only 10 examples, evenly distributed in heading angle• measured at a depression angle of 17 degrees (targets were at 15 degrees)
![Page 38: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/38.jpg)
MIT AI Lab / LIDS
BMP2-9563 BMP2-9566
BTR70-C71
T72-812 T72-S7
Target vehicles
• Five target vehicles were used.
• Vehicles which differed from the target class were included as confusion targets.
• There were roughly 200 images in each class.
![Page 39: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/39.jpg)
MIT AI Lab / LIDS
2S1 BRDM2 D7 T62
ZIL131 ZSU23
Confusion vehicles
Six additional confusion vehicles were used as well.
![Page 40: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/40.jpg)
MIT AI Lab / LIDS
BMP2-C21 BTR70-C71 T72-132
Flexible Histograms
Template Matching
De Bonet, Fisher and Viola
![Page 41: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/41.jpg)
MIT AI Lab / LIDS
Rtie-point
Rtest
parent structure
B (x,y)= 8
Measuring Visual Structure : Flexible Histogram II
![Page 42: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/42.jpg)
MIT AI Lab / LIDS
Rlandmark
B(,x,y)= 8
2= (B-B’)2/B
B’(x,y)= 3
Rtie-point
Rtest
Measuring Visual Structure : Flexible Histogram III
![Page 43: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/43.jpg)
MIT AI Lab / LIDS
Registration pipeline
Tie-pointdetermination
Multiresolution texture match:flexible histograms
Multiresolutionalignmentsearch
![Page 44: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/44.jpg)
MIT AI Lab / LIDS
Tie-point determination
![Page 45: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/45.jpg)
MIT AI Lab / LIDS
Here, only vehicles provide distinct landmarks.
When present, roads and buildings provide useful landmarks as well.
Tie-point examples
![Page 46: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/46.jpg)
MIT AI Lab / LIDS
Coarse Fine
Coarse to fine alignment
![Page 47: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/47.jpg)
MIT AI Lab / LIDS
Example Registration
![Page 48: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/48.jpg)
MIT AI Lab / LIDS
Example Registration
![Page 49: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/49.jpg)
MIT AI Lab / LIDS
Example Registration
![Page 50: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/50.jpg)
MIT AI Lab / LIDS
![Page 51: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/51.jpg)
MIT AI Lab / LIDS
![Page 52: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/52.jpg)
MIT AI Lab / LIDS
![Page 53: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/53.jpg)
MIT AI Lab / LIDS
distribution | | condition
Prior beliefs about natural images
image analysis
Image + Noise
maximum likelihood sample
Blind Image Denoising
![Page 54: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/54.jpg)
MIT AI Lab / LIDS
![Page 55: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/55.jpg)
MIT AI Lab / LIDS
![Page 56: Artificial Intelligence Laboratory & Laboratory for Information and Decision Systems](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815441550346895dc2480e/html5/thumbnails/56.jpg)
MIT AI Lab / LIDS