deep learning competence center & smart data and knowledge … · 2018. 4. 19. ·...
TRANSCRIPT
![Page 1: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/1.jpg)
…
…
…
…
…
…
…
…
…
…
Patrick Helber, Benjamin Bischke, Damian Borth, Andreas Dengel
Deep Learning Competence Center & Smart Data and Knowledge Services
German Research Center for Artificial Intelligence (DFKI)
Earth Observation From SpaceDeep Learning Based Satellite Image Analysis
…
…
…
…
![Page 2: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/2.jpg)
Satellite Data Becomes Publicly Available
2
© ESA / Astrium
![Page 3: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/3.jpg)
Global Challenges
3
![Page 4: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/4.jpg)
Satellite Analysis -> Global Challenges
4
Infrastructure & Traffic MonitoringPollution Monitoring: Oil Spills and Pipelines
Land Change Detection, DeforestationAgriculture
![Page 5: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/5.jpg)
Agriculture: Monitoring Farm Land
5
© ESA
![Page 6: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/6.jpg)
Climate Change: Deforestation
6
![Page 7: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/7.jpg)
Monitoring of Economic Factors
7
• Economic Growing and Shrinking
© DigitalGlobe
![Page 8: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/8.jpg)
Natural Disaster: Flooding, Wildfire, Earthquake …
8
![Page 9: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/9.jpg)
Earth Observing SatellitesOptical and Radar
9
![Page 10: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/10.jpg)
Satellites – Optical View
• Sentinel-2
– 10 meters per pixel
– Two-satellite constellation
– 5 days revisit time
– Global land surface coverage• Onshore, large islands, inland and coastal waters
• Worldview, GeoEye, SkySat
– Up to ca. 0.30 meters per pixel
– High-resolution images
10
Author: Raman
![Page 11: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/11.jpg)
Optical View: 10 meters/pixel
11
![Page 12: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/12.jpg)
Optical View: High-resolution up to ca. 30cm/pixel
12
![Page 13: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/13.jpg)
Radar Satellites – Sentinel 1
• Synthetic Aperture Radar (SAR)
• Sentinel-1
– 6 day repeat cycle
– Two satellite constellation
• For the first time continuous
radar scanning
13
© ESA
![Page 14: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/14.jpg)
Radar-Based Earth Observation
14
![Page 15: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/15.jpg)
Satellite Image ClassificationOptical Images
15
![Page 16: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/16.jpg)
EuroSAT Publicly Released by DFKI
16
![Page 17: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/17.jpg)
Ground Truth Data
• European Urban Atlas
– Detailed mapping for 695 cities distributed over 30 Europeancountries
– Released August 2016
– Covered time period: 2011-2013
17
P. Helber, B. Bischke, A. Dengel, and D. Borth, “Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification,” arXiv preprint arXiv:1709.00029, 2017.
![Page 18: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/18.jpg)
EuroSAT - Distributed Over 30 Countries
• 10 classes
• 27,000 images
• 64 x 64 images
• 2,000 – 3,000 images per class
• 13 spectral bands• Spatial resolution: 10 m per pixel
18
P. Helber, B. Bischke, A. Dengel, and D. Borth, “Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification,” arXiv preprint arXiv:1709.00029, 2017.
![Page 19: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/19.jpg)
Land-Use and Land-Cover Classification
19
Industrial Residential Highway Annual Crop Permanent Crop Pasture
Sea & Lake RiverForestHerbaceous vegetation
![Page 20: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/20.jpg)
Classification with ResNet and GoogLeNet
• Dataset split
– 80% Training and 20% Testing
• Convolutional Neural Networks
– ResNet-50
– GoogLeNet
20
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1–9, 2015.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
Inception Module
ResNetGoogLeNet
Residual Building Block
![Page 21: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/21.jpg)
Models Pre-trained on ImageNet
21
• Transfer learning
– Fine-tuning of pre-trained networks
• Pre-trained on ImageNet
– ILSVRC-2012
Band combination
RGB SWIR CI
Accuracy 0.9857 0.9705 0.9830
SWIR = Short-Wave-Infrared
CI = Color-Infrared
![Page 22: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/22.jpg)
Classification Results
22
• Transfer learning
– Fine-tuning of pre-trained networks
• Pre-trained on ImageNet
– ILSVRC-2012
Band combination
RGB SWIR CI
Accuracy 0.9857 0.9705 0.9830
SWIR = Short-Wave-Infrared
CI = Color-Infrared
![Page 23: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/23.jpg)
Spotting Land Use Changes
23
Residential Area Built Upin Dallas, USA
August 2015 March 2017
• Large-scale scanning and monitoring
• Time component• High frequency
• Near-real-time
• Future availability
• Innovative applications
• Building systems for future
real-time applications
![Page 24: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/24.jpg)
Spotting Land Use Changes
October 2015 September 2016
Deforestation (Forest Clearing)in Villamontes, Bolivia
24
Industrial Buildings Demolished in Shanghai, China
December 2015 December 2016
![Page 25: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/25.jpg)
Support Mapping Services
25
Melbourne, Australia Shanghai, China
![Page 26: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/26.jpg)
Look More CloselyClassification with High-Resolution Optical Images
26
![Page 27: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/27.jpg)
Functional Map of the World
• Four different satellites
• Different spatial and spectralresolutions
– 4-band and 8-band
• Temporal image sequences
• 63 land-use/-cover and building types
– Security & Defense
– Emergency Response
– Infrastructure
27
Christie, Gordon and Fendley, Neil and Wilson, James and Mukherjee, Ryan, “Functional Map of the World” CVPR, 2018.
Airport Car DealershipBorder Checkpoint
Police Station Shopping MallFlooded Road
Lighthouse DamSolar Farm
![Page 28: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/28.jpg)
Task: Sequence Classification
• Sequences determine the underlying class
– Construction sites
– Flooded road
28
![Page 29: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/29.jpg)
Classification and Fusion Approaches
• DenseNet-161
– CNN Features (I)
– Metadata Features (M)
– CNN + Metadata Features (IM)
• Sequence Classification
with LSTM
29
DenseNet
Approach LSTM-M CNN-I LSTM-I CNN-IM LSTM-IM
F1-Score 0.193 0.679 0.688 0.722 0.734
Christie, Gordon and Fendley, Neil and Wilson, James and Mukherjee, Ryan, “Functional Map of the World” CVPR, 2018.
CNN (Features)
Metadata Features
![Page 30: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/30.jpg)
Misclassification
Hard to classify
• Port Shipyard
• Office building Fire station, police station
• Hospital Educational institution
• Police station Educational institution
• Multi unit residential Single unit residential
30
Shipyard Port
HospitalEducational Institution
Approach Port Shipyard Police station
Office Building
Hospital
F1-Score 0.193 0.351 0.329 0.225 0.458
![Page 31: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/31.jpg)
Satellite Image SegmentationHigh-Resolution Optical Images (Aerial)
31
![Page 32: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/32.jpg)
Fine-Grained Segmentation With 10cm/Pixel Images
32
![Page 33: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/33.jpg)
Fine-Grained Cadastral Map for Land Change Detection
33
Work in progress!
![Page 34: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/34.jpg)
Netherlands: High-Resolution RGB+NIR
34
![Page 35: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/35.jpg)
Leverage Learning with Altitude Measurements
35
![Page 36: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/36.jpg)
Beyond the Visible SpectrumMultispectral Signatures
36
![Page 37: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/37.jpg)
Multispectral Classification
37
EuroSATRGB
FMoW
![Page 38: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/38.jpg)
Multispectral Segmentation
• Damage Estimation Caused by Wildfires
– Dataset• Satellite: Sentinel-2
• Mapping: EMS Coperncicus
• 13 spectral bands
• 1.564 scenes from 5 different countries
• 256px images
– Band combinations• Visible spectrum: RGB
• Infrared spectrum: Red Edge
• Near-Infrared: NIR
38
![Page 39: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/39.jpg)
RGB + Red Edge + NIR Spectral Information
39
SegNet
SegNet Input Pix. Accuracy mIOU
RGB 0.9878 0.7657
Red Edge 0.9885 0.7592
NIR 0.9815 0.4964
RGB + NIR 0.9864 0.7844
RGB + Red Edge 0.9855 0.7928 (+2,71)
RGB + Red Edge + NIR Prediction
loss
epoch
![Page 40: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/40.jpg)
Qualitative Results
40
GT RGB RGB + Red Edge
RGB Red Edge10m 20m
![Page 41: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/41.jpg)
Beyond the Visible SpectrumRadar Waves
41
![Page 42: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/42.jpg)
Optical Satellite Images - Obstructed View
42
![Page 43: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/43.jpg)
Optical Satellite Images - Obstructed View
43
![Page 44: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/44.jpg)
Optical Satellite Images - Obstructed View
44
![Page 45: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/45.jpg)
Optical Satellite Images - Obstructed View
45
![Page 46: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/46.jpg)
Vision Independent of Clouds
46
![Page 47: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/47.jpg)
Radar-Based Earth Observation Imagery
47
VVVH
© ESA
• Synthetic Aperture Radar (SAR)
• Satellite: Sentinel-1
• Dual-Polarization
– VV and VH
• Scenarios• Deforestation
• Find safest route for icebreakers
• Land monitoring
• Ship traffic monitoring
• Natural disasters like flooding and earthquake
![Page 48: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/48.jpg)
Dual-Polarized SAR
48
Flooding
• Satellite: Sentinel-1
• Mapping: EMS Copernicus
• 9.426 radar image scenes
• 56 flooding events in 12 countries
• Globally spread• Australia, USA, France, UK, …
![Page 49: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/49.jpg)
Segmentation with Dual-Polarized Radar Waves
• Convolutional Networks
– SegNet: Convolutional Encoder-Decoder Network
49
SegNet PredictionInput
VV + VH
Model Pix. Accuracy mIOU
FCN-6 0.9496 0.6772
VGG16 0.9413 0.6740
SegNet 0.9530 0.6541
Segmentation
![Page 50: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/50.jpg)
Natural Disasters: Damage Maps
Flooding: Damage Estimation Fire: Damage Estimation
• Biomass by radar waves
50
• Government, Insurance, Investment
Copernicus Emergency Management Service (© European Union, 2012-2017)
Destroyed
Highly damaged
Slight damage
![Page 51: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/51.jpg)
Multimedia Satellite Task 2018
• Emergency Response for Flooding Events
• Focus on Road Passibility (Impact of Infrastructure) (Road Access, blocked Road) with two subtasks
• Classification of Road-Access & Passability– in Social Multimedia
– in multiple Satellite Images (Radar, Optical)
• Registration open (http://www.multimediaeval.org)
51
Images in Tweets (Hurricane Harvey)
Satellite Images of Houston (US) for Hurricane Harvey
CombineResultvia Geo-Location
![Page 52: Deep Learning Competence Center & Smart Data and Knowledge … · 2018. 4. 19. · Classification and Fusion Approaches • DenseNet-161 –CNN Features (I) –Metadata Features (M)](https://reader036.vdocuments.site/reader036/viewer/2022071217/604abefb3a39c1047b3c7309/html5/thumbnails/52.jpg)
Thanks!
52
NVAIL partner
all networks trained on DGX-1