cloud computing & big data - morris ... outline of the course 1. cloud computing & big data 2....
Post on 20-May-2020
0 views
Embed Size (px)
TRANSCRIPT
Dr. Gabriele Cavallaro Postdoctoral Researcher High Productivity Data Processing Group Juelich Supercomputing Centre, Germany
Remote Sensing Systems and Applications (1)
October 16th, 2018 Room V02 – 138
Cloud Computing & Big Data PARALLEL & SCALABLE MACHINE LEARNING & DEEP LEARNING
PRACTICAL LECTURE 6.1
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Outline
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Outline of the Course
1. Cloud Computing & Big Data
2. Machine Learning Models in Clouds
3. Apache Spark for Cloud Applications
4. Virtualization & Data Center Design
5. Map-Reduce Computing Paradigm
6. Deep Learning driven by Big Data
7. Deep Learning Applications in Clouds
8. Infrastructure-As-A-Service (IAAS)
9. Platform-As-A-Service (PAAS)
10. Software-As-A-Service (SAAS)
11. Data Analytics & Cloud Data Mining
12. Docker & Container Management
13. OpenStack Cloud Operating System
14. Online Social Networking & Graphs
15. Data Streaming Tools & Applications
16. Epilogue
+ additional practical lectures for our
hands-on exercises in context
Practical Topics
Theoretical / Conceptual Topics
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Outline
Remote Sensing Background
Data Acquisition
Preprocessing
Feature Extraction and Selection
Copernicus: Sentinel 2 Mission
Practical Lecture 6.1 – Remote Sensing Systems and Applications
The term remote sensing was first used in the United States in the 1950s by Ms. Evelyn Pruitt of the U.S. Office of Naval Research
Remote (without physical contact) Sensing (measurement of information)
• Measurement of radiation of different wavelengths reflected or emitted from distant objects or materials
• They may be categorized by class/type, substance, and spatial distribution
[1] Satellite (1960)
Remote Sensing
[2] The Earth-Atmosphere Energy Balance Practical Lecture 6.1 – Remote Sensing Systems and Applications
• Suitable for many applications
• Non-invasive method
• Satellite platforms • Invaluable view • Repetitive and consistent
Application Domain
Practical Lecture 6.1 – Remote Sensing Systems and Applications
• How to assess the damage? • Which are the most hit points?
• How to plan the Humanitarian aid?
A devastating tsunami hit many coastal regions in the Indian Ocean (2014)
• One of the most devastating natural disasters in recorded history • 14 countries were hit
[3] The 2004 Indian Ocean Tsunami [4] Five years after Indian Ocean tsunami
Application Examples (1)
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Global change detection and monitoring: Deforestation
Environmental assessment and monitoring: Urban growth
[6] Deforestation in Bolivia from 1986 to 2001
[5] Population growth from 1975 - 2010 of Manila
Application Examples (2)
1986 2001Practical Lecture 6.1 – Remote Sensing Systems and Applications
• Perhaps the most common form of image interpretation
• Applications: environmental management, agricultural planning, health studies, climate and biodiversity monitoring, and land change detection
Classification of Remote Sensing Images
Generation of thematic maps
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Different tasks of equal importance
Successful classification results depend on all the steps
Pipeline for Classification
New analysis challenges
5Vs: Volume, Variety, Velocity, Veracity and Value
Need of scalable methods and underlying infrastructures
DATA ACQUISITION PRE-PROCESSING
EXTRACT/SELECT FEATURES CLASSIFICATION
POST- PROCESSING
CLASSIFICATION SYSTEM
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Platforms and Sensors
Active Sensor: own source of illumination
Capture image in day and night
Any weather or cloud conditions
Passive Sensor: natural light available
Great quality satellite imagery
Multispectral and Hyperspectral technology
Platform: selected according to the application
[7] Active-and-passive-remote-sensing
DATA ACQUISITION PRE-PROCESSING
EXTRACT/SELECT FEATURES CLASSIFICATION
POST- PROCESSING
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Data are complex, noisy and may contain errors
Improve image quality as the basis for later analyses that will extract information
Detection and restoration of bad lines
Geometric rectification
Image registration
Radiometric calibration
Atmospheric correction
Preprocessing
DATA ACQUISITION PRE-PROCESSING
EXTRACT/SELECT FEATURES CLASSIFICATION
POST- PROCESSING
CLASSIFICATION SYSTEM Practical Lecture 6.1 – Remote Sensing Systems and Applications
Radiometric Calibration and Correction Process
[8] Radiometric Corrections
The value recorded for a given pixel includes:
reflected or emitted radiation from the surface
radiation scattered and emitted by the atmosphere
Most of the applications are interested in the actual surface values
Practical Lecture 6.1 – Remote Sensing Systems and Applications
The Feature Domain
Select suitable features for successfully implementing an image classification
How to obtain discriminating and independent features?
[9] G. Hughes
Too many features may decrease the classification accuracy
(1) : few raw features
(2): hundreds of raw features
DATA ACQUISITION PRE-PROCESSING
EXTRACT/SELECT FEATURES CLASSIFICATION
POST- PROCESSING
CLASSIFICATION SYSTEM
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Normalized Difference Vegetation Index (NDVI)
Create additional relevant features from the existing raw features in the data
Increase the predictive power of the classifier
NDVI = (NIR – R) / (NIR + R)
[11] NDVI
R G
B NIR
Bellingham, WA , US
[10] NDVI & ClassificationPractical Lecture 6.1 – Remote Sensing Systems and Applications
Spatial Information (1) The scene complexity and the spatial resolution determines the number of mixed pixels
The spectral unmixing problem:
Identify the pure materials (endmembers)
Estimate their corresponding proportions (abundances)
mixed spectral signature mixed pixel
65% 20% 5% 6% 4%
soft classification
Two models to analyze the mixed pixel
[12] A. Plaza et al.
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Spatial Information (2)
When spatial resolution increases, structures are larger than the pixel size
The correlation between neighboring pixels increases
Adjacent pixels of a roof pixel belong to the same class with a high probability
Structures can be represented as regions of spatially connected pixels
The presence of mixed pixels can not be avoided
Spatial contextual classifiers can exploit the correlation of pixels within a subset domain
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Classification
DATA ACQUISITION PRE-PROCESSING
EXTRACT/SELECT FEATURES CLASSIFICATION
POST- PROCESSING
CLASSIFICATION SYSTEM
Model choice
Mapping between low level features and high level information
Training
Training set must be representative
Evaluation
Measure the error rate (or performances)
Computational Complexity
Scalability
Practical Lecture 6.1 – Remote Sensing Systems and Applications
World's largest single earth observation programme
Directed by the European Commission in partnership with the European Space Agency (ESA)
Monitors the Earth, its environment and ecosystems
Free and open data policy
Features: Continuity, Global coverage, Frequent updates and Huge data volumes
Served by a set of dedicated satellites (the Sentinel families)
[13] Copernicus programme
[14] Sentinel Space
Copernicus
Practical Lecture 6.1 – Remote Sensing Systems and Applications
Copernicus
[15] Sentinels for Copernicus Practical Lecture 6.1 – Remote Sensing Systems and Applications
Sentinel 2 Mission
• Platform: Twin polar-orbiting satellites, phased at 180° to each other
• Temporal resolution of 5 days at the equator in cloud-free conditions
Provides