interactive web-based geospatial big-data...
TRANSCRIPT
Shashikant A SharmaSpace Applications Centre, ISRO
Ahmedabad
https://vedas.sac.gov.on
Interactive Web-Based
Geospatial Big-Data Analytics for Vegetation Monitoring
Current Indian EO Missions
2
ScatSat
Gujarat as seen by IRS WiFS Data (180 meters)
Agriculture using High Resolution Data (1 m)
Changing Emphases:From Data to Analysis
75%
Data Conversion
10-15%
Attribute Tagging
Spatial 5% Analysis
Data Conversion
Spatial Analysis
Attribute Tagging
Past Present / Future
Changing Emphasesfrom 2-D description to 3-D, 4-D interaction
Past
• 2-D flat map displays
-User as observer
Future
• Effective 3-D visualization• Via the merger of CAD and GIS?
• New data models
• 4-D incorporation of time:
“The time has come for time.”• agents (e.g. vehicles, fires or people)
• interacting over time in a raster (cell)-
based environment according to established
rules
• User as participant • Users interact with the model
• Participatory GIS: the public as the planner
Technological Trends Underlying the Transition
• Location via GPS & Storage• millimeter accuracy • available in cellphone • super high capacity mass storage
pettabyte and more systems, SSDs
• High resolution (<1m) satellite remote sensing• High resolution: 30 cms now, 10cms soon?• Real time Google Earth?
• Communication revolution• super high capacity networks, even to the home • wireless (cellular) communication with anything that
moves anywhere on earth
Technological Trends Underlying the Transition
Information Technology Evolution
• Interoperability:
• Easier sharing of data between users, and among vendor products
• Metadata ; OpenGIS, Spatial Data Transfer Standards
• Mash-ups
• Spatial data tools in DBMS and Software Dev. Env. (e.g. OOPS )
• Oracle Spatial, PostGIS (spatial database engine)
• GRASS, GDAL .. Open source Libraries
• 3-tier computing :
• user interface (client workstation)
• analysis (applications server)
• data (multiple distributed data servers)
Need of Web GIS
Web-GIS is a Geographic Information
System distributed across networked
computer environment to integrate,
disseminate and communicate geographic
information visually on the World Wide
Web over the Internet.
Web Browser
Client
Web Server Web GIS Server
GIS software
GIS database
Server
WWW
Middle Ware
Spatial request
Maps, HTML, Image..
Google Earth – Geomatics Demystified
Thematic Data Visualisation &Dissemination
• Agriculture• Forestry• Desertification• Wetland• Snow & Glacier• Coastal zone studies• Marine Ecosystem• Polar Science• Hydrology• Climate change• Planetary Science....... .........
Administrative boundary
-National boundary-State; Districts; Tehsil; Village;Cadastral-Thematic boundaries
Base layers
-Road, Rail, River, Waterbody,Drainage, Settlements, Satellitemosaics, DEMs and more...
GIS Apps (Information Systems)
GIS based information systems forUrban; land & water resource planning,snow & glaciers, coastal zonemanagement, socio-economic planning,hydrology, disaster management ..........
Salient features :
• Satellite based geo-spatial data Archival and Dissemination
• Data visualisation in 2-D and 3-D and graphical analysis on web
• Spatial and Non-spatial search engine
• Publish Web map services and metadata of all data
• Geo-processing tools for analysis
• Mentoring development of Indigenous software (IGIS Server)
• Integrate Web Map Service from various sources
• Providing platform for Research & training to Academia by providing data, domain knowledge and infrastructure
• Website available at https://vedas.sac.gov.in
VEDAS : Glaciers in 3D draped on Carto DEM 10m
Solar Site Selection Tool(Using multi-criteria analysis)
1. Road distance2. Grid distance3. Slope4. Solar Insolation5. Landuse
Big Data Analytics of EO data
It is a field that treats ways to analyze, systematically extractinformation from, or otherwise deal with EO data sets that aretoo large or complex to be dealt with by traditional data-processing application software.
Characteristics:
○ Volume: Quantity of Data (Raster – Global coverage)○ Variety: Type and Nature of Data ( PAN, MX, Hys ….)○ Velocity: Frequency of Data Generation (Daily, Hourly …)○ Veracity: Data value and Quality of Data (Reliable as sensed)
Google Earth Engine : NDVI profile using Landsat/Sentinel
Web GIS based Crop Monitoring
Village level NDVI profile
Dynamic NDVI composite & profile for Daily OCM data
Web based Image AnalysisNDVI difference between two date (Dec 25, 2015 & Dec 25, 2010)
Vegetation Condition Index (VCI)
Dashboard for Rajasthan
RGB composite of multi-temporal NDVI dataLISS-IV NDVI images of 23-Jan-2019, 23-Dec-2018, 23-Mar-2019
RGB composite of multi-temporal Sentinel-1 SAR data12-Sep-2019, 31-Aug-2019, 26-Jul-2019
Web based Principal Component Analysis Nov 2017 - Apr 2018 (14 images)
Web based Spatio-temporal Image AnalysisNov 2017 – April 2018 : Image classification
Web based Spatio-temporal Image AnalysisJan, Feb, March 2017 : Image classification
Web based Spatio-temporal Image AnalysisSeptember 1 – 15, 2017 : 0.95 < Soil Moisture Index < 1.0
Nov 1 – Dec 31, 2019 0.4 < NDVI < 1.0
Minimum Maximum
AverageStd. Dev.
Long term statistics of NDVI for February : 2001 - 2018
Handling Big Geospatial Data: Unique Challenges and Solutions
• Data organization (different resolutions, projections,geographical areas)• Resampling and storing dataset vastly different resolutions and keeping them
in storage is inefficient.
• Parallel (multi-core and GPU) algorithms for fast on-the fly resampling.
• Contextual algorithms:• Algorithms which work on individual pixel are easy to parallelize.
• Certain algorithms such as applying median filter require contextualinformation which needs to be provided to each individual processing worker.
• Kernel evaluators which automatically tile the image with necessary overlapand provide it to each worker were developed.
• Storage Access Load• Despite using fast storage (SSDs), IO is a major bottleneck (due to file
size).
• To make storage access as sequential as possible, we performtemporal chunking ( tiling in time dimension ) so each tile containsinformation in a range of 3 dimensions. (x1 to x2, y1 to y2, and t1 tot2). This drastically reduces storage access load.
• Protocol Issues• Browsers keep at most 5 requests pending for accessing data.
• While CPU is busy serving those requests, the disks remain free.
• We keep disks busy by pre-loading datasets which have the highestprobability of next access.
Handling Big Geospatial Data: Unique Challenges and Solutions
Processing Framework
• We do not use Spark or Hadoop - In our experiments they showed bottleneck in network IO
• We have developed our own frameworks which handle task distribution and aggregation.
• We have sacrificed task level resiliency in favour of simplicity and speed as we do not have long running tasks.
AI Application. Urban Built-up area Detection from IRS LISS-IV
• Architecture Used: ASPP-Unet
• Training Data: 10 images of Resourcesat-2 for Indian cities
• Validation Data: 2 images of Resourcesat-2 for Indian cities
• Accuracy Metrics: 72% IoU, 95.7% pixel accuracy
• Limitations: Concrete pavements are sometimes misclassified.
Input Output
• The Charter is a worldwide collaboration, through which satellite dataare made available for the benefit of disaster management.
• Combining Earth Observation assets from different space agencies,the Charter allows resources and expertise to be coordinated forrapid response to major disaster situations.
• This unique initiative is able to mobilise agencies around the worldand benefit from their know-how and their satellites through a singleaccess point that operates 24x7 at no cost to the user.
634 Activations : 125 Countries
17 Charter members : 34 satellites
International Charter : Space and Major Disasters
Floods as seen in Sentinel-SAR data (6 May 2019)Odisha cyclone & floods (Activation 608 on 2 May 2019)
Floods as seen in FCC of DMC data (6 May 2019)Odisha cyclone & floods (Activation 608 on 2 May 2019)
Flood area extracted using Sentinel-SAR data (6 May 2019)Odisha cyclone & floods (Activation 608 on 2 May 2019)
Architecture
Storage Node
Storage Node
Storage Node
Processing Node
Processing Node
Processing Node
Serving Node
Serving Node
Load Balancer
(HA)