interactive web-based geospatial big-data...

Shashikant A SharmaSpace Applications Centre, ISRO

Ahmedabad

sasharma@sac.isro.gov.in

https://vedas.sac.gov.on

Interactive Web-Based

Geospatial Big-Data Analytics for Vegetation Monitoring

Current Indian EO Missions

2

ScatSat

Gujarat as seen by IRS WiFS Data (180 meters)

Agriculture using High Resolution Data (1 m)

Changing Emphases:From Data to Analysis

75%

Data Conversion

10-15%

Attribute Tagging

Spatial 5% Analysis

Data Conversion

Spatial Analysis

Attribute Tagging

Past Present / Future

Changing Emphasesfrom 2-D description to 3-D, 4-D interaction

Past

• 2-D flat map displays

-User as observer

Future

• Effective 3-D visualization• Via the merger of CAD and GIS?

• New data models

• 4-D incorporation of time:

“The time has come for time.”• agents (e.g. vehicles, fires or people)

• interacting over time in a raster (cell)-

based environment according to established

rules

• User as participant • Users interact with the model

• Participatory GIS: the public as the planner

Technological Trends Underlying the Transition

• Location via GPS & Storage• millimeter accuracy • available in cellphone • super high capacity mass storage

pettabyte and more systems, SSDs

• High resolution (<1m) satellite remote sensing• High resolution: 30 cms now, 10cms soon?• Real time Google Earth?

• Communication revolution• super high capacity networks, even to the home • wireless (cellular) communication with anything that

moves anywhere on earth

Technological Trends Underlying the Transition

Information Technology Evolution

• Interoperability:

• Easier sharing of data between users, and among vendor products

• Metadata ; OpenGIS, Spatial Data Transfer Standards

• Mash-ups

• Spatial data tools in DBMS and Software Dev. Env. (e.g. OOPS )

• Oracle Spatial, PostGIS (spatial database engine)

• GRASS, GDAL .. Open source Libraries

• 3-tier computing :

• user interface (client workstation)

• analysis (applications server)

• data (multiple distributed data servers)

Need of Web GIS

Web-GIS is a Geographic Information

System distributed across networked

computer environment to integrate,

disseminate and communicate geographic

information visually on the World Wide

Web over the Internet.

Web Browser

Client

Web Server Web GIS Server

GIS software

GIS database

Server

WWW

Middle Ware

Spatial request

Maps, HTML, Image..

Google Earth – Geomatics Demystified

NASA – Real Time data

Thematic Data Visualisation &Dissemination

• Agriculture• Forestry• Desertification• Wetland• Snow & Glacier• Coastal zone studies• Marine Ecosystem• Polar Science• Hydrology• Climate change• Planetary Science....... .........

Administrative boundary

-National boundary-State; Districts; Tehsil; Village;Cadastral-Thematic boundaries

Base layers

-Road, Rail, River, Waterbody,Drainage, Settlements, Satellitemosaics, DEMs and more...

GIS Apps (Information Systems)

GIS based information systems forUrban; land & water resource planning,snow & glaciers, coastal zonemanagement, socio-economic planning,hydrology, disaster management ..........

Salient features :

• Satellite based geo-spatial data Archival and Dissemination

• Data visualisation in 2-D and 3-D and graphical analysis on web

• Spatial and Non-spatial search engine

• Publish Web map services and metadata of all data

• Geo-processing tools for analysis

• Mentoring development of Indigenous software (IGIS Server)

• Integrate Web Map Service from various sources

• Providing platform for Research & training to Academia by providing data, domain knowledge and infrastructure

• Website available at https://vedas.sac.gov.in

VEDAS : Glaciers in 3D draped on Carto DEM 10m

Solar Site Selection Tool(Using multi-criteria analysis)

1. Road distance2. Grid distance3. Slope4. Solar Insolation5. Landuse

Big Data Analytics of EO data

It is a field that treats ways to analyze, systematically extractinformation from, or otherwise deal with EO data sets that aretoo large or complex to be dealt with by traditional data-processing application software.

Characteristics:

○ Volume: Quantity of Data (Raster – Global coverage)○ Variety: Type and Nature of Data ( PAN, MX, Hys ….)○ Velocity: Frequency of Data Generation (Daily, Hourly …)○ Veracity: Data value and Quality of Data (Reliable as sensed)

Google Earth Engine : NDVI profile using Landsat/Sentinel

Web GIS based Crop Monitoring

Village level NDVI profile

Dynamic NDVI composite & profile for Daily OCM data

Web based Image AnalysisNDVI difference between two date (Dec 25, 2015 & Dec 25, 2010)

Vegetation Condition Index (VCI)

Dashboard for Rajasthan

RGB composite of multi-temporal NDVI dataLISS-IV NDVI images of 23-Jan-2019, 23-Dec-2018, 23-Mar-2019

RGB composite of multi-temporal Sentinel-1 SAR data12-Sep-2019, 31-Aug-2019, 26-Jul-2019

Web based Principal Component Analysis Nov 2017 - Apr 2018 (14 images)

Web based Spatio-temporal Image AnalysisNov 2017 – April 2018 : Image classification

Web based Spatio-temporal Image AnalysisJan, Feb, March 2017 : Image classification

Web based Spatio-temporal Image AnalysisSeptember 1 – 15, 2017 : 0.95 < Soil Moisture Index < 1.0

Nov 1 – Dec 31, 2019 0.4 < NDVI < 1.0

Minimum Maximum

AverageStd. Dev.

Long term statistics of NDVI for February : 2001 - 2018

Handling Big Geospatial Data: Unique Challenges and Solutions

• Data organization (different resolutions, projections,geographical areas)• Resampling and storing dataset vastly different resolutions and keeping them

in storage is inefficient.

• Parallel (multi-core and GPU) algorithms for fast on-the fly resampling.

• Contextual algorithms:• Algorithms which work on individual pixel are easy to parallelize.

• Certain algorithms such as applying median filter require contextualinformation which needs to be provided to each individual processing worker.

• Kernel evaluators which automatically tile the image with necessary overlapand provide it to each worker were developed.

• Storage Access Load• Despite using fast storage (SSDs), IO is a major bottleneck (due to file

size).

• To make storage access as sequential as possible, we performtemporal chunking ( tiling in time dimension ) so each tile containsinformation in a range of 3 dimensions. (x1 to x2, y1 to y2, and t1 tot2). This drastically reduces storage access load.

• Protocol Issues• Browsers keep at most 5 requests pending for accessing data.

• While CPU is busy serving those requests, the disks remain free.

• We keep disks busy by pre-loading datasets which have the highestprobability of next access.

Handling Big Geospatial Data: Unique Challenges and Solutions

Processing Framework

• We do not use Spark or Hadoop - In our experiments they showed bottleneck in network IO

• We have developed our own frameworks which handle task distribution and aggregation.

• We have sacrificed task level resiliency in favour of simplicity and speed as we do not have long running tasks.

AI Application. Urban Built-up area Detection from IRS LISS-IV

• Architecture Used: ASPP-Unet

• Training Data: 10 images of Resourcesat-2 for Indian cities

• Validation Data: 2 images of Resourcesat-2 for Indian cities

• Accuracy Metrics: 72% IoU, 95.7% pixel accuracy

• Limitations: Concrete pavements are sometimes misclassified.

Input Output

• The Charter is a worldwide collaboration, through which satellite dataare made available for the benefit of disaster management.

• Combining Earth Observation assets from different space agencies,the Charter allows resources and expertise to be coordinated forrapid response to major disaster situations.

• This unique initiative is able to mobilise agencies around the worldand benefit from their know-how and their satellites through a singleaccess point that operates 24x7 at no cost to the user.

634 Activations : 125 Countries

17 Charter members : 34 satellites

International Charter : Space and Major Disasters

Floods as seen in Sentinel-SAR data (6 May 2019)Odisha cyclone & floods (Activation 608 on 2 May 2019)

Floods as seen in FCC of DMC data (6 May 2019)Odisha cyclone & floods (Activation 608 on 2 May 2019)

Flood area extracted using Sentinel-SAR data (6 May 2019)Odisha cyclone & floods (Activation 608 on 2 May 2019)

sasharma@sac.isro.gov.in

https://vedas.sac.gov.in

Thanks

Architecture

Storage Node

Storage Node

Storage Node

Processing Node

Processing Node

Processing Node

Serving Node

Serving Node

Load Balancer

(HA)