http://ukclimateprojections-ui.defra.gov.uk uk climate projections user interface centre for...

http://ukclimateprojections-ui.defra.gov.uk

UK Climate Projections User Interface

Centre for Environmental Data ArchivalRAL, UK

Stephen Pascoe, Ag Stephens, Alan Iwi, Peter Norton, David Alderson, Philip James, Simon Abele


Introduction

Peter Norton

• A contractor from Tessella

• Worked on the Web Processing Service (WPS) and the data processing code

www.tessella.com


Overview

• What are the UK Climate Projections

• Quick look at the UKCP User Interface

• A Closer look at the WPS

• The UKCP-UI Deployment Architecture


UK Climate Projections

• Funded by UK Department of Environment DEFRA

• Designed to serve as the evidence base for climate change mitigation decisions by public and private UK bodies

• 7 years work by UK Met. Office

• 2 year project at CEDA to build the user interface UKCP-UI


UKCP09 Data Product

• Probabilistic:

• Ensemble of model runs

• Summarised into CDFs and PDFs

• Regional and sub-regional resolution

• 25km grid over land

• Administrative regions and river basins

• Weather Generator


UKCP09 Reports


UKCP-UI a quick look


Starting the request


Selecting a datasource


Emissions Scenario and Time


Selecting a location

• Different methods of selection

25Km Rotated Grid


Selecting a location

5Km Grid

Regions


Choosing your output

• Select from different output types• Can name the request


Graphical Output

Preview Image

Adjust Variables

Download Options

Plot Settings


Graphical Output (Map)


User jobs summary


Data product delivery

• The UKCP output formats• Publication quality plots: PNG, PDF, PS, JPG• CF-NetCDF. via subsetting with cdat_lite

(cdms)• CSV via nappy NetCDF<->NASA Ames library• Shapefiles via shapelib

• Delivered in a zipfile with project-specific metadata

• Duplicate outputs avoided using caching• Resubmit jobs via the “MyJobs” page• Share some jobs via URLs


Architecture

Geoserver (Java)Geoserver (Java)

UI (PHP)UI (PHP)

COWS-WPS (Python)

COWS-WPS (Python) COWS-WMS

(Python)

COWS-WMS(Python)

UKCP Data libUKCP Data lib

geoplotgeoplot

chartplotchartplot

Weather GeneratorWeather Generator

Threshold DetectorThreshold Detector

Sun Grid EngineSun Grid Engine

cdat_lite, matplotlib, rpy, nappycdat_lite, matplotlib, rpy, nappy

User Interface Layer

Application Layer

Library Layer

Processing Layer


The Application Layer


Web Processing Service

• OGC Standard for asynchronous processing

• A WPS service offers a set of Processes that operate on Inputs producing Outputs

• Supports process status polling

• Percent complete

• Outputs available


COWS WPS

• COWS handles OGC interfaces• “Plug-In” process modules

• Synchronous processes run in the server• Asynchronous processes run via Sun Grid

Engine• Cost estimation

• SGE scheduling• Configurable queues for fast and slow jobs• Control number of simultaneous requests• Processing occurs on separate nodes


WPS Protocol


Evaluating the use of a WPS

Advantages:• Someone has thought about the interface

before• Useful access (via http) directly to a process• Clear division between UI and Application layer

code

Disadvantages:• Need to negotiate a clear interface (XML)

between UI and Application layers• Had to follow (or extend) a specification


Deployment


Original Server Structure

ddp-ps1 ddp-ps2 ddp-ps3

ddp-ui1

UI (php)geoserver (tomcat)spatialdb (postgres)

ddp-app1

WMS (python)WPS (python)

ddp-store1

SGE execd

archive

ddp-u2i


ddp-app2


ddp-store2

SGE execd

archive

ddp-ui3


ddp-app3


ddp-store3

SGE execd

archive

ddp-ps5

ddp-admin1

haproxySGE master

userdb (postgres)

cache

ddp-ps4

ddp-adminbak1

mirror statecachebak

acache1

acache2

acache3

Master server and backup

Physical servers

Virtual Machines

Temporary servers for high-demand launch period


Requirements shift

“We need 1,000 simultaneous users” from 50

• Extensive system testing:

• Login, Request Builder, WMS, WPS, download

• Scaled up Xen VM system x3

• Found several critical optimisations


Modified Server Structure

ddp-ps1 ddp-ps2 ddp-ps3

ddp-ps5 ddp-ps4

ddp-ui1


ddp-app1


ddp-store1

SGE execd

archive

ddp-u2i


ddp-app2


ddp-store2

SGE execd

archive

ddp-ui3


ddp-app3


ddp-store3

SGE execd

archive

ddp-ui6


ddp-app6


ddp-ps6

ddp-ui9


ddp-app9


ddp-ps9

ddp-ui7


ddp-app7


ddp-ps7

ddp-ui8


ddp-app8


ddp-ps8

ddp-ui10


ddp-app10


ddp-ps10

ddp-ui11


ddp-app11


ddp-ps11

ddp-admin1

haproxySGE master

userdb (postgres)

ddp-adminbak1

mirror statecache cachebak

acache1

acache2

acache3

Master server and backup

Physical servers

Virtual Machines

Temporary servers for high-demand launch period


Thank you

Stephen Pascoe, Ag Stephens, Alan Iwi, Peter Norton, David Alderson, Philip James, Simon Abele

http://ukclimateprojections-ui.defra.gov.uk uk climate projections user interface centre for...

Documents