astrogrid status soft the vo the future astrogrid presentation to gsc andy lawrence july 2003
TRANSCRIPT
![Page 1: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/1.jpg)
AstroGrid AstroGrid
• status
• SOFT
• the VO
• the future
AstroGrid presentation to GSC Andy Lawrence July 2003
AstroGrid presentation to GSC Andy Lawrence July 2003
![Page 2: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/2.jpg)
![Page 3: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/3.jpg)
status status
• six months into Phase B • team of 26 people (23.4 FTEs)• spent 1.5M (40% of budget) • 10,000 lines of code • working s/w released • working on infrastructure, standards
– not tools
![Page 4: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/4.jpg)
Delivered in Phase-A Delivered in Phase-A
• requirements analysis• technology evaluation• comprehensive Phase A report• demonstration s/w (with AVO)• initial VO standards (with AVO and NVO)• respected position in UK e-sci programme• world ranking position in VO work
![Page 5: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/5.jpg)
Delivered in Itn-01,02 Delivered in Itn-01,02 • basic engine room components
– registry schema, implementation, and query service– data access and job control methods– MySpace manager, explorer, and registry
• preliminary interface and contents– simple portal, no tools– populated with a handful of DCs and datasets
• further international/esci visibility– use of OGSI, OGSA-DAI, formed GGF-RG– defined IVOA standards process
• OGSI experiments– intercontinental visualisation with grid services
![Page 6: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/6.jpg)
Planned for Itn03,04 Planned for Itn03,04
• engine room continued– consolidate Registry, Data Access, MySpace – add Auth/Auth, workflow tool (poss Triana)– Registry harvesting
• interface and contents– astronomer useable interface– meaningful contents– data federation services
• international programme– standards for Registry, VOQL
![Page 7: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/7.jpg)
demonstrations demonstrations
• Jan 2003 : with AVO– modest technical advance – new tools, well integrated, excellent PR
• July 2003 : AGOC and Sydney– large technical advance– no new tools, minimal interface
• next steps– feedback from AGSAG beta testers crucial– wrap tools for next demo
![Page 8: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/8.jpg)
TargetTarget
![Page 9: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/9.jpg)
CurrentCurrent
![Page 10: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/10.jpg)
![Page 11: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/11.jpg)
successes successes
• well motivated hardworking team
• rigorous s/w eng processes
• code cut and released on schedule
• basic engine room components in place
• status in international VO
• status in astronomical community
![Page 12: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/12.jpg)
opportunities opportunities
• our infrastructure could become standard
• additional tool access via AVO collaboration
![Page 13: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/13.jpg)
failures failures
• portal work failed : GUI hacked fixed
• process take-up slow fixed
![Page 14: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/14.jpg)
threats threats
• US diverging on registry standard• OGSA-DAI vs standard Grid• too little Data Centres effort
– incorporating datasets– writing services
![Page 15: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/15.jpg)
Progress on AstroGrid goalsProgress on AstroGrid goals(1) develop standards for data, metadata, data exchange and provenance GOOD(2) develop a software infrastructure for data services GOOD(3) establish a physical grid of resources GOOD but DC effort worrying(4) construct and maintain a Service and Resource Registry GOOD(5) implement a working VO system of real scientific use to astronomers TBD NEXT SIX MONTHS(6) provide a user interface to that VO system BEHIND(7) provide or adapt a set of science user tools to work with the VO LACK OF FUNDING(8) establish a leading position for the UK in VO work GOOD
![Page 16: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/16.jpg)
![Page 17: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/17.jpg)
The VO is a system that will ..The VO is a system that will ..• allows users to interrogate multiple data centres in a
seamless and transparent way
• provide new powerful analysis and visualisation tools within that system
• give data centres a standard framework for publishing and delivering services using their data.
AstroGrid Phase A ReportOctober 2002
![Page 18: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/18.jpg)
multi- views of a Supernova Remnant
Shocks seen in the X-ray
Heavy elementsseen in the optical
Dust seen in the IR
Relativistic electrons seen in the radio
![Page 19: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/19.jpg)
What happens to the Earth's magnetosphere during a coronal mass ejection ?
Event imaged by space-based solar observatory
Effect detected later bysatellites and ground radar
![Page 20: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/20.jpg)
needles in a haystackneedles in a haystack Hambly et al 2001
- faint moving object is a cool white dwarf- may be solution to the dark matter problem- but hard to find : one in a million- even harder across multiple archives
![Page 21: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/21.jpg)
The VO is a system that will ..The VO is a system that will ..• allows users to interrogate multiple data centres in a
seamless and transparent way
• provide new powerful analysis and visualisation tools within that system,
• give data centres a standard framework for publishing and delivering services using their data.
VO-1
VO-2
VO-3
![Page 22: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/22.jpg)
The VO is a system that will ..The VO is a system that will ..• allows users to interrogate multiple data centres in a
seamless and transparent way
• provide new powerful analysis and visualisation tools within that system,
• give data centres a standard framework for publishing and delivering services using their data.
VO-1
VO-2
VO-3
looking good
need effort for DCs to "publish and deliver"
almost untouched
![Page 23: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/23.jpg)
the future : data servicesthe future : data services
• VO : geometry– not a warehouse; not a hierarchy; not P2P – small set of service centres + large popn of end-users
• VO : content– not software monolith : framework + standards – needs content : data services and user tools
• implies work for Data Centres– writing services; framework uptake; resource management– upcoming opportunities internationally competitive – establish a few centres as professionally competitive
![Page 24: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/24.jpg)
yesterdayyesterday
browserfrontend
CGIrequest
html
web page
DBengine
SQL
data
![Page 25: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/25.jpg)
todaytoday
appl
icat
ion
webservice
SOAP/XML request
SOAP/XML data
DBengine
SQL
nativedata
anyt
hin
g
standard formats
![Page 26: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/26.jpg)
tomorrowtomorrow
appl
icat
ion
webservice
job
results
anyt
hin
g
webservice
webservice
webservice
webservice
webservice
Registry Workflow
GLUE AstroPass MySpace
standard semantics
publ
ish W
SDL
![Page 27: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/27.jpg)
day after tomorrowday after tomorrow
appl
icat
ion
gridservice
job
results
anyt
hin
g
gridservice
gridservice
gridservice
gridservice
gridservice
Registry
Workflow GLUE AstroPass MySpace
poo
led
res
ourc
e
standard semantics
agents
ontology
![Page 28: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/28.jpg)
work neededwork needed
appl
icat
ion
gridservice
job
results
anyt
hin
g
gridservice
gridservice
gridservice
gridservice
gridservice
Registry
Workflow GLUE AstroPass MySpace
poo
led
res
ourc
e
standard semantics
agents
TOOLS
STANDARDS
INFRASTRUCTURE
TECHNOLOGYRESEARCH
DATA SERVICES(access and analysis)
GRID UPTAKE
DA
TA
PIP
EL
INE
S
ontology
![Page 29: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/29.jpg)
![Page 30: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/30.jpg)
expected status 2004expected status 2004
• tools : ok but minimal
• standards : good but evolving
• glue : good but could be better
• data access services : ok but under-resourced
• data analysis services : almost untouched
• grid uptake : ready to deploy but resource worry
• technology research : treading water
![Page 31: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/31.jpg)
AstroGrid-2 AstroGrid-2
• priority areas– core : evolve and improve, build tools
– establish content : data services, resource grid
– analysis services : visualisation, datamining : research and deploy
– next technology wave : ontology, agents
• expertise needed– existing AstroGrid developer team
– alliance of Data Centres
– add Computer Science expertise
![Page 32: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/32.jpg)
CS expertiseCS expertise
• Leeds : visualisation experts
• Portsmouth : kd-tree algorithms
• Exeter : Intelligent Agents for astronomy
• commercial experience : Linde, Noddle
• local links : Bristol, Edinburgh, Cambridge
• external links : OGSA-DAI, GGF, MS, IBM
![Page 33: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/33.jpg)
InfrastructureInfrastructure
• framework largely in place but..
• needs to be robust and reliable
• needs continuing work on standards– IVOA and Grid
• needs suite of tools
• key idea is personalisation– tool requests from competitive calls
delivers VO-1
![Page 34: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/34.jpg)
R&DR&D
• really mostly D– assess, customise, integrate
• keep on the next wave– ontology, agents
• keep promise that everybody can be a power user– large volume and multi-D visualisation and datamining services
delivers VO-2and re-newed VO-1
![Page 35: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/35.jpg)
Data CentresData Centres
• Data Centres are the heart of the VOthere is no UK VO without well resourced DCs
• top priority is support for VO/uptake
• second priority is core support for DCs– not mission specific resources
– not pipeline or archive development
– basic physical resource management and data curation
delivers VO-1
delivers VO-3
![Page 36: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/36.jpg)
Costed Model OptionsCosted Model Options
• M1 : full programme £9.25M• M2 : full research £6.68M
– lose DCA core support
– world beating VO infrastructure but UK content compromised
• M3 : partial research £5.28M– lose some lines and trim others; minimise new algorithms
– good working VO but loses opportunity for UK lead in new technology
– approximately flat continuation of AG1-PhaseB
• M4 : minimum viable £3.75M– no research; unlikely to lead any third wave of VO development
– working VO but relies on external components
![Page 37: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/37.jpg)
![Page 38: AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003](https://reader030.vdocuments.site/reader030/viewer/2022032612/56649ee65503460f94bf5c72/html5/thumbnails/38.jpg)
FIN