May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 1

VO Applications @ JHUOpen SkyQuery and more

…T. Budavari, S. Carliles, L. Dobos, G. Fekete, J. Gupchup, V. Haridas, N. Li,

T. Malik, M. A. Nieto-Santisteban, W. O’Mullane, J. Raddick, A. Szalay, & A. Thakar

The Johns Hopkins University

Baltimore, MD, USA.

THE US NATIONAL VIRTUAL OBSERVATORY

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 2

NVO Core Applications

• NVO Registry (STScI & JHU)– Search and Registration of astronomical

resources (Organizations, Catalogs, Images, Services, etc.)

• NVO Spectrum Services– Search, Plotting, and Spectra upload

• Open SkyQuery – Query and Cross-match of catalog data

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 3

Related to NVO

• CasJobs– Personal database (MyDB) on the server

side– Scheduling and tracking of long SQL Jobs

• VO Enabled Mirage– Wrapper to Mirage (Bell Laboratories)– Takes input from existing VO services (Cone

and SIAP) in the form of VOTable

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 4

NVO Registry

• Distributed database of astronomical resources

• Registries share data by harvesting

• Searching & Publishing

• SOAP WSCaltech

NCSA

STScI/JHUHEASARC

Astrogrid

CDS

JapanVO

ESOCfA

NOAOESACNRAO

Registries as of April 2005

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 5

NVO Spectrum Services • Search and access > 500k spectra

• Plots, normalizes, combines spectra

• Export data in VOTable

• SOAP WS• Mirrored and

extended by MAST (STScI)

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 6

Open SkyQuery

• SkyNode individual and cross-match queries

• Import/Export VOTable

• Plots with VOPlot

• Automatic Query Builder

• SOAP WS

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 7

Virtual Observatory Query Language Layers

VOQL2 Federation

VOQL1 Web services

Portals working across nodes –SQL-like query language and federation system i.e. Open SkyQuery

SkyNodes using Astronomical Data Query Language (ADQL) and VOTABLE

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 8

Open SkyQuery Architecture

Users

OSQ Portal

Uses Registry and full SkyNodes.

May use services at any level

VOQL2

ADQL

VO

VORegistryQueryable

Archives

ExecPlan

FULL

SkyNode

SkyNode

SkyNode

SkyNode

SkyNode

register

lookup

lookupBASIC

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 9

Data Flow – Execution plan

SkyNode 1

Open SkyQuery

SkyNode 2

SkyNode 3

query

http://www.openskyquery.net

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 10

Limitations

• Intended to work with small result sets constrained by Region or other attributes– Currently nodes limited to 5k rows– Intention to remove restriction – requires

change in operation

• Upload (MyData as a SkyNode) – Size limit is ~10k rows– No Region support on MyData at the moment

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 11

Contributors & Adaptors

• Contributors– Tamas Budavari and Tanu Malik (original prototype)– Nolan Li – main .Net development (Basic & Full)– Ramon Williamson (Java Basic SkyNode)– Tom McGlynn (Java Full SkyNode)– Iñaki Ortiz ESA/ESAC (Java “Full” SkyNode)

• Adaptors– NOAO

• Open SkyQuery portal for proprietary data using a private registry

– NDWFS, Galex, GOODS, HST, UDF, Phoenix, POSSUM …

– JVO (Subaro XMM), ESAC (XMM & ISO), India VO

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 12

VO Enabled Mirage

• IVOA Client Package (Sam Carliles @JHU)– VOTable parser based on JAVOT and SAVOT

• IVOA FITS Package (Tom McGlynn @Hearsac)

• Wrapper to Mirage (Bell Labs by Tin Kam Ho)

• Simultaneous visualization of tabular and image data

• Connects to Cone and SIAP services

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 13

CasJobs• Batch system• Personal DB on

server side: Tables, Procedures, ...

• Sharing env.• Long queries• Import/Export

VOTable• will(/is) support

VOStore• Adopted by

Galex• Mirrored by VO

India (SDSSDR2)

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 14

Future Work

• Reach the big scale– Parallel Open SkyQuery: framework capable

of processing cross-match queries involving catalogs with billion of rows in less than one hour

• Integrate CasJobs and Open SkyQuery

• CasJobs systems federation

May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 15

Links of interest

• Registry– http://voservices.net/registry/– http://nvo.stsci.edu/voregistry/

• Spectrum Services– http://voservices.net/spectrum/

• Open SkyQuery– http://openskyquery.net

• CasJobs– http://casjobs.sdss.org & http://galex.stsci.edu/casjobs &

http://skynode.iucca.ernet.in/casjobs• VO Enable Mirage & IVOA Client Package

– http://skyservice.pha.jhu.edu/develop/vo/mirage/• IVOA Client & FITS Packages

– http://skyservice.pha.jhu.edu/develop/vo/ivoa/– http://skyservice.pha.jhu.edu/develop/vo/ivoafits/

• First NVO Summer School– http://www.us-vo.org/summer-school/2004/proceedings/