an international virtual observatory data exchange format
DESCRIPTION
an International Virtual Observatory data exchange format. Roy Williams François Ochsenbein Clive Davenhall Daniel Durand Pierre Fernique David Giaretta Robert Hanisch Tom McGlynn Alex Szalay Andreas Wicenec. VOTable. XML: Structured Information. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/1.jpg)
an International Virtual Observatoryan International Virtual Observatory
data exchange format data exchange format
VOTable
Roy WilliamsFrançois Ochsenbein
Clive DavenhallDaniel Durand
Pierre FerniqueDavid Giaretta
Robert HanischTom McGlynn
Alex Szalay Andreas Wicenec
![Page 2: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/2.jpg)
XML: Structured Information
<From>Antonio Stadivarius</From><To>Domenico Scarlatti</To><Date>
<Day>13</Day><Month>4</Month><Year>1723</Year>
</Date><Body>Io bisogno una appartamento acoglienti a Cremona …</Body>
4/13/23
April 13, 1723
17.iv.1723
Separation of structure from presentation
The computer can read the document:“Find all memos from April 1723”
![Page 3: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/3.jpg)
VOTable
• Full metadata representation• Hierarchy of RESOURCEs• containing PARAMs and TABLEs• UCD (unified content descriptor)
– a has unit meter
– a has UCD ORBIT_SIZE_SMAJ (Semi-major axis of the orbit )
• Can reference remote and/or binary streams• Table can be
– Pure XML
– "Simple Binary"
– FITS Binary Table
![Page 4: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/4.jpg)
VOTable Parentage
• Astrores• XML format for tables• Developed at CDS Strasbourg• Presented at ADASS 1999• Vizier implementation
• XSIL• XML format for Tables and Arrays• Developed at LIGO Caltech 2000• Extensible through Type-Class dynamic loading• Java parsing, browsing, editing• Matlab interface
![Page 5: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/5.jpg)
Sample VOTable
<?xml version="1.0"?><!DOCTYPE VOTABLE SYSTEM "http://us-vo.org/xml/VOTable.dtd"><VOTABLE version="1.0"> <DEFINITIONS> <COOSYS ID="myJ2000" equinox="2000." epoch="2000." system="eq_FK5"/> </DEFINITIONS> <RESOURCE> <PARAM name="Observer" datatype="char" arraysize="*" value="William Herschel"> <DESCRIPTION>This parameter is designed to store the observer's name </DESCRIPTION> </PARAM> <TABLE name="Stars"> <DESCRIPTION>Some bright stars</DESCRIPTION> <FIELD name="Star-Name" ucd="ID_MAIN" datatype="char" arraysize="10"/> <FIELD name="RA" ucd="POS_EQ_RA" ref="myJ2000" unit="deg" datatype="float" precision="F3" width="7"/> <FIELD name="Dec" ucd="POS_EQ_DEC" ref="myJ2000" unit="deg" datatype="float" precision="F3" width="7"/> <FIELD name="Counts" ucd="NUMBER" datatype="int" arraysize="2x3x*"/> <DATA> <TABLEDATA> <TR> <TD>Procyon</TD><TD>114.827</TD><TD>5.227</TD> <TD>4 5 3 4 3 2 1 2 3 3 5 6</TD> </TR> <TR> <TD>Vega</TD><TD>279.234</TD> <TD>38.782</TD><TD>8 7 8 6 8 6</TD> </TR> </TABLEDATA> </DATA> </TABLE> </RESOURCE></VOTABLE>
<DATA> <FITS> <STREAM href="ftp://server.com/mydata.fits" expires="2002-02-22" actuate="onRequest"/> </FITS></DATA>
![Page 6: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/6.jpg)
Table Cell
scalar
arrays
variable length arrays
etc
etc
boolean
bit
unsignedByte
short
int
long
char
unicodeChar
float
double
floatComplex
doubleComplex
Primitives
![Page 7: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/7.jpg)
VOTable is Flexy
• eg Table of images• UCD="JPEG_IMAGE"
datatype="unsignedByte" arraysize="*"
• eg Table of URL links• UCD=“DATA_LINK"
datatype="char" arraysize="*"
![Page 8: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/8.jpg)
VOTable Schema (xsd)
![Page 9: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/9.jpg)
Table Data Model
• Metadata• Class definition for Row• FIELD
– data type
– semantic type
• Data• Each Row is a list of Cells• Each Cell is an array of Primitives
– may be variable length
![Page 10: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/10.jpg)
Table Data Layout
• All metadata first – small, complex, XML
• Class definition for table record• + params, description, etc etc
• Then data– (may be) large, remote– XML | binary | FITS
• Instantiations of table record• All records MUST have same format
![Page 11: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/11.jpg)
Param Data Model
• Param is “Table with one cell”• Like a FIELD value• But with a “value” attribute
![Page 12: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/12.jpg)
Primitives
• All have fixed binary length
• Same as FITS primitives
• Except Unicode
datatype Meaning FIT
S Bytes
"boolean" Logical "L" 1
"bit" Bit "X" *
"unsignedByte" Byte (0 to 255) "B" 1
"short" Short Integer "I" 2
"int" Integer "J" 4
"long" Long integer "K" 8
"char" ASCII Character "A" 1
"unicodeChar" Unicode Character 2
"float" Floating point "E" 4
"double" Double "D" 8
"floatComplex" Float Complex "C" 8
"doubleComplex" Double Complex "M" 16
![Page 13: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/13.jpg)
Multidimensional Array Cell
• A table cell can have lots of Primitives• Example: WCS parameters are arrays
– <FIELD name=“CRVAL”datatype=“double”arraysize=“2”/>
• Example: up to 10 images, each 64x64– <FIELD name="thumbs"
datatype="unsignedByte" arraysize="64x64x10*"/>
![Page 14: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/14.jpg)
Hierarchy
• A VOTable contains RESOURCES– RESOURCE can contain:
• TABLE• RESOURCE• etc etc
• Usage example• Many observations in the file,
– each is a RESOURCE
• Each observation is– Parameters
– Calibration table
– Raw data table
![Page 15: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/15.jpg)
Hierarchy
• New feature: GROUP
<TABLE name=“Nutation and Aberration”> <GROUP name=“Nutation”> <FIELD name=“Longitude”/> <FIELD name=“Obliquity”/> </GROUP> <GROUP name=“Aberration”> <GROUP name=“Equinox 1950.0”> <FIELD name=“C”/> <FIELD name=“D”/> </GROUP> <GROUP name=“Equinox 1955.0”> <FIELD name=“C”/> <FIELD name=“D”/> </GROUP> </GROUP></TABLE>
![Page 16: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/16.jpg)
Unified Content Descriptors
• UCD is a “semantic type”
PHOT.INT-MAG.B Integrated total blue magnitude ORBIT.ECCENTRICITY Orbital eccentricity STAT.MEDIAN Statistics Median Value INST.QE Detector's Quantum Efficiency
• Can be resolved by web service– to description, examples, etc
• Base + Specifiers• eg error in default right ascension• POS.EQ.RA, MAIN, ERROR
![Page 17: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/17.jpg)
VOTable Friends
XML BinaryStreaming
Table Datacube Semantics
VOTable√ √ √ √
BinX√ √ √ √
MS Dataset √ √
HDF√ √
XDF√ √
Some “self-describing” file formats
![Page 18: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/18.jpg)
XML Parsing
SAX: Event-BasedHandlers for StartElement, Text, EndElement, etc.
Found element BookCatalogueFound element BookFound Element TitleFound Text The Cambridge Star AtlasFound End Element Title….
![Page 19: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/19.jpg)
Parsing
DOM: Document Object ModelReturns a tree-like Document object with data attached
BookCatalogue
BookBook
Title
ISBN Parallel Computing Works!
Cambridge Star Atlas
Title
Author
Wil Tirion
![Page 20: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/20.jpg)
Binding to make a Parser
for(int i=0; i<table.getFieldCount(); i++){ Field field = (Field)table.getFieldAt(i); String u = field.getUcd(); if(u != null && u.equals("POS_EQ_RA_MAIN")) System.out.println("Field " + i + " is for RA"); }
From the Schema an API and library is generatedJAXBBreezeCastor
This is JAVOT (Caltech)
![Page 21: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/21.jpg)
VOTable Software
Treeview from UK-VO
![Page 22: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/22.jpg)
VOTable Software
VOPlot from India-VO
![Page 23: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/23.jpg)
VOTable Software
VOTool from US-VO
![Page 24: an International Virtual Observatory data exchange format](https://reader036.vdocuments.site/reader036/viewer/2022081513/568133f9550346895d9aeca0/html5/thumbnails/24.jpg)
VOTable software
Mirage from Bell Labs