![Page 1: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/1.jpg)
raw data recipes SGDP
Science Grade Data ProductsWolfram Freudling
DPD / SDP Group
![Page 2: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/2.jpg)
Science grade data products (SGDPs) are data products which can be used as-is to extract scientific conclusions, or to carry out quantitative measurements.
Instrument signature removed physical units signal-to-noise ratio close to the optimum error estimates all aspect of data taking and processing fully documented
![Page 3: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/3.jpg)
Data Product Survey Nov 2008
AMBER: 4
CRIRES: 11
FORS1: 3
FORS2: 3
FORS12: 16
GIRAFFE: 9HAWK-I: 3ISAAC: 12
MIDI: 6
NACO: 2
SINFONI: 4
UVES: 6
VIMOS: 5
VISIR: 10
About 50% overlap with DFS tickets
![Page 4: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/4.jpg)
Proposed Projects
![Page 5: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/5.jpg)
Science Ready Pipelinesas reported by Survey
• CRIRES LS Spectroscopy • GIRAFFE/MEDUSA• Hawk-I• UVES• VISIR
![Page 6: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/6.jpg)
address issues which apply to several instruments
• Multi-OB image combination • Telluric lines tool• illumination correction • distortions • background subtraction• detector “monitoring”
Austrian contribution
Austrian contribution
![Page 7: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/7.jpg)
![Page 8: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/8.jpg)
Looking for science-ready instrument to develop Reflex
workflow• Hawk-I:– Major problem with background subtraction
=> fixed– Distortions:
• not enough calibration data• needed recipe to analyze distortions• distortions not stable
=> Austrian in-kind
![Page 9: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/9.jpg)
Looking for science-ready instrument to develop Reflex
workflow
• UVES:– error-bars– Combination of calibrations– CR-like artefacts
=> most issues fixed draft of report by Daniel Bramich
![Page 10: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/10.jpg)
• Extrapolation: Every instrument needs substantial work before data products can be considered science-grade
• Many easy to spot problems• Problems often involve both calibration data
and pipeline• Bottle-neck is the resources available for
validation and verification• Only 2.5 astronomers work in SDP group => need IOT involvement
Conclusions so far
![Page 11: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/11.jpg)
New process to set priorities
![Page 12: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/12.jpg)
http://eso.org/observing/dfo/sdp/
![Page 13: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/13.jpg)
Why?• Tickets driven development not good way to set
priorities• Rush to submit something to pipeline priority
meeting• PSD resources are spread too thin:
– do fewer things with more resources
• Projects are too software-centric– full support of projects by IOT/SDP
• Goals of projects are ill-defined– start with good specifications, test plan
![Page 14: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/14.jpg)
“Proposals should be articulated, comprehensive, and set some specific measurable goals to improve data quality. “
Proposals will be assessed by DData PProducts BBoard.
Instrument upgrades should use the same route.
SDP Proposals
![Page 15: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/15.jpg)
The Board
• The Data Product Board is chaired by the head of SDP
• meets twice per year, monthly progress meetings
• Members: IOT Coordinator, head of LPO/PSO, the head of DMO/DPD, head of SDD/PSD, representative from INS/IPD
• First meeting: Feb 2011
![Page 16: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/16.jpg)
but...
• IOT/IS does not have the time, resources, motivation ... to write a SDP proposals
• It is de-motivating if IOT proposal needs DPB approval
• We will not be able to process some data if operational critical tickets are not addressed right away
• I prefer to talk directly to software developer
We should work only on projects which we can complete.
Approved projects get lots of resources.
We need to use our resources efficiently.
This is understood and accepted.
![Page 17: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/17.jpg)
What should be in a proposal / data products requirements
• Specific requirements for data quality• Proposal is not compilation of tickets• Separate requirements from
implementation suggestions• Prototypes for algorithms are great, but...• Detail resources
![Page 18: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/18.jpg)
Effect on Pipeline Development
• All pipeline development frozen be default• Tickets should be submitted but will not be
attended immediately.• Exceptions have to be approved on a case-
by-case basis• “new” pipelines under development not
affected.
![Page 19: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/19.jpg)
Pipelines not frozen• VIMOS: support for upgrade• XSHOOTER: still in development for several
modes• VISIR: basic support for the new detector
only; additional development must go through an enhancement proposal
• OMEGACAM: support for commissioning• VIRCAM: pipeline still in development for
several modes• PRIMA: still in development
![Page 20: Raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group](https://reader036.vdocuments.site/reader036/viewer/2022062314/56649ebc5503460f94bc4b4c/html5/thumbnails/20.jpg)
Questions?
Comments?