synthesis of loi's f. mignard oca/cassiopée
DESCRIPTION
Synthesis of LoI's F. Mignard OCA/Cassiopée. Main Statistics. 162 answers received and validated 29 data processing in general 7 for calibration 10 double and multiple stars 16 photometry 10 planetary systems 13 radial velocity 8 relativity and reference frame 5 science alerts - PowerPoint PPT PresentationTRANSCRIPT
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory1
Synthesis of LoI's
F. Mignard
OCA/Cassiopée
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory2
Main StatisticsMain Statistics
•162 answers received and validated
29 data processing in general
7 for calibration
10 double and multiple stars
16 photometry
10 planetary systems
13 radial velocity
8 relativity and reference frame
5 science alerts
12 simulations
17 solar system objects
20 source classification
17 variable stars
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory3
Major Coordinated answers Major Coordinated answers
• A set from Cambridge to lead the photometric processing
• A combined answer from Cambridge and Geneva for the science alert
•One answers from Geneva coordinating the involvement of ISDC
• One series from ARI to cover the first look, bright stars analysis
• Two coordinated letters from CNES covering support for simulation, major
involvement in the data analysis, system architecture
• One global answer from Italy for a quality and validation system
• An international answer coordinated by Meudon for the RVS analysis
•One global answer from the SSWG to carry out the analysis for solar system
objects
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory4
Answers for the Variable stars
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory5
Synthesis of answersSynthesis of answers
•Number of answers Collective : 6
Individual : 11 but some may have considered a personal reply unecessary
•Areas covered Detection of variability
Analysis of light curves
Period determination
Classification
Follow-up
Science alert
•Areas marginally covered Simulation
overall management
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory6
# institute Contact FTE Areas of interest
99 Geneva Obs. G. Burki Commitment of Geneva Obs.
45 Inst. Tech. Lausanne S. Morgenthaler 1 Statistical modelling, classification
98 Geneva Obs. L. Eyer 2 to 10 ISDC involvement, study of variability
22 Cambridge G. Gilmore 1-2 Detection and analysis of variables
146 OHP M. Boer 4 Follow-up, science alert
165 Univ. Cathol. Louvain C. Aerts 5 ? Automated classification of lightcurves
Answers to LoI : collective answers Answers to LoI : collective answers
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory7
# Name institute FTE/yr Areas of interest
2 E. Milone Calgary 2-4 EB : modeling, simulation
13 L. Szabados Budapest 2 Variable stars analysis
15 Gomez de Castro Madrid ? Data mining, mathematical modeling
77 J. Fabregat Valencia 0.5 O-B Star classification
87 P; Mathias Nice 0.2 Light curves of Gamma Dor
111 P. de Laverny Nice 0.5 Long period Miras
113 T. Zwitter Lublijana 0.5 EB and Spectro binaries
116 A. Prsa Lublijana 0.5 Period determination, EB
145 M. Chadid Nice 0.1 Pulsating stars
149 E. Poretti Milan 0.2 Pulsating stars
177 G. Gilmore Cambridge 1-2 Light curves and planetary transits
Answers to LoI : individual answers Answers to LoI : individual answers
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory8
# Name institute FTE/yr Areas of interest
18 G. Gilmore Cambridge 5-7 S/W for the photometric processing
51 A. Brown Leiden 2-3 Core photometric processing
94 J. Carme Barcelone 2 Photometric simulation, calibration
144 A. Vidal Madjar Paris (IAP) 3 Periodic variables, micro-lensing,
planetary transits
30 C. Bailer-Jones Heidelberg 5.5 Stellar classification
Related proposalsRelated proposals
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory9
Organisation of the Data
Processing
F. Mignard
From GST and DACC inputs
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory10
Objectives and challengesObjectives and challenges
•Gaia will observe over 5 years with three instruments
•Gaia will produce ~ 200 TB of raw data
At the lowest level they are primarily CCD counts
•The DP must cope with
A large treatment both in volume and computation
A complex and iterative treatment
Interconnected data and tasks
•It is a community task, neither organised, nor funded by ESA
•If must be ready by the launch time
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory11
Objectives and challengesObjectives and challenges
•It requires specific expertises and commitment :
Overall analysis of a complex project
Professional S/W developers
Quality control
Strong commitment of teams over ~ 10 to 15 years
Ability to work with large team under pressure with tight schedule
•The basic organisation does not exist yet
This is very different from the WG structure
The level of coordination is much more demanding
Science will not be the primary activity in the DP
The organisation must have clearly defined responsibilities
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory12
Overall ConceptsOverall Concepts
•There are three obvious main steps in the DP
The initial data treatment of the CCD images up to centroiding
The astrometric, photometric and spectroscopic iterative solution
The special non iterative processes
•The global structure should be
Must be as simple as possible
Clear hierarchy with non overlapping responsibilities
GST must have the full visibility and retain control of the DP
•The current workplan
Global system called Gaia Data Analysis Consortium (DAC)
Central concept around Coordination Units (CU)
Within each CU there are Development Units (DU)
The lower level is the Work Package (WP)
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory13
Coordination UnitsCoordination Units
•Small in number ( < 10)
•Each must be in charge of a significant fraction of the DP
•Interfaces betweens CU's are limited
•Each CU coordinates the developments of many tasks
It defines the structure in DU
It follows the development, testing and documentation of the S/W
It defines and implement the interfaces between the DUs
•A CU is placed under the responsibility of a manager and his
deputy
They manage their CU
They provide contact and interfaces with the other CUs
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory14
Identified Cus (provisional)Identified Cus (provisional)
•C1 : Overall system architecture and central DB
•C2 : Data simulation ( three levels)
•C3 : Astrometric core processing
•C4 : Object processing
•C5 : Photometric reduction
•C6 : Spectroscopic reduction
•C7 : [Scientific exploration and Catalog access]
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory15
Identified Cus leaders and Identified Cus leaders and deputiesdeputies
•C1 : Overall system architecture W. O'Mullane, U. Lammers
•C2 : Data simulation X. Luri, C. Babusiaux, F. Mignard,
•C3 : Astrometric core processing U. Bastian, M. Lattanzi, J. Torra
•C4 : Object processing D. Pourbaix, C. Bailer Jones, P.
Tanga
•C5 : Photometric reduction F. van Leeuwen, A. Brown, C.
Jordi
•C6 : Spectroscopic reduction D. Katz, M. Cropper, U. Munari
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory16
Developpement UnitsDeveloppement Units
•Responsibility for the development of S/W
Conceptual development
Algorithm testing on simulated data
Preparation of the implementation of the operational versions
Process the Gaia data and provide outputs for the DB
•A DU should be based on a small number of well connected groups
Typically 5 people in each place
•Examples of Dus :
PSF calibration, RV determination, orbit determination, taxonomic
classification
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory17
Architecture and H/WArchitecture and H/W
•Overall architecture not yet defined
•Several approaches possible :
All computational H/W and DB at one place serious funding problem
All core processing at one place + distributed shell tasks data transfer
Centralised data base and core tasks ; separate location for integration
of shell tasks and centralised or distributed execution
Local data bases with shell tasks processing and coordination units
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory18
Astro-1: 20 TB
Astro-2: 20 TB
RVS : 30 TB
MBP : 25 TB
50 GB/day
11h mean GS visibility per day
5 Mbps downlink rate
MOC
TMArchive
GAIA
primary
Data
Base
Initial treatementCalibrations
First lookSystem determination (GIS)
C3
C2
Simulation
Data Streams
Images
Objects
C5Photometry
C6Spectro
C4Object
Photo DB Spectro DB Object DB
MBPBBP
VariabilityAlerts
Calibrations
SGISRVAP
DMSSolar System
PlanetsGalaxies
QSOs
Class. DB
APAge
Global Class.
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory19
Astro-1: 20 TB
Astro-2: 20 TB
RVS : 30 TB
MBP : 25 TB
50 GB/day
11h mean GS visibility per day
5 Mbps downlink rate
MOC
TMArchive
GAIA
primary
Data
Base
Initial treatementCalibrations
First lookSystem determination (GIS)
C3
Simulation
C2
Data Streams
Images
Objects Photo DB
C5Photometry
Spectro DB
C6Spectro
Object DB
C4Object
MBPBBP
VariabilityAlerts
Calibrations
SGISRVAP
DMSSolar System
PlanetsGalaxies
QSOs
Class. DB
APAge
Global Class.
CambridgeGenève
CNES
ESAC, UB , ISDC ?
UB,CNES
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory20
How to form the Consortium How to form the Consortium ??
•Setting of a DACC (Data Analysis Coordination Committee)
•Must establish the structure of the consortium :
Breakdown into CU compatible with data flow
List of DUs within each CU
Examine the funding issues, data flow problem, system architecture
Identify institutes, teams or individuals for the WPs
•DACC established for one year
Will be replaced by the DACE (Data Analysis Consortium Executive)
•An AO will be issued to formalise the structure
no competitive AO is foreseen
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory21
Composition of the DACCComposition of the DACC
•Chair : F. Mignard
•Co-chair : C. Bailer-Jones
•CU managers :
W. O'Mullane, X. Luri, U. Bastian, D. Pourbaix, F. Van Leeuwen, D. Katz
•Project scientists : M. Perryman
•+ Experts : A. Brown, U. Lammers, L. Lindegren, J. Torra
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory22
Pyramidal structurePyramidal structure
•GST (includes all the CU managers)
•Data Analysis Consortium Executive (DACE)
Consists of the CU managers
•Coordination Units
Manager, deputy manager, S/W engineers, DUs
•Development Units
Leader, includes several work packages
•Work Packages
From individual to small groups within one location
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory23
Next StepsNext Steps
•List of work-packages of each CU
Compilation document in preparation
•Construct the data flow constraints
Functional analysis
List of processes and dependencies
•CU managers build their architecture of DUs
Should reflect the answers to the LoI
Large contact with the community needed
• Next meeting of the DACC : 6-7/10 in Heidelberg
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory24
MOC: Telemetry ArchiveHousekeeping, Orbit Files
Operations Reports
Astrometry+ Data Base
(sample tasks in GDB)
Object Tasks& Classification
+ Data Base
Spectroscopy+ Data Base
(sample tasks TBD)
Photometry+ Data Base (position and flux information)
(sample tasks in GDB)
DecodingDecompression
Pass Archive(flat files, t order)
Spatial Ordering(HTM)
Cross Matching(A/P/S)
INTEGRATOR
Raw DataBase
[Oracle] Update LSF centroidPosition + Flux
Element AstromElement Photom
Match Table
Source (+RV)Attitude
CalibrationGlobal
Gaia Main Data Base
VN VN-1
Source (+RV)Attitude
CalibrationGlobal
Element AstromElement Photom
Match Table
SAGC VN-1
AstrometricGlobal Iterative
Solution
SourceAttitude
CalibrationGlobal
SpectroscopicGlobal Iterative
Solution
Calibration
Cross CorrelationCCD/FoV/Mission
ASPC results communicated via ID + parameters + data base flags + other flags
120 618
V0 (ground) V1V2
GIS + shell
Integrator
Timeline
months
8-1 weeks
Gaia Data Flow, V1.1, 20-6-05 (MP, WOM, UL, LL, JdB +UB, FvL)
Construct and applyPre-pass
Catalogue (VN-1)
1 iterationearlier
Solar system objects
Radial velocitiesCalibrated spectra
Metallicity, Rotation
Photometric Calibration Model
Source Updating(secondary stars)
AC new
SGC VN-1
Image parametersInitial cross-matchingOn-ground att. 50 mas
Cross Matching
First Look
Science Alerts
Real-Time Attitude(~6 arcsec)
PSF/LSF calibrationCCD calibrationA/P/S calibration
Heidelberg
ASM imaging(disturbing stars)
Reference Frame
Photometric CalibrationParameters
Photometric CalibrationStars
Visual binariesAstrometric binaries
Planets
Source classificationAstrophysical parameters
Calibration Monitoring Reduced photometric data
Epoch photometry
Constant stars
Accumulating statisticsVariability analysis
Period type?
No: Unsolved variables
Period search
Period type?
Yes: Periodic variables
Alert filter
Unexpected change inbrightness/classification
Alerts: SNe, NEO
ESAC
ESAC
Cambridge ?
CNES? CNES?
ESAC
Lund, Barcelona Cambridge, Leiden, Geneva Meudon, MSSL
ESAC
Heidelberg, Bruxelles, Paris
Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory25