LAT ISOC DOE Status Review, 12 Oct 2004

1

GLAST Large Area TelescopeGLAST Large Area Telescope

LATInstrument Science Operations Center

Rob CameronStanford Linear Accelerator Centerrac@slac.stanford.edu650-926-2989

Gamma-ray Large Gamma-ray Large Area Space Area Space TelescopeTelescope

LAT ISOC DOE Status Review, 12 Oct 2004

2

OutlineOutline

Overview ISOC Organization ISOC Operations Software Architecture Operations Facility Planning Development Schedule Current Status

LAT ISOC DOE Status Review, 12 Oct 2004

3

LAT ISOC in the GLAST GDSLAT ISOC in the GLAST GDS

Analysis Software

LAT ISOC DOE Status Review, 12 Oct 2004

4

LAT ISOC ObjectivesLAT ISOC Objectives

The LAT ISOC is organized to:– safely operate the instrument– produce the LAT science data products

Functions:– Command generation; health and safety monitoring– Maintain and modify flight software and testbed– LAT performance verification and optimization– Process and archive Level 1 and Level 2 data– Maintain and optimize the software pipeline that produces

science data products These functions are organized as teams that share personnel

LAT ISOC DOE Status Review, 12 Oct 2004

5

ISOC ReviewsISOC Reviews

ISOC Peer Review: 2 March 2004 ISOC CDR: 4 August 2004 GLAST GSDR: 18-19 August 2004

CDR: very successful review– 8 RFAs and 8 Recommendations received– 6 RFA responses submitted for review, others in work– All recommendations completed

GSDR: very successful review– No ISOC-specific RFAs– Ground-system RFAs will be applied to ISOC as needed

LAT ISOC DOE Status Review, 12 Oct 2004

6

CDR RFA StatusCDR RFA Status

Section RFA Title Requestor Actionee ECD/CommentsSent to

Originator for Review

Submitted to Chairpersons

Closed

1ISOC Documentation Tree

E. Andrews L. BatorResponse submitted

8/12/04    

2Requirements Traceability Tool

E. AndrewsB. CraigJ. Martin

Response submitted

8/27/04 9/07/04   

3.1Add ISOC Architecture Diagram

E. Andrews B. CraigResponse submitted

8/16/04    

3.4ISOC Requirements & Testing (Overall)

E. Andrews J. MartinResponsesubmitted  9/14/04    

4.2 SAA Handling Approach N. Johnson L. BatorResponse submitted

8/30/04    

4.2Launch Critical Support Plan    

M. RackleyJ. Leibee

S. CulpAwaiting GSFC

input      

4.2Red/Yellow Limit Philosophy

J. Leibee L. BatorAwaiting SAI

definitions      

8SAS Verification Approach

M. Rackley J. MartinResponsesubmitted 9/14/04     

LAT ISOC DOE Status Review, 12 Oct 2004

7

ISOC OrganizationISOC Organization

ISOC ManagerR Cameron

Commanding, Health & Safety

Flight Software Performance V&O Science ProductsScience Analysis

Tools

TimelinePlanning

CommandGeneration

Command V&V

CodeMaintenance

CodeV&V

TestbedMaintenance

Calibration

PerformanceTrending

Standard SourceMonitoring

IRF Generation

Level1 CRs, gammas,diagnostics

Level 2Transients, GRBs

PipelineOperation

Simulation/Recon

Science Tools

Infrastructure

Database

Health & Safety

ConfigurationTracking

Documentation

LAT ISOC DOE Status Review, 12 Oct 2004

8

Commanding, Health and SafetyCommanding, Health and Safety

The Commanding, Health and Safety (CHS) team is responsible for:– LAT mission planning support– generation and validation of LAT commands– passing LAT commands to the GSSC – verifying commands were correctly executed – receiving Level 0 data from the MOC– logging and archiving of all commands and Level 0 data– monitoring data to ascertain and track the health and safety

of the instrument– continuous knowledge of the configuration of the LAT

LAT ISOC DOE Status Review, 12 Oct 2004

9

CHS SystemCHS System

ITOS used for command, health and safety functions– HK data limit checking– Telemetry and command definition file validation– Command load verification and validation

ITOS provides compatibility with MOC Other tools

– Level 0 receipt and archiving– HK trending– Data transmission– Mission planning and generation of file uploads– Anomaly tracking and notification– Relational database queries for trending and analysis– Configuration management tools

LAT ISOC DOE Status Review, 12 Oct 2004

10

Flight SoftwareFlight Software

The Flight Software (FSW) team is responsible for:– Updating and validation of all flight software files – Debug or problem fixes to the FSW– Implementation, and validation on the instrument test bed

of authorized upgrades to FSW– Continuing maintenance of the instrument testbed to

ensure it is available to validate code and command sequences as well as to investigate any anomalies seen on orbit

FSW team moves into ISOC after FSQ (April 2005)– Continue code/test/CM cycle as part of the ISOC operations– Maintenance of Test Bed– Review commanding, HK, and performance on a frequent

and regular basis

LAT ISOC DOE Status Review, 12 Oct 2004

11

Performance Verification and OptimizationPerformance Verification and Optimization

The Performance Validation and Optimization (PVO) team is responsible for:– instrument calibration from low level through to IRFs – continuous monitoring of the LAT science performance,

identification of instrument performance trends and resolution of performance anomalies

– generation of performance reports– generation and initial validation of algorithms that improve

on-orbit performance of the LAT– management of pre-launch test and calibration data– configuration and maintenance of the LAT reference

geometry and the LAT Monte Carlo Model The basic structure for the PVO team is already in place for

LAT I&T activities

LAT ISOC DOE Status Review, 12 Oct 2004

12

Science ProductsScience Products

The Science Products Team is responsible for– Generation, archiving and distribution of the Level 1 data– Generation, archiving and distribution of specific Level 2

data needed for transient source detection and GRB parameter determination

– Configuration control of the pipeline and generated data The functions (pipeline etc) used by the Science Products

team are developed by the Science Analysis Software team

LAT ISOC DOE Status Review, 12 Oct 2004

13

Science Analysis SoftwareScience Analysis Software

The Science Analysis Software Team is responsible for– Development and maintenance of the pipeline machinery– Development and maintenance of the simulation,

reconstruction and event classification software– Development and maintenance of the calibration

algorithms, including low level cal and Instrument Response Function generation

– Development and maintenance of the quicklook transient analysis tools

– Development and maintenance of the high-level diagnostics derived from reconstruction and classification

– Development and maintenance of the high-level analysis tools

ISOC will have responsibility for tools used to produce its deliverables

LAT ISOC DOE Status Review, 12 Oct 2004

14

ISOC Staffing PlanISOC Staffing Plan

ISOC Staffing by Job Function

0

10

20

30

40

50

60

1 5 9 13 17 21 25 29 33 37 41 45 49 53

Months since March 04

FTE

Support

Scientist

Technician

Software

Engineering

Management

ISOC staff will grow mostly by team transfer, plus some new hires

LAT ISOC DOE Status Review, 12 Oct 2004

15

ISOC OperationsISOC Operations

ISOC will be involved in all phases of LAT operation: LAT I&T

– Pre-FSW (EM)– Post-FSW (Flight unit level)– TVAC test at NRL

Observatory I&T at Spectrum Astro Launch and Early Orbit (L&EO) – Phase 0

– LAT power-on and configuration– Initial checkout and calibration

First year – Phase 1– Survey mode

Second and subsequent years – Phase 2– Pointed observations– Survey mode

LAT ISOC DOE Status Review, 12 Oct 2004

16

Initial Turn-On and Checkout (Phase 0) Initial Turn-On and Checkout (Phase 0)

Launch and Early Orbit (L&EO) phase currently scheduled for 60 days

Instrument turn-on will not be executed automatically– Humans required to check environmental conditions prior

to significant steps– Must establish correct LAT configuration at each step

Functional checkout of DAQ, ACD, CAL and TKR Initial calibrations ISOC will have staff at MOC during L&EO, but data will also

flow to west coast ISOC for processing

LAT ISOC DOE Status Review, 12 Oct 2004

17

Science Operations (Phases 1 and 2)Science Operations (Phases 1 and 2)

Data taking

– Continuous

– Few commands needed Calibration

– Weekly, biweekly and monthly

– A few commands to initiate Load changes to tables and FSW

– Infrequent

– A few commands and/or file uploads which may be large Load new tables and files

– Infrequent

– A few commands and/or file uploads which may be large Perform Diagnostics

– Infrequent

– A few commands and/or file uploads which may be large

LAT ISOC DOE Status Review, 12 Oct 2004

18

Operations CoverageOperations Coverage

Weekday, day-time operations– Mission planning, command building– Operator shift coverage

• 5 am to 2 pm to cover MOC shift times on East Coast• 9 am to 6 pm to cover ISOC internal coordination needs

Telemetry monitoring and science data processing– Continuous automatic processing

On-call support available 24/7– Support for targets of opportunity, anomalies– Operators for real-time commanding or anomaly support– Software engineer for emergency software support– Computer technician for computer/network support– Team Lead and Instrument Physicist for anomalies

LAT ISOC DOE Status Review, 12 Oct 2004

19

LAT Mission Planning TimelineLAT Mission Planning Timeline

*Minor changes that don’t affect observing schedule or TDRS contacts

July 30

Monday Tuesday Wednesday Thursday Friday Saturday Sunday

31 August 1 2

GSSC Sends Preliminary

Activity Timeline to MOC

3

MOC Requests TDRSS Contacts

for the Week

4 5

6

ISOC Generates Command Plan

7 8 9 10

ISOC Sends Command Plan to

GSSC

11 12

13 14 15 16 17 18 19

Deadline for ISOC to Send Changes* to

Week

20

GSSC Sends Final Activity

Timeline to MOC

21 22

MOC Uploads ATS

23

ATS Goes Active

24 25 26

27 28 29 30 31 September 1 2

Weekly mission planning cycle, with 3 week lead time

LAT ISOC DOE Status Review, 12 Oct 2004

20

Contingency: LAT Instrument Commands, LAT Instrument Loads,

Level 0 Data,LAT Alerts,

Integrated Observ. Timeline,Spacecraft Timeline,As-Flown Timeline,

Flight Dynamics Products,SC Incident Reports,

PDB updates

Software Architecture: Primary Data FlowsSoftware Architecture: Primary Data Flows

MOC

WebAccess

AnomalyTracking &Notification

System

MissionPlanning

CHSmonitoring& trending

Database

LAT Test Bed

Science Data Analysis

Pipeline

Commands to Verify

Tested OK

Receive& Archive

Integrated Observ. Timeline,Spacecraft Timeline,As-Flown Timeline,

Flight Dynamics Products

Level 0Sci Data

Level 0Hkpg Data,LAT Alerts

Level 1&2Sci Data Products

Level 1&2 Sci Data Products,LAT Timeline

LAT Timeline

Prelim. ScienceTimeline, Long TermScience Schedule,

Observ. Timeline Package

GSSCArchive& Send

LAT ISOC

Alerts, Logs,Process status,

File status

Receive& Archive

All data into ISOC,and products out

GSSC

LAT Incident Reports,LAT T&C Database Updates

MOCArchive& Send

LAT ISOC DOE Status Review, 12 Oct 2004

21

Automatic Telemetry ProcessingAutomatic Telemetry Processing

MOC

LAT ISOC

Receive& Archive

Level 0sci & hkpg telemetry, LAT Alerts

Level1 & 2data

products

Level 0 hkpg telemetry,LAT Alerts

GINO & FastCopy/DTS

Monitor hkpg parameters

ITOS

AnomalyTracking &Notification

System

FASAT

Beeper / email message

Level 0 science

telemetry

Realtime Level 0 hkpg telemetry

(during contacts),

MonitorData

STOL Config monitor scripts

Beeper / email message

Text & Stripchart displays (optional)

Science Data Analysis Pipeline

Subsystem-level

monitor

ApplyCalibration

Recon-struction

Transient Search/GRB Refinement

GCN

Level 1 processing

Level 2 processing

GSSCArchive& Send

GINO & FastCopy/DTS

System-levelMonitor

ITOS

Calib.DB

Calibration Trending

MonitorTrends

Alerts, Logs,Process status,File status

Database

Reports, Log entries, Data Products received

and sentLAT Alerts

LAT ISOC DOE Status Review, 12 Oct 2004

22

Requirements Drive ISOC DesignRequirements Drive ISOC Design

ISOC requirements tracing is complete Each ISOC Level 3 requirement was identified as:

1. Procedural (non-software) 42

2. Existing software 277

3. Under development by SAS 39

4. To be developed by others (I&T, FSW, ITOS) 21

5. New development needed 88

Total467

Some requirements map to multiple categories or SW tools, so total exceeds total number of Level 3 requirements (379)

Top-down planning and scheduling of new software development is underway

LAT ISOC DOE Status Review, 12 Oct 2004

23

ISOC Operations FacilityISOC Operations Facility

Working with SLAC management and Facilities group to define near-term ISOC operations facility. Coordinating ISOC space requirements with LAT project management.

Near-term ISOC facility: a new temporary building at SLAC Long-term ISOC facility: discussions are starting with SLAC

and KIPAC management on ISOC facility in new Kavli building at SLAC, available 2006. Also ISOC presence on Stanford campus.

LAT ISOC DOE Status Review, 12 Oct 2004

24

Proposed ISOC Layout for 60Proposed ISOC Layout for 60΄́x36x36΄́ Building Building

Operations Center

59'-6"

35'-6

"

20'-0

"15

'-0"

No window (wall for projection screen)

Glass window in door

Glass window in door

Glass doors or glass windows in doors

Cipher locked

15'-0

"11'-0" 25'-0"

14'-6"

Viewing wiindow

LAT ISOC DOE Status Review, 12 Oct 2004

25

ISOC Trailer LocationISOC Trailer Location

LAT ISOC DOE Status Review, 12 Oct 2004

26

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

                                

SW Dev Slack Times

ISOC Development ScheduleISOC Development Schedule

Launch

Mission Simulations

ISOC testing at NRL

Environmental tests at NRL

ISOC SW Releases

ISOC Simulation Testing

ISOC Demos

ISOC Verif. with Test Bed

Plotting/Trending Dev

DB/Web/E-logbook Dev

Mission Planning Dev

Diagnostic Tool Dev

End-to-End Tests

Ground Readiness Tests

ISOC CDR

Mar-07

Feb-07

Jan-07

Dec-06

Nov-06

Oct-06

Sep-06

Aug-06

Jul-06

Jun-06

May-06

Apr-06

Mar-06

Feb-06

Jan-06

Dec-05

Nov-05

Oct-05

Sep-05

Aug-05

Jul-05

Jun-05

May-05

Apr-05

Mar-05

Feb-05

Jan-05

Dec-04

Nov-04

Oct-04

Sep-04Phase / Milestone

Aug-04

2 3 4 5 6 7

ITOS setup/configuration

1 2 3 4 5 6

1 32

ISOC ready to take over LAT operation

ISOC used to I/F with LAT (front-door only)

4

GSDR

LAT ISOC DOE Status Review, 12 Oct 2004

27

Current StatusCurrent Status

ISOC team structure and development plan is in place – System development underway– Software releases, GRTs and ETE tests as milestones

ISOC activity is ramping up– ISOC staffing plan is being refined– Recent hires in July, August– CHS software developer starting in November– Test engineer to be hired in 2004– ISOC facility and infrastructure being developed

Lots of coordination effort needed between various project elements to bring together all the elements of the ISOC– e.g. inter-team meetings, ICDs