data management for mars exploration rovers

Mars Exploration Rover

DES/JFS - 1

Data Management for Mars Exploration Rovers

David E. SmythJoseph F. Snyder

Jet Propulsion LaboratoryCalifornia Institute of Technology

Pasadena, California

Spacecraft Flight Software WorkshopNovember 2009


Flight Software Workshop, November 2009DES/JFS - 2

Mars Exploration Rovers

Spirit Landed 3 January 2004

Operating in excess of 2027 Sols

Over 30 gigabits data returned

Spirit Opportunity

MGS Odyssey Mars Express

X-band Direct-to-Earth

UHF Relay Link

Backup/Demo UHF Link

Opportunity Landed 24 January 2004Operating in excess of

2006 SolsOver 28 gigabits data

returned


Introduction: Data Management

• MER goal was to leverage as much as possible from Mars Pathfinder mission (MPF)– MER Surface Mission far more complicated and demanding than MPF

– Data Management was a significant departure from MPF

• MER Data Management defined as follows:– “Data” refers to Data Products generate by the Rovers for possible telemetry to Earth, as well as those actually received on Earth· Discrete set of data meant to be logically treated as a unit – a file

· Images, spectra, sets of time-ordered measurements, related set of states, etc.

– “Management” covers the handling of Data Products on-board, on Earth, and the interface between the two

• MER Data Management treated as a System, during development and operations




A Day in the Life of MER

• A Typical Sol on Mars:– Once per Sol Uplink: 100KB– One DTE, one ODY downlink before command generation begins for next Sol.– Limited time to plan for next sol.– The rovers are on opposite sides of Mars…

• MER-specific Flight System Constraints:– Frequent Shutdown/Wakeup Cycles– Data Generation and Storage Capacity Limitations– Asynchronous link bandwidth– Most data not down yet when we start to plan for next Sol– MGS data drop-outs add additional latency

06:00 08:00 10:00 12:00 14:00 16:00 18:00

Earthset

Sunrise Sunset

20:00 22:00 00:00 02:00 04:00 06:00

Events

Rover

Telecom

DSN

Earthrise

Sequenced Activities

(example)

Wake -up

Shutdown

Wake -up

Shutdown

Autonomous Activities

(example)

UHF OdysseyMGS OdysseyMGS

Shutdown

New Sequences

Wake -up

Shutdown

Flight Team

Evaluate telemetry

Plan next sol

Build new sequences

Margin

Mars Local Solar Time

Driving, Imaging, or Spectroscopy

Spectroscopy/ Atmospheric Imaging

Earliest Latest Earliest Latest

Sequence Mode

Autonomous Mode

Shutdown Mode

Legend for Rover Activities

Wake -up Odyssey UHF Relay

Wake -up

Shutdown

MGS UHF Relay

Solar Arrays illuminated & battery potentially charging

Shutdown

DFE/DTE

DTE

Downlink Allocation

View Period

Legend for Telecom

Uplink Allocation

Communication

Window

Margin

DFE - Direct from EarthDTE - Direct to Earth

06:00 08:00 10:00 12:00 14:00 16:00 18:00

Earthset

Sunrise Sunset

20:00 22:00 00:00 02:00 04:00 06:00

Events

Rover

Telecom

DSN

Earthrise

Sequenced Activities

(example)

Wake -up

Shutdown

Wake -up

Shutdown

Autonomous Activities

(example)

UHF OdysseyMGS OdysseyMGS

Shutdown

New Sequences

Wake -up

Shutdown

Flight Team

Evaluate telemetry

Plan next sol

Build new sequences

Margin

Mars Local Solar Time

Driving, Imaging, or Spectroscopy

Spectroscopy/ Atmospheric Imaging

Earliest Latest Earliest Latest

Sequence Mode

Autonomous Mode

Shutdown Mode


Sequence Mode

Autonomous Mode

Shutdown Mode


Wake -up Odyssey UHF Relay

Wake -up

Shutdown

MGS UHF Relay

Solar Arrays illuminated & battery potentially charging

Shutdown

DFE/DTE

DTE

Downlink Allocation

View Period

Legend for Telecom

Uplink Allocation

Communication

Window

Margin

DFE - Direct from EarthDTE - Direct to Earth



Summary Data Management Objective

• Accommodate “human-in-the-loop” Tactical Operations – While simultaneously automating standard operations

– While respecting the unique flight system constraints of MER

– While respecting unique characteristics of each type of communications link

– While supporting changes in downlink opportunities, changes in data collection, and changes in data criticality



Specific Objectives

• Insight into On-board Data

• Explicit Downlink Priority Control

• Transmission Path Control

• Retransmission Control

• Deletion Control

• Data created each Sol is MUCH greater than can be transmitted

• Maintain compact Synopsis of on-board data: metadata– Source– Type– Generating Event– Sample Time

• Metadata must be predictable

• Create and telemeter a Data Product Summary Report containing a listing of metadata of all on-board data products



Specific Objectives






• Some data is critical: needed for planning the next sol

• Individual Data Products have “Dynamic Usefulness”

• Implement fine priority control– Priority set at creation time per command parameters

– Reprioritize individual DP

– Reprioritize sets of DPs



Specific Objectives

• Link characteristics vary considerably– Direct to Earth

· Low bandwidth, least latency, high reliability

– Relay· High bandwidth, fixed

availability, variable latency and reliability

• Implement transmission path control– Assign at discrete data product level

– Assign to sets of data products

– Allow specification of preferred path, multiple paths, or first available








Specific Objectives






• Make partially received data useful– Self-identifying at part level

– Delivered and “viewable” even with holes

• Fine retransmission control– Rexmit parts of a DP– Rexmit entire DP– Rexmit sets of DPs– Change priorities– Change transmit path



Specific Objectives






• Fine deletion control– Delete individual DP

– Delete sets of DPs



Development History

• MPF Data Architecture:Store and Transmit Packets– Event Reporting

Service (EVR)– Channelized Telemetry

Service (EHA)– Science and all other

Engineering Data generated as raw packets

– CCSDS Packet Telemetry Service (DWN)· Packet priority based on APID

· Packet storage in RAM· No other persistent storage

· Assumes no shutdowns (shutdowns result in loss of all data in RAM)



Development History

• MER Data Architecture:

Data Products = file+metadataTelemetry is Packets– Data Products stored

in persistent memory (FLASH, not RAM)

– MPF EVR and EHA services retained, but now produce Data Products

– Science and all other Engineering Data generated as specific Data Products

– CCSDS Packet Telemetry Service· Products turned to Packets “Just In Time” in product priority order



Key System Components

• Flight Software– MRF : Massively multi-threaded Real-time File system

· Client API, similar to POSIX fopen(), fwrite(), and fclose() calls

– FME : File Metadata Engine· Supports operations on sets of data products, individual data products, and portions of individual data products

· Provides prioritized list of data products to PDP for packetization

– PDP : Packetized Data Products· Converts Data Products to Packets during communications sessions

• Ground Software– MDP : MER Data Product Build

· Reconstitutes Data Products from received packets

· Accounts for missing packets

– MDPV : MER Data Product View· Allows simple viewing of most Data Products



Data Management Surface Process

• Current Surface Process performs the following tasks once each uplink cycle (1 per Rover per Sol)– Retransmit un-received, partially received, and corrupt data products· Typically results in tens of commands

– Delete all other data products that have been sent· Typically results in hundreds of DPs deleted per command

– Delete unsent data products no longer considered to be of value· Typical results in a half dozen commands deleting hundreds of DPs per sol

– Reprioritize, retransmit, and/or delete data products per requests



Data Management Surface Process Details

1. Identify all partially and completely received Data Products since the last cycle completed– Includes all data products received from all relay assets

and direct to earth links

2. Generate individual Retransmit Commands for each identified missing or partially received Data Product– Applies configurable heuristics:

· To not retransmit certain Data Products· To change priority and/or transmission path of certain Data

Products

3. Perform automatic corruption checking– Generate individual Retransmit Commands for corrupted

parts – Update original corrupt data when the retransmitted data

is received

4. Generate “Group” Delete Commands for each communications pass



Data Management Surface Process Details (contd)

5. Generate “Group” Delete Commands for unsent data products that meet configurable age and size parameters– Small PMA and HGA log files (no motion), old images,

…

6. Generate single and “Group” Reprioritization Commands for classes of Data Products that meet configurable age and size parameters– Increase priority of biggest PMA and HGA log files

(during movement)

7. Accommodate highly custom requests– Delete sets of data products no longer needed or

wanted– Reprioritize data products from specific sequences,

time ranges, or other criteria



Conclusions

• Use of standard stdio-like API greatly simplified flight and ground software for science instruments and engineering subsystems

• On-board metadata object base enabled operations on sets of data products– Effective use of downlink bandwidth across multiple types of links

– Dramatically reduced uplink bandwidth needs

• Designed for “mission-operations-in-the-loop” from the beginning– System is highly automated while still able to respond to unforeseen pitfalls and opportunities

• Well suited for future autonomous spacecraft.• “Version 2” being applied to Mars Science

Laboratory



Conclusion: Key Capabilities of MER DP

• MER DP is multi-route end-to-end• MER DP has predictable and unique-over-life-of-

mission DP IDs• MER DP transaction state is persistent and

auditable• MER DP obeys FLP:

– MER DP can use protocol level “immediate” data (packet headers) for non-destructive actions (recognizing when a DP arrives, detecting holes in DPs that need re-transmission)

– MER DP uses an auditable “synchronous” artifact (DPSR) to make destructive actions (delete



What is FLP?

• “Impossibility of distributed consensus with one faulty process”– Journal of the ACM, Volume 32, Issue 2 (April 1985)– Pages: 374 - 382, ISSN:0004-5411– by Michael J Fisher, Yale,– Nancy A. Lynch, MIT,– and Michael S. Paterson, University of Coventry.– In this paper, it is shown that every protocol for this problem has the possibility of non-termination, even with only one faulty process. By way of contrast, solutions are known for the synchronous case, the Byzantine Generals problem.

• ACM Symposium on Principles of Distributed Computing Influential Paper Award: 2001– The PODC Influential Paper Award was created to acknowledge an

outstanding paper on the principles of distributed computing, whose significance and impact on the theory and/or practice of distributed computing has been evident for at least a decade.

data management for mars exploration rovers

Documents