flight software through the looking glass
DESCRIPTION
Flight Software Through the Looking Glass. Scott Burleigh Jet Propulsion Laboratory California Institute of Technology 12 Dec ember 2013. - PowerPoint PPT PresentationTRANSCRIPT
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
Flight SoftwareThrough the Looking Glass
Scott BurleighJet Propulsion Laboratory
California Institute of Technology
12 December 2013
This research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. (c) 2013 California Institute of Technology. Government sponsorship acknowledged.
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
2
Flight Software is TWO systems
• We all know that real-time flight software is indispensable to spacecraft operations. It drives the hardware of a robotic vehicle and its instruments, and it has been present since deployment of the first flight computer.
• As missions become more capable, a second system emerges. It’s a non-real-time system that manages data.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
3
The second system
• The software we usually think of when we say “flight software” is the real-time, safety-critical, interrupt-driven foreground system.
• The other part of “flight software” – which we typically don’t recognize as being different in nature – is the non-real-time, discretionary, time-sharing background system.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
4
Where it lives
• Scheduling theory tells us that the real-time tasks’ time allocations should add up to no more than about 60% of CPU time, to ensure they never miss deadlines.
• The remaining 40% of CPU – “idle time” – isn’t necessarily idle. It’s time that is available for interruptible tasks, the second system.
• That second system should include everything that is not truly real-time.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
5
Second system overview
• All tasks run at the same priority, the lowest priority supported by the O/S.– Possibly multiple background subsystems.– To the real-time FSW they all look like “idle”.
• Tasks have no deadlines.– Locking one another out for prolonged periods is okay, so
long as everybody gets a chance to run eventually.– Each task must release the CPU on finishing a unit of work,
usually blocks on something.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
6
A Practical Example
• The Flight Software architecture of the Curiosity rover is clearly an excellent approach to the design of a FSW real-time system (but other good approaches are possible).
• Postulate: the Interplanetary Overlay Network (ION) architecture, an implementation of Delay-Tolerant Networking, is a good approach to the design of a FSW non-real-time system (but again other good approaches are possible).
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
7
ION Design Overview
Application
Object database
Routing
Sender Receiver
issue deliver
transmit receive dispatch
bundles for delivery
bundles to be forwarded
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
8
• Use shared memory.– Often there’s no protected memory, so we have no option.– But this can be turned to advantage: shared memory is a highly efficient way to pass
data between flight software tasks.
ION Design Principles
sender receiver
• Zero-copy procedures: leverage shared memory to minimize processing overhead.– Encapsulation in layers of protocol overhead (headers and trailers) can be done
by reference rather than by copy.– The same data object can be shared by multiple tasks, provided reference
counting prevents premature deletion.• Portability: this is an unfamiliar programming model, so we must make it
easy to develop in an environment with good programming support (e.g., Linux) and then deploy – without change – in the target RTOS environment.
S
linked list
give take
enqueue dequeue
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
9
Through the looking glass
• The foreground and background systems of a spacecraft’s FSW share many qualities that distinguish them – both – from workstation or PC software.
• But in some ways the character of the background system is the exact inverse of the character of the real-time FSW.
• And the two can coexist in perfect compatibility. It’s been demonstrated.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
10
What they have in common…
• …is most of the flight project cultural values:– Project schedule can’t be jeopardized.– Module coupling must be minimized.– No dynamic system memory allocation.– Make maximum use of available resources.– Contain and, as possible, tolerate faults.– Test as you’ll fly, fly as you tested.– Formal, controlled development process.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
11
Where they differ
• What is optimized– Determinism– Portability
• Mutual exclusion• Data sharing• Message passing• Memory management
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
12
What is optimized
• The foreground system must be reliable, else you lose the spacecraft.– Fixed scope enables minimal, mission-specific design
which enables comprehensive testing.• The background system must be efficient, else not
enough work gets done.– Minimize wasted space and cycles.– This adds complexity, so it’s important to support
portability: reliability comes from extensive multi-mission testing history.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
13
Mutual exclusion
• Real-time FSW can’t tolerate lengthy mutual exclusions: tasks miss deadlines.
• In the background system, no problem. You can serialize an entire subsystem on a single mutex, because there are no deadlines.– Only one task runs at a time anyway. (For now; multi-core
flight computers are on the horizon.)– Loops and function calls in critical sections are okay, so
long as they eventually terminate and let other tasks run.– Minimize cycles spent on task switching.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
14
Data sharing
• Data sharing requires mutual exclusion, so it’s no good in the foreground system.
• In the background system, lengthy mutual exclusion is okay – so shared access to data is okay.
• Which is good, because shared access is also the fastest way multiple tasks can operate on the same data.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
15
Message passing
• Because shared access is excluded, the real-time system uses message passing to enable multiple tasks to operate on common data.
• But because shared access is okay in the background system, message passing is not needed.– No cycles wasted in copying anything.– Signal “data ready” by giving a semaphore.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
16
Memory management
• Because mission scope is fixed and is known at launch time, all foreground FSW memory can be in fixed-length arrays, each with margin.
• But the background system gains reliability from being multi-mission, hence portable and evolvable.– Management of private common heap: pooled
resource, pooled margin, efficient use of space.– Automatically adapts to mission scope change.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
17
How they collaborate
• The foreground system never calls the background system’s library functions – never blocks, ignores background tasks.
• Background tasks are interrupted whenever foreground tasks need to run.
• For communication between the two: operating system (e.g., VxWorks) message queues.– Non-blocking at the foreground end.– Blocking at the background end.
12 December 2013
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
18
cfdp_manager
pxlso
pxlsi
PX
LTP
LTP segs
S
S
PxVdb
libfdput_px.so
To telecomsystem and earth
OutFPDU
InFPDU
S
S
pxout
pxin
PxDb
Outbound FPDUs
Inbound FPDUs
How we know this works
12 December 2013
• The Deep Impact Network (DINET) project validated ION in flight, on the EPOXI (formerly Deep Impact flyby) spacecraft in October-November 2008. The ION non-real-time data management system was integrated with DI’s real-time flight system, written by Ball Aerospace. No modification to either the Ball flight software or the ION multi-mission software was required; a small “link service adapter” system, named “PX”, provided the interface between the foreground and background systems.
• The ION software enabled EPOXI to function as a DTN router in deep space, about 10 million miles from Earth, for four weeks while the spacecraft was in cruise to comet Hartley 2. About 14.5 MB of data (292 images) were routed through the spacecraft over 8 low-rate DSN tracking passes, without data loss and without software failure.
National Aeronautics and Space AdministrationJet Propulsion Laboratory, California Institute of Technology
19
Thanks!
12 December 2013