breakout session: how to test flight software

10
2007 Workshop on Spacecraft Flight Software (FSW-07) November 5-6, 2007

Upload: softwarecentral

Post on 11-May-2015

2.994 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Breakout Session: How to Test Flight Software

2007 Workshop on Spacecraft Flight Software

(FSW-07)November 5-6, 2007

Page 2: Breakout Session: How to Test Flight Software

2

Standard Flight Software Architectures

Various agencies and organizations have been seeking a standard for flight software. Standards would enable software to be reused across organizations and would reduce software costs from mission to mission. However, where should standardization occur? At the hardware level? The OS (if so, which OS? And does Linux have a place?)? At a messaging layer? Is it even possible to standardize software? One of the goals of this breakout session is answer some of these questions and to also judge the feasibility of applying a standard to organizations that already have a large amount invested in their current architectures.

Page 3: Breakout Session: How to Test Flight Software

3

Preventing and Responding to Flight Software Errors

All developers have war stories of how they had a bad index in an array or made a particular programming error. With flight software, that can be a significant event. This session delves into some of the gotchas in flight software and what can be done when they aren’t caught before launch. For instance, what do we need to focus when developing code or conducting code walkthroughs? And once on-orbit, there are questions of how to handle updates or anomalies. Are there benefits to patching code as opposed to doing full builds? Should processor resets be avoided at all costs or are they acceptable in some cases?

Page 4: Breakout Session: How to Test Flight Software

4

How to Test Flight Software

Flight software testing is a key element in the development process. This session focuses on defining the best techniques for delivering solid code. Potential discussion areas include: how should stress testing be performed? How should code from other organizations or auto-generated code be tested? What is the developers role in I&T and FSW testing? With limited resources and testing seemingly always pushed to the end of the process, what provides the best bang for the buck?

Page 5: Breakout Session: How to Test Flight Software

5

Stress testing Is . . .

Test SW until it breaks (although customers don’t require this – but we think they probably should)

Spacecraft – ensure it degrades gracefully Instrument – know when to do a reset Understand what will break it (and write operational

procedures to preclude Ops breaking it) What is stress?

Use as much of the processor, I/O, TLM as possible Stress using all possible loads (worst case scenarios)

– then go beyond Stress definition evolves as you proceed through

development

Page 6: Breakout Session: How to Test Flight Software

6

How Should Stress Testing Be Done?

Build a worst-on-worst test case Capture each “oops” in the lab/test bed to add to the

suite of stress tests

Page 7: Breakout Session: How to Test Flight Software

7

How Do We Test Autocode?

Black Box Test against requirements, in such a way as to ensure

the software is robust Test for critical timing

White/Clear Box So many paths through autogenerated code that

complete path testing may not be practical Issue with combination of GFE/reused code combined

with autogenerated code Mostly rely on testing at the model level (especially for

algorithms and functionality) For “dead code” from OTS, reuse or autogenerated, just

use hand analysis to assure it is benign

Page 8: Breakout Session: How to Test Flight Software

8

Role of Developers in Testing

Testers participate in design reviews Developers do subsystem testing Independent testers for qualification testing Challenge folks to “break” the software

Ability to both “break” the software/system AND give this info as a “gift” to the developers that they cherish

Page 9: Breakout Session: How to Test Flight Software

9

Biggest “Bang for the Buck” in Testing

Make sure all requirements are tested Use model-based requirements testing wherever possible

Desktop software simulation/testbed Robust testbed with flight-like hardware Trade-off between making fixes quickly and doing lots of

regression testing Do system testing and stress testing on mature version of the

software Robust regression testing Patch testing (like you fly) Day-in-the-life testing Week (or month)-in-the-life characterization Get all changes in at least 1 release before the final release

Page 10: Breakout Session: How to Test Flight Software

10

Autonomous Flight Software

Modern spacecraft are becoming increasingly reliant upon autonomous systems for operations. This includes command and control as well as data processing and fault protection. However, a price is paid in computing costs with this technology and some argue that it hasn’t been needed in the past, so why do we need it now. Questions for to answer for this session can include: What functionality is appropriate for flight and for the ground? How does fault protection differ between mission objectives? Is a “black box” a solution for spacecraft fault protection? What is the impact of more onboard processing?