framework for risk reduction in gas turbine product
TRANSCRIPT
Framework for Risk Reduction in Gas Turbine Product Development
by
Jonathan K. Niemeyer
M.S. Mechanical Engineering, Rensselaer Polytechnic Institute, 1999B.E. Engineering Sciences, Dartmouth College, 1992A.B. Engineering Sciences, Dartmouth College, 1991
Submitted to the System Design & Management Programin Partial Fulfillment of the Requirements for the Degree of
Master of Science in Engineering and Managementat the
Massachusetts Institute of Technology
February 2002
0 2002 Jonathan K. Niemeyer. All rights reserved.
The author hereby grants to MIT permission to reproduce and to distribute publicly paper and electroniccopies of this thesis document in whole or in part.
Signature of AuthorJonauiai Ph. NIFMycl
System Design and Management ProgramFebruary 2002
Certified byDr. Dai Whitney
Senior Research ScientistCenter for Technology, Policy, and Industrial Development
Thesis Supervisor
Accepted bySteven D. Eppinger
Co-Director, LFM/SDMGM LFM Professor of Management Science and Engineering Systems
Accepted by
MASSACH SETTS INSTITUTE Paul A. LagaceOFT TEHLOGY LFM/SDM Co-Director
Professor of Aeronautics & Astronautics and Engineering Systems
LIBRARIES
[This page is intentionally left blank.]
2
Framework for Risk Reduction in Gas Turbine Product Development
by
Jonathan K. Niemeyer
Submitted to the System Design & Management Programon January 11, 2002 in Partial Fulfillment of the
Requirements for the Degree of Master of Science inEngineering and Management
ABSTRACT
This work looks at the product development process as an exercise in risk reduction and performsa critical analysis of how gas turbine engine manufacturers weigh the competing risks associatedwith on-time delivery, product quality, and development costs. Risk focuses on decision pointswhen future outcomes are still uncertain, and is defined as:
Risk = (probability of failure) x (severity of failure)
Three frameworks are used to focus the analysis:
* Iteration by using multiple attempts to converge to an acceptable solution.* Maintaining options in development, and delaying convergence to a single design.* Improving the organization's predictive capability prior to committing to a particular set of
performance goals, designs, or technologies for a product. This is explored from theperspective of "technology readiness."
For six gas turbine engine development programs, case studies were performed to assess theeffectiveness of the product development process by measuring how well the engine met itsguaranteed level of fuel consumption. For each development program, performance againstguarantees was compared against technology readiness levels (TRL) at program initiation andagainst the degree of flexibility provided to designers to react. Decomposition of the enginesystem into sub-systems was necessary to specifically define TRL, parallel efforts, and iteration.Risk strategies were compared in light of the time sensitivity of the quality of information, thecost of engineering changes, contractual penalties, and lead times associated with implementingimprovements.
Recommendations are provided for future engine programs based on past successes and failures.The decisions to begin engine programs with a given level of technology readiness, or to planparallel design efforts, or even to rely solely on iteration, are strategic choices. This workattempts to provide a framework for theseidecisions.
Thesis Supervisor: Daniel WhitneyTitle: Senior Research Scientist
Center for Technology, Policy, and Industrial Development
3
ACKNOWLEDGEMENTS
I would like to thank my thesis supervisor, Dr. Daniel Whitney, for his guidance in thepursuit of this thesis. His experience, insight, and persistent questioning both expandedmy perspective and focused my efforts. Our weekly meetings helped to make my termon-campus a rich experience.
I am thankful for the generous support I have received from Pratt & Whitney throughoutthe duration of the SDM program. In particular,
Merrill Kratz, William Beyerly, David Haas, Andrea Borondy-Kitts, Robert Saia,Rachel Rosenfeld, and David Crow for supporting my application to this program.
Thomas Auxier, Stanley Balamucki, Glenn Bartkowski, William Beyerly, Craig Bolt,Richard Carlton, Ray Carmichael, David Carter, Peter Chenard, Russell Grace, KarlHasel, Joseph Latour, Charles Lejambre, Lewis Mackechnie, Shankar Magge, JackMosley, Kurt Noe, Jeffrey Pearson, Thomas Pelland, Thomas Rogers, Gary Stetson,Jean Wright, and Larry Zeidner for their help and input into this thesis.
Finally, thank you to my wife, Missy, for patiently supporting me for the last two years. Ilook forward to finally fixing the pipes that froze two winters ago...
4
TABLE OF CONTENTSACKNOW LEDGEM ENTS ........................................................................................... 4TABLE OF CONTENTS ................................................................................................ 5LIST OF FIGURES ........................................................................................................ 71. INTRODUCTION ................................................................................................ 8
1.1 Problem Statement and M otivation .................................................................... 81.1.1 Engine Performance S-Curve ................................................................. 81.1.2 M otivating Examples.............................................................................. 10
1.2 Definition of Risk ......................................................................................... 141 .3 S co p e ................................................................................................................. 161.4 Framework for Analysis ............................................................................... 201.5 Thesis Overview ........................................................................................... 23
2 GAS TURBINE PRODUCT BACKGROUND .................................................... 252.1 M odularity and Integrality ........................................................................... 252.2 Gas Turbine Engine Fundamentals................................................................ 28
2.2.1 TSFC: The Key M easure of Engine Performance ................................ 292.2.2 Compression System.............................................................................. 322.2.3 Combustion........................................................................................... 352.2.4 Expansion (Turbines)............................................................................ 362.2.5 Section Summary .................................................................................. 37
2.3 Gas Turbine Product Development Process ................................................. 382.3.1 Pratt & Whitney Product Development Process................. 382.3.2 Pratt & W hitney Gated Decision Process ................................................. 442.3.3 General Electric Gated Decision Process ............................................. 47
2.4 Chapter Summary ......................................................................................... 483 RELATED W ORK ................................................................................................ 49
3.1 NASA and Technology Readiness................................................................ 493.2 Value of Options ........................................................................................... 533.3 Toyota and Set Based Design ...................................................................... 543.4 Cost of Rework ............................................................................................. 563.5 Iteration to Achieve Convergence on Requirements .................................... 57
3 .5 .1 Iteration ................................................................................................. . 573.5.2 T esting .................................................................................................. . 58
3.6 Chapter Summary ......................................................................................... 614 PRATT & W HITNEY CASE STUDY.................................................................. 62
4.1 Technology Readiness .................................................................................. 624.1.1 TSFC = f(Component Performance)..................................................... 644.1.2 Technology Readiness leads to Simulation Readiness .......................... 644.1.3 Increasing the TRL/SRL for New Technologies ................. 694.1.4 'Derivative' Technologies .................................................................... 714.1.5 Interview Process to Define TRL/SRL .................................................. 734.1.6 Aggregating Component TRL's to Engine TRL ................................... 74
4.2 Design Flexibility & Iteration....................................................................... 784.3 Chapter Summary ......................................................................................... 83
5 CONCLUSIONS and FOLLOW -ON ACTIVITY ............................................... 855.1 Applicability of Case Study........................................................................... 85
5
5.1.1 Severity of Failure ..................................................................................... 855.1.2 TRL - Subjective M etric ........................................................................... 86
5.2 Follow -On W ork ............................................................................................... 87GLO SSORY ..................................................................................................................... 89BIBLIOGRAPHY ............................................................................................................. 90
6
LIST OF FIGURESFigure 1-1. Commercial aircraft propulsion S-curve from 1900 to present. [Adapted from
Mattingly (1996).] Overall efficiency is inversely proportional to fuel consumption(T S F C ). ....................................................................................................................... 9
Figure 1-2. Rate of TSFC improvement over time for the three major enginemanufacturers: Pratt & Whitney, General Electric, and Rolls Royce. [Anonymous].................................................................................................................................... 1 0
Figure 1-3. Pratt & Whitney Product Development Process (PDP). Note that the scaledoes not reflect duration......................................................................................... 12
Figure 1-4. Comparison of TSFC gap when engine enters service (EIS) to gap after firstfull engine test (FE T T)........................................................................................... 13
Figure 1-5. Probability of failure as a function of mean (pt) and variation (a)............. 16Figure 1-6. Thesis focuses on decisions made at (1), and validated at (2) and (3)..... 18Figure 1-7. Requirement dependencies after engine concept defined. [Adapted from Moy
(2 0 0 0 ).]...................................................................................................................... 19Figure 1-8. Decision tree for product development process........................................ 21Figure 2-1. Cut-away view of a high bypass turbofan engine. [http://www.pratt-
whitney.com/engines/gallery/g.pw4000.94cut.html]........................................... 26Figure 2-2. Brayton thermodynamic cycle for gas turbine engines............................... 32Figure 2-3. FAA Testing Requirem ents......................................................................... 43Figure 2-4. Decision Gates in PDP (Passport review numbers identified)................... 44Figure 2-5. General Electric Tollgate Process. [Adapted from Wheelwright (1992).]..... 47Figure 3-1. NASA Technology Readiness Levels........................................................ 50Figure 3-2. TRL, variation, and m ean........................................................................... 52Figure 3-3. Example of benefit of real options............................................................. 54Figure 3-4. Convergent design process used by Toyota. [Adapted from Wheelwright
(19 9 2 ).]...................................................................................................................... 5 6Figure 4-1. TRL, variation, and commitment............................................................... 64Figure 4-2. Technology and Simulation Readiness Levels. ......................................... 66Figure 4-3. Iterative Improvement of TRL/SRL for new HPC's or derivative designs... 68Figure 4-4. Illustration of the use of a derivative technology....................................... 72Figure 4-5. FETT TSFC gap as a function of TRL....................................................... 77Figure 4-6. Flexibility for iterations on engine component design................................ 80Figure 4-7. TSFC Improvement versus Iteration Rigidity............................................. 82Figure 4-8. Sum m ary of results. .................................................................................. 84
7
1. INTRODUCTION
1.1 Problem Statement and Motivation
Effective product development requires organizations to balance the often-conflicting
demands of cost, schedule, and quality. NASA describes this goal as "faster-better-
cheaper." In the aircraft propulsion industry, although the jet engine represents a mature
technology, intense market pressures have driven the demand for continuous incremental
improvements to engine performance. In 1998, fuel represented 12% of an average
airline's operating costs. This fraction was as high as 28% during the fuel crisis of the
early 1980's.1 In this environment, engine manufacturers are competing to provide the
airlines the most fuel-efficient engines, the fastest, while incurring the lowest
development costs.
1.1.1 Engine Performance S-Curve
At the end of World War II, the gas turbine engine emerged as the 'dominant design'
for aircraft propulsion as a result of innovative work by Frank Whittle in the United
Kingdom and Hans von Ohain in Germany. Utterback defines a dominant design as "one
that wins the allegiance of the marketplace, [and] one that competitors and innovators
must adhere to if they hope to command significant market following." 2 Gas turbines
represented a technology discontinuity relative to the existing piston-powered engines
and were capable of greatly improved thrust-to-weight ratios and thrust specific fuel
1 Greenslet, Ed. "World Airlines: Year in Review." Interavia, Vol 54, Issue 632 (June 1999), 42-45.2 Utterback, James M. Mastering the Dynamics of Innovation. Boston: Harvard Business School Press,
1994.
8
consumption (TSFC). Figure 1-1 below, summarizes the rate of engine fuel consumption
improvement over time.
) 50%- EMERGENCE OFDOMINANT DESIGN
40%-
- 30%-
S20%- Propeller / Piston Engines
'Discontinuous' innovations at sub-system level
10%- drive 'incremental' improvements to fuelconsumption at system level (TSFC)
1900 1945 2000
Figure 1-1. Commercial aircraft propulsion S-curve from 1900 to present. [Adapted fromMattingly (1996).] Overall efficiency is inversely proportional to fuel consumption(TSFC).
Today, the key architectural elements of the gas turbine engine remain largely the same
as in the late 1940's. Within the architectural framework of the jet engine,
'discontinuous' innovations to engine sub-systems have fostered incremental
improvements to engine fuel consumption. Since the 1960's, TSFC has improved at a
rate of 1% per year improvement. See Figure 1-2 below.
9
JT3O-1
4 JT8D 17* JT8D-15A
JT8D 9 JT8D- 17A
C CJ805
JT8D2 19 A TAY 650
A TAY 620JT9D 7A JT9D59 A
JT9D. 3A J 9D7Q N FM56- 3C
L) J9D-70A *RB 211-535C CFM56-5B1 CFK6-7B26LL UCF6-S06JC R&21 1-22B1
CFO- 6D CFM56-2 A CFM56-5AJT9D7R4 A FO 215240 \25 00 A5 T BR715
J9D-7R4C 0 A 11 A 5 3 6-80C2-BEF TN768CF6-80A -2155 *P144062
FW ie-211J CZ -+ 0 043
RN12037 PVV 406 0 - 880EI -A 2PN 2040 P06A098
2 W 4056 C FM 5FW 44 A
W4084 TRENT895
GE90-858 09 0-908
perye ar
1950 1960 1970 1983 1990 2000 2010 2020
Certification Date
Figure 1-2. Rate of TSFC improvement over time for the three major enginemanufacturers: Pratt & Whitney, General Electric, and Rolls Royce. [Anonymous].
Within the context of these schedule and quality (TSFC) pressures, engine manufacturers
are competing to 'beat' the curve.
1.1.2 Motivating Examples
Beating the curve, or even matching it, is no easy task; program failures, quality
shortfalls, and cost overruns seem to be commonplace. The following two examples
illustrate the types of problems that provide the motivation for this work.
GE90 Turbofan Engine
In the mid- 1990's General Electric was developing the GE90 turbofan engine to
power the Boeing 777. Relative to the original four-year development budget of $1.5
billion, Forbes reported that development costs had grown to at least $1.8 billion and
10
perhaps more than $2 billion. Cost overruns, in part, could be attributed to stretching the
technology envelope without appreciating the risks. Traditionally, GE reduced the risk
and cost of new commercial engine programs by reusing technologies developed in
military engines. The GE90 did not follow this strategy and was "pushing the envelope",
running to higher overall pressures and temperatures than competitors' engines.
Furthermore, GE was experiencing unanticipated problems with incorporation of
composite fan blades. "Such technology bets.. .often take longer and soak up more
investment before they pay off. That has been the case with the GE90." 3 Fixing problems
associated with the fan blades had negative cost and schedule impacts, delaying Federal
Aviation Administration (FAA) development testing and requiring technical fixes that
hurt fuel consumption. Forbes reported that "even if nothing else goes wrong at all, the
GE90 will be six months behind schedule getting its permit for extended over-water
flying, creating yet more costly penalties. If there are more problems, watch for Welch
[former GE CEO] to lop heads at his engine division." 4
Pratt & Whitney Gas Turbine Development
When an engine manufacturer commits to the development of a new engine, an
agreement is made that specifies the performance of the product that will be delivered to
the airframer and the airlines. This specification includes parameters such as engine
thrust, noise, emissions, weight, and TSFC. During the product development process,
engine tests are performed as a means to improve product quality as well as to satisfy
regulatory requirements. Figure 1-3 below illustrates Pratt & Whitney's product
3 Banks, Howard, "Engine Trouble", Forbes, New York; September 11, 1995; Vol. 156, Iss. 6; pg 156.4 Banks, Howard, "Hit the Fan", Forbes, New York; January 1, 1996, Vol 157, Iss. 1, pg 14.
11
development process (PDP). Typical segment durations are provided. In light of the
market demand for 1% TSFC improvement per year, there is pressure to compress this
development cycle. Note that the commitment to the customer takes place at the
beginning of Product Definition and precedes the execution of the first, full-engine
system test. Engines often remain in revenue service for longer than 30 years before
being retired.
PRODUCT PLANNING PRODUCT DEFINITION PRODUCT VALIDATION PRODUCT DELIVERY
COMMITMENT: ENGINE TESTING ENTRY INTO SERVICE"LAUNCH" PROGRAM
ON-GOING 1 /2 years -- 2 /2 years -- 4+--- > 30 years --
Figure 1-3. Pratt & Whitney Product Development Process (PDP). Note that the scaledoes not reflect duration.
For six historical engine development programs at Pratt & Whitney, Figure 1-4 below
compares the fuel consumption demonstrated by the first engine to test (FETT) with that
of the completed engine at entry into service (EIS). Fuel consumption is expressed as a
normalized delta relative to the level promised in the engine specification (SPEC). A
value of 1.0 reflects the maximum TSFC miss relative to requirement and a value of 0.0
indicates that the engine meets requirement.
12
PRATT & WHITNEY GAS TURBINE ENGINE DEVELOPMENTPROGRAMS
NO IMPROVEMENTAFTER FETT
EI~
7-Ogap = 0.46 (FETT gap)
R2= 0.90
0.0 0.2 0.4 0.6
TSFC gap [FETT - SPEC], normalized
0.8 1.0
Figure 1-4. Comparison of TSFC gap when engine enters service (EIS) to gap after firstfull engine test (FETT).
A linear least squares regression fit shows the TSFC gap at EIS to be on average 54%
of the gap demonstrated at FETT. Several engine programs fall near the solid line,
indicating that the testing and validation phase of the product development process had
difficulty in correcting any of the performance shortfalls that were identified at FETT.
Both General Electric and Pratt & Whitney have decades of experience in the
development of aircraft propulsion systems. These examples highlight the potential
13
1.0
0.8
0.6
0.4
0.2 -
N
0
H
0.0
4
+ 0.05
Zi -
i
0
impact of uncertainty on the ability to deliver promised engine performance on time and
within budget.
Engine manufacturers are directly responsible for the performance of the engines they
deliver. Before the 1980's, airframers and engine makers shared the performance risks
associated with a new airframe-engine combination. Range and fuel bum guarantees were
made to the airlines based on a predicted aircraft drag and engine performance. There
were no penalties, for example, if a shortfall in engine performance was offset by an
aircraft exhibiting less than expected drag. This luxury was removed with the advent of
the Ortega Principle, named after an Airbus Industrie manager.5 Engine makers became
financially accountable for their performance. This means that if an engine fails to deliver
the contractually promised TSFC, the engine maker pays penalty fees to the airframer.
The airframers typically pass these on to the airlines as part of their contractual range
guarantees.
1.2 Definition of Risk
As mentioned above, engine manufacturers are attempting to 'beat the TSFC curve'
of 1% improvement per year. One strategy is to compress the existing four-year product
development cycle while delivering the same TSFC improvement. A second strategy is to
accelerate the TSFC improvement for a fixed four-year cycle. Either strategy has the
potential to increase the risk that engine makers will fail to deliver the promised level of
TSFC on the promised date.
14
5 Personal interview.
This thesis looks at the product development process as an exercise in risk reduction,
and seeks to offer explanations for the pattern reflected by the engine programs in Figure
1-4. Risk focuses on decision points when future outcomes are still uncertain, and is
defined as 6
Risk = (probability offailure) x (severity offailure)
Examples from everyday life can help to illustrate the dependence of risk on both
probability and severity. The decision to purchase a $1 lottery ticket has low risk. The
high probability of failure is offset by the minimal impact on the buyer ($1). Likewise,
being struck by lightning on a sunny day has low risk. The severe consequence of being
struck is offset by the extremely low probability of occurrence.
This thesis seeks to develop an understanding of the factors that influence the
probability of failure and derives its approach from Taguchi's methods for process
control; in order to control a parameter (and avoid failure), the designer needs to
understand the separate factors that affect its mean (pt) and the variation (G).7
Analogously, mean and variation are often expressed as signal and noise. Understanding
how to control the signal and the noise are separate tasks, but both affect the outcome.
See Figure 1-5 below.
6 Browning, Tyson R. "Modeling and Analyzing Cost, Schedule, and Performance in Complex SystemProduct Development." PhD Thesis, Massachusetts Institute of Technology, 1999. Also, Zeidner,Lawrence, personal communication, May 2001.7 Whitney, Daniel, personal communication, December 2001.
15
REQUIREMENT
I -. I
I II I
I II II I
Figure 1-5. Probability of failure as a function of mean (pt) and variation (G)
This Figure illustrates a possible evolution of a parameter's mean (pt) and variation
over time (G) using a series of probability density functions. The mean of the initial
distribution is the same as the requirement. There is a 50% chance of failure and of
success. The ensuing density functions illustrate both a mean shift and a reduction in
variation. The final distribution function shows a parameter that is well understood (low
Y) but fails to meet the requirement.
It follows that a discussion of risk can be segmented into understanding the causes of
mean shift and the causes of variation.
1.3 Scope
From the perspective of product development, we are concerned with the key
decision points that have the greatest impact on the probability of success for the engine
16
program meeting its requirements. Success for a typical engine program is expressed in
terms of the following functional requirement categories -
* Development costs, which are non-recurring and 'internal' to Pratt & Whitney.
* Manufacturing costs, which are recurring and 'internal' to Pratt & Whitney.
* Normal maintenance costs based on refurbishment interval. That is, how fast does
the engine deteriorate? For this requirement and the ones that follow, Pratt &
Whitney can be responsible for a portion of the costs when engines fail to meet
requirements.
* Reliability, which is often indicated by the metrics of In-flight Shutdown Rate
(IFSD) and shop visit rate (SVR).
" Engine weight, which is critical to proper design of the interface with the aircraft
and to the maximum allowable aircraft payload.
* Noise. Many airports and communities around airports impose restrictions.
* Emissions.
* On-time completion of development. Implicit in this requirement is receiving
certification from regulatory authorities: Federal Aviation Administration (US)
and Joint Airworthiness Administration (Europe).
* Engine performance. [Thrust specific fuel consumption].
These requirements are interdependent, and their relationships with one another
change during each phase of the product development process.9 For example, during the
product-planning phase of development, designers need to select a diameter for the
engine, which has a strong impact on manufacturing costs, engine weight, noise, and
engine performance. Once an engine concept (including the diameter) is selected, weight
Moy, Habs M. "Commercial Gas Turbine Platform Strategy and Design." SM Thesis, MassachusettsInstitute of Technology, 2000.9 Hague, Douglas C. "Description of a Turbofan Engine Product Development Process." SM Thesis,Massachusetts Institute of Technology, 2001.
17
and performance become less interdependent. That is, there are far fewer decisions that
impact both parameters.
This thesis is concerned only with the key decisions made at the end of the product
planning phase that impact how effectively the PDP allows Pratt & Whitney to meet its
requirements for TSFC within a fixed schedule while minimizing development costs.
Referring to Figure 1-6, this thesis assesses the effectiveness of decisions made at (1) by
measuring TSFC at the first engine test (2) and at the end of the validation phase (3). A
more detailed description of these three points follows:
1. Engine is launched based on an expected value of TSFC (p).' 0 As the engine hasnot been built, a simulation is used to predict this level of TSFC, which has avariation (o) based on the quality of information used in the prediction. Acontractual commitment is made based on this prediction.
2. The first full engine test provides a measurement of TSFC. At this point, thevariation (u) is approximately = 0 (we have data now). The mean (p), however,may not meet expectation. What actions are now possible during the validationphase to drive the mean closer to the original commitment?
3. Engine is delivered to customer. Failure to meet required level of TSFC results inpenalty fees.
0 0PRODUCT PLANNING PRODUCT DEFINITION PRODUCT VALIDATION PRODUCT DELIVERY
1 PROUCT ALIDTION & SUPPORT
COMMITMENT:"LAUNCH" PROGRAM
ON-GOING -
ENGINE TESTING ENTRY INTO SERVICE
/2 years --- 4---- 2 /2 years > 30 years -*
Figure 1-6. Thesis focuses on decisions made at (1), and validated at (2) and (3).
1 Launch refers to when the engine maker conmirnits to delivering the product. Note that the engine is notyet available. This is different than the more conventional NASA notion of launching a spacecraft intoouter space.
18
Given that this analysis is focusing only on TSFC after engine concept selection, it is
important to understand any interdependencies between TSFC and requirements omitted
from this analysis. The dependencies illustrated in Figure 1-7 are appropriate for
development phases after product planning, once the engine concept is defined. An 'X'
represents a strong coupling. The absence of a mark indicates a weak coupling. The
matrix is symmetric, so marks are only included below the diagonal.
ri~0
0a)
a)
0
0
a)
M)a)
Cl)
0
0
0
Ha)0
a)
Development CostsManufacturing CostsMaintenance CostsReliabilityEngine weightNoiseEmissionsOn-time completionPerformance (TSFC)
XX XX XX X XX X X XX X XX XX X X
Figure 1-7. Requirement dependencies after engine concept defined. [Adapted from Moy(2000).]
19
Reading across the bottom row in Figure 1-7, we can see that after an engine concept
has been selected, decisions regarding the risk of delivering engine performance (TSFC)
are strongly coupled with engine development costs, manufacturing costs, maintenance
costs, and risk of on-time delivery. Decisions aimed at meeting the requirements of
reliability, weight, noise, and emissions are unlikely to have a strong influence on TSFC.
Their omission from this work is grounded in their weak correlation.
1.4 Framework for Analysis
Performance-schedule-cost trades are explored through analysis of the decisions
made after engine concept definition and by focusing on three frameworks for managing
risk:
1. Iteration (use of multiple attempts to converge to an acceptable solution)
2. Maximizing design flexibility and options during development, potentially with
parallel efforts.
3. Improving predictive capability prior to committing to development of a product.
This will be explored from the perspective of technology readiness.
Figure 1-8 provides graphical representations of how these frameworks will be used to
look at key decisions during engine development.
20
CI)HCI)0UCI)
HCI)0
HH
0/A
b-
............................
CI)HCI)
* 0U
CI)C
I)
000r-L~H0
C/)
0
ISO
D
Figure 1-8. D
ecision tree for product development process.
21
z00
r-L~
U00z........ .
.. .... ...
...
.H
z
0,
.... .... .... ... .... .... .
..
> I
... .. ... ..
...
..
...
..
...
..;
UHHz00
C/)
000H0
z w
U
0000
Explanation of Figure:
It is assumed that underlying the development process, Pratt & Whitney is continuously
investing in improving the readiness and decreasing the variation (G) of performance
enhancing technologies. This simplified illustration shows four possible paths for the
product development process (labeled A-D), each with a different combination of
projected TSFC, schedule, and development costs. These paths are differentiated based
on their approaches to two major decisions:
DECISION #1: Whether or not to launch the engine program. This decision is polarized
into one of two options. Launch the engine program 'early' with 'low' technology
readiness, or delay the program investing in technology readiness (decreased (5) and
launch with 'high' readiness.
DECISION #2: Once committed to launching, decide whether to invest in having
increased design flexibility to allow a reaction to unexpected levels of TSFC at the
FETT. In other words, if the first engine to test demonstrates an inadequate level of
TSFC, this investment would permit designers more latitude to react to the shortfall
and fix the problem. The concept of flexibility combines the risk reduction
frameworks of parallel options and iteration into a single metric that allows designers
to shift the (p) of TSFC. Specific examples of how engine programs can invest in
design flexibility will be provided in Chapter 4.
The spending profiles of paths A-D are each comprised of four different slopes: investing
in technology readiness, pre-validation spending, validation, and post sale liabilities.
PATH A: Launch engine with "low" technology readiness and choose to not provide
additional design flexibility to react to problems. This is the riskiest product
22
development process. Although this path has the smallest projected requirement for
development spending, the increased technical risk translates into increased expected
penalty fees associated with missing the promised level of TSFC.
PATH B: Launch engine with "low" technology readiness and invest in additional design
flexibility. The rate of post sale costs is better than A.
PATH C: Delay launch while improving technology readiness. Launch the engine
program and make contractual commitments using improved information regarding
the engine's performance (less &). Choose not to provide additional design flexibility
to react to problems.
PATH D: Delay launch while improving technology readiness. Choose also to provide
additional design flexibility to react to problems. This path has the least risk and
assumes no post sale costs. Both Paths C and D could result in the engine coming to
market later than Paths A and B. It depends on when technology development occurs
and to what degree the company is willing to spend on technology that is not linked to
a specific engine program.
1.5 Thesis Overview
Chapter 2 provides a synopsis of how a gas turbine engine operates, including
descriptions of the major sub-systems and how they relate to each other. This section
summarizes in a more detailed manner than Figure 1-3, the activities that take place
during engine product development. The formal structures for dealing with risk at both
GE and PW are reviewed.
23
Chapter 3 provides the background for the risk reduction frameworks of iteration,
options, design flexibility, and technology readiness. Learning is based on examples from
the aerospace and automotive industries.
In Chapter 4, these frameworks are applied to a case study of gas turbine
development at Pratt & Whitney. Data from (6) engine development programs are
analyzed to assess how different programs were able to manage risk.
Chapter 5 attempts to identify the limitations of the case study, identifies
opportunities for future work, and where possible, provides recommendations for engine
development programs.
24
2 GAS TURBINE PRODUCT BACKGROUND
The purpose of this section is to provide the context necessary for understanding the
risks associated with meeting the required level of TSFC, on time and within budget.
Background information is provided in the following areas:
* Description of important engine sub-systems and to what degree they constitute a
modular or integral architecture.
" How the engine operates with attention focused on how sub-systems contribute to
TSFC.
* Detailed description of the product development process at Pratt & Whitney. This
includes a summary of the formal mechanisms in place for dealing with risk in
product development.
2.1 Modularity and Integrality
To deal with system level risk (e.g. delivering promised level of TSFC), it is useful to
be able to decompose the system into smaller and more manageable pieces. An ideal
decomposition minimizes the degree to which subsystems are coupled with each other
and allows each part to be managed independently. The system level risks associated with
a completely modular architecture would simply be the sum of each part's contribution.
An integral or coupled architecture, however, needs to understand both the risks
associated with individual modules as well as the risks associated with the interfaces.
A high bypass turbofan engine is commonly decomposed into eight subsystems or
modules.
" Fan
* Low pressure compressor (LPC)
25
High pressure compressor (HPC)
Diffuser and combustor
High pressure turbine (HPT)
Low pressure turbine (LPT)
Mechanical components (including bearings, shafts, oil system)
Externals and controls (including Full Authority Digital Electronic Control
[FADEC], actuators, sensors)
Figure 2-1. Cut-away view of a high bypass turbofan engine. [http://www.pratt-whitney.com/engines/gallery/lg.pw4000.94cut.html].
26
S
S
S
S
S
S
Previous Pratt & Whitney MIT-SDM S.M. theses have attempted to understand the
degree to which an engine represents a modular design. These works employed Design
Structure Matrices (DSM) to analyze the dependencies that exist internal to an engine's
physical architecture, the company's organization, or how product development activities
are organized. Generally speaking, Browning identifies four types of DSM:"
* Component: Break-down based on physical architecture
" Organization: Models dependence between different groups in organization based on
flows such as information or hardware.
" Activity: Models dependency of product development activities based on information,
or people, or hardware.
" Parameter: Models relationships between design decisions and parameters, systems of
equations
Mascoli's parameter-based DSM showed that a turbofan engine has elements of both
modular and integral architecture. Firstly, Mascoli's work suggested that the engine
decomposition described above (FAN, LPC, HPC...) minimized the coupling across sub-
systems. The majority of the dependencies between design parameters existed internal to
each of these sub-systems. The DSM also showed, however, that the gas turbine engine is
far from being perfectly modular with many key system-level parameters cutting across
sub-system boundaries. For example, engine weight and TSFC are a function of all the
subsystems listed above. During development, if an early prototype is too heavy or not
fuel efficient, solutions will assess the impact at the local level, but also the impact of that
local change on the engine.
" Browning, Tyson R. "Modeling and Analyzing Cost, Schedule, and Performance in Complex SystemProduct Development." PhD Thesis, Massachusetts Institute of Technology, 1999.
27
Relative to a modular design, this integral engine behavior is a fundamental source of
risk. Incremental TSFC improvements are possible through discontinuous innovations at
the sub-system level. The insertion of new technologies is riskier because designers need
to be concerned not only about the performance of the innovation, but also how to
successfully integrate it with the rest of the engine.
The next section will help to identify functional and mechanical dependencies
between subsystems.
2.2 Gas Turbine Engine Fundamentals
This section provides an overview of how a gas turbine engine functions, with the
specific goal of providing a more detailed understanding of the major influences on
TSFC. The discussion focuses on four themes:
* How the performance of engine sub-systems aggregates to define the overall
performance of the engine (TSFC)
" How TSFC can be 'traded' to provide additional compressor stability. On-time
delivery is contingent on receiving certification from the FAA/JAA, asserting the
flight-worthiness of the engine. One of the certification requirements is for the
engine maker to demonstrate that the compression system will perform properly
throughout the life of the engine. TSFC can be forfeited to improve compressor
stability and to help ensure on-time delivery.
" Why decisions affecting TSFC are unlikely to affect emissions, noise, and weight.
28
The most common form of commercial engine today is the two-spool high-bypass
turbofan (illustrated in Figure 2-1, above). 'Two-spool' and 'high-bypass' will be
explained shortly.
2.2.1 TSFC: The Key Measure of Engine Performance
At the simplest level, a gas turbine engine generates thrust for the aircraft by
swallowing air and increasing that air's momentum with energy derived from fuel. It
follows that the most efficient engines are the ones that can generate the most thrust with
the least fuel. Thrust specific fuel consumption (TSFC) measures this efficiency and
directly impacts airlines' daily operating costs. TSFC is a function of two factors: 1) how
well the engine converts chemical fuel energy into useful work (thermal efficiency (1T))
and 2) how well that useful work is converted into thrust (propulsive efficiency (rjp)). 12
TSFC= Vo
77Tf P HV
Vo: forward airspeed of aircraft
HV: heating value of the fuel (typically ~ 18500 BTU/pound mass of fuel)
For a given flight speed and type of jet fuel, TSFC is driven only by propulsive and
thermal efficiencies. Lower values of TSFC are better.
2.2.1.1 Propulsive Efficiency, Byjass Ratio, and Noise
Propulsive efficiency [flp] is maximized as the velocity of air leaving the engine (Vj)
approaches the forward airspeed of the aircraft (Vo). In this condition, the engine would
12 Mattingly, Jack D. Elements of Gas Turbine Propulsion. New York: McGraw-Hill, 1996.
29
have converted all of the available work into thrust. The ratio of Vj/Vo is set mostly by
the engine's bypass ratio, a parameter defined during product planning when the engine
concept is selected. Air passing through the engine splits into two annular ducts. See
Figure 2-1. More than 80% of the air is pumped through the outer annulus, or bypass
duct. The 'bypass ratio' is the mass ratio of airflow in the bypass duct to airflow in the
inner annulus or 'core'. 'High bypass' turbofans push a large mass of air through the
bypass duct with a low pressure ratio, minimizing (Vj-Vo) and maximizing propulsive
efficiency. Once the basic geometry of the engine (including bypass ratio) is defined
during the product planning phase, there is little that can be done to change the propulsive
efficiency. Engine noise, also a strong function of Vj, is mostly defined by configuration
decisions made during product planning. Small adjustments to noise can be made later
with acoustic liners and mixing technologies. For this reason, noise and TSFC are
relatively uncoupled during the development phases after planning.
2.2.1.2 Thermal Efficiency
Because this thesis is concerned with decisions made after bypass ratio is selected, it
focuses on the other key driver of TSFC, thermal efficiency ('9T). Thermal efficiency is
perhaps best viewed as a function of the thermodynamic cycle processes that an engine
13 Bartkowski, Glenn D. "Accounting for System Level Interactions in Knowledge ManagementInitiatives." SM Thesis, Massachusetts Institute of Technology, 2001.14 Bypass ratio helps explain a fundamental difference between large commercial engines and the smallerengines powering high-speed military airframes. The former delivers thrust by increasing by a smallamount the momentum of a large mass of air. The most powerful engine in the world in service today, thePratt & Whitney PW4098 with 98000 pounds of thrust at sea level, is capable of sucking all the air out of a2000 square foot house in approximately second. Military engines, on the other hand, generate most oftheir thrust in the core by increasing the momentum of a much smaller mass of air a great deal. Militaryengines have poorer TSFC because of poor propulsive efficiencies.
30
performs on the air passing through: compression, combustion, and expansion. Together,
these processes generate useful work. In the case of a gas turbine engine, they constitute a
Brayton cycle, as illustrated in Figure 2-2, below. Thermal efficiency is maximized by a
cycle that creates the most work with the least entropy generation. For a given pressure
ratio, this would be defined in Figure 2-2 by the path 1-2i-3-4i [isentropic compression,
isobaric combustion, and isentropic expansion]. In reality, the compression and
expansion processes are not isentropic and follow the path 1-2-3-4. The compression
system complements the combustion system by maximizing the area inside the cycle and
the amount of work available. Note that although the air can be expanded back to ambient
pressure conditions, the entropy generation prevents the exhaust temperature (Point 4 in
Figure 2-2) from returning to ambient levels.
31
BRAYTON CYCLEWORK AVAILABLE = AREA INSIDE CURVE
3
EXPANSIONHPT, LPT, nozzles
Exit Temperature
22i-
COMPRESSIONFAN, LPC, HPC
Inlet Temperature
ENTROPY
Figure 2-2. Brayton thermodynamic cycle for gas turbine engines.
2.2.2 Compression System
The purpose of the compression system is to increase the pressure of air the prior to
entering the combustion area. Air leaving the compression system has the highest
pressure in the engine. Some of today's large engines operate at total pressure ratios
higher than 40:1 (> 600 pounds per square inch).
2.2.2.1 Functionality
The ideal compression system would minimize the temperature rise for a given
pressure ratio (path 1-2i in Figure 2-2). As the engine swallows ambient air, the first
32
turbomachinery it encounters is the fan. The engine splitter, which separates the bypass
duct and the core, is immediately after the fan. Therefore, the outer span of the fan
creates the pressure rise for the bypass duct. The inner span of the fan represents the
beginning of compression in the core. The inner span of the fan, the low pressure
compressor (LPC), and high pressure compressor (HPC), together, generate the pressure
rise noted by path 1-2 in Figure 2-2.
The compression system increases pressure by forcing air through a cross sectional
area that decreases forward to aft. Alternating rows of rotating and stationary airfoils add
energy and pressure to the air. Pressure rise is created as the rotating airfoils accelerate
the air and the stationary airfoils turn the air to align it with the next stage of airfoils.1 5
Airfoil designers use the blades' areas, angles-of-attack, and geometry to create a stable
laminar flow over each stage of airfoils. Compressor designers would also opt to have
each row of airfoils spinning at a unique speed. To accomplish this, each row would need
to be attached to its own shaft that was supported by its own bearings. An optimal two-
shaft design has emerged that balances the airfoil designers' needs, the weight of
bearings, the requirements of the bearing-oil system, and the complexity of having
concentric shafts. 16 The fan and LPC are attached to the 'low spool' spinning at speed
N1, and the HPC is attached to the 'high spool' spinning at speed N2.
2.2.2.2 Compressor Surge
As the total pressure of the air rises across each stage, the compressors are working
against an adverse pressure gradient. There are limits to the amount of turning (and
pressure rise) possible over a given airfoil. If the air is turned too rapidly, it tends to
1 A rotating and stationary row of airfoils together is termed a stage.16 "Two spool" = "two shaft".
33
separate from the airfoil and become turbulent and inefficient. At the system level, this
can result in a failure to continue pumping, which in the face of an adverse pressure
gradient causes compressor surge. The flow reversal typically results in a momentary loss
in power associated with the deceleration of the low and high spools. Power is restored as
the compression system restores its ability to turn and pump air. Surge can be caused by
external influences such as ingested birds or vortices, or can be caused by a design that
relies on too much pressure rise per stage. The latter cause becomes an important variable
in selecting a compressor design.
2.2.2.3 Efficiency versus Surge Margin
Compressor design needs to balance the often-conflicting requirements of surge
margin17, efficiency, manufacturing cost, and refurbishment cost. A large number of
stages typically require a lower pressure rise per stage, yielding better surge margin,
better efficiency, but increased weight, worse manufacturing and worse refurbishment
costs (more airfoils).
Once the stage count is established (typically during product planning), surge margin
and efficiency can still be traded. The fan, LPC, and HPC each have a unique airflow and
pressure ratio at which they are the most efficient, known as the design point. All other
flow and pressure ratio combinations are considered "off-design". The family of airflows
and pressure ratios over which each component operates, is known as its operating line.
Ideally, the operating line passes through the design point to maximize component
efficiency, thermal efficiency, and TSFC. If the operating line has insufficient surge
17 Surge margin is defined by the difference between the pressure ratio where the engine is operating andthe pressure ratio at which surge occurs. The difference is measured at constant airflow.
34
margin, it needs to be 'dropped', sacrificing some efficiency. There are two common
mechanisms for dropping a compressor's operating line:
1. Airflow at the inlet to the high pressure turbine is effectively choked. The HPT
inlet area, therefore, controls the amount of airflow through the core. Opening this
controlling area allows more flow to pass through the machine at a lower pressure
ratio.
2. Bleed air from the compressor downstream of the stage that has insufficient surge
margin, effectively allowing more flow at lower pressure ratio.
2.2.2.4 Variable Stator Vanes - Flexibility
Most high compressors include variable-geometry stators. These stator vanes are
attached to a control system that permits the engine to change the vanes' angle-of-attack
inside the gaspath. Because the engine operates over a wide range of speeds, the variable
vanes permit an optimization of flow as a function of speed. When compressor
performance at the FETT is poor, modifying the variable stator vane schedule is a rapid
change that can often improve efficiency and/or surge margin.
2.2.3 Combustion
Air leaving the compression system enters the combustion system. The purpose of the
combustion system is to add energy to air leaving the compressor. Burner efficiency
reflects how well fuel energy is added to the gaspath, where inefficiencies arise as a result
of incomplete combustion. Burner efficiencies in modern gas turbine are -100%.
35
In order to maintain a stable combustion process, the high velocity air from the HPC
is decelerated through a diffuser. The cross-sectional area of the gaspath expands,
allowing the conversion of kinetic energy into increased static pressure. Fuel from the
aircraft passes through several pumps that raise its pressure to a level that is higher than
the air in the combustor. Fuel system plumbing distributes the fuel to multiple fuel
nozzles in the combustor, which disperses the fuel into tiny droplets to enhance
combustion. The combustion of this fuel-air mixture increases the enthalpy of the air to
temperatures typically >3000F, significantly above the melting points of the metals in
this area. To permit operation at these high temperatures, cooler air, which is often hotter
than l000F, is 'bled' from the compressor and 'sprayed' onto the exposed metal
providing a protective film of cool air.
Environmental regulations today place limits on the production of nitrous oxides,
carbon monoxide, and unburned hydrocarbons during combustion. The details of the
combustor design determine the level of emissions output. Emissions, as a system level
parameter of concern, are largely controllable by focusing on this modular element of the
engine. For this reason, decisions impacting emissions typically can be segmented from
those affecting TSFC.
2.2.4 Expansion (Turbines)
Air leaving the combustor enters the expansion system. The purpose of the expansion
system is to extract work from the high enthalpy air, and mechanically drive the
compression system (through the shafts). The expansion system is composed of the high
pressure turbine attached to high spool (HPT), low pressure turbine attached to the low
spool (LPT), and the nozzles at the aft of the engine. The ideal expansion process would
36
generate no entropy and would follow path 3-4i in Figure 2-2. Actual turbines are less
efficient, and hence drop the temperature less, indicated by path 3-4 in Figure 2-2.
Like the compression system, the turbines are composed of alternating rows of
rotating and stationary airfoils. The rotating airfoils, which are coupled to the
compressors with shafts, are accelerated by the hot air leaving the combustor. The
stationary airfoils serve to straighten the flow for each subsequent rotating stage. As air
moves through the turbines, it delivers the work required by compression, and hence
expands, dropping both its temperature and pressure. Gas leaving the turbines is
accelerated through a nozzle with a convergent area. This process, which is the opposite
of the function of the diffuser, acts to convert static pressure into increased gas velocity
and increased thrust.
2.2.5 Section Summary
This section illustrated how engine component efficiencies contribute to TSFC. It was
shown that the failure to achieve a desired level of TSFC could be attributed to efficiency
shortfalls at the component level. This section provided examples for three categories of
component shortfalls. 1) The efficiency of a given component is less than expected,
hurting both thermal efficiency and TSFC. 2) Varation in one component's performance
can push other components 'off-design', damaging efficiency and TSFC. 3) Operating
lines can be consciously shifted 'off-design' to facilitate meeting FAA/JAA
requirements. The specific example, of trading compressor efficiency for stability by
modifying the HPT inlet area, was discussed.
37
2.3 Gas Turbine Product Development Process
Recall from Chapter 1 that engine makers are competing to deliver the best
performing engine, the fastest, for the least cost. This section provides a breakdown of
the activities in the product development process and an overview of the formalized
mechanisms in place at Pratt & Whitney and General Electric for dealing with
programmatic risk.
2.3.1 Pratt & Whitney Product Development Process
The gas turbine PDP is typically illustrated in a non-iterative, linear manner, with
four major phases: planning, development, validation, and support. Historically, the cycle
time from the end of planning to the end of validation is approximately 4 years. Even
though it is easy to see why a shorter development time is desirable, some experienced
managers at Pratt & Whitney question the practicality of trying to compress it below 4
years.18 One provided an analogy between pregnancy and product development
explaining that "you may want it to take less than 9 months, but it just doesn't work that
way." These managers value the important role that testing plays in creating a robust
product. Chapter 4 will offer some of the arguments that support this conventional
wisdom.
2.3.1.1 Product Planning (Conceptual and Preliminary Design)
This phase represents the step at which the engine architects analyze market needs
(includes projected TSFC for when the engine would enter service), airframer and airline
requirements, Pratt & Whitney's capabilities, and opportunities for new technology
insertion. There exists a great deal of design flexibility. Various engine concepts are
38
is Personal interview.
generated and discussed with airframers, potential airline customers, and with Pratt &
Whitney designers and system engineers. The following parameters are defined for each
engine concept: net thrust (FNT), bypass ratio (BPR), overall pressure ratio (OPR),
compressor exit temperature (T3), turbine inlet temperature (T4), fan flow capacity, fan
pressure ratio, LPC pressure ratio, HPC pressure ratio, HPT pressure ratio, and LPT
pressure ratio.
Iterative discussions with all stakeholders yield a single most feasible concept that
best balances internal capabilities and projected market needs. At the end of the product-
planning phase, the engine program is launched. Typically at this point, the engine
manufacturer, airframer, and initial airline customers agree to a contract that specifies the
engine's TSFC, weight, emissions, noise, reliability, maintenance interval, and
availability date. A rare exception is when the engine maker decides to build an engine
without a specific market defined. This scenario does not have a forward commitment to
a specific value of TSFC.
At the end of product planning, although requirements are frozen in the interest of
motivating the design effort, there still remains a reasonable amount of design flexibility
at the component level. At this time, "estimations of design and off-design performance
are generated but fidelity tends to be poor." Models tend to be empirical and cannot
predict "outside" the design experience of past products. 19 The use of derivative engine
technologies (re-use from earlier programs) can have a significantly positive impact on
the fidelity of simulation predictions.
'9 Sullivan, John P. "The Relationship Between Organizational Architecture, Product Architecture andProduct Complexity", SM Thesis, Massachusetts Institute of Technology, 1999.
39
2.3.1.2 Product Definition (Detailed Design)
During this phase, a detailed design for the engine is produced. The most recent
engine development programs at Pratt & Whitney have aggressively pursued
improvements in the speed of design. Design times have been reduced from 1 2 years
(associated with traditional 4 year engine development cycle time) to less than a year.
This was accomplished by eliminating design iterations through three major efforts:
1. Earlier involvement of personnel from manufacturing, PW overhaul, and even
airline mechanics to help focus designers on the final product. Their early
involvement, however, introduced new tensions into the design process. Many
designers acknowledge the role of engine testing in honing their designs and resist
pressure from manufacturing to deliver final prints.
2. Improved component design through the use of finite element and computational
fluid dynamics (CFD) models. Use of software has reduced the number of costly
integration problems associated with initial engine design. The design can also be
enhanced by component or 'rig' testing that allows iteration internal to this
development phase.
3. Improved engine system modeling capability during the product planning phase
eliminated some of the iterations that formerly happened during detailed design.
2.3.1.3 Product Validation
As mentioned above, the strategy for engine cycle time improvements has been to
move iterations to earlier parts of the development process where they are quicker and
less costly. In this spirit, Pratt & Whitney management changed the name of this phase
from Test to Validation. Goals have also been set for the duration of validation to be
40
reduced from 2 /2 years (associated with 4 year engine cycle time) to less than 2 years.
As will be discussed in the case study in Chapter 4, this strategy has implications for the
level of risk in development. The product validation phase seeks to achieve three goals:
* Validate that the intent of the design was met. In other words, confirm that theengine meets the contractual requirements for performance, noise, emissions, etc.
" Receive regulatory approval for release of the engine into revenue service. This isaccomplished through successful completion of FAA/JAA mandated testing andanalysis.
* Integrate the engine with the airframe through flight-testing.
The tension between designers and manufacturing regarding maintaining design
flexibility persists. Hague described this tension from the designers' perspective by
stating that "enforcing requirements and not allowing for iteration will most often result
in sub-optimal design...."20 A second aspect of this tension can be seen as an application
of the Heisenberg Uncertainty Principle: the act of measurement can change the system
you are trying to measure. The act of validation requires that measurements be made. The
instrumentation, however, creates non-production holes, flanges, welds, and taps that can
affect natural frequencies, thermal growth, and flow conditions. As a result, there is a
possibility that the engine will perform differently when it is not instrumented.
Simulations attempt to predict the impacts of instrumentation.
A discussion of each of the three validation goals listed above follows, with particular
focus on delivering the required level df TSFC.
Much of the scheduled engine testing in fact focuses not on validating TSFC but
rather on identifying minor design problems that would become a source of warranty
41
expense. Minor design problems are identified by having engines perform hundreds of
hours of endurance testing, simulating typical in-service operation. This testing uncovers
"infant mortality" problems and identifies parts that prematurely wear-out. There arefew
formal mechanisms in place, however, that allow engineers to deal with misses in engine
performance (TSFC). Some buffer is retained in the schedule to react, but this occurs in a
fairly ad hoc manner. In the political environment that stresses cycle time reduction, the
value of this ad hoc buffer is often under-appreciated.2 1 Details with regard to how this
affects reducing misses in TSFC are discussed in Chapter 4.
The second major goal of the validation phase is to successfully complete a series of
tests required by the FAA and JAA prior to receiving certification and entering service.
Figure 2-3 summarizes the key tests required by these regulatory authorities. With regard
to reducing misses in TSFC, these tests should be seen as a potential source of costly
rework. That is, if the engine's configuration is modified late in the development program
to improve TSFC, any completed FAA tests need to be re-assessed and often repeated.
Figure 2-3 summarizes, for each mandated test, engine modules that the FAA sees as
critical in the proof that the test was successful. For example, if the engine program
decides to redesign a fan blade for improved efficiency and TSFC after the FAA Fan
Blade Containment test was completed, it is likely that the test would need to be repeated.
This particular test involves liberating a fan blade and proving that the blade was
contained by the engine. This test is very expensive, as the entire engine is unusable
afterwards.
42
20 Hague, Douglas C. "Description of a Turbofan Engine Product Development Process." SM Thesis,Massachusetts Institute of Technology, 2001.21 Interviews with product validation managers.
Test Modules Most Directly Affected
Blade Loss Containment x x
Cooling x x
Emissions x x x
Endurance (150 hours at maximum exhaust gas x x x x x x x xtemperature and rotor speeds)
Gearbox Endurance x
High Rotor Stress x x x x
Icing x x x
Ingestion (Birds) x x x
Ingestion (Hail Ball) x x x
Ingestion (Ice Slab) x x x
Ingestion (Water / Hail) x x x x x
Initial Maintenance Inspection Endurance x x x x x x x x
Low Rotor Stress x x x x x
Noise xx x x x x x x
Operations (Thrust response, compressor x x x x x x x xstability, burner stability....)
Overspeed x x x x x
Overtemperature x x x
Starts / False Starts x x x x
Vibratory Stress x x x x x
Figure 2-3. FAA Testing Requirements.
43
The last major goal of the validation phase is to install the engine on the aircraft and
perform a flight-test. Hardware and software integration issues are addressed. Dedicated
testing is performed to assess the drag of the aircraft and the TSFC of the engines. These
tests play a role in establishing whether the engine meets contractual guarantees for
TSFC. In a 4-year development cycle, 9 months are dedicated to flight testing.
2.3.2 Pratt & Whitney Gated Decision Process
Within the framework of Integrated Product Development (IPD) at Pratt & Whitney,
the product development process is controlled through a gated review process that intends
to "ensure that customer requirements and company commitments are identified and met.
The reviews are timely gates within the product cycle and help reduce inherent risks." 22
The timing of the gates, termed Passport reviews, are shown in the Figure below.
PRODUCT PLANNING PRODUCT DEFINITION PRODUCT VALIDATION PRODUCT DELIVERY
r r r-A0 1 II iH IV V VI VII
Figure 2-4. Decision Gates in PDP (Passport review numbers identified).
At each review, data is presented to senior management that facilitates an assessment of
engine risk. Risk is calculated based on the answers to the following types of questions.
What percentage of the allocated product development budget has been spent? What
percentage of anticipated work is complete? To what degree does the forecasted state of
2 Anonymous. Pratt & Whitney process PW-SLP 4.5.1, Product Review Process.
44
the engine meet requirements? What is the anticipated level of spending? As a result of
these discussions, management can take one of three possible courses of action:
1) Proceed to the next gate. Management is satisfied with the current level of risk.
2) Recommend further work be performed prior to advancing through the gate. This
entails improving the uncertainty ((Y), or
3) Cancel the program due to unacceptably high levels of risk of achieving a cost
effective solution.
The following is a brief description of what officially happens at each gate:
GATE 0: Engine system review during which several concepts are evaluated with regard
to marketplace opportunities and Pratt & Whitney's technical capabilities (including
technology readiness).
GATE I: Engine system review that converges to an attractive concept that meets market
needs, complies with the company's strategic plan, uses appropriate technologies, and
identifies capital and manpower requirements.
GATE II: Engine system review that marks the end of "product planning." Customer and
company requirements have evolved into an engine requirements document. If
warranted, the engine program is launched defining a detailed schedule with risk
mitigation activities. Historically, there has been a reluctance to commit to the
increased spending associated with risk reduction activities. This can be explained
partially by seeing Pratt & Whitney in the context of its corporate parent (and source
of funding), United Technologies Corporation (UTC). Engine programs are
competing with Otis elevators, Carrier air-conditioners, Sikorsky helicopters, etc. for
corporate funding. In an environment where cash flow is a critical metric, engine
45
program managers are driven to propose 'success-oriented' programs that require less
cash. At the end of this Passport gate, the company is contractually committed to
delivering an engine that meets requirements (including TSFC).
GATES III and IV: Subsystem (module) and part reviews that support detailed design,
manufacturing, and overhaul considerations. When appropriate, design reviews
include discussions of component-specific (rig) testing.
GATE V: Engine system review that takes place when data from the First Engine to Test
(FETT) becomes available. Demonstrated performance is compared to requirements,
and risk mitigation plans are updated when appropriate. Additional Passport reviews
are often inserted prior to Gate VI to address key technical decisions. It is important
to note that commitments for both engine performance and schedule were made at
GATE II. This fact significantly reduces management's options in dealing with risk
identified at this point in time. Yet, this is the first time that highly reliable
information is available.
GATE VI: Engine system review that takes place after certification is received from the
FAA/JAA and prior to entering revenue service.
GATE VII: Engine system review of in-service experience.
Fundamentally, a gated decision process attempts to provide clear guidance for what
tasks and risk mitigation activities need to be performed prior to moving through the next
review gate. A process with gates aims to prevent launching a program with unacceptable
levels of risk and seeks to ensure that the development process is sufficiently capable of
retiring risk. This thesis sees the Passport process as an important and useful mechanism
46
for controlling risk. It will seek to offer suggestions for risk management within the
framework of a gated review process.
2.3.3 General Electric Gated Decision Process
General Electric also uses a gated decision process, termed "tollgates" in all of their
divisions (from lightbulbs to engines). The fundamental similarity between PW and GE
gated decision-making is in the understanding that the cost of rework increases over time.
Both companies seek to retire risk early when the penalties of fixing problems are less.
GE's generic process is shown below.
I Customer needs definedII Concept reviewIII Feasibility reviewIV Preliminary design reviewV Final design reviewVI Critical productivity reviewVII Field Test ReviewVIII Manufacturing Feasibility reviewIX Market readiness review
DEVELOPMENT
DESIGN
MANUFACTURING/PLANNING
PRE-PRODUCTION , PROD
Figure 2-5. General Electric Tollgate Process. [Adapted from Wheelwright (1992).]
47
2.4 Chapter Summary
This chapter summarized the central importance of delivering engines that have a
competitive level of TSFC. For decisions that are made after an engine concept has been
selected, it was shown that thermal efficiency has more leverage on TSFC than does
propulsive efficiency. Thermal efficiency was explained in the context of the Brayton
thermodynamic cycle to be a function of the engine component efficiencies (fan, LPC,
HPC, HPT, and LPT).
The product development process was described in detail. Each phase of development
was described from the perspective of how it influences the TSFC of the engine when it
enters revenue service. Lastly, the mechanism of a gated decision process was shown as
the key formal control over programmatic risk at PW and GE. The degree to which the
formal process is able to serve this purpose was also discussed.
48
3 RELATED WORK
This thesis developed its theoretical understanding of technology readiness, design
flexibility, and iteration by studying how other enterprises in the aerospace and
automotive industries develop products.
3.1 NASA and Technology Readiness
One of NASA's goals is to "extend the commercial application of NASA technology
for economic benefit and improved quality of life."2 3 NASA's Aerospace division states
that "we must work with our industry and FAA partners to ensure that our technologies
are implemented."24 In the early 1990's, NASA identified slow product development
cycles as a roadblock to the advancement and commercialization of technologies. In the
early 1990's, NASA was becoming concerned that complex projects with long
development times were not able to keep pace with the rapid technological advances in
many fields. In this environment, space programs would reach a launch-ready state with
outdated and obsolete technologies.
In response to this problem, NASA identified a core need of being able to mature
technologies more quickly, and launched an initiative called the Integrated Technology
Program (ITP).25 As a result, Technology Readiness Levels (TRL) were created as a
standardized approach for assessing the technical maturity level of a system or
2 http://www.aero-space.nasa.gov/goals/ct.htm24 http://www.aero-space.nasa.gov/goals/index.htm
2 Sarsfield, Liam. The Cosmos on a Shoestring: Small Spacecraft for Space and Earth Science. SantaMonica, California: RAND, 1998.
49
subsystem. See Figure 3-1 below. Note that TRL increases with improved understanding
of a technology's capability.
STAGE TRL OBJECTIVEBasic Technology 1 Basic principles observed and reportedFeasibility Research 2 Technology concept and/or application
formulated3 Analytical and experimental critical
function and/or characteristic proof-of-concept
Technology Development 4 Component and/or breadboard validationin laboratory environment
Technology Demonstration 5 Component and/or breadboard validationin relevant environment
6 System/subsystem model or prototypedemonstration in a relevant environment(ground or space)
System/subsystem 7 System prototype in a space environmentdevelopmentSystem test, launch, and 8 Actual system completed and "flightoperations qualified" through test demonstration
(ground or space)9 Actual system "flight proven" through
successful mission operations.
Figure 3-1. NASA Technology Readiness Levels. 26
The idea of TRL was a subtle part of a cultural shift from viewing technical risk
as something to be avoided at all costs to seeing risk as a parameter that needs to be
managed. Higher TRL's reflected less technical uncertainty, and this metric could be
used to track how investments in analysis and testing could retire risk. TRL could
serve as a mechanism for the comparative assessment of competing technologies.
26 Mankins, John C. "Technology Readiness Levels: A White Paper". NASA Advanced Concepts Office,(http://www.hq.nasa.gov/office/codeq/trl/tr.pdf). April 6, 1995.
50
Perhaps most importantly, it could provide technologists and product developers a
common language for discussions regarding technical uncertainty.
In 1998, Liam Sarsfield at the RAND Corporation identified some shortcomings
in how small spacecraft development programs at NASA were dealing with the
integration of new technologies:
* Programs still lacked mature methods for risk quantification.
" NASA still lacked a clear tie between technology development and specific
mission requirements. "Reliance on labels such as 'flight qualified' and 'flight
proven' without careful review of applicability can introduce unanticipated
risk." 27 Just knowing the TRL does not necessarily make it appropriate for a given
mission. This concept will be highlighted in the Pratt & Whitney case study,
where it will be shown that interpretation of TRL can be subjective. Different
engineers can have a range of opinions regarding the readiness of a technology.
The following fictitious example serves to illustrate the important relationships
between technology readiness, standard deviation (s), and mean value (p). Improving
technology readiness yields a reduction in (a), but does not have a predictable
relationship with the mean (p).
A new technology, assessed to be TRL3, is predicted by simulations on average to be
capable of meeting the requirements of a new program. Figure 3-2 below shows the
technology's predicted performance (x-axis) using a probability density function.
When no real testing on the technology has been performed, there exists a relatively
high level of uncertainty about its performance, and its TRL is 3. Further investment
2 Sarsfield, Liam. The Cosmos on a Shoestring: Small Spacecraft for Space and Earth Science. SantaMonica, California: RAND, 1998.
51
in the component, and then system level testing eventually raises the technology to
TRL6. At this point, the new technology's performance has significantly less
uncertainty (smaller cy), but it exhibits a mean shift (p) that represents a shortfall in
actual performance. This example illustrated the scenario in which the technology
remained constant throughout a series of tests. In the end, the investment in testing
helped to gain control over (Y) but not (g). An implication of this example is that
'high TRL' helps to make more realistic a priori promises regarding a technology's
performance.
I I REQUIREMENT
TRL6 .
I '. I
I -. I
I I-
TI 5
TRL4
TRL3
Figure 3-2. TRL, variation, and mean.
As shown in Figure 1-8 this thesis plans to investigate four possible paths for gas
turbine engine development. Two of these paths (C & D) attempt to reduce risk by
reducing (G) prior to launch. Paths (B & D) attempt to decrease risk by investing in
maintaining additional design flexibility to react to testing results, permitting a change in
(p.). In these latter scenarios, consideration must be given to the nature of the changes that
52
are made as a result of testing. There is a possibility that a design change (intended mean
shift (pt)) will make a previous test result invalid for the new design. If so, design changes
could lower the performance certainty (G) and TRL. This reflects the often-unpredictable
nature of maturing new technologies to a useable state.
3.2 Value of Options
This section uses 'financial options' to illustrate the idea that options can provide a
decision-maker a useful mechanism for hedging or controlling risk. Financial options are
introduced here only as an analogy for the choices available in product development.
Financial options work as follows. In return for a small investment today, an option
allows the purchaser to exercise the option only when it provides a profit. Consider a
simple example in which the potential future outcomes of an investment are highly
uncertain and risky. In Case #1 below, there is a 50% of either winning or losing $100.
The expected value of this scenario is $0 = ((0.5*100) + (0.5*-100)). In Case #2, the
investor pays $10 for an option to invest. Both the upside and downside are less severe,
but the expected value is now larger ($40).
53
CASE #1 CASE #2 (OPTION)Expected value = $0 Expected value = $40
$100 $100-10 = 9050% 50%
50% 50%
-$100 -$10
Figure 3-3. Example of benefit of real options.
The idea of real options extends to product development in the sense that having
choices in the face of uncertainty can improve expected value. Within this framework, a
product developer needs to be concerned with the expected values of both the 'success'
path and the 'fallback' path. The gated decision making processes at PW and GE provide
options at each gate. Managers can choose to proceed, recommend additional work that
addresses risk (e.g. parallel development paths), or they can cancel the program. The next
section looks at design flexibility at Toyota as an application of real options, and
identifies additional early investments in product development as a means for hedging
risk. These options have the effect of increasing flexibility after testing and thus apply to
Paths B and D.
3.3 Toyota and Set Based Design
Similar to the gas turbine product development process, many American and Japanese
automobile manufacturers use a product development process that seeks to define clear
and specific requirements both at the system and detailed design level as early as
54
possible.28 When successful, this approach facilitates earlier commitment to production
tooling and shorter cycle times. From this perspective, Toyota's product development
process appears to be inefficient, as it delays commitment to important configuration
decisions and invests in parallel efforts or design 'sets'.
This delay, however, allows Toyota and its suppliers to perform more component
testing that decreases their likelihood of having to perform rework. As the architecture of
an automobile is more modular than a gas turbine engine, this early component testing
can provide an accurate representation of how the component will perform when
integrated into the automobile. When Toyota finally commits to a product architecture, it
is able to select a design set that includes well-tested and robust components. 29 In the
context of Wheelwright's design funnel, Toyota keeps the mouth of the funnel wide
(considering many sets) and the neck narrow (converging to a single 'optimal' design).
28 Ward, Allen, Jeffrey K. Liker, John J. Cristiano, and Durward K Sobeck 1I. "The Second ToyotaParadox: How Delaying Decisions Can Make Better Cars Faster." Sloan Management Review, Vol. 36,Issue 3 (Spring 1995), 43.29 Ward et al (1995).
55
Figure 3-4. Convergent design process used by Toyota. [Adapted from Wheelwright(1992).]
3.4 Cost of Rework
Implicit in Toyota's approach is the principle that the cost of rework increases as a
function of time and the phase of product development.
There is evidence of this in other industries. A program manager at General
Dynamics Bath Irons Works shipyard cites the exponential growth in the cost of rework
as a destroyer moves through its development cycle (design, fabrication, modular
assembly, system assembly "on the ways", customer delivery, operational service at sea).
He explained that a $1 change in the drydock "on the ways" would cost $10 at berth, and
$100 at sea.30
A similar trend applies in the development of gas turbine engines. Recall the four
product development paths (A-D) illustrated in Figure 1-8. Path A 'launched' the
development program with a high risk that the engine at Entry into Service (EIS) would
30 Personal communication with Arleighe Burke Class Destroyer Program Manager at Bath Iron Works,
March 2000.
56
not meet its performance guarantees. Corrective action takes place when rework is the
most expensive (when the engine is in the customer's hands). Path B costs more upfront,
attempting to move rework from post-EIS into the validation phase, where rework is
cheaper. Path C attempts to invest in the earliest phase of development (planning), prior
to engine launch, where rework is the cheapest. Path D attempts to use all possible
resources to avoid the most costly form of rework (post EIS).
The next section discusses perhaps the most straight-forward approach to dealing
with failures in program development: trail-and-error, or iteration.
3.5 Iteration to Achieve Convergence on Requirements
Information about a product decreases uncertainty and risk. This section explores
testing as an information gathering exercise that promotes iterative learning and risk
reduction.
3.5.1 Iteration
Browning defines an iterative design process as "one where multiple passes are
required for the design to converge to suit an array of sometimes-conflicting
specifications."31 Iterations may arise from changing requirements or simply from the
failure to meet performance objectives. If we acknowledge the occurrence of planned or
unplanned iterations, cycle time can be improved by makingfewer orfaster iterations. In
the extreme, fewer equates to "do it right the first time", and testing is simply a validation
of what was expected. In the eyes of many engine program planners at Pratt & Whitney,
this is the ideal role for testing.
3 Browning, Tyson R. "Modeling and Analyzing Cost, Schedule, and Performance in Complex SystemProduct Development." PhD Thesis, Massachusetts Institute of Technology, 1999.
57
3.5.2 Testing
One Pratt & Whitney manager painted a picture of the future in which engine
simulations and design tools had evolved to the point that the President of the company
could automatically design an engine with a few keystrokes. Providing input for several
key design parameters such as FNT and TSFC would launch an engine design with high
certainty of meeting its promised performance. In this ideal world, engine testing would
be performed to satisfy FAA requirements only. It would not be seen as a critical source
of information for development.
In the absence of perfectly predictive simulations and design tools, testing at the
component and engine system level is a critical source of information and learning. For
testing to achieve this additional purpose, an organization needs to be self aware
regarding the quality of the predictions of its simulations and design tools. Organizations
that think that they are better than their true capabilities, risk using inefficient testing
strategies.
When viewing a single isolated test, the most efficient test is the one that yields the
most information for the least cost in the shortest period of time. Bernstein identifies the
potential paradox of organizations that seek to "do it right the first time." They may fall
into the trap of executing the least efficient test - one that generates the least amount of
learning about the product's capabilityv32 This is perhaps understood more clearly by
comparing two testing scenarios in the gas turbine engine validation phase.
32 Bernstein, Joshua I. "Design Methods in the Aerospace Industry: Looking for Evidence of Set-BasedPractices." SM Thesis, Massachusetts Institute of Technology, 1998.
58
SCENARIO #1: 'Assumes' that the first engine to test (FETT) will perform as
expected. Instrumentation is installed to measure TSFC, but none of the internal
instrumentation, required to calculate component efficiencies, is installed. This
strategy works if the engine 'does it right the first time' and meets guarantees. If
the engine performs worse than expected, the lack of instrumentation makes it
impossible to identify the root cause of the problem. The engine would need to be
rebuilt (time and $$) with an improved array of instrumentation and re-tested.
SCENARIO #2: This program 'assumes' that there will be problems and installs a
maximum amount of instrumentation. Any problems will be able to be understood
and fixes will be quickly identified. But this scenario costs more and delays
design of production-ready parts.
The latter scenario was likely able to deliver more information to the engine
development program for a lower net cost.
If the goal is to maximize learning and reduce the uncertainty associated with a new
product, Thomke and Bell saw testing as a task that could be optimized by selecting the
appropriate timing, frequency, and testing fidelity.3 3 Their work yielded testing heuristics
based on several basic principles:
* As described above, the cost of rework increases as a function of time into the
product development process.
* The earlier information is available, the greater its value.
1 Thomke, Stefan and David Bell. "Optimal Testing Under Uncertainty." Unpublished. Harvard BusinessSchool, 1998.
59
* Higher fidelity tests cost more, where fidelity represents the quality of
information possible. For gas turbine engine development, an engine with a lot of
instrumentation that performs a wide range of throttle maneuvers is higher fidelity
than one that has no instrumentation and performs a small range of tests.
Furthermore, when the configuration of the engine is changed to react to problems
identified during testing, the maximum possible fidelity of a test increases over
time. That is, it may not be possible to simply invest more money for increased
fidelity at FETT. There is an upper limit. As the configuration becomes more
static, higher fidelity tests are possible.
Thomke and Bell generated mathematical relationships based on these assumptions
and focused on the degree to which sequential tests provide overlapping information.
From this, the following recommendations regarding optimal testing are made:
1. For multiple variable-fidelity overlapping tests, perform (dv/2m)0 5 tests, where d
is the cost of rework over time, v is the number of cumulated problems over time,
and m is the cost of given test as a function of fidelity.
2. For multiple variable-fidelity independent tests, they recommend to perform as
many tests as is possible. 34
These recommendations can be placed in the context of the development paths A-D
first referenced in Figure 1-8. Most gas turbine engine tests fall in the category of
'variable-fidelity overlapping tests', where Thomke and Bell recommend that the amount
of testing should be:
14 Thomke, Stefan and David Bell. "Optimal Testing Under Uncertainty." Unpublished. Harvard BusinessSchool, 1998.
60
" proportional to the cost of rework over time [d in the equation above]. This
variable is the same for each of the development paths A-D.
" inversely proportional to the cost of testing [m in the equation above]. This
variable is the same for each the development paths A-D.
" proportional to the number of anticipated problems with the design [v in the
equation above]. Thomke and Bell would differentiate paths A-D based on this
variable. Paths A and B, which represent programs launched with low technology
readiness, should have more problems than paths C and D, which have higher
readiness. Thomke and Bell would recommend more testing for Paths A and B.
Furthermore, if we assume that problems need to be identified during
development, paths B and D, which allow for more testing, would be superior to
paths A and C.
3.6 Chapter Summary
This chapter provided background for this work's use of technology readiness, design
flexibility, and iteration as frameworks for risk reduction. NASA's formalized technology
readiness level metric can be used as a proxy for predicting the uncertainty of a
technology. Toyota has demonstrated a willingness to invest more heavily than other
automobile manufacturers to maintain design flexibility. This represents a conscious risk-
reduction strategy and not simply inefficiency. Lastly, the roles of testing and iteration
were explored in a generic sense. This investigation helped to underscore the need to
develop a testing strategy that meets an organization's needs. In the presence of high
uncertainty, testing should be optimized for its ability to generate useful information.
61
4 PRATT & WHITNEY CASE STUDY
This section analyzes six engine development programs at Pratt & Whitney and
investigates the degree to which technology readiness and design flexibility were used to
deal with the risk of meeting the levels of TSFC promised in the engine specification.
This is motivated by the data in Figure 1-4 which showed the degree to which the
validation phase was (or was not) able to offset the TSFC misses demonstrated by the
First Engine to Test (FETT). As described in Figure 1-8, there are two major decisions
that are made after the end of the product planning phase that influence how risk is
managed.
1. Whether or not to launch the engine program. Launch the engine program 'early' with
'low' technology readiness, or delay the program while investing in technology
readiness (decreased () and launch later with 'high' readiness.
2. Once committed to launching, decide whether to invest in having increased design
flexibility to allow a reaction (ability to shift pt) to unexpected levels of TSFC.
4.1 Technology Readiness
For each the six engine programs, this section tests the hypothesis that the technology
readiness of the engine components can serve as a proxy for the probability that the
performance of the First Engine to Test (FETT) will meet the guaranteed level of TSFC.
This hypothesis is based on the following logic:
* Technology readiness correlate with the quality of information that is available
to be incorporated into aero-thermal engine simulations. These simulations are
used to predict TSFC and form the basis for understanding the engine and for
making contractual commitments.
62
" The quality of information available at the time of decision-making correlates
with expected variation of TSFC (G).
" It is possible to formulate an engine's TRL by suitably combining the TRL's of
the engine components.
* Interviewing component and system design engineers allows an assessment of
component and engine TRL.
Note that a large value of (-) provides decision-makers more 'latitude' to set 'stretch
goals'. As illustrated in Figure 1-2, engine manufacturers are under intense market
pressures to offer an engine that has a competitive level of TSFC. Refer to Figure 4-1
below. If the predicted capability of a new technology (labeled 'designer's
expectation') fails to meet the market demand (labeled 'commitment') the large value
of (a) associated with a TRL3 technology allows management to set a 'stretch goal'
for the engine program. After an engine test is executed, the technology is TRL6,
with a significantly smaller (a). Figure 4-1 illustrates one possible outcome of the
test, where the mean proves to be lower than expected. In this scenario, the
technology is incapable of meeting the stretch goal. Even if the mean were the same
as predicted when the technology was TRL3, it would have failed to meet the stretch
goal.
In summary, poor information can lead to high (a) and imposes less constraints on
decisions-makers choosing to move thy commitment above the mean (p) for marketing or
business reasons. The scenario of setting stretch goals increases the probability of failure.
One way to avoid them is to perform testing to reduce (a).
63
TRL3 (DESIGNER'S EXPECTATION)
TRL6 COMMITMENT (BASED ON'STRETCH GOAL')
TSFC GAP @ FETT
I II II II II I
G TRL6 I II II II II I
ITRL
Figure 4-1. TRL, variation, and commitment.
4.1.1 TSFC = f(Component Performance)
This thesis is most interested in the Product Definition and Product Validation phases
of the PDP when TSFC is most directly influenced by component efficiencies' impact on
thermal efficiency. This work makes the simplifying assumption that TSFC (and lT) can
be modeled as a function of the 'first-order' influences: fan, LPC, HPC, HPT, and LPT
efficiencies. 'Second-order' sources of loss are ignored and should have minimal
influence on this analysis. An example would be the pressure loss associated with guide
vanes that straighten the flow but are not part of the above-mentioned components.
4.1.2 Technology Readiness leads to Simulation Readiness
Technology readiness is only a useful concept if it is able to influence decision-
making. In this light, the testing associated with Technology Readiness Levels (TRL)
64
leads to incremental learning, which is then captured in an aero-thermal model of the
component and engine. This model can be assigned a corresponding Simulation
Readiness Level (SRL).
When the decision is made to launch an engine program, the engine's TSFC is
predicted using an aero-thermal simulation. From the perspective of impacting this
important decision, TRL is not enough. The testing associated with advancing the TRL of
a technology needs to be 'captured' in a simulation. This view sees the functionally
separate tasks of testing and building predictive simulations as two halves of the same
process. Figure 4-2 below summarizes how NASA TRL's have been adapted to TRL's
for gas turbine engine development and then extended to Simulation Readiness Levels.
65
TECHNOLOGY and SIMULATION READINESS LEVELS
Readiness NASA Technology Readiness Pratt & Whitney Technology Readiness Pratt & WhitneySimulation Readiness
Level
S1 Basic principles observed and reported
-------------------------------------------------------........
2 Technology concept and/or application formulated Simulation concept formulated
3 Analytical and experimental function and/or characteristic proof-of-concept Initial modeling and definition of design space
--- -- -------------------------------------------- -- ------ - - --- -- -- -- - -
4 Component and/or breadboard validation in Component validation in laboratory or rig Improved model captures the results from rig
laboratory environment Environment (test facility for (1) comp.) test
--------------------------------- -----------------------------------Component and/or breadboard validation in Component validation on prototype or core Improved model captures the results from core
5relevent environment engine (group of components) test
CD System/subsystem model or prototypeS mSystem level demonstration in prototype or Improved model captures the results from
6 demonstration in a relevent environment full engine(ground or space)
System level demonstration prototype or engine Improved model captures the results from flight7 System prototype in a space environment flightIin flgttest Itest
8 Acutal system completed and "flight qualified" Actual system certification (FAA/JAA) N/Athrogh test demonstration (ground or space)-------------------- 4----------------- ---------------------------
Actual system "flight proven" through I Improved model captures the results (including9 A ccssel"might Ipre"to gh Actual system in service I deterioration trending) from revenue service
successful mission operations I data
Prior to flight-testing an engine (TRL7), all simulations and tests in the development
process fall short of representing the exact environment in which the engine will operate.
Even at flight-test, instrumentation is limited and often fails to provide an accurate
picture of what the engine is doing. The learning process is iterative:
" create a 'pre-test' simulation
* execute the test,
" analyze the data and incorporate new learning into the simulation.
In Thomke and Bell's language, thefidelity of the test will drive the quality and amount
of new information. TRL is assigned based on the completion of different fidelity tests. A
full engine test has a higher fidelity and TRL than a component rig test.
SRL is assigned once the results of the test have been understood and captured in a
model. At this point, the test data has improved the predictive capability of model.
Decisions regarding engine performance are now based on higher quality information
with lower anticipated variation. The following iterative spiral is an example of how a
new high pressure compressor (HPC) might increase its TRL and its SRL. See Figure 4-3
below. The two sections that follow describe how this spiral applies to new technologies
and to derivative technologies.
67
CD
CD
CD
rA
TECHNOLOGY READINESS LEVEL
00
SIMULATION READINESS LEVEL
SRL3 or SRL6
SRL3or SRL5
SRL3
SRIJ or SRL4
erate / Install HPC
librate Install HPC into into a core rig
n of HPC rig, and executeInstall HPC
nance & test to measure combustor and engine, anctdes H testto mHPT), and test to meas ent esig HPCexecute test to perform
based on performance eeue t orst reultsmeasure HPC
performance
TRL4
TRL5
TRIL6
Design (or
modify) HPCbased on 3D-aerocodes and finiteelement analysis
Genreca
simulatiperfon
implemchanges
any te
into full
executeure HPCance
4.1.3 Increasing the TRL/SRL for New Technologies
Examples of new technologies that would initially be assessed to be TRL3
" new compressor airfoil shapes designed to perform more turning and work per
stage; increased work per stage permits reduced number of stages.
* materials changes in the cases surrounding the rotating blades designed to allow
improved control over operating clearances.
* 'high work' turbine airfoils designed to maximize efficiency and work extraction
from the gaspath.
* shroudless fan blades (some fans have part span shrouds designed to prevent
resonant flutter; shroudless designs can yield improved efficiencies).
* improved HPT materials allowing hotter turbine inlet temperatures.
Figure 4-3 illustrates how testing and analysis improve the TRL and SRL for an HPC
with new technologies (such as blade design or case materials). The process begins on the
left side of the Figure, as 3D-aerodynamic codes and finite element models are used to
generate a new compressor design. Hardware is acquired for the purpose of testing and an
aero-thermal model is created that predicts the compressor's efficiency and flow capacity
as a function of airflow, pressure ratio, high rotor speed, and the position of the variable
angle stator vanes. At this point, compressor performance is based on analytical models
only and is TRL3/SRL3. This example will shows how subsequent testing can improve
the understanding of the component's capability to TRL6/SRL6. The component will
69
take the following improvement path: TRL3/SRL3 - TRL4 - SRL4 - TRL5 - SRL5 -
TRL6 - SRL6.35
The compressor is installed into a rig that attempts to simulate the physical and
functional boundary conditions of the compressor. After tests have been executed that
test the validity of the model, the HPC is TRL4. The data is analyzed and the model is re-
calibrated to capture new information learned in the rig test. The predictive capability of
the simulation is now SRL4. Typically, the rig test provides improved understanding of
the HPC that leads to modifications of the design. The readiness level after the changes is
no longer easy to determine. The design modifications are discussed to answer the
following questions. Do we anticipate that the changes will have significant impacts on
HPC performance? Can we accurately predict these performance impacts in the
simulation? If so, the readiness of the HPC and its simulation would remain at
TRL4/SRL4. If not, readiness would return to TRL3/SRL3. The discussion of 'how well
we understand the changes' is not always precise and reveals how TRL/SRL can become
a subjective measure.
The second design iteration is based on a core engine test where the newly modified
HPC is integrated with a combustor and HPT (the part of the engine attached to the same
shaft as the HPC). This test is superior to the rig test in terms of realistic boundary
conditions, but costs more. After test results have been modeled, the HPC and its
simulation are TRL5/SRL5. If design modifications take place, the discussions of the
specific changes are required.
3 In reality, engine programs may choose to skip intermediate tests and follow a path such as TRL3/SRL3- TRL6 - SRL6. This example covers each step for illustrative purposes.
70
Incorporating the HPC into a full engine, testing, and recalibrating the simulation
yields a TRL6/SRL6 component and simulation. Once again, this test is more expensive,
but yields better information. Design modifications need to be understood. If major
changes are incorporated into the design with large uncertainty, the readiness returns to
TRL3/SRL3.
4.1.4 'Derivative' Technologies
A 'derivative' technology is created by tailoring/modifying the design of a pre-
existing component for use in a new engine. In the previous example, if the HPC
proceeded through testing with no design modifications, the TRL/SRL would be
unambiguous.TRL/SRL, however, is harder to evaluate when design changes are made.
'Derivative' components follow the same process described in the example above. This
scenario requires detailed discussions of similarities and differences between the original
component and the derivative. Do the differences lie in a functional area where the design
tools and simulations are strong or where they are weak? Fundamentally, the question
reduces to whether testing is required to validate the concept or not. Figure 4-4 provides
an illustration of how derivatives may be migrated from one engine program to another,
increasing the effective TRL/SRL.
71
TRL/SRL of(1) component -- 3 ---------------- 6
PLANNING DEFINITION VALIDATION
LAUNCH ENGINE TESTING EIS
0
0
CD
C,,CD0
CD
CD
CDC-)
PLANNING DEFINITION VALIDATION
LAUNCHENGINE TESTING
TRL/SRL of (1) component ---- 3 or 6 --------- * 6
Some % of the engine components in Program #2 arederivatives of components in Program #1. They areTRL6/SRL6 (have been tested and modeled). The
TRL/SRL of the derivative will be equal to 3 or 6 basedon the nature of the design changes.
z7
tIl
EIS
4.1.5 Interview Process to Define TRL/SRL
In order to assess the technology and simulation readiness for the components of the
six Pratt & Whitney engine programs, interviews were conducted with two types of
engineers. The first category of people included engineers who had responsibility for the
design and aerodynamic performances of their respective components. Five people were
interviewed (an expert for each component). The second category of people included
system engineers responsible for the functional integration of the engine during the
product planning and validation phases. Four people were interviewed based on their
experience on these engine programs.
For each engine/component combination, interviewees were asked to assess the
TRL/SRL at the time of engine launch. Questions included: What was the best source of
test data available at the time (none, rig, core, or full engine)? Was the data analyzed and
captured in a model prior to launch? If the component was a derivative, in what
functional areas did the new design change the component's performance? How strong
are the models in the areas that were changed? These questions were discussed in the
context of the processes outlined in Figure 4-3 and in Figure 4-4. As a result, all
components were assigned a TRL/SRL level that ranged from 3 to 6. Note that
interviewees were asked to recall information regarding engine programs that took place
many years prior. It is the author's belief, however, that this is not a source of error in this
analysis. Interviewees were selected based on their personal experience in these engine
programs. It was difficult to specify whether there was any difference between TRL and
SRL. Interviewees provided a single number to represent both TRL and SRL. The
73
forthcoming analysis does not differentiate between technology and simulation readiness,
and uses TRL to represent both indices.
Results from interviews reflected two schools of thought regarding derivative
technologies. Design engineers tended to see derivative technologies as existing within
the knowledge base that was incorporated into the design system. System integration
engineers tended to be more skeptical regarding the ability of the design tools and
simulations to capture the differences associated with derivatives. When technologies
were not derivatives, the two groups agreed on the appropriate level of TRL/SRL.
The differing interpretations of TRL are consistent with their respective areas of
responsibility. An HPC designer focuses on performing his design task with boundary
conditions that are assumed to be accurate. Typical assumptions include the shape of the
pressure profile leaving the LPC, the thermal environment around the case (affecting
clearances between case and rotating blades), and the operating line (partly a function of
the HPT). Component designers are experts and are in the best position to make technical
assessments of component performance. The system integration engineer is more likely
to be thinking about the interfaces and boundary conditions. These differences highlight
the need for different groups to understand the other group's 'language'.
4.1.6 Aggregating Component TRL's to Engine TRL
Functionally, the gas turbine engine is highly integral (non-modular) as each
component's performance is dependent on how the other components are performing. As
mentioned in Chapter 2, each component has a design point at which it is most efficient.
Operation at that point is contingent on the other components establishing the boundary
conditions that are expected. To capture this dependency, for each component, a new
74
TRL was calculated reflecting weighted influences of the component TRL itself, and the
impact of the TRL's of the other components on it.
TRLADJFAN (WFAN)( TRLFAN) + (WLPC-FAN)( TRLLPC) + (WHPC-FAN)( TRLHPC) + (WHPT-
FAN)(TRLHPT) + (WLPT-FAN)( TRLLPT)
TRLADJLPC = (WFAN-LPC)( TRLFAN) + (WLPC)( TRLLPC) + (WHPC-LPC)( TRLHPC) + (WHPT-
LPC)(TRLHPT) + (WLPT-LPC)( TRLLPT)
TRLADJHPC = (WFAN-HPC)( TRLFAN) + (WLPC-HPC)( TRLLPC) + (WHPC)( TRLHPC) + (WHPT-
HPC)(TRLHPT) + (WLPT-HPC)( TRLLPT)
TRLADJHPT = (WFAN-HPT)( TRLFAN) + (WLPC-HPT)( TRLLPC) + (WHPC-HPT)( TRLHPC) +
(WHPT)(TRLHPT) + (WLPT-HPT)( TRLLPT)
TRLADJLPT = (WFAN-LPT)( TRLFAN) + (WLPC-LPT)( TRLLPC) + (WHPC-LPT)( TRLHPC) +
(WHPT-LPT)(TRLHPT) + (WLPT)( TRLLPT)
TRL are acquired from personal interviews. The calculations were performed oncefor the input from designers and once for the input from system integrators.
wx-y is the weighting of the influence of component X on component Y. Weightingsare generated from aero-thermal engine simulations that model the dependencies ofone component's performance on another. These simulations are based on actualengine test data. The sum of the weightings for a given component = 1.
TRLADJ is the weighted TRL.
As an example of the dependency between components, the HPT design efficiency is
a strong function of high rotor speed (N2) and turbine inlet temperature (T4). If the HPC
has a different efficiency than expected, its exhaust temperature (T3) will be different. As
a result, all downstream temperatures including T4 will change. The change in T4 will
push the point where the engine is operating off of the HPT's design point.
75
Each component impacts the engine's TSFC differently. To reflect the unequal
dependence of TSFC on components, the engine TRL is a weighted average of the
component TRLADJ's calculated above:
TRLTSFC ENGINE = (WFAN-TSFC)( TRLADJFAN) + (WLPC-TSFC)( TRLADJLPC) + (WHPC-TSFC)(
TRLADJHPC) + (WHPT-TSFC)(TRLADJHPT) + (WLPT-TSFC)( TRLADJLPT)
TRLADJ are calculated above.
WA-TSFC is the weighting of the influence of component A on engine TSFC.Weightings are generated from same aero-thermal engine simulations describedabove. This relationship is separate from the relationship above for intercomponentdependencies. The sum of the weightings = 1.
The resulting engine TRL for each of the six engine programs is plotted relative to the
TSFC gap measured at the First Engine To Test (FETT) in Figure 4-5, below.
76
PRATT & WHITNEY GAS TURBINE ENGINE DEVELOPMENTPROGRAMS
1.0 r I
S0.8
0.6
0.4
0.2
3 3.5 4 4.5 5 5.5 6Engine Technology and Simulation Readiness Level
Figure 4-5. FETT TSFC gap as a function of TRL.
The interview results support the hypothesis that TRL can serve as proxy for
uncertainty of predictions for engine performance. Residual uncertainties associated with
a linear data fit are approximately 0.5 for both sets of data. Recall that TSFCSPEC refers to
the performance promised to the airframers and airlines after the Product Planning phase
at Passport Review II. TSFCFETT refers to the performance demonstrated at the first
engine to test. The normalized difference, therefore, represents the gap between
commitment and actual engine capability. When commitments are made with low
certainty regarding component performance, the engine is more likely to fall short, and
by a somewhat predictable amount.
77
Engine Designers' Definition of TRLTSFC gap = -0.31 (TRL) + 1.9, R2
= 0.49
System Integrators' Definition of TRLTSFC gap = -0.45 (TRL) + 2.0, R2 = 0.53
- --
U +
Notice that although the two engineering groups participating in interviews disagreed
over the absolute level of TRL, they agreed over which engines possessed higher or
lower TRL. Design engineers would predict that engines launched with a TRL ~ 6 are
likely to meet their commitments. System integration engineers would predict that
engines launched with a TRL of 4.5 would meet commitments.
4.2 Design Flexibility & Iteration
This section tests the hypothesis that once the (FETT) has demonstrated its
performance relative to requirement and problems are identified at the component level,
the ability to react is driven by the flexibility given to each component center. Note that
after the FETT, the variation (a) of TSFC is approximately 0. All recovery efforts
seeking to lower risk need to drive a mean shift (ji). Flexibility can be segmented into two
types.
1. If parallel design efforts were pursued (and parallel engine tests), there exists more
than one option to choose from. This would be a direct application of real options.
2. Given a fixed amount of time available for design improvement, maximize the quality
and frequency of iterations.
There is minimal evidence of parallel options in any of these six programs. Design
engineers explained that this would be expected. Design begins for each component with
specific requirements in terms of pressure ratio, temperature ratio, and airflow.
Standardized design tools promote the convergence towards a single "optimal" design.
Similarly to Toyota, Pratt & Whitney entertains design-sets during conceptual design and
typically constructs only a single full system design for validation. In this environment,
flexibility during the validation phase comes from the quality and frequency of iterations.
78
For the six engine programs under review, the type of work that is required during
iteration in the validation phase is a function of the specific problems identified during
the FETT. Nevertheless, flexibility in the face of these problems can be defined as the
product of the frequency and quality of iterations. In a typical development program, test
planners can react to a performance shortfall at FETT and allocate roughly six months for
design iteration without impacting the Entry Into Service (EIS) date. This time for
iteration, which is maintained in the validation phase schedule, is based on the
recommendations of planners with years of engine development experience. 36
The frequency of iteration is a function of the analysis and hardware lead times
required to generate an improved design. The quality of the iteration is significantly
more subjective. Sometimes, the single 'data point' from the FETT is not enough to fully
understand problems with a component. Iteration will help to provide more data but
might not 'fix' the problem. On a relative basis, when designers are given more time and
more freedom to make 'large' changes, there is more opportunity to have 'large' impacts
on the component's performance. Figure 4-6 helps to highlight the lead times associated
with various changes in each component.
36 Interviews with PW program planners.
79
Column #1 lists the engine components.Column #2 lists for each component, the major types of design iterations possible.Column #3 lists the process lead times associated with each change, including
analysis. This is the time required to exercise the option. 37They are listed in orderof 'to what degree' a given design can be modified by the iteration. For the fan, aforging change permits the designer significantly greater latitude in modifying thefan's design.
Column #4 divides the lead time into the amount of time available (6 months) andreports the integer number of iterations possible in that time. A (0) for example,indicates that there is insufficient time allocated for that type of change. In eachcase, the number of 'high impact' changes was maximized.
Column #5 assigns a subjective scale (1-3) to the 'impact capability' of each change.See fan forging versus machining example above. A (3) indicates a large changeis possible, (2) - medium, and (1) - minor.
Column #6 reports the product of column #4 and column #5. This reflects the potencyof a given change.
Column #7 sums the values in Column #6 for each component. This sum is asubjective representation of the design flexibility for each component.
CHANGE OPTION LEAD # POSS. SCALE (#ITER)x FLXTIME ITER. (SCALE)
FAN Forging 8 mo. 0 3 0Machining 5 mo. 1 2 2 2Minor mach. 2 mo. 0 1 0
LPC Forging / mach. 5 mo. 1 3 3 41st stator chng 1 mo. 1 1 1
HPC Moderate aero chng. 5 mo. 1 2 2 4Variable vane optimization ~0 1 2 2
HPT Tooling / casting / machining 8 mo. 0 3 0 1Cutback / restagger '/2 mo. 1 1 1
LPT Tooling / casting / machining 8 mo. 0 3 0Cutback / restagger '/ mo. 1 1 1
Figure 4-6. Flexibility for iterations on engine component design.
As illustrated by product development paths B and D in Figure 1-8, soon after
product launch, program managers can decide to 'build-in' additional design flexibility.
37 This represents a significant departure from financial options, which can be purchased instantaneously.
80
The most common example of this is to modify the production process for airfoils used in
the compressors and turbines. When producing airfoils in a production process, it is
cheapest to create airfoil forgings that minimize the amount of machining required. If
more metal is left on the forgings, more costly machining is required. As is seen in Figure
4-6, the lead time for changing forgings is significantly longer than changing the
machining process. Therefore, it follows that by using forgings with more metal, the
additional machining cost gives design engineers more flexibility to change the airfoil
shapes.
There is relatively more design change flexibility in the compression system than in
the turbines. These flexibilities are now compared to actual engine program data to test if
a subjective flexibility scale can serve as a proxy for predicting the effectiveness of the
validation phase in improving TSFC.
For each engine program, a rigidity index (opposite of flexibility) was calculated to
reflect the potential effectiveness of iteration. The index is a ratio of the TSFC gap
measured at the FETT relative to the flexibility numbers reported in Figure 4-6. Because
flexibility is specific to a given component, analysis had to be performed to understand
which components caused the TSFC gap. A fictitious example is used to illustrate how
the calculation is performed.
The FETT misses its TSFC guarantee by 1%. Through analysis of engine data 50% ofTSFC miss was attributed to the fan, 25% to the LPC, and the remaining 25% to theLPT. The HPC and HPT contributed nothing. The sum of the contributions = 100%.Divide each component % by its flexibility.
Rigidity = (25%) / (flexFAN) + (25%) / (flexLPC) + (0%) / (flexHpc) + (0%) / (fleXHPT)+ (25%) / (flexLPT) = (50%/2) + (25% / 4) + 0 + 0 + (25% / 1) = 56.25%
81
The hypothesis was that engines with high rigidity numbers (close to 100%) would be
relatively unable to improve the performance of the engine during the validation phase.
For five of the six engine programs rigidity is plotted relative to the TSFC improvement
that was accomplished during the validation phase.
PRATT & WHITNEY GAS TURBINE ENGINE DEVELOPMENTPROGRAMS
1.0
TSFC Improvement =-0.64 (Rigidity)+ 0.66, R 0.38
0.8
0.6
0.4
0.2
n
0.0 0.2 0.4 0.6 0.8 1.0
Rigidity [Performance gap / flexibility]
Figure 4-7. TSFC Improvement versus Iteration Rigidity
The data shows a loose (R2 = 0.38) correlation between TSFC improvement and
iteration rigidity. [Note that one engine was omitted from the grouping. Its performance
at FETT met its required fuel consumption. Later some of this performance was "traded"
to improve compressor stability.] This data suggests that allowing time for iteration
82
improves the chances of being able to improve the TSFC during the validation phase. It
also suggests that deliberately providing flexibility (paths B & D), even at increased cost,
can be rewarded by permitting more ways to cut the TSFC gap.
4.3 Chapter Summary
This chapter investigated six engine development programs at Pratt & Whitney and
tested the degree to which technology readiness, design flexibility, and iteration impacted
the risk of meeting TSFC guarantees.
The concept of Simulation Readiness was introduced as a required intermediate step
between Technology Readiness and decision-making. Technology readiness levels were
assigned to each engine program that corresponded to the quality of information at the
end of the Product Planning phase. Component TRL/SRL's were based on interviews
with component designers and systems engineers. These were aggregated to an engine
TRL/SRL based on the thermodynamic relationships between components and the impact
of the components on TSFC. The data supported the hypothesis that TRL/SRL can serve
as a proxy for the probability that the FETT will meet its performance guarantees.
The concepts of design flexibility and iteration were combined into a single metric
that aimed at reflecting Pratt & Whitney's ability to react within a fixed schedule. This
rigidity index reflected how much iteration was possible in the areas that needed to be
fixed in order to improve TSFC. The rigidity index was a function of the specific
problems that caused the TSFC gap. The data supported the hypothesis that increased
83
iteration flexibility correlates with the ability to improve TSFC. It is important, therefore,
to attempt to anticipate areas of risk. These areas should be given additional flexibility.
When viewing this as a control process that attempts to converge on a required level
of TSFC, some efforts focus on decreasing signal noise (variation) and some focus on
controlling the signal (TSFC) itself. Variation was shown to be a function of the quality
of information. Controlling the signal was contingent on eliminating noise and retaining
the flexibility to modify it.
Recall from the gated decision process present in product development, there is
always the option to cancel the engine program if the costs and risks rise above an
unacceptable level. Using this historical data can provide some quantitative insight into
what the expected TSFC gap will be and whether the engine program should be allowed
to proceed.
In summary, this case study provided a more formal mechanism for systematically
addressing the risk associated with meeting TSFC guarantees:
Path Main Goal Method Risks Most Useful WhenA Baseline Baseline BaselineB Allow p shift Spend to get TSFC gap is too High use of
after FETT options and large within derivativeflexibility schedule and $ components
constraintsC Reduce cy at Use TRL/SRL to Simulations are Low use of
Passport Gate make better not accurate derivativeII promises components
D Combine B&C Both Both Least schedulep pressures
Figure 4-8. Summary of results.
84
5 CONCLUSIONS and FOLLOW-ON ACTIVITY
5.1 Applicability of Case Study
5.1.1 Severity of Failure
The previous section provided data that supported the hypotheses that
" engine programs that are launched with higher technology readiness levels are
more likely to meet TSFC commitments, and
" validation programs that maximize the quality and frequency of iterations are
more likely to be able to react to performance shortfalls
These statements address the probability offailure. The case study did not deal directly
with the severity of missing TSFC guarantees. It did not provide guidance as to how
much Pratt & Whitney should be willing to invest to avoid this problem. To understand
the cost-benefit and marginal reduction of risk associated with improving TSFC 1%, we
need to revisit the comprehensive list of objectives for an engine program:
" Cost of delivery.
" Cost of ownership.
" Engine weight.
* Environmental impact.
" On-time completion of development.
* TSFC
For example, consider the relationship between TSFC and cost of ownership at the
two decision points discussed in the case study. When deciding whether to launch a new
engine program, Pratt & Whitney considers the airlines' operating cost structure. In 1998,
85
12.5% of an average airline's costs were in fuel and 10% were associated with
maintenance. 38
This point can be further illustrated by comparing airlines that operate a high
frequency of flights on short flight missions relative to airlines that operate low
frequencies on longer flight legs (e.g. trans-Atlantic). The latter airline is more concerned
with fuel burn, while the former airline is perhaps more concerned with the maintenance
interval. When making a configuration decision, Pratt & Whitney can trade engine
performance for maintenance cost. For reasons such as this, the cost-benefit of 1% of
TSFC is different between engines, and even different as a function of time.
5.1.2 TRL - Subjective Metric
Interviews with members of the systems engineering organizations that are
responsible for establishing fuel consumption targets during conceptual design and for
tracking performance throughout testing revealed a perception of strong political and
cultural barriers to adopting an honest process of committing to feasible technical goals.
The current process allows for inflation of component and system performance targets
that meet market demands, but are not supported by a technical development plan that
rationally and quantifiably assesses risk.
This tension reinforces the need for Pratt & Whitney to continually develop
technologies that will allow competitive market offerings. The failure to invest in
performance improving technologies will further increase the tensions between what the
market demands and what the company is capable of.
TRL/SRL are subjective, particularly with regard to derivative technologies. It is
important to see TRL/SRL as a communication tool for discussing risk. It has the
38 Greenslet, Ed. "World Airlines: Year in Review." Interavia, Vol 54, Issue 632 (June 1999), 42-45.
86
potential to allow discussions to migrate from pass/fail to a discussion of 'how much risk
can we tolerate.'
5.2 Follow-On Work
Engine performance has been improving consistently since the 1960's at a rate of 1%
per year. With engines in service for more than 30 years, the bulk of the revenue stream
comes from overhaul activities associated with normal maintenance. Engine companies
are often willing to promise more than they are capable of in order to gain entry to a
market. This highly competitive environment requires that engine makers understand the
relative cost-benefit of several technology insertion strategies:
1. Delay engine launch, but invest in increasing the technology and simulation
readiness levels. This would reflect a cultural shift at Pratt & Whitney, where
money is typically allocated through specific engine programs that present
business plans that aim to increase shareholder value. Increasing the readiness
level of different engine technologies would occur prior to a commitment and
would not have the luxury of being part of an engine-specific business plan. They
would have to be funded separately. PW needs to be careful to protect 'research'
budgets for maturing performance enhancing technologies.
2. Performance shortfalls can also be addressed with follow-on engine improvement
programs that aim to cut into the liabilities associated with the initial engine
offering. Future work should lok into the cost-benefit of promising more than the
company is currently capable of. Costs would include the development costs
associated with all follow-on engine development work required to improve the
performance, penalty fees during that time, and the loss in reputation. The benefit
87
would be in terms of the size of the initial market share combined with future
overhaul revenue.
A second area of follow-on work would look to see how risk can be managed with
regard to other major engine requirements such as engine weight and maintenance
interval. To what degree are these parameters defined by decisions made in the product
planning phase? If so, what is the quality of information used in these predictions? What
is the set of simulation tools available and what is their SRL? What methods are used to
increase their SRL? To what degree are designers and system engineers able to react to
shortfalls in maintenance interval and engines that are over weight?
88
GLOSSORY
EIS Entry into ServiceFAA Federal Aviation AdministrationFETT First Engine to Test (First engine system test)FNT Net Thrust (normally expressed in pounds force)GE General ElectricHPC High Pressure CompressorHPT High Pressure TurbineJAA Joint Airworthiness AuthorityLPC Low Pressure CompressorLPT Low Pressure TurbineNI Low Rotor Speed (normally expressed in rpm)N2 High Rotor Speed (normally expressed in rpm)PDP Product Development ProcessPW Pratt & WhitneySPEC Engine specification (requirements)SRL Simulation Readiness LevelTRL Technology Readiness LevelTSFC Thrust Specific Fuel ConsumptionG standard deviationp meanrjP Propulsive EfficiencynT Thermal Efficiency
89
BIBLIOGRAPHY
Augustine, Norman R. Augustine's Laws. New York: Viking Penguin, 1983.
Banks, Howard. "Engine Trouble." Forbes, Vol. 156, Issue 6 (September 11, 1995), 156.
Banks, Howard. "Hit the Fan." Forbes, Vol. 157, Issue 1 (January 1, 1996), 14.
Bartkowski, Glenn D. "Accounting for System Level Interactions in KnowledgeManagement Initiatives." SM Thesis, Massachusetts Institute of Technology, 2001.
Bernstein, Joshua I. "Design Methods in the Aerospace Industry: Looking for Evidenceof Set-Based Practices." SM Thesis, Massachusetts Institute of Technology, 1998.
Browning, Tyson R. "Modeling and Analyzing Cost, Schedule, and Performance inComplex System Product Development." PhD Thesis, Massachusetts Institute ofTechnology, 1999.
Czarnecki, Sharon H. and Wayne A. Johnson. "Complex Organizational Metric ForProgrammatic Risk Environments (COMPRE): A Model Assessment Overview."NASA.
Dornheim, Michael A. "Aerospace Corp. Study Shows Limits of Faster-Better-Cheaper."Aviation Week & Space Technology, (June 12, 2000), 47.
Glynn, Steven V. and Thomas Pelland. "Information Flow & Knowledge CaptureLessons for Distributed Integrated Product Teams." SM Thesis, MassachusettsInstitute of Technology, 2000.
Greenslet, Ed. "World Airlines: Year in Review." Interavia, Vol 54, Issue 632 (June1999), 42-45.
Hague, Douglas C. "Description of a Turbofan Engine Product Development Process."SM Thesis, Massachusetts Institute of Technology, 2001.
Kandebo, Stanley W. "Pratt & Whitney Suspends PW4173 Engine Program." AviationWeek & Space Technology, Vol. 152, Issue 7 (February 14, 2000), 63.
Mascoli, Gregory J. "A Systems Engineering Approach to Aero Engine Development in aHighly Distributed Engineering and Manufacturing Environment." SM Thesis,Massachusetts Institute of Technology, 1999.
Mattingly, Jack D. Elements of Gas Turbine Propulsion. New York: McGraw-Hill, 1996.Moy, Habs M. "Commercial Gas Turbine Platform Strategy and Design." SM Thesis,
Massachusetts Institute of Technology, 2000.Pugh, S. Total Design. Reading, Massachusetts: Addison Wesley, 1991.Rowles, Craig M. "System Integration Analysis of a Large Commercial Aircraft Engine."
SM Thesis, Massachusetts Institutq of Technology, 1999.Sarsfield, Liam. The Cosmos on a Shoestring: Small Spacecraft for Space and Earth
Science. Santa Monica, California: RAND, 1998.Shigley, J.E. and C.R. Mischke. Mechanical Engineering Design. 5th edition. New York:
McGraw-Hill, 1989.Slack, Robert A. "The Application of Lean Principles to the Military Aerospace Product
Development Process." SM Thesis, Massachusetts Institute of Technology, 1999.
90
Smith, Preston G. and Donald G. Reinertsen. Developing Products in Half the Time. NewYork: Van Nostrand Reinhold, 1991.
Sullivan, John P. "The Relationship Between Organizational Architecture, ProductArchitecture and Product Complexity", SM Thesis, Massachusetts Institute ofTechnology, 1999.
Thomas, L. Dale and Robert A. Mog. "A Quantitative Metric of System DevelopmentComplexity." NASA.
Thomas, L. Dale and Robert A. Mog. "A Quantitative Metric of System DevelopmentComplexity: Validation Results." NASA.
Thomke, Stefan and David Bell. "Optimal Testing Under Uncertainty." Unpublished.Harvard Business School, 1998.
Utterback, James M. Mastering the Dynamics of Innovation. Boston: Harvard BusinessSchool Press, 1994.
Ward, Allen, Jeffrey K. Liker, John J. Cristiano, and Durward K Sobeck II. "The SecondToyota Paradox: How Delaying Decisions Can Make Better Cars Faster." SloanManagement Review, Vol. 36, Issue 3 (Spring 1995), 43.
Wheelwright, Steven C. and Kim B. Clark. Revolutionizing Product Development. NewYork: The Free Press, 1992.
91