aditya p. mathur professor, department of computer science, associate dean, graduate education and...
Post on 20-Dec-2015
220 views
TRANSCRIPT
Aditya P. Mathur
Professor, Department of Computer Science,
Associate Dean, Graduate Education and International
Programs
Purdue University
Monday December 20, 2004. University of Paderborn, Paderborn, Germany.
Software Cybernetics:Progress and Challenges
Cybernetics
[n] the field of science concerned with processes
of communication and control (especially the
comparison of these processes in biological and
artificial systems)
Software Cybernetics
[n] the field of science concerned with processes of communication and control in software systems.
Sample Problems
Control of the software test process:How much and what additional effort is to be applied to achieve the desired quality objective under time/cost constraints?
Optimal selection of tests:What is an optimal set of tests for achieving the desired quality objective given time constraints?
Sample Problems (continued)
Software performance controlHow best to adjust software parameters so that an optimal level of performance is maintained?
Control of the software development process:What is an optimal set of process variables required to achieve delivery objectives within cost/time constraints?
Approaches
1. Use instinct and experience.
2. Use (1) supported by quantitative tools.
(b) Use simulation: “forward” approach.
(c) Use (a) plus feedback control: “inverse” approach.
Software cybernetic approach..
(a) Use simulation: “forward” approach.
Closed Loop (feedback) Control
Specifications
ProgramEffort +
f(e)Additionaleffort What is f ?
-
RequiredQuality
rQ
ObservedQuality
oQ
oQr
Qe −=
Sample Problem Scenario
cp1 cp2 cp3 cp4 cp5 cp6 cp7 cp8 cp9
cpi = check point i
rf
schedule set bythe manager
Approximation of how r is likely to change
r0 observed
deadline
r -
num
ber
ofre
mai
ning
err
ors
t- time
t0
Our Approach
Controllerrerror(t)
’
w’f
+
+
wf+wf
+
wf+wf
+
robserved(t)
rexpected(t)
Actual STP
sc r0
STP State Model
sc r0
Initial Settings(wf,)
wf
Test Manager
wf: workforce : quality of the test process
Decision Support via Feedback
ActualProcess
ParameterEstimation
ProgressMetrics
Process Model
ModelParameters
FeedbackController Estimated Future
Schedule Deviation
Sug
gest
edD
ecis
ion
Cha
nges
Mgmt.Control
ManagementDecisions
-Predicted Schedule
-
+Desired Schedule
Act
ual
Sch
edul
e
+
EstimationError
Challenges (Sources of difficulty)Physical Logical (software)
Laws of physics Yes No
Object controlled Physical Logical (and Humans)
Time dependence Stationary and non-stationary
Non-stationary
Rate of change Very slow to very fast
Slow
Parameter variability Relatively low High
Overcoming the challenge: Understanding the problem
Partnership amongst researchers and practitioners.
Is the problem real?
Are the existing solutions adequate?
Overcoming the challenge: Solving the problem
Parternership amongst researchers and practitioners.
Develop realistic models
Develop parameter estimation methods.
Develop ways to incorporate parameter estimation into the development process.
Details and Case Studies
Razorfish
Guidant Corporation
Sun Microsystems
Physical and Software Systems: An Analogy
Dashpot
Rigid surface
External force
Xequilibrium
X: Position
Number of remainingerrors
Spring Force
Effective Test Effort
Block
Software
Mass of the blockSoftware
complexity
Quality of thetest process
Viscosity
Xcurrent
SpringTo err isHuman
Physical Systems: Laws of Motion [1]
First Law:
Every object in a state of uniform motion tends to remain in that state of motion unless an external force is applied to it.
Does not (seem to) apply to testing because the number of errors does not change when no external effort is applied to the application.
Physical Systems: Laws of Motion [2]
Newton’s Second Law:
The relationship between an object's mass m, its acceleration a, and the applied force F is F = ma.
CDM First Postulate:
The relationship between the complexity Sc of an application, its rate of reduction in the number of remaining errors, and the applied effort E is E=Sc .
r..
Physical Systems: Laws of Motion [3]
Third Law:
For every action force, there is an equal and opposite reaction force.
When an effort is applied to test software, it leads to (mental) fatigue on the tester.
Unable to quantify this relationship.
CDM First Postulate
The magnitude of the rate of decrease of the remaining errorsis directly proportional to the net applied effort and inverselyproportional to the complexity of the program under test.
€
˙ ̇ r = E
sc⇒ E = ˙ ̇ r sc
This is analogous to Newton’s Second Law of motion.
CDM Second Postulate
The magnitude of the effective test effort is proportional to theproduct of the applied work force and the number of remaining errors.
for an appropriate .
Analogy with the spring:
Note: While keeping the effective test effort constant, a reduction in r requires an increase in workforce.
€
eet = ζwf r
CDM Third Postulate
The error reduction resistance is proportional to the errorreduction velocity and inversely proportional to the overallquality of the test phase.
€
er = ξ1
γ˙ r
for an appropriate .
Analogy with the dashpot:
Note: For a given quality of the test phase, a larger error reduction velocity leads to larger resistance.
State Model
€
˙ r
˙ ̇ r
⎡
⎣ ⎢
⎤
⎦ ⎥=
0 1
−ζ wf
sc−
ξ
γ sc
⎡
⎣
⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥
r
˙ r
⎡
⎣ ⎢
⎤
⎦ ⎥+
0
1
sc
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥Fd
r
˙ r
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥=
1 0
0 1
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
r
˙ r
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
Fd: Disturbance
x(t) = Ax(t) + B u(t).
etr eeE −=Force (effort) balance equation:
Computing the feedback: Question
Question:
What changes to the process parameters will achieve the desired r(T+T) ?
r(T): the number of remaining errors at time T
r(T+T): the desired number of remaining errors attime T+T
Given:
Computing the feedback: Answer
From basic matrix theory:
The largest eigenvalue of a linear system dominates the rate of convergence.
Hence we need to adjust the largest eigenvalue of the system so that the response converges to the desired value within the remaining weeks (T). This can be achieved by maintaining:
teTrtTr −=+ max)()( λObtain the desired eigenvalue.
Computing the feedback-Calculations (λmax)
Compute the desired λmax
teTrtTr −=+ max)()( λGiven the constraint:
We know that the eigenvalues of our model are the roots of its characteristic polynomial of the A matrix.
Computing the feedback-Calculations (λmax)
[ ]
c
f
c
cc
f
s
w
s
ss
wAI
ˆ
ˆ
ˆ
ˆ1
detdet
2 ζλ
γ
ξλ
γ
ξλ
ζλ
λ
++=
⎥⎥
⎦
⎤
⎢⎢
⎣
⎡
+
−=−
fwff wwandwhere +=+= ˆˆ
We use the above equation to calculate the space of changes to w and such that the system maintains its desired eigenvalue.
f
A Flow Model of Incremental Software Development/Test
TestSpecs Test
Authoring
Feat.Specs
TestCode
Coding
TestVerification
& Correction Reg.Cases
RegressionTesting
KnownDefects
CodeDebugging
ProjectCode
LatentDefects
Workforce Allocation
Workforce allocated to particular tasks Effort is split across all active tasks
State-Model [Example] Equations
tkPkudtkud ddTT ⋅−=+ )()()(
System State Progress
Feature Coding (fc) Code Debugging (dr) Test Authoring (ta) Test Debugging (td) Regression Testing (rr)
Defect Model Development Testing
tkPkfctkfc fc ⋅+=+ )()()(
tkPkudtkud diDD ⋅+=+ )()()(
cn
bb w
w
rcrcP
⋅+−= )1(
Variable Productivity Equation
Human Productivity Workload Dependent (Csikszentmihalyi,’88)
•rb – Base Work rate
•c – Fractional size-dependent increase
•wc – Current workload size
•wn – Nominal workload size
The “Productivity” Eqns.
[ ]))()(()1()()( kfckfcwrcrcdkwfkP goalfc
fcfcfcfcifcfc −⋅−= +
Process Productivity (E.g. Feature Coding)
Defect Introduction
Defect Detection (Cangussu et al., ’02)
)()()( kPikPikP drdrfcfcdi ⋅+⋅=
[ ])()()()( . kPkPkudckP rrtdTedd +⋅=
Control Strategy
Model Predictive Control
timek k+Pk0
Desired behavior
Actual Behavior
Control Input Prediction Horizon
Predicted FutureBehavior
CalculatedControl Input
2,2,
21
,)(Q
Pkk
Q
Pkkd uxxkJ Pkk
p
++ +−= +
Model Predictive Control Select Cost Functional
E.g. Q1,Q2 := positive definite
Calculate
where S[xk, uk,k+P] xpk,k+P
)(min,
kJPkku +