A Model-based Approach to Reactive Self-Configuring Systems

Brian

AbstractThis paper describes Livingstone, an implemented ker-nel for a self-reconfiguring autonomous system, that isreactive and uses component-based declarative mod-els . The paper presents a formal characterizationof the representation formalism used in Livingstone,and reports on our experience with the implementa-tion in a variety of domains . Livingstone's represen-tation formalism achieves broad coverage of hybridsoftware/hardware systems by coupling the concur-rent transition system models underlying concurrentreactive languages with the discrete qualitative rep-resentations developed in model-based reasoning . Weachieve a reactive system that performs significant de-ductions in the sense/response loop by drawing on ourpast experience at building fast propositional conflict-based algorithms for model-based diagnosis, and byframing a model-based configuration manager as apropositional, conflict-basted feedback controller thatgenerates focussed, optimal responses . Livingstoneautomates all these tasks using a single model and asingle core deductive engine, thus making significantprogress towards achieving a central goal of model-based reasoning . Livingstone, together with the HSTSplanning and scheduling engine and the RAPS exec-utive, has been selected as the core autonomy archi-tecture for Deep Space One, the first spacecraft forNASA's New Millenium program .

Introduction and DesiderataNASA has put forth the challenge of establishing a"virtual presence" in space through a fleet of intelli-gent space probes that autonomously explore the nooksand crannies of the solar system . This "presence" isto be established at an Apollo-era pace, with softwarefor the first probe to be completed late 1996 and theprobe (Deep Space One) to be launched in early 1998 .The final pressure, low cost:, is of an equal magnitude .Together this poses an extraordinary opportunity andchallenge for Al . To achieve robustness during years inthe harsh environs of space the spacecraft will need toradically reconfigure itself in response to failures, andthen navigate around these failures during its remain-ing days . To achieve low cost and fast deployment, one-of-a-kind space probes will need to be plugged together

274 QR-96

C . Williams and P. Pandurang NayakRecorn Technologies

NASA Ames Research Center, MS 269-2Moffett Field, CA 94305 USA

E-mail : will iams,nayak(aptolemy .arc . nasa .gov

quickly, using component-based models wherever pos-sible to automatically generate the control softwarethat coordinates interactions between subsystems. Fi-nally, the space of failure scenarios a spacecraft willneed to entertain over its lifespan will be far too largeto generate software before flight that explicitly enu-merates all contingencies . Hence the spacecraft willneed to think through the consequences of reconfigu-ration options on the fly while ensuring reactivity .

We made substantial progress on each of these frontsthrough a system called Livingstone, an implementedkernel for a self-reconfiguring autonomous system, thatis reactive and uses component-based declarative mod-els . This paper presents a formal characterization of areactive, model-based configuration management sys-tem underlying Livingstone . Several contributions arekey : First, our modeling formalism represents a radicalshift from first order logic, traditionally used to char-acterize model-based diagnostic systems . Our repre-sentation formalism achieves broad coverage of hybridsoftware/}rardware systems by coupling the concurrenttransition system models underlying concurrent reac-tive languages (Manna & Pnueli 1992) with the dis-crete qualitative representations developed in model-based reasoning . Reactivity is respected by restrict-ing the model to concurrent propositional transitionsysterns that are synchronous . Second, this approachhelps to unify the classical dichotomy within Al be-tween deduction and reactivity. We achieve a reac-tive system that performs significant deductions in thesense/response loop by drawing on our past experi-ence at building fast propositional conflict-based al-gorithms for model-based diagnosis, and by framing amodel-based configuration manager as a propositional,conflict-based feedback controller that generates fo-cussed, optimal responses . Third, the long held visionof the model-based reasoning community has been touse a single central model to support a diversity ofengineering tasks . For model-based autonomous sys-tems this means using a single model to support tasksincluding : monitoring, tracking planner goal activa-t,ions, confirming hardware modes, reconfiguring hard-ware, detecting anomalies, isolating faults, diagnosis,

Figure 1 : Engine schematic

fault. recovery ; safing and fault avoidance . Livingstoneautomates all these tasks using a single model and asingle core deductive engine, thus making significantprogress towards achieving the model-based vision .

Livingstone, tightly integrated with the HSTS plan-ning/scheduling system (Muscettola 1994) and theRAPS executive (Firby 1995), was demonstrated tosuccessfully navigate the simulated NewMaap space-craft into Saturn orbit during its one hour insertionwindow, despite half a dozen or more failures, includ-ing unanticipated bugs in the simulator . Consequently,Livingstone, together with RAPS and HSTS have beenselected as the core autonomy architecture of the NewMillenium program, and will fly Deep Space One early1998 .The rest of the paper is organized as follows . The

next section introduces part of the spacecraft domainand the problem of configuration management, in thisdomain . Section introduces transition systems, thekey formalism for modeling hybrid concurrent systems .It also introduces the basic formalization of configu-ration management . Section discusses model-basedconfiguration management, and discusses its key com-ponents : mode identification and mode reconfigura-tion . Section introduces algorithms for statisticallyoptimal model-based configuration management usingconflict-directed best-first search . Section presents anempirical evaluation of Livingstone using a suite of sce-narios and domains . Conclusions and related work arediscussed in Section .

Example : Autonomous SpaceExploration

Figure 1 shows the schematic of the main engine sub-system of Cassini, the most complex spacecraft built todate . The main engine subsystem consists of a helium

tank, a fuel tank, an oxidizer tank, a pair of main en-gines, regulators, latch valves, pyro valves, and pipes .The helium tank pressurizes the two propellant tanks,with the regulators acting to reduce the high heliumtank pressure to a lower working pressure . When pro-pellant paths to a main engine are open, the pressur-ization of the propellant tanks forces fuel and oxidizerinto the main engine, where they combine and sponta-neously ignite, producing thrust . The pyro valves canbe fired exactly once, i .e ., they can change state (ei-ther from open to closed or vice versa) exactly once .Their function is to isolate parts of the main enginesubsystem until needed, or to isolate failed parts . Thelatch valve states are controlled using valve drivers (notshown), and the accelerometer (Acc) senses the thrustgenerated by the main engines . In the figure, valves insolid black are closed, while the others are open .

Starting from the configuration shown in the figure,the high level goal of producing thrust can be achievedusing a variety of different configurations : thrust canbe provided by either main engine, and there are anumber of different ways of opening propellant pathsto either main engine . For example, thrust can be pro-vided by opening the latch valves leading to the engineon the left, or by firing a pair of pyros and opening aset of latch valves leading to the engine on the right .There are a number of other configurations correspond-ing to various combinations of pyro firings . The differ-ent configurations have different characteristics sincepyro firings are irreversible actions and since firing pyrovalves requires significantly more power than changingthe state of latch valves .Suppose that the main engine subsystem has been

configured to provide thrust from the left main engineby opening the latch valves leading to it . Suppose nowthat this engine fails, e.g ., by overheating, so that it.fails to provide the desired thrust . To ensure that thedesired thrust is provided even in this situation, thespacecraft must be transitioned to a new configurationin which thrust is now provided by the main engineon the right. . Ideally, this is achieved by firing the twopyro valves leading to the right side, and opening theremaining latch valves (rather than firing additionalpyro valves) .A configuration manager for the spacecraft con-

stantly attempts to move the spacecraft into lowest,cost configurations that achieve the desired high-levelgoals . When the spacecraft. strays from the chosen con-figuration due to failures, the configuration manageranalyzes sensor data to identify the current configura-tion of the spacecraft, and then moves the spacecraftto a new configuration which, once again, achieves thedesired configuration goals . In this sense a configura-tion manager is a discrete control system that ensuresthat the spacecraft's configuration always achieves theset point defined by the configuration goals .

Configuration goals are generated dyamicallythrough onboard planning, scheduling and execu-

Williams 275

tion capabilities . High-level goals are translated topartially-ordered resource timelines by the HSTS plan-ner/scheduler . RAPS then executes these plans, dy-namically decomposing them into sequences of config-uration goals that are passed to Livingstone . RAPScomplements Livingstone, for example, decomposingplanner tokens, monitoring plan temporal constraintsand coordinating replanning when configuration goalsor temporal constraints cannot be satisfied .

Models of Concurrent ProcessesSelecting a restricted, but appropriately expressive for-malism for describing the plant is essential to achiev-ing the competing goals of achieving reactivity on theone hand and richly expressing the properties of hy-brid software/hardware systems . Extensive experienceapplying model-based diagnosis to causal systems sug-gests that propositional deductive engines with a focus-ing algorithm can be made extremely fast . We know ofno first order formalism that achieves these properties,thus operating over fixed, finite domains is essential .

Reasoning about a component's configurations andautonomous repair requires the concepts of operatingmodes, failure modes, unmodeled failures, operatingmodes, repairable failures and configuration changes .These concepts can be expressed in a state diagram .Note in particular that repairable failures are repre-sented by state transitions from a failure state to anominal state, configuration changes are between norn-inal states, and failures are transitions from a nominalto a failure state .Components operate simultaneously, communicat-

ing over wires . Hence we model components throughconcurrent, communicating transitions systems . Like-wise, for software routines, it is well established thata broad class of reactive languages can be representednaturally as concurrent transition systems communi-cating through shared variables . We use the concur-rent transition system model of Manna & Pnueli andits specification through temporal logic as a startingpoint (Manna & Pnueli 1992) for modeling both soft-ware and hardware .Where our model differs from that, of Nlanna &

Pnueli, is that reactive software modifies its statethrough explicit variable assignments . On the otherhand, a hardware component's behavior in a givenstate is governed by a set of constraints between vari-ables . For digital systems these constraints are overfinite domains, while for continuous components thesedomains are infinite . A variety of experiences apply-ing qualitative modeling to diagnostic tasks for digi-tal systems (Harrischer 1991), copiers and spacecraftpropulsion, suggest that extremely simple qualitativerepresentations over finite domains are quite adequatefor tackling many hardware diagnostic problems . Theadded advantage of using qualitative models is thatthe models nominal behavior are extremely robust. todetailed modeling errors and changes, avoiding false di-

276 QR-96

agnoses . Hence behaviors within states are representedby constraints over finite domains, which in turn areencoded as propositional formula .

Other authors such as Zhang and Mackworth, McIl-raith . Levesque and Reiter, and Poole have been con-sidering related issues in reactive autonomous systems .The major difference between their work and ours isour focus on fast. reactive inference using propositionalencodings over finite domain .

Transition systemsWe model a concurrent process as a transition. system .Intuitively, a transition system consists of a set of statevariables defining the system's state space and a set oftransitions between the states in the state space . Moreprecisely,Definition 1 A transition system S is a tuple(H, E, T), where" H is a finite set of state variables. Each state variableranges over a finite domain .

" E is the feasible subset of the state space .

Eachstate in the state space assigns to each variable in Ha value from its domain .

" T is a finite set of transitions between states . Eachtransition T E T is a function T : E -+ 2E represent-ing a state transforming action, where r(s) denotesthe set of possible states obtained by applying tran-sition T in state s .A trajectory for S is a sequence of feasible states

o- : s o , sl . . . . such that for all i >_ 0, si + l E T(si) forsome T E T. In this paper we assume that. one ofthe transitions of S is designated the nominal tran-sition, with all other transitions being failure tran-sitions . Hence in any state a component may non-deterministically choose to perform either its nominaltransition, corresponding to correct functioning, or afailure transition, which moves to a set of failure states .Furthermore in response to a successful repair action,the nominal transition will move the system from afailure state to a nominal state .A transition system S = (H, E, T) can be naturally

specified using a propositional temporal logic . Suchspecifications are built using state formulae and the Qoperator . A state formula is an ordinary propositionalformula in which all propositions are of the form yk =ek, where yk is a state variable and ek is an element ofyk's domain . Q is the next operator of temporal logicdenoting truth in the next state in a trajectory .A state s defines a truth assignment in the natural

way : proposition yk = ek is true iff the value of yk isek in s . A state s satisfies a state formula 0 preciselywhen the truth assignment corresponding to s satisfiesq . The set of states characterized by a state formula ¢is the set of all states that satisfy 0 . Hence, we specifythe set of feasible states of S by a state formula pS .

Similarly, a transition T is specified by a formulapr, which is a conjunction of formulae p, of the form

(pi =~> OF,, where (D and T are state formulae . Afeasible state sk can follow a feasible state s j in a tra-jectory of S using transition T iff for all formulae p,;,if sj satisfies the antecedent of p,, , then sk satisfiesthe consequent. of p� . A transition 'ri that models aformulae p,, is called a subtransition . Hence taking atransition T corresponds to taking all its subtransitionsTi .Note that specifying transition systems only draws

upon the O operator, above and beyond standardpropositional logic . This severely constrained use oftemporal logic is an essential property of transitionsystems that will allow us to perform deductions re-actively .Example 1 The transition system correspondingto a valve driver consists of 3 state variables{mode, cmdin, cmdout}, where mode represents thedriver's mode (on, off, resettable or failed), cmdinrepresents commands to the driver and its associatedvalve (on, off, reset, open, close, none), and cmdoutrepresents the commands output to its valve (open,close, or none). The feasible states of the driver arespecified by the formula

mode = on

(cmdin = open =t~ cmdout = open)/(cmdin = close =~, cmdout = close)A- (cmdin = open V cmdin = close)

cmdout = nonemode= off

cmdout = none

together with formulae like -(mode = on) V

(mode =of , . . Ahat assert that variables have unique values .The driver's nominal transition is specified by the fol-lowing set of formulae :

((mode = on) V (mode = of) n cmdin = off erOmode = off

((mode = on) V (mode = off) A cmdin - onOmode = on

(mode = failed) A cmdin = reset => Omode = onmode = reset A-(cmdin = reset) =~- Omode = resetmode = failed =t~ Omode = failed

The driver also has a failure transition specified bythe formula Omode = failed.

Configuration managementGiven a transition system and an initial state, config-uration management involves evolving the transitionsystem along a desired trajectory. The combinationof a transition system and a configuration manager iscalled a configuration system . More precisely,

Definition 2 A

configuration

system

is a tuple(S, O, o-),

where S is a transition system,

O is afeasible state of S representing its initial state, and

go, gi, . . - is a sequence of state formulae calledgoal configurations . A configuration system generatesa configuration trajectory o' : s o , sl . . . for S such thatso is O and either si +1 satisfies gi or si + l E r(si) forsorne failure transition T.

Configuration management is achieved by sensingand controlling the state of a transition system . Thestate of a transition system is (partially) observablethrough a set of variables C C_ II . The next. state ofa transition system can be controlled through an ex-ogenous set of variables p C 11 . We assume that p areexogenous so that the transitions of the system do notdetermine the values of variables in p . We also assumethat the values of C in a given state are independent ofthe values of p at that. state, though they may dependon the values of p at the previous state .

Definition 3 A configuration manager C for a transi-tion system S is an online controller that takes as inputan initial state, a sequence of goal configurations, anda sequence of values for sensed variables C, and in-crementally generates a sequence of values for controlvariables p such that the combination of C and S is aconfiguration system .A model-based configuration manager is a configura-

tion manager that uses a specification of the transitionsystem to compute the desired sequence of control val-ues . We discuss this in detail shortly .

Plant transition systemWe model a plant as a transition system composedof a set of concurrent component transition systemsthat communicate through shared variables . The com-ponent. transition systems of a plant operate syn-chronously, that is, at each plant transition every corn-ponent performs a state transition . The motivationfor imposing synchrony is given in the next section .We require that the specification of the plant's tran-sition system be composed out of the specification forits component transition systems as follows :

Definition 4 A plant transition system S = (H, E, T)composed of a set CD of component. transition systemsis a transition system such that ;" The set. of state variables of each transition system

in CD is a subset of II . The plant transition systemmay introduce additional variables not in any of itscomponent transition systems .

" Each state in E, when restricted to the appropri-ate subset of variables, is a feasible state for eachtransition system in CD. This means that for eachC E CD, pS ~_ pe . However, pS can be strongerthan the conjunction of the pc .

" Each transition r E T performs one transition rcfor each transition system C E CD . This means that

Pr t=5 A PTcCECD

The concept of synchronous, concurrent actions iscaptured by requiring that each component performsa transition for each state change . Nondeterminism liesin the fact that each component can traverse either thenominal transition or any of the failure transitions at.

Williams 277

Figure 2 : Model-based configuration management

each state change . Note that the nominal transitionof a plant performs the nominal transition for each ofits components, and that multiple simultaneous fail-ures correspond to traversing multiple component fail-ure transitions .The set of trajectories for a plant transition system

are specified in temporal logic by the formula p, t :

Pat

-

Po n O (PE n Vi (Ai ((DijA Oi (Pobs, n pu,)

where O i (D is used to specify that. (D holds in the ithstate, and is a syntactic short form defined by Oi(OOi-,(D and 00 (D _ (D .

Returning to the example, each hardware compo-nent in the schematic (figure 1) is modeled by a com-ponent transition system similar to that given in Sec-tion . Component communication, denoted by wires inthe schematic, is modeled by shared variables betweenthe corresponding component transition systems .

Model-based configuration managementWe presume that the state of the plant is partiallyobservable, and that the results of actions on the plantinfluence the observables and internal state variablesthrough a set of physical processes . Our focus is onreactive configuration management systems that usea model to infer a plant's current state and to selectoptimal control actions to meet configuration goals .More specifically, a model-based configuration man-

ager uses a plant transition model A4 to determine thedesired control sequence in two stages-mode identi-fication (MI) and mode reconfiguration (MR) . MI in-crementally generates the set of all plant trajectoriesconsistent with the plant transition model and the se-quence of plant control and sensed values . M R usesa plant transition model and the partial trajectoriesgenerated by MI up to the current state to determine

278 QR-96

a set of control values such that all predicted trajecto-ries achieve the configuration goal in the next state .Both MI and MR are reactive . MI infers the current

state from knowledge of the previous state and obser-vations within the current state . M R only considersactions that achieve the configuration goal within thenext state . Given these commitments, the decision tomodel component transitions as synchronous is key.An alternative is to model multiple concurrent transi-tions through interleaving . This, however, would placean arbitrary distance between the current. state and thestate in which the goal is achieved, defeating a desire tolimit inference to a small fixed number of states . Hencewe use an abstraction in which multiple commandsare given synchronously. In the Newmaap spacecraftdemonstration, for example, these synchronous com-mands were sent by a single processor through inter-leaving, and the next state sensor information returnedto Livingstone is the state following the execution ofall these commands .We now develop a formal characterization of MI and

MR. Recall that taking a transition -r corresponds totaking all of a set of subtransitions rj . A transitionr can be defined to apply over a set of states S in thenatural way :

Ti (S) = U Ti (S)3ES

Similarly we define rid (S) for each subtransition rid of-ri . We can show that

Ti (S) g n 7i)(S)In the following characterizations, Si denotes the set ofpossible states at time i before any control values areasserted by MR, lei denotes the control values assertedat time i, Oi denotes the observations at time i, andS,, ; and SO denote the set of states in which controland sensed variables have values specified in lei andOi, respectively. Hence, Si n S, is the set of possiblestates at time i .We characterize both MI and MR in two ways-first

model theoretically and then using state formulas .

Mode IdentificationMI incrementally generate the sequence So, Si, . . . us-ing a model of the transitions and knowledge of thecontrol actions lei as follows :

So

Si+1

C U (nT, k(Sin S,,,)~ n E n So,+, (4)k

{U) (2)

( Ui (Si n Su,) I n E n so,, (3)

where the final inclusion follows from Equation 1 .Equation 4 is useful because it is a characterizationof Si+1 in terms of the subtransitions Tik . This allowsus to develop the following characterization of Si+I interms of state formulae :

PST+1 = V ~

l \

`f j kT1

PS, APS,, 4`=D,k

Mode Reconfiguration

Mi ;?

{pj I

ps, A p,, ; is consistent and

PE ^ P(,),+ ,

(5)

This is a sound but potentially incomplete character-ization of the set of states in ,5'i+i, i .e ., every state inSi+1 satisfies ps,+ , but not all states that satisfy ps,+ ,are necessarily in Si+ , . However, generating ps,+ , re-quires only that. the entailment of the antecedent ofeach subtransition be checked . On the other hand,generating a complete characterization based on Equa-tion 3 would require enumerating all the states in Si,which can be computationally expensive if Si containsa lot of states .

MR incrementally generates the next set, of control val-ues pi using a model of the nominal transition Tn , thedesired goal configuration gi, and the current set ofpossible states Si . The model-theoretic characteriza-tion of Mi, the set of possible control actions that MRcan take at time i, is as follows :

mi

=

fp; I T.( .Si fl 5,.,.) fl E C gi}

(6 )

fp,inTttk(Si(15,)(1ECgi}k

where, once again, the latter inclusion follows fromEquation 1 . As with MI, this weaker characterizationof Mi is useful because it is in terms of the subtran-sitions Tnk . This allows us to develop the followingcharacterization of ,Mi, in terms of state formulae :

/ \

T.k A PE k jo g , }

(8)Ps,APa l I--'Pnk

The first part of the characterization says that the con-trol actions must be consistent with the current state,since without this condition the goals can be simplyachieved by making the world inconsistent . Equation 8is a sound but potentially incomplete characterizationof the set of control actions in ,Mi, i .e ., every controlaction that. satisfies the condition on the right handside is in Mi, but not necessarily vice versa . However,checking whether a given pj is an adequate control ac-tion only requires that the entailment ofthe antecedentof each subtransition be checked . On the other hand,generating a complete characterization based on Equa-tion 6 would require enumerating all the states in Si,which can be computationally expensive if Si containsa lot, of states .

Statistically optimal configurationmanagement

The previous section characterized the set of all feasi-ble trajectories and control actions that. can be gener-ated by MI and MR. However, in practice, not. all suchtrajectories and control actions need to be generated .Rather, just the likely trajectories and an optimal con-trol action needs to be generated . We efficiently gen-erate these by recasting MI and MR as combinatorialoptimization problems .A combinatorial optimization problem is a tuple

(X, C, f), where X is a finite set of variables with finitedomains, C is set of constraints over X, and f is an ob-jective function . A feasible solution is an assignment. toeach variable in X a value from its domain such thatall constraints in C are satisfied . The problem is tofind one or more of the leading feasible solutions, i .e .,to generate a prefix of the sequence of feasible solutionsordered in decreasing order of f.

Mode IdentificationEquation 3 characterizes the trajectory generationproblem as identifying the set of all transitions fromthe previous state that yield current states consistentwith the current observations . Recall that, a transi-tion system has one nominal transition and a set offailure transitions . In any state, the transition systemnon-deterministically selects exactly one of these tran-sitions to evolve to the next state . We quantify thisnon-deterministic choice by associating a probabilitywith each transition : p(T) is the probability that thetransition system will select transition T to evolve tothe next state . 1With this viewpoint, we recast MI's task to be one

of identifying the likely trajectories of the plant . Inkeeping with the reactive nature of configuration man-agement, MI will incrementally track the likely trajec-tories by always extending the current set of trajecto-ries by the likely transitions . The only change requiredin Equation 5 is that, rather than the disjunct rang-ing over all transitions Tj, it ranges over the subset oflikely transitions .The likelihood of a transition is its posterior proba-

bility p(TIOi) . This posterior is estimated in the stan-dard way using Bayes Rule :

P(710i) = p(Oi IT)P(T) a P(Oi 17)p(T)PA)

If T('Si_1) and Oi are disjoint sets then clearlyp(Oi JT) = 0 . Similarly, if T('Si_1) C_ Oi then Oi is en-tailed and P(OiJT) = 1, and hence the posterior prob-ability of T is proportional to the prior . If neither ofthe above two situations arises then p(OiJT) < 1 . Es-timating this probability is difficult and requires more

'We have made the simplifying assumption that theprobability of a transition is independent of the current.state of the transition system .

Williams 279

research, but see (de Kleer & Williams 1987) for someideas .

Finally, to view MI as a combinatorial optimizationproblem, recall that each plant transition consists ofa single transition for each of its component transi-tion systems . Hence, we introduce a variable into Xfor each component in the plant whose values are thepossible component transitions . Each plant transitioncorresponds to an assignment of values to variables inX . C is the constraint that the set of states resultingfrom taking a plant transition is consistent with theobserved values . The objective function f is just theprobability of a plant transition . The resulting combi-natorial optimization problem hence identifies the lead-ing transitions at each state, allowing MI to track theset of likely trajectories .

Mode reconfigurationEquation 6 characterizes the reconfiguration problemas one of identifying a control action that ensures thatthe result of taking the nominal transition yields statesin which the configuration goal is satisfied . Recast-ing MR as a combinatorial optimization problem isstraightforward . The variables X are just the controlvariables le with identical domains . C is the constraintin Equation 5 that lcj must satisfy to be in Mi . Fi-nally, as discussed in Section , different control actionscan have different costs that reflect differing resourcerequirements, etc . We take f to be negative of thecost of a control action . The resulting combinatorialoptimization problem hence identifies the lowest costcontrol action that achieves the goal configuration inthe next state .

Conflict-directed best first searchWe solve the above combinatorial optimization prob-lems using a conflict directed best first search, similar inspirit to (de Kleer & Williams 1989 ; Dressler & Struss1994) . A conflict is a partial solution such that anysolution containing the conflict is guaranteed to be in-feasible . Hence, a single conflict can rule out the feasi-bility of a large number of solutions, thereby focusingthe best-first search . Conflicts are usually generatedwhile checking to see whether a solution Xi satisfies theconstraints C . In the case of MI and MR, we check Cusing unit propagation for propositional inference, sothat simple LTMS-style dependency recording sufficesto generate conflicts (Forbus & de Kleer 1993) .Our conflict-directed best-first search algorithm,

CBFS, is shown in in Figure 3 . It has two major com-ponents : (a) an agenda that holds unprocessed solu-tions in decreasing order of f; and (b) a procedure togenerate the immediate successors of a solution . Themain loop of the algorithm removes the first solutionfrom the agenda, checks whether it is feasible, and addsin the solution's immediate successors to the agenda .When a solution Xti is infeasible, we assume that theprocess of checking the constraints C returns a part

280 QR-96

function CBFS(X, C, f)Agenda = f {best- candidate (X)}} ; Result = 0;while Agenda is not empty do

Soln = pop(Agenda);if Soln satisfies C thenAdd Soln to Result ;if enough solutions have been found thenreturn Result ;

else Succs = immediate successors Cand ;elseConf - a conflict that subsumes Cand ;Succs - immediate successors of Cand notsubsumed by Conf

endifInsert each solution in Succs into Agenda

in decreasing f order ;endwhilereturn Result ;

end CBFS

Figure 3 : Conflict directed best first search algorithmfor combinatorial optimization

of Xi as a conflict Ni . We focus the search by gener-ating only those immediate successors of Xi that. arenot subsumed by Ni, i .e ., do not agree with Ni on allvariables .

Intuitively, solution Xj is an immediate successor ofsolution Xi only if f(Xi ) > f(Xj) and Xi and Xj differonly in the value assignedto a single variable (ties arebroken consistently to prevent loops in the successorgraph) . One can show this definition of the immediatesuccessors of a solution suffice to prove the correct-ness of CBFS, i.e ., to show that even with conflict-directed focusing, all feasible solutions are generatedin decreasing order of f. Our implemented algorithmfurther refines the notion of an immediate successorwhile preserving correctness . The major benefit of thisrefinement is that each time a solution is removed fromthe agenda, at most two new solutions are added on,so that the size of the agenda is always bounded bythe total number of solutions that have been checkedfor feasibility . The details of this refinement are be-yond the scope of this paper and appear in the longerversion .

Implementation and experimentsWe have implemented Livingstone, a model-based con-figuration manager, based on the ideas described inthis paper . Livingstone was part of a rapid prototyp-ing demonstration of an autonomous architecture forspacecraft control . To evaluate the architecture, space-craft engineers at JPL defined the Newmaap spacecraftand scenario . The Newmaap spacecraft is a scaleddown version of the Cassim spacecraft that retainsmost challenging aspects of spacecraft control . The

Table 1 : Newmaap spacecraft. model properties

Table 2 : Results from the seven Newrnaap failure re-covery scenarios

Newmaap scenario was based on the most complex mis-sion phase of the Cassini spacecraft.-successful inser-tion into Saturn's orbit even in the event of any singlepoint of failure . Table 1 provides summary informationabout Livingstone's model of the Newmaap spacecraft,demonstrating its complexity.The Newmaap scenario included seven failure sce-

narios . From Livingstone's viewpoint, each scenario re-quired identifying the failure transitions using MI anddeciding on a set of control actions to recover from thefailure using MR (initial nominal hardware configura-tions were established by RAPS) . Table 2 shows theresults of running Livingstone on these scenarios . Thefirst column narnes each of the scenarios ; a discussionof the details of these scenarios is beyond the scopeof this paper . The second and fifth columns show thenumber of solutions checked by algorithm CBFS whenapplied to MI and MR, respectively . On can see thateven though the spacecraft model is large, the use ofconflicts dramatically focuses the search . The thirdcolumn shows the number of leading trajectory exten-sions identified by MI . The limited sensing available onthe Newmaap spacecraft often makes it impossible toidentify unique trajectories . This is generally true onspacecraft., since adding sensors is undesirable becauseit increases spacecraft, weight . The fourth and sixthcolumns show the time in seconds on a Sparc 5 spentby MI and MR on each scenario, once again demon-strating the efficiency of our approach . Furthermore,initial experiments with the use of ideas from truthmaintenance has demonstrated an order of magnitudespeed-up .

Livingstone's MI component was also tested on tencombinational circuits from a standard test. suite (Br-glez & Fujiwara 1985) . Each component in these cir-cuits was assumed to be in one of four modes : ok,stuck-at-1, stuck-at-0, and unknown . The probabil-ity of transitioning to the stuck-at modes was set at

Table 3 : Testing MI on a standard suite of combina-t.ional circuits

0 .099 and transitioning to the unknown mode was setto 0 .002 . We ran 20 experiments on each circuit usinga random fault and a random input. vector sensitiveto this fault . MI stopped generating trajectories af-ter either 10 leading trajectories had been generated,or when the next trajectory was 100 times more un-likely than the most likely trajectory . Table 3 showsthe results of our experiments . The columns are self-explanatory, except that the time is the number of sec-onds on a Sparc 2 rather than on a Sparc 5 . Note onceagain the power of conflict-directed search to drarnati-c:ally focus search . Interestingly, it is worth noting thatthese results are comparable to the results from thevery best ATMS-based implementations, even thoughLivingstone uses no ATMS.

Livingstone is also being applied to two other prob-lems . The first is ASSAP, another rapid prototyp-ing effort demonstrating other aspects of spacecraftautonomy including autonomous science and naviga-tion . The second is the autonomous real-time controlof a scientific instrument called a Bioreactor . Boththese projects are still underway, and final results areforthcoming . More excitingly, the success of the New-maap demonstration has launched Livingstone to newheights : Livingstone is going to be part. of the flightsoftware of the first New Millennium mission, calledDeep Space One, to be launched in early 1998 . Weexpect final delivery of Livingstone to this project latein 1996 .

ConclusionsIn this paper we introduced Livingstone, a fast, reac-tive, model-based self-configuring system, which pro-vides a kernel for model-based autonomy . Livingstone,the HSTS planning/scheduling system, and the RAPSexecutive, have been selected to form the core auton-omy architecture of Deep Space One, the first. flight ofNASA's New Millennium program .

Three technical features of Livingstone are particu-larly worth highlighting . First., Livingstone uses con-current transition systems as its underlying modelingparadigm . Transition systems provide the appropri-

Williams 28 1

Devices# of

components# of

clauses I Checked Timec17 6 18 18 0.1c432 160 514 58 4.7c499 202 714 43 4.5c880 383 1112 36 4.0c1355 546 1610 52 12 .3c1908 880 2378 64 22 .8c2670 1193 3269 93 28 .8c3540 1669 4608 140 113.3c5315 2307 6693 84 61 .2c7552 3512 9656 71 61 .5

Failure M I M RScenario Click Accpt ' ime ( ime

EGA preaim 7 2 2.2 4 1 .7BPLVD 5 2 2.7 8 2 .9IRU 4 2 1 .5 4 1 .6EGA burn - - 7 -

2.2 11- 3 .6

ACC 4 2 2.5 5 1 .9ME hot 6 2 2.4 13 3 .8Acc low 16 3 5.5 11 20 6 .1

Number of components 80Average modes/component 3.5Number of propositions 3424Number of clauses 11101

ate modeling paradigm because autonomous systemsare concurrent hardware/software hybrids that funda-mentally need to reason about their state and how thestate evolves over time . Furthermore, transition sys-tems provide a natural characterization of configura-tion management as a kind of discrete control system .

Second, Livingstone is a fast reactive configurationmanager . Reactivity is achieved by restricting reason-ing to just the current state and the next state . Fastinference is achieved by focusing on propositional tran-sition systems, unit propagation, and qualitative mod-eling . The interesting and important result of applyingLivingstone to Newmaap, Deep Space One, ASSAP,and the Bioreactor project is that Livingstone's modelsand restricted inference are still expressive enough tosolve important problems in a diverse set of domains.

Third, Livingstone casts mode identification andmode reconfiguration as combinatorial optimizationproblems, and uses a core conflict-directed best-firstsearch to solve them . The ubiquity of combinato-rial optimization problems and the power of conflict-directed search are central themes in Livingstone .

ReferencesBrglez, F ., and Fujiwara, H . 1985 . A neutral netlistof 10 combinational benchmark circuits and a tar-get translator in FORTRAN . Distributed on a tapeto participants of the Special Session on ATPG andFault Simulation, Int . Symposium on Circuits andSystems .de Kleer, J ., and Williams, B . C . 1987 . Diagnosingmultiple faults . Artificial Intelligence 32(1) :97-130 .Reprinted in (Hamscher, Console, & de Kleer 1992) .de Kleer, J ., and Williams, B . C . 1989 . Diagnosiswith behavioral modes . In Proceedings of IJCAI-89,1324-1330 . Reprinted in (Hamscher, Console, & deKleer 1992) .Dressler, O., and Struss, P. 1994 . Model-based di-agnosis with the default-based diagnosis engine : Ef-fective control strategies that work in practice . InProceedings of ECAI-9/, .Firby, R . J . 1995 . The RAP language manual . Ani-rnate Agent Project. Working Note AAP-6, Universityof Chicago .Forbus, K . D ., and de Kleer, J . 1993 . Building Prob-lem Solvers. MIT Press .Hamscher, W . ; Console, L. ; and de Kleer, J . 1992 .Readings in Model-Based Diagnosis. San Mateo, CA:Morgan Kaufmann,Hamscher, W. C . 1991 . Modeling digital circuits fortroubleshooting . Artificial Intelligence 51 :223-271 .Manna, Z ., and Pnueli, A . 1992 . The Temporal Logicof Reactive and Concurrent Systems: Specification .Springer-Verlag .

282 QR-96

Muscettola, N . 1994 . HSTS : Integrating planningand scheduling . In Fox, M ., and Zweben, M ., eds .,Intelligent Scheduling . Morgan Kaufmann .