pervasive self-regeneration through concurrent model-based execution

Approved for Public Release, Distribution Unlimited

Pervasive Self-Regeneration through Concurrent Model-Based

Execution

Brian Williams (PI)Paul Robertson

MITComputer Science and Artificial Intelligence Laboratory

7/20/04 2Approved for Public Release, Distribution Unlimited

Outline

• What we are trying to do.

• Demonstration scenario and test bed.

• Implications of successful results.

• Technical approach.

• What is new.

• Anticipated challenges.

• Technology transition.

• Looking forward: next steps.


What we are trying to do

• Why software fails:– Software assumptions about the environment

become invalid because of changes in the environment.

– Software changes introduce incompatibilities.– Software is attacked by a hostile agent.

• What can be done when software fails:– Recognize that a failure has occurred.– Diagnose what has failed – and why.– Find an alternative way of achieving the

intended behavior. Runtime

Models


Building upon a proven technology base.

• By extending RMPL to support software failure, we can extend robustness in the face of hardware failures to robustness in the face of software failures.

• Many of the same issues pertain:– Detection of faulty behavior– Diagnosis of the fault given faulty behavior– Reconfiguration of the software to achieve intended

behavior using different software components.– Select among alternatives to maximize utility.


Deliverables

• Model-based programming tools– A language for modeling (RMPL):

• The process in terms of desired state evolutions;• The components; and• The environment.

• Model-based executives that provide:– Safe optimal dispatch.– Redundant methods.– Continuous monitoring and diagnosis.– Regeneration and optimization.


Architectural overview

SPlant

Obs Cntrl

Model-basedEmbedded Programs

S

ContinuousReactive

Commanding

Continuous Mode/StateEstimation

Model

Desiderata: languages that are•Suspicious

•Monitor intentions and plans

•Self-Adaptive•Exploits and generates contingencies

•State and Fault Aware•Anticipatory

–“Model-predictive languages”–Plans and verifies into the future–Predicts future states

–Plans contingencies


Basic assumptions for our approach

• Objectives for the RMPL language– Low overhead

• The effort required of the programmers to achieve robustness should be small compared to the effort required to implement the base capability.

– Pervasive• The technology should be applied not only to major

components but to all components in the system.

– Incremental• Increased robustness can be achieved

incrementally by adding greater modeling.


Innovative claims

• Features of our approach:– Fault-aware processes.– Fault-adaptive.– Model-based.– Synthesizes a fault-adaptive process to achieve state evolutions.– Reasons from MODELS of correct and faulty behavior of

supporting service components.– Constructs novel recovery actions in the face of novel faults.

• Specific approaches:– Dynamic selection from redundant methods.– Self-optimization: select the optimal candidates.– Continuous monitoring.– Incremental addition of robustness by adding monitoring

procedures incrementally.– Synthesis of repair procedures from models.


Demonstration scenarioEnd to End Self-regeneration of Command & Control Systems:

• Robot must plan and execute motion to one or more targets.• Robot must perform designated tasks at selected destinations.• Robot utilizes various sensors and actuators in achieving its

task.• Robot utilizes various software components in interpreting its

sensor data and manipulating its actuators.• Changing environment will cause software components to fail.

Failed software components will be detected and diagnosed. Alternate configurations of the software will be found that can maintain mission objectives while maximizing utility—in real-time.

• Fault injection testing: We will inject faults into the test bed including (1) environment changes, (2) incompatibilities, (3) software attacks.


Test bed summary• Robot scenario involves:

– Path planning and execution.– Goal selection with risks and rewards.– Visual and other sensors and actuators utilized in

navigation and task execution.– Reacts to:

• Failures in the software.– Exploiting redundant methods.– By re-planning to maintain optimality.

• Discoveries in the environment.– Obstacles– Suitability of using selected sensors given terrain lighting etc.

• Attack


Rover test bed

Allows real-world testing of planning and execution software

Consists of a reconfigurable environment with one ATRV-2 and three ATRV-JRs.


Rover test bed setup

Differential drive

Laser range scannerSonar sensors

Wheel encoders(odometry)

Stereo camera

Inclinometer

GPS receiver

Compass

Antennas for wireless LAN

rFLEXcontroller

rFLEXscreen

Sonarcontrolboard

Sonar sensors

SICK LMS 200 laser scanner

Onboard PC

Firewire card

ttyR ports

Stereo camera

Serial port

Inclinometer

802.11a wireless network adapter

Ethernet card

Left motor

Right motor

• Sensors give information on motion and environment.

• Onboard PC allows for real-time computation and command processing.


Implications of successful results

• Robotic systems that can operate autonomously to achieve goals in a complex and changing environment.– Modeling environment

• Software that detects and works around “bugs” resulting from incompatible software changes.– Modeling software components

• Software that detects and recovers from software attacks.– Modeling attack scenarios

• Software that automatically improves as better software components and models are added.

• Higher level of command and control for robotic missions.


Military roles for autonomous robots

• Reconnaissance– Rover makes its way to designated places and

reports back—such as with pictures.

• Search and Rescue– Rover enters a dangerous area and reports back on

the presence of wounded people.

• Mapping– Rover explores (a building) producing a map

annotated with pictures in support of ground forces.

• Munitions Delivery– Rover goes to designated locations and delivers

munitions. (like predator UAV)


Technical approach

• We will extend the technology developed for execution, hardware fault detection, diagnosis and reconfiguration successfully used in Deep Space One to be applied to software fault awareness and reconfiguration.

• Software components look like hardware components but reconfiguration is less restricted.

• A greater emphasis on environment modeling is necessary because most software faults will be because of environmental changes.


Systems Should be Suspicious

Sensor Interpretation:

• Nominal:

• Command Confirmation

• Fault Detection

• Failure:

• Fault Isolation

• Fault Diagnosis

Traditional Commanding is Open Loop

Continuous Estimation of

Modes and State

Model

RealTime

Robust Systems Should be Fully State AwareAnd Should Use To Dynamically Monitor Specifications


Commanding InvolvesRepairing a Correct State

For long-lived systems, failure is the rule:

• Must navigate around failures.

• Must operate with varying resources and capabilities.

• Should select best means amongst multiple alternatives.

Nominal commanding is Traditionally pre-determined

Robust systems should select the best action

in context

ContinuousReactive

Commanding


Model


How Do We Guide Self-Regenerative Systems? - By Interacting Directly with State

Embedded programs

interact with sensors and actuators.

Model-based programs

interact with state

Embedded Program

SServices

Obs Cntrl

Model-basedEmbedded Program

SServices

Programmers map between sensors, actuators to states.

Model-based executives map between sensors, actuators to states.


Model-based Programs Interact Directly with State

SPlant

Obs Cntrl


Model-basedExecutive

S

PlantModel

State estimates

ReactiveCommanding

ModeEstimation

State goals

s

RMPL State assertion State query Conditional Execution Preemption Iteration Concurrent execution

Fault OccursCurrent Belief State

S T

X0 X1 XN-1 XN

Reconfigure

S T

X0 X1 XN-1 XN

least cost reachable goal stateFirst Action


Real-Time Real-Time ExecutionExecution


Flight Flight H/WH/W

Flight Flight H/WH/WFault Fault

MonitorsMonitorsFault Fault

MonitorsMonitorsPlanning Experts Planning Experts (incl. Navigation)(incl. Navigation)Planning Experts Planning Experts (incl. Navigation)(incl. Navigation)

Model-based Autonomy Architecture

Ground Ground SystemSystemGround Ground SystemSystem

RAX ManagerRAX Manager

Model-Model-based Faultbased FaultProtectionProtection

Mission Mission ManagerManager ExecutiveExecutive

Planner/Planner/SchedulerScheduler










Model-based Model-based FaultFault

ProtectionProtection

Mission Mission ManagerManager

ExecutiveExecutive


Remote AgentRemote AgentGoal States











Model-basedModel-basedFault Fault

ProtectionProtection


ExecutiveExecutive


Remote AgentRemote Agent











Model-Model-based Fault based Fault ProtectionProtection


Procedural Procedural ExecutiveExecutive


Remote AgentRemote Agent

Low-Level Fault

Protection











Model-Model-based based

ExecutiveExecutive


Procedural Procedural ExecutiveExecutive


Remote AgentRemote Agent High-Level Fault

Protection



Remote Agent ExperimentMay, 1999

May 17-18th experiment: High-level Fault Protection• Generate plan for course correction and thrust • Diagnose camera as stuck on

– Power constraints violated, abort current plan and replan

• Perform optical navigation• Perform ion propulsion thrust

May 21th experiment: Low-level Fault Protection• Diagnose faulty device and

– Repair by issuing reset.• Diagnose switch sensor failure.

– Determine harmless, and continue plan. • Diagnose thruster stuck closed and

– Repair by switching to alternate method of thrusting. • Back to back planning

RA was a toolbox, want a seamless language


Technology transition

• The structure of our solution facilitates transition. – Implements self-regenerative system using

language+executive, as opposed to a set of regeneration utilities.

– Easily wraps self-regeneration around existing components, through a clean separation of the “executive” and “plant.”

– Has supported a long history of successful technology transition.

• Papers and other publications


Looking forward: next steps

• Reasoning about complex software models.• Coordination while preserving privacy of

subsystem information.• Extending the technology to work with complex

distributed self-regenerative systems.– Regeneration requires peer to peer coordination of

systems.– Subtle faults that are distributed across multiple

systems.– Faults that are detected in systems in which we do

not have direct control—and must negotiate fault resolution.


Appendix A


Model-based Execution Kernels as Stochastic Optimal Controllers

DeductiveController

Plant

modeestimation

mode reconfiguration

s’(t)

commands(t)

ffs (t)

gg

Observations(t)

ModelGoalStates


Operators and programmers reason through system-wide interactions to:

• isolate faultsisolate faults• diagnose causesdiagnose causes

Diagnosis


OPSAT

Conflict-directed A*

Conflict-directed A*

OptimalOptimalfeasiblefeasiblemodesmodes

ConflictConflict

CheckedCheckedkernelkernel

ISATISAT

Generate Most-likely Candidate Diagnoses: • conflict-directed A* (< 10 states visited)

Test Against Model and Observables: • Incremental Satisfiability (avg. < 10 % off ideal)• Learn from counter examples (conflicts) to guide generation.


Compare Most Likely Candidate to Observations

Helium tank

Fuel tankOxidizer tank

MainEngines

Flow1 = zeroPressure1 = nominal

Pressure2= nominal

Acceleration = zero


Isolate Conflicting Modes

Helium tank


MainEngines

Flow 1= zero

A conflict, C, is an assignment to a subset of the mode variables that is inconsistent with the model and observations.


Helium tank


MainEngines

Flow 1= zero

Generate Next Most Likely Candidate

Every consistent mode assignment must differ from the conflict for at least one mode variable



Every consistent mode assignment must differ from the conflict for at least one mode variable

Helium tank


MainEngines


Testing Candidate Detects Another Conflict

Pressure1 = nominal Pressure2= nominal

Acceleration = zero

Helium tank


MainEngines


Pressure1 = nominal Pressure2= nominal

Acceleration = zero

Helium tank


MainEngines

Consistent mode assignment must differ from both conflicts



Consistent: Most Likely Diagnosis

Helium tank


MainEngines

Pressure1 = nominalFlow1 = zero

Pressure2= nominalFlow2 = positive

Acceleration = zero



• isolate faultsisolate faults• diagnose causesdiagnose causes

Diagnosis

• repairrepair• reconfigurereconfigure

ConfigurationConfigurationManagementManagement


Conflicts Focus MRGoal: Achieve Thrust

Find least cost modes that entail the current goal.• A conflict, C, is an assignment to a subset of the mode variables

that entails the negation of the goal.


Goal: Achieve Thrust

Conflicts Focus MR




Closed

Open Stuckopen

Stuckclosed

OpenCloseCost 5

Prob .9

Models are Concurrent, Constraint Encodings of Partially Observable Markov Decision Process

Vlv = closed => Outflow = 0;

vlv=open => [Outflow]S = [Inflow]S and[Outflow]R = [Inflow]R

vlv=stuck open => [Outflow]S = [Inflow]S and[Outflow]R = [Inflow]R

vlv=stuck closed=> Outflow = 0;

Unknown

• One automaton for each device.• Communication through shared variables.



• isolate faultsisolate faults• diagnose causesdiagnose causes• monitormonitor• confirm commandsconfirm commands • track goalstrack goals

Estimating Modes


Mode Estimation

Left engine on Observe“no thrust”

Enumerated by decreasing probability.

Find most likely reachable states consistent with observations.

Every component transitions at each time step.


Operators and programmers reason through system-wide interactions to :

Controlling ModesControlling ModesEstimating Modes

• monitormonitor• track goalstrack goals• confirm commandsconfirm commands• isolat faultsisolat faults• diagnose faults diagnose faults

• repairrepair• reconfigurereconfigure• executeexecute• avoid failuresavoid failures• change control change control

policiespolicies


1. Find least cost state that entails the current goal and is reachable from the current state.

2. Find first action that moves towards this state.

Model-based Reactive Planning

ModeReconf.

ModeEst.

Command

Configuration goals

Model

goal statecurrent state

ReactivePlanning

Burton Model-based Executive IJCAI97


Architectural overview

SPlant

Obs Cntrl


S

ContinuousReactive

Commanding


Model

Desiderata: languages that are•Suspicious

•Monitor intentions and plans

•Self-Adaptive•Exploits and generates contingencies

•State and Fault Aware•Anticipatory

–“Model-predictive languages”–Plans and verifies into the future–Predicts future states

–Plans contingencies


Anticipated Challenges

• Software is harder than hardware:– Hardware recovers by leveraging backups,

while software leverages alternative methods. – Models of software are more complex to

specify.– While hardware’s topology is static,

software’s topology dynamically changes.

• Performance– Software systems result in larger state

spaces, making real-time response a challenge.

pervasive self-regeneration through concurrent model-based execution

Documents