1

BLACKBOX: A New Paradigm for Planning

BLACKBOX: A New Paradigm for Planning

Bart SelmanCornell University

2

Search as Inference: DirectSearch as Inference: Direct

Abstract problem specification

General inference (NP complete)

Solution

Model in propositional logic

3

State-space PlanningState-space Planning

Find a sequence of operators that transform an initial state to a goal state

State = complete truth assignment to a set of variables (fluents)

Goal = partial truth assignment (set of states)

Operator = a partial function State State

• specified by three sets of variables:precondition, add list, delete

list

4

Some Applications of PlanningSome Applications of Planning

Autonomous systems

• NASA Deep Space One Remote Agent

Softbots - software robots

• Internet agents, program assistants

• Bots, characters in games

Program verification

• Jackson (1998) - finding bugs in protocols

- is there a sequence of actions that reaches an error state?

5

SATPLAN(Kautz & Selman 1996)SATPLAN

(Kautz & Selman 1996)

STRIPSModel in propositional logic

Walksat SAT engine Solution

6

Lessons from SATPLANLessons from SATPLAN

A general propositional theorem prover outperformed traditional AI planning systems (UCPOP, Nonlin, Prodigy, ...)

• Power of propositional logic– much better scaling than attempts in 1970’s using

first-order theorem proving

• Fast SAT engines– stochastic search - walksat

– large SAT/CSP community sharing ideas and code

– older planning systems can be viewed as adhoc, incomplete, poorly understood theorem provers!

• Importance of modeling– different axiomatizations can have vastly different

computational properties

7

Graphplan(Blum & Furst 1996)Graphplan(Blum & Furst 1996)

Planning as graph searchLike SATPLAN...

• Two phases: instantiation of propositional structure, followed by search

• Plan graph is very close to CNF

Unlike SATPLAN…

• Takes STRIPS operators directly as input

• Interleaves instantiation and pruning of plan graph

– results in much smaller structure

• Employs specialized search engine

• Graphplan - better instantiation

• SATPLAN - better search

Goal: Combine best features of both systems

8

Where Graphplan Gets its PowerWhere Graphplan Gets its Power

During instantiation, Graphplan computes mutex relationships between incompatible actions

• used for pruning, and later speeding search

• mutex algorithm is actually a form of limited resolution on binary negative clauses!

• polytime preprocessing O(n2)

Issue:

• research on graphplan failed to discover any useful extensions to mutex algorithm

• Can general polytime limited inference algorithms discover other kinds of useful local information?

9

Multistep Problem ReformulationMultistep Problem Reformulation

Domain specific model

Polytime domain specific inference

Combinatorial core - general language

Full general inference(NP complete)

SolutionPolytime general inference

Abstract problem specification

10

BlackboxBlackbox

STRIPSPlan Graph

Mutex computation

CNF Translation

Stochastic / Systematic SAT engines

SolutionLimited resolution - failed literal rule

11

IntuitionIntuition

Many real-world problems not tractable, but are nearly so

• domain specific polytime inference takes advance of special kinds of structure

• small number of practical methods for combinatorial core– can be highly optimized

– limited inference: variations of constraint propagation

– full inference: local search, smart backtracking, *randomized backtracking

12

Translation to CNFTranslation to CNF

Fact Act1 Act2

Act1 Pre1 Pre2

¬Act1 ¬Act2

Act1

Act2

Fact

Pre1

Pre2

Alternating layers of facts and actions• fully factored (nodes are propositions, not states!)

Not all atoms in a layer can hold simultaneously• solution = subgraph containing all goals, all supports, no mutexes

13

General Limited InferenceGeneral Limited Inference

Generated wff can be further simplified by consistency propagation techniques

Compact (Crawford & Auton 1996)

• unit propagation: is Wff inconsistant by resolution against unit clauses?

O(n)

• failed literal rule: is Wff + { P } inconsistant by unit propagation?

O(n2)

• binary failed literal rule: is Wff + { P V Q } inconsistant by unit propagation?

O(n3)

Complements domain specific limited inference

Discovers hidden local structure!

14

General Limited InferenceGeneral Limited Inference

Percent vars set byProblem Varsunitprop

failedlit

binaryfailed

bw.a 2452 10% 100% 100%bw.b 6358 5% 43% 99%bw.c 19158 2% 33% 99%log.a 2709 2% 36% 45%log.b 3287 2% 24% 30%log.c 4197 2% 23% 27%log.d 6151 1% 25% 33%

15

Randomized Sytematic SolversRandomized Sytematic Solvers

Stochastic local search solvers (walksat)

• when they work, scale well

• cannot show unsat

• fail on some domains

Systematic solvers (Davis Putnam)

• complete

• seem to scale badly

Can we combine best features of each approach?

16

Heavy TailsHeavy Tails

Bad scaling of systematic solvers can be caused by heavy tailed distributions

Deterministic algorithms get stuck on particular instances

• but that same instance might be easy for a different deterministic algorithm!

Expected (mean) solution time increases without limit over large distributions

17

Heavy Tailed Cost DistributionHeavy Tailed Cost Distribution

0.1

1

1 10 100 1000 10000 100000

log( Backtracks )

log

( 1

- F

(x)

)

18

Randomized RestartsRandomized Restarts

Solution: randomize the systematic solver

• Add noise to the heuristic branching (variable choice) function

• Cutoff and restart search after a fixed number of backtracks

Eliminates heavy tails

In practice: rapid restarts with low cutoff can dramatically improve performance

19

Rapid Restart SpeedupRapid Restart Speedup

1000

10000

100000

1000000

1 10 100 1000 10000 100000 1000000

log( cutoff )

log

( b

ackt

rack

s )

20

Blackbox as Experimental TestbedBlackbox as Experimental Testbed

All components of blackbox are parameterized

Can experiment with different schedules for instantiating, simplifying, and solving problems

blackbox -solver -maxsec 20 graphplan

-then compact -l

-then satz -cutoff 20 -restart 100

-then walksat -cutoff 1000000 -restart 10

21

blackbox version 9Bcommand line: blackbox -o logistics.pddl -f logistics_prob_d_len.pddl -solver compact -l -then satz -cutoff 25 -restart 10

----------------------------------------------------Converting graph to wff6151 variables243652 clausesInvoking simplifier compactVariables undetermined: 4633Non-unary clauses output: 139866----------------------------------------------------Invoking solver satz version satz-rand-2.1Wff loaded[1] begin restart[1] reached cutoff 25 --- back to root[2] begin restart[2] reached cutoff 25 --- back to root [3] begin restart[3] reached cutoff 25 --- back to root [4] begin restart[4] reached cutoff 25 --- back to root [5] begin restart**** the instance is satisfiable ********* verification of solution is OK ****

total elapsed seconds = 25.930000----------------------------------------------------Begin plan1 drive-truck_ny-truck_ny-central_ny-po_ny

22

Begin plan1 drive-truck_ny-truck_ny-central_ny-po_ny1 drive-truck_sf-truck_sf-airport_sf-po_sf1 load-truck_package5_bos-truck_bos-po1 drive-truck_pgh-truck_pgh-airport_pgh-central_pgh1 fly-airplane_airplane2_pgh-airport_sf-airport1 load-truck_package6_bos-truck_bos-po2 load-truck_package2_pgh-truck_pgh-central2 load-truck_package4_ny-truck_ny-po2 load-truck_package7_ny-truck_ny-po2 load-truck_package3_pgh-truck_pgh-central2 drive-truck_bos-truck_bos-po_bos-airport_bos2 load-airplane_package8_airplane2_sf-airport2 fly-airplane_airplane1_pgh-airport_sf-airport2 drive-truck_la-truck_la-po_la-airport_la3 fly-airplane_airplane2_sf-airport_bos-airport3 unload-truck_package6_bos-truck_bos-airport3 drive-truck_pgh-truck_pgh-central_pgh-airport_pgh3 fly-airplane_airplane1_sf-airport_pgh-airport3 unload-truck_package5_bos-truck_bos-airport3 drive-truck_ny-truck_ny-po_ny-airport_ny3 drive-truck_sf-truck_sf-po_sf-airport_sf4 unload-truck_package3_pgh-truck_pgh-airport4 unload-truck_package2_pgh-truck_pgh-airport4 unload-truck_package4_ny-truck_ny-airport4 load-airplane_package6_airplane2_bos-airport4 load-airplane_package5_airplane2_bos-airport4 drive-truck_la-truck_la-airport_la-po_la4 drive-truck_bos-truck_bos-airport_bos-central_bos4 unload-truck_package7_ny-truck_ny-airport5 drive-truck_ny-truck_ny-airport_ny-po_ny5 drive-truck_bos-truck_bos-central_bos-po_bos5 load-airplane_package2_airplane1_pgh-airport5 drive-truck_la-truck_la-po_la-central_la5 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh5 load-airplane_package3_airplane1_pgh-airport5 fly-airplane_airplane2_bos-airport_ny-airport6 drive-truck_sf-truck_sf-airport_sf-central_sf6 unload-airplane_package6_airplane2_ny-airport6 load-airplane_package4_airplane2_ny-airport6 drive-truck_la-truck_la-central_la-po_la6 drive-truck_bos-truck_bos-po_bos-airport_bos6 load-airplane_package7_airplane2_ny-airport6 drive-truck_ny-truck_ny-po_ny-airport_ny6 unload-airplane_package8_airplane2_ny-airport6 fly-airplane_airplane1_pgh-airport_sf-airport6 load-truck_package1_pgh-truck_pgh-po7 fly-airplane_airplane2_ny-airport_la-airport7 fly-airplane_airplane1_sf-airport_bos-airport7 load-truck_package9_sf-truck_sf-central7 load-truck_package6_ny-truck_ny-airport

7 drive-truck_bos-truck_bos-airport_bos-central_bos7 drive-truck_pgh-truck_pgh-po_pgh-airport_pgh7 load-truck_package8_ny-truck_ny-airport8 drive-truck_sf-truck_sf-central_sf-po_sf8 fly-airplane_airplane2_la-airport_pgh-airport8 unload-truck_package1_pgh-truck_pgh-airport8 drive-truck_bos-truck_bos-central_bos-po_bos8 drive-truck_ny-truck_ny-airport_ny-central_ny8 fly-airplane_airplane1_bos-airport_la-airport8 drive-truck_la-truck_la-po_la-airport_la9 unload-airplane_package7_airplane2_pgh-airport9 unload-truck_package8_ny-truck_ny-central9 unload-airplane_package5_airplane2_pgh-airport9 unload-truck_package9_sf-truck_sf-po9 unload-airplane_package3_airplane1_la-airport9 unload-truck_package6_ny-truck_ny-central9 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh9 load-airplane_package1_airplane2_pgh-airport10 drive-truck_ny-truck_ny-central_ny-po_ny10 fly-airplane_airplane2_pgh-airport_bos-airport10 load-truck_package3_la-truck_la-airport10 fly-airplane_airplane1_la-airport_ny-airport10 drive-truck_pgh-truck_pgh-po_pgh-airport_pgh11 drive-truck_bos-truck_bos-po_bos-airport_bos11 drive-truck_ny-truck_ny-po_ny-airport_ny11 unload-airplane_package2_airplane1_ny-airport11 drive-truck_la-truck_la-airport_la-central_la11 drive-truck_sf-truck_sf-po_sf-airport_sf11 unload-airplane_package1_airplane2_bos-airport11 load-truck_package7_pgh-truck_pgh-airport11 load-truck_package5_pgh-truck_pgh-airport12 drive-truck_sf-truck_sf-airport_sf-po_sf12 load-truck_package1_bos-truck_bos-airport12 fly-airplane_airplane2_bos-airport_la-airport12 load-truck_package2_ny-truck_ny-airport12 fly-airplane_airplane1_ny-airport_pgh-airport12 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh12 unload-truck_package3_la-truck_la-central13 drive-truck_ny-truck_ny-airport_ny-po_ny13 load-truck_package3_la-truck_la-central13 load-truck_package9_sf-truck_sf-po13 drive-truck_bos-truck_bos-airport_bos-po_bos13 unload-truck_package5_pgh-truck_pgh-po13 unload-airplane_package4_airplane2_la-airport14 unload-truck_package9_sf-truck_sf-po14 unload-truck_package1_bos-truck_bos-po14 unload-truck_package7_pgh-truck_pgh-po14 unload-truck_package2_ny-truck_ny-po14 unload-truck_package3_la-truck_la-centralEnd plan

23

Blackbox ResultsBlackbox Results

0.01

0.1

1

10

100

1000

10000

rocket.a rocket.b log.a log.b log.c log.d

Graphplan

BB-walksat

BB-rand-sys

Handcoded-walksat

1016 states6,000 variables125,000 clauses

24

AI Planning Systems CompetitionCMU, 1998

AI Planning Systems CompetitionCMU, 1998

Team Number of Average FastestShortest

problems solution onsolutions

solved time (msec) for

Blackbox 10 3171 3 6

(AT&T Labs)

HSP 9 25875 1 5

(Venezuela)

IPP 8 (11) 11036 1(3) 6(8)

(Germany)

STAN 7 20947 5 4

(UK)

25

NotesNotes

All finalists based on SATPLAN, Graphplan, or A* !

• Traditional non-linear planning no longer competitive

• Knowledge-intensive approaches require too much human effort

Other new techniques• Type-theoretic analysis of operators: can infer

state invariants (package only in one vehicle, etc.)– powerful, generally applicable pre-processor

• Compilation of more expressive languages (conditional effects) to STRIPS

• Recent extensions to MDP’s of A* (Geffner), Graphplan (Blum), SATPLAN (Littman)

26

SummarySummary

Blackbox combines best features of Graphplan, SATPLAN, and new randomized systematic search engines

Automatic generation of wffs from standard STRIPS input

• No performance penalty over hand-encodings!

Testbed for bridging different planning paradigms

27

Current Research IssuesCurrent Research Issues

Incorporating explicit domain knowledge (Kautz & Selman, 1998)

• state invariants

• optimality conditions

• declarative constraints - independent of search engine

More expressive planning languages: optimizing resources

• can view bounded integer linear programming as generalization of SAT

• ILPPLAN - adapts SATPLAN framework to ILP, solve with WSAT(OIP) (local search for ILP)

• Initial results - can find better quality solutions (counting action costs) than previously known for benchmark logistics & scheduling problems(Kautz & Walser 1999)