factored planning

Factored Planning 1

Eyal Amir (UC Berkeley)Eyal Amir (UC Berkeley)

Barbara Engelhardt (UC Berkeley)Barbara Engelhardt (UC Berkeley)

Factored PlanningFactored Planning

Factored Planning 2

MotivationMotivation• Planning in structured domainsPlanning in structured domains

• Scale up planners to large domainsScale up planners to large domains

• Avoid backtracking of search for plansAvoid backtracking of search for plans

• TechniquesTechniques: Similar to structured exact : Similar to structured exact logical and probabilistic reasoninglogical and probabilistic reasoning

• (Pearl’88), (Dechter & Rish’94), (Amir & (Pearl’88), (Dechter & Rish’94), (Amir & McIlraith ’00,’01), (McCartney etal. ’03) McIlraith ’00,’01), (McCartney etal. ’03)

Factored Planning 3

Talk OutlineTalk Outline• Factored domainsFactored domains

• Factored planning algorithmFactored planning algorithm

• Decomposition algorithmDecomposition algorithm

Factored Planning 4

Example Planning DomainExample Planning Domain

Room 1Room 1(20 windows)(20 windows)


FluentsFluents: closed(x), done1, done2, at1, at2: closed(x), done1, done2, at1, at2ActionsActions: close(x), move-R, move-L: close(x), move-R, move-L GoalGoal: done1,done2 (all windows closed): done1,done2 (all windows closed)Initial stateInitial state: at2: at2

Factored Planning 5



Plan:Plan:1.1. move-leftmove-left2. close(1), close(2), …2. close(1), close(2), …3. move-right3. move-right4.4. close(21), close(22), …close(21), close(22), …

Factored Planning 6





at2at2

Example plan:Example plan:

Factored Planning 7




FluentsFluents: closed(x), done1, done2, at1, at2: closed(x), done1, done2, at1, at2ActionsActions: close(x), move-R, : close(x), move-R, move-Lmove-L GoalGoal: done1,done2 (all windows closed): done1,done2 (all windows closed)Initial stateInitial state: at2: at2

at2at2


Factored Planning 8




FluentsFluents: closed(x), done1, done2, at1, at2: closed(x), done1, done2, at1, at2ActionsActions: close(x), move-R, : close(x), move-R, move-Lmove-L GoalGoal: done1,done2 (all windows closed): done1,done2 (all windows closed)Initial stateInitial state: at2: at2

at1at1


Factored Planning 9




FluentsFluents: closed(x), done1, done2, at1, at2: closed(x), done1, done2, at1, at2ActionsActions: : close(x),close(x), move-R, move-L move-R, move-L GoalGoal: done1,done2 (all windows closed): done1,done2 (all windows closed)Initial stateInitial state: at2: at2

at1at1


Factored Planning 10





at1at1







at1at1


done1done1





FluentsFluents: closed(x), done1, done2, at1, at2: closed(x), done1, done2, at1, at2ActionsActions: close(x), : close(x), move-Rmove-R, move-L, move-L GoalGoal: done1,done2 (all windows closed): done1,done2 (all windows closed)Initial stateInitial state: at2: at2

at1at1


done1done1





FluentsFluents: closed(x), done1, done2, at1, at2: closed(x), done1, done2, at1, at2ActionsActions: close(x), : close(x), move-Rmove-R, move-L, move-L GoalGoal: done1,done2 (all windows closed): done1,done2 (all windows closed)Initial stateInitial state: at2: at2

at2at2


done1done1






at2at2


done1done1






at2at2


done1done1 done2done2




Plan:Plan:1.1. move-leftmove-left2. close(1), close(2), …2. close(1), close(2), …3. move-right3. move-right4.4. close(21), close(22), …close(21), close(22), …


Factored Planning DomainFactored Planning DomainPlan:Plan:1.1. move-leftmove-left2. close(1), close(2), …2. close(1), close(2), …3. move-right3. move-right4.4. close(21), close(22), …close(21), close(22), …

Fluents:Fluents:closed(x) (1closed(x) (1xx20)20)done1, at1, at2done1, at1, at2

Actions:Actions:Close(x) (1Close(x) (1xx20)20)Move-R, move-LMove-R, move-L

Fluents:Fluents:closed(x) (21closed(x) (21xx40)40)done1done1,done2,,done2,at1,at2at1,at2

Actions:Actions:Close(x) (21Close(x) (21xx40)40)Move-R, move-LMove-R, move-L

at1,at2at1,at2done1done1


IntuitionIntuition• Planning separately in different partsPlanning separately in different parts

• Combine separate plans to form Combine separate plans to form complete plancomplete plan

• DifficultyDifficulty: want a sound + complete : want a sound + complete procedure that is always applicableprocedure that is always applicable

• Potential Potential benefits:benefits:– Fast planning & replanningFast planning & replanning– Scaling to very large domainsScaling to very large domains


Planning via Dynamic ProgramPlanning via Dynamic Program



Do someDo someProcessingProcessing

In subdomain 1In subdomain 1

Do someDo someProcessingProcessing

In subdomain 2In subdomain 2+ messages+ messages

SendSendMessagesMessages(one way)(one way)


Planning Algorithm (example)Planning Algorithm (example)



Find Find capabilitiescapabilitiesof subdomain 1of subdomain 1

relevant torelevant tosubdomain 2subdomain 2



Room 1Room 1““if if at1at1, then there is, then there is

a way to makea way to makedone1done1 true” true”










Create a new actionCreate a new actionIn subdomain 2:In subdomain 2:


““make done1 true”make done1 true”Precond: at1Precond: at1Effect: done1Effect: done1









Find plan in subdomain 2Find plan in subdomain 2







Find plan in subdomain 2Find plan in subdomain 2

Plan:Plan:1.1. move-leftmove-left2.2. ““make done1 true”make done1 true”3.3. move-rightmove-right4.4. close(21),close(21),5.5. close(22), …close(22), …


Room 2Room 2







Expand plan toExpand plan toInclude only atomicInclude only atomic

actionsactions

Plan:Plan:1.1. move-leftmove-left2.2. ““make done1 true”make done1 true”3.3. move-rightmove-right4.4. close(21),close(21),5.5. close(22), …close(22), …

Room 2Room 2







Expand plan toExpand plan toInclude only atomicInclude only atomic

actionsactions

Plan:Plan:1.1. move-leftmove-left2.2. A. close(1),A. close(1),

B. close(2), …B. close(2), …3. move-right3. move-right4.4. close(21),close(21),5.5. close(22), …close(22), …

Room 2Room 2


Summary & OutlookSummary & Outlook• So far: example of algorithmSo far: example of algorithm

• Next: algorithm + generalizationNext: algorithm + generalization

Fluents:Fluents:closed(x) (0<x<21)closed(x) (0<x<21)done1, at1, at2done1, at1, at2

Actions:Actions:Close(x) (0<x<21)Close(x) (0<x<21)Move-R, move-LMove-R, move-L

Fluents:Fluents:closed(x) (19<x<41)closed(x) (19<x<41)done1done1,done2,,done2,at1,at2at1,at2


at1,at2at1,at2done1done1

SeparatorSeparator


Planning Algorithm (2 partitions)Planning Algorithm (2 partitions)INPUT: 2 Partitions of fluents and actionsINPUT: 2 Partitions of fluents and actions

Initial state, Goal conditionInitial state, Goal condition

1.1. Iterate over values V for separator:Iterate over values V for separator:A.A. Find achievable goals for part 1 from V Find achievable goals for part 1 from V

B.B. Send found plans to part 2 Send found plans to part 2

2.2. Add Add new actionsnew actions for plans of part 1 for plans of part 1

3.3. Find plan for goal in part 2Find plan for goal in part 2

4.4. Replace Replace new actionsnew actions with atomic ones with atomic ones


General MessagesGeneral Messages




What if we need more interactions between rooms?What if we need more interactions between rooms?

Example: robot must recharge batteriesExample: robot must recharge batteries

powerpower

Not true anymoreNot true anymore


General MessagesGeneral Messages

Room 1Room 1““if if at1at1, and when, and when

needed needed battery_okbattery_ok,,then we can makethen we can make

done1done1 true” true”


powerpower

What if we need more interactions between rooms?What if we need more interactions between rooms?

Example: robot must recharge batteriesExample: robot must recharge batteries


Planning AlgorithmPlanning AlgorithmINPUT: INPUT: mm Partitions of fluents and actions Partitions of fluents and actions





Find plan in final partitionFind plan in final partitionReplace new actionsReplace new actions

with atomic actionswith atomic actions


Analysis of AlgorithmAnalysis of Algorithm• TheoremTheorem: Planning algorithm is : Planning algorithm is sound sound

and and completecomplete..• TheoremTheorem: Algorithm runs in time : Algorithm runs in time

O(m O(m •• 2 2kk11••kk22))m = number of partitionsm = number of partitions

kk11 = # fluents in largest partition = # fluents in largest partition

kk22 = “plan width”: = “plan width”:

# back-and-forth interactions# back-and-forth interactionsbetween partitions in a plan between partitions in a plan


Analysis of AlgorithmAnalysis of Algorithm• TheoremTheorem: Planning algorithm is : Planning algorithm is sound sound

and and completecomplete..• TheoremTheorem: Algorithm runs in time : Algorithm runs in time

O(m O(m •• 2 2kk11••kk22))• Two Rooms Two Rooms ExampleExample::

m = 2m = 2 kk11 = 24 = 24 kk22 = 1 = 1

FactoredFactored planning: O( planning: O(2 2 •• 2 22424))TraditionalTraditional planning: O( planning: O(224444))


Experiment: Ring of RoomsExperiment: Ring of Rooms

Number of roomsNumber of rooms

Tim

e (in

mill

isec

onds

)T

ime

(in m

illis

econ

ds)


Decomposition AlgorithmDecomposition Algorithm1.1. Convert into a graph problemConvert into a graph problem

2.2. Find a tree decomposition of low widthFind a tree decomposition of low width

3.3. Decompose the problem using treeDecompose the problem using tree


Decomposition AlgorithmDecomposition Algorithm1. Convert into a graph problem1. Convert into a graph problem

closed(1)closed(1)

closed(2)closed(2)

closed(21)closed(21)


close(1)close(1)

close(2)close(2)

at1at1 at2at2

move-Rmove-R

move-Lmove-L


close(21)close(21)

close(22)close(22)



closed(1)closed(1)

closed(2)closed(2)



close(1)close(1)

close(2)close(2)

at1at1 at2at2

move-Rmove-R

move-Lmove-L


close(21)close(21)

close(22)close(22)


Decomposition AlgorithmDecomposition Algorithm2. Find a tree decomposition of low width2. Find a tree decomposition of low width

closed(1)closed(1)

closed(2)closed(2)



close(1)close(1)

close(2)close(2)

at1at1 at2at2

move-Rmove-R

move-Lmove-L


close(21)close(21)

close(22)close(22)


Decomposition AlgorithmDecomposition Algorithm3. Decompose planning problem accordingly3. Decompose planning problem accordingly

closed(1)closed(1)

closed(2)closed(2)



close(1)close(1)

close(2)close(2)

at1at1 at2at2

move-Rmove-R

move-Lmove-L


close(21)close(21)

close(22)close(22)



Fluents:Fluents:closed(x) (0<x<21)closed(x) (0<x<21)done1, at1, at2done1, at1, at2


Fluents:Fluents:closed(x) (19<x<41)closed(x) (19<x<41)done1done1,done2,,done2,at1,at2at1,at2


at1at1

at2at2

done1done1


Decomposition AlgorithmDecomposition Algorithm1.1. Convert into a graph problemConvert into a graph problem

2.2. Find a tree decomposition of low widthFind a tree decomposition of low width– Can be done using known algorithms Can be done using known algorithms

(e.g., (Rose ’76), (Amir 2001))(e.g., (Rose ’76), (Amir 2001))

3.3. Decompose the problem using treeDecompose the problem using tree


ConclusionsConclusions• Factored planning algorithmFactored planning algorithm

• Decomposition algorithm relies on Decomposition algorithm relies on known graph algorithmsknown graph algorithms

• Algorithm applicable to all domainsAlgorithm applicable to all domains

• Always Always sound and complete sound and complete (unlike (unlike (Guestrin etal. ’01) - MDPs)(Guestrin etal. ’01) - MDPs)

• Tractability of algorithm depends on Tractability of algorithm depends on quality of domain decompositionquality of domain decomposition and and decomposition decomposition properties of solutionproperties of solution


Applications & Future WorkApplications & Future Work• Nondeterministic domainsNondeterministic domains

• Stochastic domainsStochastic domains

• First-order decompositionsFirst-order decompositions

• Automatic generation of subsumption Automatic generation of subsumption architecturesarchitectures


THE THE ENDEND


Related WorksRelated Works• Factored reasoning [Pearl’88], [Amir & Factored reasoning [Pearl’88], [Amir &

McIlraith’00,’01], [MacCartney etal.’03]McIlraith’00,’01], [MacCartney etal.’03]

• Structured planning: Hierarchical Structured planning: Hierarchical planning, structured planning [Lanskey planning, structured planning [Lanskey & Getoor ’95], Agents [Lansky’91], …& Getoor ’95], Agents [Lansky’91], …

• Stochastic planning: [Parr’99], [Guestrin Stochastic planning: [Parr’99], [Guestrin etal. ’01]etal. ’01]

factored planning

Documents