continuous time and resource uncertainty cse 574 lecture spring ’03 stefan b. sigurdsson
Post on 08-Jan-2018
215 Views
Preview:
DESCRIPTION
TRANSCRIPT
Continuous Time and Resource Uncertainty
CSE 574 LectureSpring ’03
Stefan B. Sigurdsson
(Big Mars Rover Picture)
Lecture Overview
Context– Classical planning– The Mars Rover domain– Relaxing the assumptions– Q: What’s so different?
InnovationDiscussion
(Shakey Picture)
Slide shamelessly lifted from http://www.cs.nott.ac.uk/~bsl/G53DIA/Slides/Deliberative-architectures-I.pdf
STRIPS-Like Planning
Propositional logicClosed world assumptionFinite and staticComplete knowledgeDiscrete timeNo exogenous effects
World Description
Attainment – “Win or lose”Conjunctions of positive literals
Goal Description
Conjunctive preconditionSTRIPS operatorsConj. effect (add/delete)InstantaneousSequentialDeterministic
Actions
Plan…
(Big Mars Rover Picture)
The Mars Rover Domain
Robot control, with…– Positioning and navigation– Complex choices (goals and actions)– Rich utility model– Continuous time and concurrency – Uncertain resource consumption– Metric quantities– Very high stakes!
But alone in a finite, static universe
Resources? Metric Quantities?
What are those?
Various flavors:– Exclusive (camera arm)– Shared (OS scheduling) – Metric quantity (fuel, power, disk space)
Uncertainty
Alright, Whatsit Really Mean?
Is This Really A Planning Problem?Better suited to OR/DT-type scheduling?
– Time, resources, metric quantities, concurrency, complicated goals/rewards…
Complex, inter-dependent activities– Select, calibrate, use, reuse, recalibrate sensors– OR-type scheduling can’t handle rich choices
Insight: Maybe we can borrow some tricks?
Can Planners Scale Up?
Large plans– Sequences of ~ 100 actions
Where do we start?– POP? – MDP? – Graph/SATplan?
Can Planners Scale Up?
Large plans– Sequences of ~ 100 actions
Where do we start?– POP? (Branch factors are too big)– MDP? – Graph/SATplan?
Can Planners Scale Up?
Large plans– Sequences of ~ 100 actions
Where do we start?– POP? (Branch factors are too big)– MDP? (Complete policy is too large)– Graph/SATplan?
Can Planners Scale Up?
Large plans– Sequences of ~ 100 actions
Where do we start?– POP? (Branch factors are too big)– MDP? (Complete policy is too large)– Graph/SATplan? (Discrete representations)
Which Extensions First?
Metric quantities– Time– Resources
Resource UncertaintyConcurrency
What about non-determinism? Reasonable for Graphplan?
A (Very Incomplete)Research Timeline
1971 STRIPS (Fikes/Nilson)1989 ADL (Pednault)1991 PEDESTAL (McDermott)1992 UCPOP (Penberthy/Weld) 1992 SENSp (Etzioni et al.) CNLP (Peot/Smith)1993 Buridan (Kushmerick et al.)1994 C-Buridan (Draper et al.) JIC Scheduling (Drummond et al.) HSTS (Muscettola) Zeno (Penb./Weld) Softbots (Weld/Etzioni) MDP (Williamson/Hanks)1995 DRIPS (Haddawy et al.) IxTeT (Laborie/Ghallab)1997 IPP (Koehler et al.)
Not implemented ADL impl.
SensingConformant
Contingent
Planning + schedulingMetric time/resources
Safe planningDec. theory goalsUncertain utility
Shared resources
1998 PGraphplan (Blum/Langford) Weaver (Blythe) PUCCINI (Golden) CGP (Smith/Weld) SGP (Weld et al.) Pgraphplan (Blum/Langford)1999 Mahinur (Onder/Pollack) ILP-PLAN (Kautz/Walzer) TGP (Smith/Weld) LPSAT (Wolfman/Weld)2000 T-MDP (Boyan/Littman) HSTS/RA (Jónsson et al.)
Since then?
Uncertain/dynamicSensing
Conformant
ContingentResources
Resources
Domain Assumptions
Expressive logicNon-determinism
ObservationGoal modelPlan utility
Durative actionsComplex concurrence
Continuous timeMetric quantitiesBranching factor
Resource uncertaintyResource constraints
Goal selectionSafe planning
Exogenous events
STR
IPS
UC
POP
CG
PC
NLP
SEN
SpB
urid
anW
eave
rC
-Bur
idan
MD
PPO
-MD
PS-
MD
PT-
MD
PF-
MD
PLP
SAT
Mar
s Rov
er
Classical
Bleeding edge
Select contingencies
Serialized goals?
Brain-teaser: Domain Spec
State space S– Cartesian product of continuous and discrete axes
(time, position, achievements, energy…)
Initial state si– Probability distribution
Domain theory– Concurrent, non-deterministic, uncertain
What else?(S, si, , …)
Brain-teaser: Kalman Filters
Curiously missing from the paper we read (?)
1983 Kalman filters paper: Voyager enters Jupiter orbit through a 30 second window after 11 years in space
Hugh Durrant-Whyte’s robots
Why not for the Mars Rover?
Context Summary
Complex, exciting domainPushes the planning envelope
– Expression– Scaling
Where do we start?
Lecture Overview
ContextInnovation
– Just-in-case planning– Incremental contingency planning
Discussion
Just-In-Case Planning
Motivated by domain characteristics– Metric quantities – Large branch factors
Implications – Not plan, not policy– Expanded plan
What about concurrency?
Branch Heuristics
Most probable failure point (scheduling)Highest utility branch point (planning)
What is the intrinsic difference?
When To Execute A Contingency?
Incremental Contingency Planning AlgorithmInput: Domain description and master planOutput: Highest-utility branch pointAlgorithm:
– Compute value, estimate resources during master plan– Approximate branch point utilities– Select highest-utility branch point– Solve w/ new initial, goal conditions– Repeat while necessary
Branch Utility Approximation
… without constructing plan– Construct a plan graph– Back-propagate utility functions through plan
graph, instead of regression searching– Compute branch point utilities throughout input
plan
Back-Propagating Distributions
Mausam:
“Some parts of the paper are tersely written, which make it a little harder to understand. I got quite confused in the discussion of utility propagation. It would have been nicer had they given some theorems about the soundness of their method.”
Well, me too
Back-Propagating Distributions
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
5
Back-Propagating Distributions
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
5
Back-Propagating Distributions
5
15
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
5
5
15
5
25
Back-Propagating Distributions
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
5
5
15
12
5
25
Back-Propagating Distributions
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
5
5
15
12
r
12
t
12
5
25
Back-Propagating Distributions
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
5
25
Back-Propagating Distributions
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
15
t
Back-Propagating Distributions
5
25
15
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
5
251
15
t+
Back-Propagating Distributions
15
t
25
6
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+
15
t
15
t
25
6
15
5
251
15
t+
25
6
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+
15
t
15
t
25
6
15
5
251
15
t+
25
6
15
r+
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+1
5
t
25
6
15
5
251
15
t+
25
6
15+
15
t
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+
5
t
15
t
25
6
15
5
251
15
t+
25
6
15+
1 18+
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+
5
t
15
t
25
6
15
5
251
15
t+
25
6
15+
1 18+
15 25
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+
5
t
15
t
25
6
15
5
251
15
t+
25
6
15+
1 18+
8
15 25
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+
5
t
15
t
25
6
15
5
251
15
t+
25
6
15+
1 18+
8
15 25
18
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+
5
t
15
t
25
6
15
5
251
15
t+
25
6
15+
1 18+
8
15 25
18
(CDE)
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
15
5
25+
5
t
15
t
25
6
15
5
251
15
t+
25
6
15+
1 18+
8
15 25
18
(CDE)
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
5
t
15
5
251
15
t+
25
6
15+
1 18+
15 25
[(CDE)(ABDE)]
[(DCE)(AB)(DABE)]
A
C
D
B
E
(1, 5)
(3, 3)
(10, 15)
(10, 15)
(2, 2)
p
s
q
r
t
g
g’
1
15
5
5
15
12
rr
12
t
12
Back-Propagating Distributions
5
t
15
5
251
15
t+
25
6
15+
1 18+
15 25
18
(CDE, ABDE)
6
25
16
6
25 26
(DCE, AB, DABE)
5
Utility Estimation
p
s
18
(CDE, ABDE)
6
25
16
6
25 26
(DCE, AB, DABE)
5
Utility Estimation
p
s
18
(CDE, ABDE)
6
25
16
6
25 26
(DCE, AB, DABE)
5
16
6
25
(DCE, ABDE)
MAX operator:
Utility Estimation
p
s
18
(CDE, ABDE)
6
25
16
6
25 26
(DCE, AB, DABE)
5
16
6
25
(DCE, ABDE)
MAX operator:
(Then combine w/Monte Carlo results)
Lecture Overview
ContextInnovationDiscussion
– Q: Evaluation? Inference?
Evaluation
Optimal branch selection? (Greedy…)
Incremental Contingencies…
Sometimes adding one contingency at a timeis non-optimal
Examples?
Incremental Contingencies…
1.0
0
Rain
Shine
0.5
0.5
0
1.0
Work Go clim
bing
Exercis
e
Sometimes adding one contingency at a timeis non-optimal
Evaluation
Optimal branch selection?What else?
Inference
Where can we take these ideas?What can we add to them?
Inference
Where can we take these ideas?What can we add to them?
Optimal branch selectionOptimistic branchingMutexes in plan graphNoisy/costly sensors
top related