combining novelty-guided and heuristic-guided search · combining novelty-guided and...

Combining Novelty-Guided and Heuristic-Guided Search

Master ThesisArtificial Intelligence Group

Dario Maggi 29th July 2016

Examiner: Prof. Dr. Malte HelmertSupervisor: Dr. Thomas Keller

29.07.16 Dario Maggi 2

Overview

● 1. Introduction– Planning Task

– Heuristic Guided Search: Greedy Best-First Search

● 2. Novelty– What is novel?

– Iterated Width

● 3. Heuristic- and Novelty- Guided Search● 4. Experiments● 5. Conclusion

29.07.16 Dario Maggi 3

1. Introduction

A Z

B

C

D

E

29.07.16 Dario Maggi 4

Planning Task

A Z

B

C

D

E

● VariablesTruck: A, B, C, D, E, ZPackage 1: A, Truck, Z...

● OperatorsdriveA→B, driveA→C ...load1, load2, load3unload1, unload2, unload3

● Initial State● Goal State

Truck at APackage 1 in APackage 2 at APackage 3 at A

AAAAload1

ATAA ZZZZ

Packet 1 in Truck

29.07.16 Dario Maggi 5

Breadth-First Search

A Z

B

C

D

E

AAAA

ATAA

load1 load2 load3

AATA AAAT

BTAA CTAA

ETAA

DTAA ZTAA

BATA CATA

EATA

DATA ZATA

BAAT CAAT

EAAT

DAAT ZAAT

ZZAA

BAAA CAAA

EAAA

DAAA ZAAA

ZAZA ZAAZ

driveA→B driveA→C

ZZZZ

29.07.16 Dario Maggi 6

Breadth-First Search

A Z

B

C

D

E

AAAA

ATAA

load1 load2 load3

AATA AAAT

BTAA CTAA

ETAA

DTAA ZTAA

BATA CATA

EATA

DATA ZATA

BAAT CAAT

EAAT

DAAT ZAAT

ZZAA

BAAA CAAA

EAAA

DAAA ZAAA

ZAZA ZAAZ

driveA→B driveA→C

ZZZZ

Plan:load1,driveA→C,driveC→E,driveE→Z,unload1,...

29.07.16 Dario Maggi 7

Heuristic Guided Search

● Heuristic h: Distance to Goal Estimator

● h: #Packets + #Packets in A - #Packets in Z

ATAA

AAAAh( ) = 4 + 4 - 0 = 8

h( ) = 4 + 3 - 0 = 7

A Z

B

C

D

E

29.07.16 Dario Maggi 8

Greedy Best-First Search (GBFS)

h = 8

h = 7

AAAA

ATAA

load1 load2 load3

AATA AAAT

BTAA CTAA

ETAA

DTAA ZTAA

BATA CATA

EATA

DATA ZATA

BAAT CAAT

EAAT

DAAT ZAAT

ZZAA

BAAA

CAAA

driveA→B

driveA→C

h = 6

A Z

B

C

D

E

29.07.16 Dario Maggi 9

Greedy Best-First Search (GBFS)

h = 8

h = 7

AAAA

ATAA

load1 load2 load3

AATA AAAT

BTAA CTAA

ETAA

DTAA ZTAA

BATA CATA

EATA

DATA ZATA

BAAT CAAT

EAAT

DAAT ZAAT

ZZAA

BAAA

CAAA

driveA→B

driveA→C

h = 6

A Z

B

C

D

E

Uninformed Heuristic Region (UHR)

29.07.16 Dario Maggi 10

GBFS and UHRs

● Two Problems:– No Guidance in UHRs

– GBFS might expand multiple UHRs simultaneously

● What do we do?

Change the approach in UHRs

29.07.16 Dario Maggi 11

2. Novelty

● How ''new'' is a state?

Truck Package 1 Package 2 Package 3 Novelty

A A A A 1

A T A A 1

A A T A 1

A A A T 1

B T A A 1

B A T A 2

A A A A 5

29.07.16 Dario Maggi 12

Iterated Width

29.07.16 Dario Maggi 13

Iterated Width

AAAA

ATAA

load1 load2 load3

AATA AAAT

BTAA CTAA

ETAA

DTAA ZTAA

BATA CATA

EATA

DATA ZATA

BAAT CAAT

EAAT

DAAT ZAAT

ZZAA

BAAA CAAA

EAAA

DAAA ZAAA

ZAZA ZAAZ

driveA→B driveA→C

IW(1)

A Z

B

C

D

E

29.07.16 Dario Maggi 14

Iterated Width

29.07.16 Dario Maggi 15

Iterated Width +

● Extension for IW● Compute relaxed plan● Novelty Check on different Layers● Layer is the number of the relaxed plan facts that

were made true on a path

Iterated Width +

29.07.16 Dario Maggi 16

Iterated Width +

AAAA

ATAA

load1 load2 load3

AATA AAAT

BTAA CTAA BATA CATA BAAT CAAT

BAAACAAA

EAAA

driveA→B driveA→C

Relaxed Plan:

Truck: B, E, D, ZPacket1: T Packet2: TPacket3: T

EAAA

A Z

B

C

D

E

layer: 0

layer: 1

layer: 2

29.07.16 Dario Maggi 17

3. Heuristic- and Novelty- Guided Search

● Detect UHRs → UHR Threshold● In UHR: Start local IW● Expand nodes at maximum once

GBFS

29.07.16 Dario Maggi 18

3. Heuristic- and Novelty- Guided Search

● Detect UHRs● In UHR: Start local IW● Expand nodes at maximum once

● Star Approach

GBFS

29.07.16 Dario Maggi 19

Star Approach

s0 g

UHR

UHR

29.07.16 Dario Maggi 20

Star Approach

s0 g

UHR

UHR

29.07.16 Dario Maggi 21

Star Approach

s0 g

IW(1)

UHR

UHR

Szenario 1

29.07.16 Dario Maggi 22

Star Approach

s0 g

UHR

UHR

Szenario I

29.07.16 Dario Maggi 23

Star Approach

s0 g

UHR

UHR

Szenario I

29.07.16 Dario Maggi 24

s0 g

IW(1)

UHR

UHR

Star Approach

Szenario 2

29.07.16 Dario Maggi 25

Star Approach

s0 g

IW(1)

IW(2)

UHR

UHR

Szenario 2

29.07.16 Dario Maggi 26

Star Approach

s0 g

IW(1)

IW(2)

UHR

UHR

Szenario 2

29.07.16 Dario Maggi 27

Star Approach

s0 g

IW(1)

IW(2)

UHR

UHR

Szenario 3

29.07.16 Dario Maggi 28

Star Approach

s0 g

UHR

UHR

Szenario 3,GBFS takes over again

IW(1)

IW(2)

29.07.16 Dario Maggi 29

4. Experiments

● Implemented in Fast Downward● 46 Domains, total of 1456 Problems● Star Approach

● vs GBFS and GBFSpreferred (ff-heuristic)

29.07.16 Dario Maggi 30

Coverage: Star Approach

29.07.16 Dario Maggi 31

Comparison

1'456

1'000

GBFS1'075

GBFSpreferred1'171

Star: IW+(1), UHRsize(10)1'251

29.07.16 Dario Maggi 32

5. Conclusion

● Helpful to switch to IW● Start more, smaller local searches

● Even Closer Coupling by only using IW to escape UHRs?

29.07.16 Dario Maggi 33

Lessons Learned

● Fast Downward, Novelty, Data Structure

● Lessons I wish I had finally learned– Branching Strategy

– Setup Testing Environment

– Carefulness

– Don't Panic

– Daily Routine

Thank You

Thank You for your attention