episodic memory (epmem)web.eecs.umich.edu/~soar/ijcai16/tutorial-2016-epmem.pdf · episodic memory...
TRANSCRIPT
![Page 1: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/1.jpg)
EpisodicMemory(EpMem)IJCAI2016
NateDerbinsky
![Page 2: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/2.jpg)
Agenda
• Bigpicture• Basicusage• Exampleagents
July11,2016 EpisodicMemory(EpMem) 2
![Page 3: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/3.jpg)
Soar9
July11,2016 EpisodicMemory(EpMem) 3
SymbolicLong-TermMemories
SymbolicWorkingMemory
Procedural
Decision
Procedure
ChunkingReinforcementLearning
Action
Semantic
SemanticLearning
Episodic
EpisodicLearning
Spatial Visual SystemObject-basedcontinuousmetricspace
Perception
![Page 4: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/4.jpg)
SoarBasicFunctions
1. Input fromenvironment2. Elaboratecurrentsituation:parallelrules3. Proposeoperatorsviaacceptable preferences4. Evaluateoperatorsviapreferences:Numericindifferentpreference5. Selectoperator6. Applyoperator:Modifyinternaldatastructures:parallelrules7. Output tomotorsystem[andaccesstolong-termmemories]
OperatorDecisionInput Output
StateElaboration
OperatorProposal
OperatorEvaluation
OperatorElaboration
OperatorApplication
July11,2016 EpisodicMemory(EpMem) 4
![Page 5: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/5.jpg)
EpisodicMemory:BigPicture
Episodicmemoryisaweak learningmechanism– Automaticallycaptures,stores,andtemporallyindexesagentstate
– Supportscontent-addressableagentinterfacetoautobiographicalpriorexperience
July11,2016 EpisodicMemory(EpMem) 5
![Page 6: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/6.jpg)
ArchitecturalIntegration
July11,2016 6EpisodicMemory(EpMem)
![Page 7: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/7.jpg)
ArchitecturalIntegration
July11,2016 7EpisodicMemory(EpMem)
![Page 8: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/8.jpg)
ArchitecturalIntegration
July11,2016 8EpisodicMemory(EpMem)
![Page 9: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/9.jpg)
ArchitecturalIntegration
July11,2016 9EpisodicMemory(EpMem)
epmem “link”availableoneach
state!
![Page 10: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/10.jpg)
BasicUsage
• Episodic-memoryrepresentation• Storingknowledge• Retrievingknowledge
July11,2016 EpisodicMemory(EpMem) 10
![Page 11: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/11.jpg)
Episodic-MemoryRepresentation
Similartoworkingmemory:symbolictriples– Structureswithinanepisodeareconnected;separateepisodesaredisconnected
July11,2016 EpisodicMemory(EpMem) 11
(<id0> ^epmem <id4> ^io <id1> ^reward-link <id5> ^smem <id3> ^superstate nil ^svs <id2> ^type state)
(<id1> ^input-link <id7> ^output-link <id6>)
(<id2> ^command <id9> ^spatial-scene <id8>)
(<id3> ^command <id11> ^result <id10>)(<id4> ^command <id13> ^present-id 1 ^result <id12>)(<id8> ^id world)
ID_0
ID_1
io
ID_2
svs
ID_3
smem
ID_4
epmem
ID_5
reward-link
nil
superstate
state
type
ID_6
output-link
ID_7
input-link
ID_8
spatial-scene
ID_9
command
ID_10
result
ID_11
command
ID_12
result
ID_13
command
1
present-id
world
id
![Page 12: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/12.jpg)
StoringKnowledge• AutomaticstoragerequiresEpMem tobeenabled (seeslide12)
• Storagecapturesthetopstateofworkingmemory
• Events triggerstorageofnewepisodes– epmem --set trigger << dc output >>
• dc:decisioncycle(default)• output:newaugmentation ofoutput-link
• Storagetakesplaceattheendofaphase– epmem --set phase << output selection >>
• output isdefault• selectionmaybeuseful forin-the-headagents
July11,2016 EpisodicMemory(EpMem) 12
![Page 13: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/13.jpg)
RetrievingKnowledgeCue-Based
Findtheepisode thatbestmatchesacueandaddittoworkingmemory
TemporalProgressionReplace thecurrentlyretrievedepisodewiththenext/previously encodedepisode
Non-Cue-BasedAddanepisode toworkingmemoryfromepisode #
CommonConstraints:– RequiresthatEpMem isenabled– Onlyoneperstateperdecision– Processedduringphase– Onlyre-processed ifWMchangestocommands– Meta-data(status,etc)automatically cleaned bythearchitecture
July11,2016 EpisodicMemory(EpMem) 13
![Page 14: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/14.jpg)
Cue-BasedRetrieval:Syntax(<epmem> ^command <cmd>)(<cmd> ^query <q>
^neg-query <nq>)
• Theneg-query isoptional• Cuesmustbeacyclic• The<q> and<nq> identifiersformtherootsofepisodesub-graphcues– query representsdesiredstructures– neg-query representsundesiredstructures
July11,2016 EpisodicMemory(EpMem) 14
![Page 15: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/15.jpg)
Cue-BasedRetrieval:CueSemantics
ValuesofcueWMEsareinterpretedbytype– Constant:exactmatch– Long-TermID:exactmatch,stop– Short-TermID:wildcard(butmustbeidentifier)
Cuematchingwillreturnthemostrecentgraph-matchedepisode,orthemostrecentnon-graph-matchedcandidateepisodewiththemaximalepisodescore(w.r.t.root-to-leafpath)
July11,2016 EpisodicMemory(EpMem) 15
![Page 16: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/16.jpg)
Cue-BasedRetrieval:ExampleResult
July11,2016 EpisodicMemory(EpMem) 16
ID_0
ID_1
io
ID_2
svs
ID_3
smem
ID_4
epmem
ID_5
reward-link
nil
superstate
state
type
ID_6
output-link
ID_7
input-link
ID_8
spatial-scene
ID_9
command
ID_10
result
ID_11
command
ID_12
result
ID_13
command
2
present-id
world
id
ID_14
success
ID_15
mapping
ID_16
retrieved
1
cue-size
1
match-cardinality
1
match-score
1
normalized-match-score
1
graph-match
1
memory-id
2
present-id query
nil
superstate
ID_17
io
ID_18
svs
ID_19
smem
ID_20
epmem
ID_21
reward-link
nil
superstate
state
type
ID_22
output-link
ID_23
input-link
ID_24
spatial-scene
ID_25
command
ID_26
result
ID_27
command
ID_28
result
ID_29
command
1
present-id
world
id
ID_30
query
nil
superstate
![Page 17: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/17.jpg)
Cue-BasedRetrievalOptionalModifiers
(<cmd> ^before time-id)(<cmd> ^after time-id)(<cmd> ^prohibit time-id1 time-id2 …)
Hardconstraintsontheepisodesthatcanberetrieved.
July11,2016 EpisodicMemory(EpMem) 17
![Page 18: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/18.jpg)
TemporalProgression
(<cmd> ^next <new-id>)(<cmd> ^previous <new-id>)
Retrievesthenext/previousepisode,temporally,withrespecttothelastthatwasretrieved
July11,2016 EpisodicMemory(EpMem) 18
![Page 19: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/19.jpg)
EpMem Task:VirtualSensingepmem-virtual-sensing.soar
1. Producearandom numberinWMEpMemautomaticallyrecordsthisepisode
2. RemovethenumberfromWMWritetothetrace(forlaterverification)
3. QueryepisodicmemoryWhendidIlastseearandomnumber?
4. ReasonabouttheretrievedepisodeExtractandprintthenumber
July11,2016 19EpisodicMemory(EpMem)
![Page 20: Episodic Memory (EpMem)web.eecs.umich.edu/~soar/ijcai16/Tutorial-2016-epmem.pdf · Episodic Memory (EpMem) IJCAI 2016 Nate Derbinsky. Agenda • Big picture • Basic usage • Example](https://reader030.vdocuments.site/reader030/viewer/2022041023/5ed4743264cb9d0fda746f56/html5/thumbnails/20.jpg)
Eaters!
July11,2016 EpisodicMemory(EpMem) 20