a new approach to the design of interactive storytelling engines
DESCRIPTION
A New Approach to the Design of Interactive Storytelling Engines. TeongJoo Ong and John Leggett Texas A&M University. Overview. Introduction and Background Motivation and Scenarios of Use Architecture of the Interactive Storytelling Engine Story Generation - PowerPoint PPT PresentationTRANSCRIPT
A New Approach to A New Approach to the Design of the Design of Interactive Interactive Storytelling Storytelling
EnginesEnginesTeongJoo Ong and John LeggettTeongJoo Ong and John Leggett
Texas A&M UniversityTexas A&M University
OverviewOverview
Introduction and BackgroundIntroduction and Background Motivation and Scenarios of UseMotivation and Scenarios of Use Architecture of the Interactive Architecture of the Interactive
Storytelling EngineStorytelling Engine Story GenerationStory Generation Encoding and Evaluating the Story Encoding and Evaluating the Story
ComponentsComponents Story ExampleStory Example Evaluation Criteria, Contributions Evaluation Criteria, Contributions
and Conclusionsand Conclusions
OverviewOverview
Introduction and BackgroundIntroduction and Background Motivation and Scenarios of UseMotivation and Scenarios of Use Architecture of the Interactive Architecture of the Interactive
Storytelling EngineStorytelling Engine Story GenerationStory Generation Encoding and Evaluating the Story Encoding and Evaluating the Story
ComponentsComponents Story ExampleStory Example Evaluation Criteria, Contributions Evaluation Criteria, Contributions
and Conclusionsand Conclusions
IntroductionIntroduction Stories are used to convey information, Stories are used to convey information,
cultural values, and experiencescultural values, and experiences New technologies have constantly New technologies have constantly
provided increasingly sophisticated means provided increasingly sophisticated means to tell storiesto tell stories
Recent trend is the convergence of Recent trend is the convergence of technology, entertainment, and art in the technology, entertainment, and art in the computercomputer
The ancient art of storytelling can be used The ancient art of storytelling can be used to efficiently convey information in our to efficiently convey information in our increasingly computerized worldincreasingly computerized world
BackgroundBackground
Interactive storytelling is a major research Interactive storytelling is a major research areaarea
Many overlapping approaches have been Many overlapping approaches have been used:used:
Immersive storytellingImmersive storytelling Emergent storytellingEmergent storytelling Plot-based systemsPlot-based systems Interactive authoring of storiesInteractive authoring of stories Character-based systemsCharacter-based systems
These approaches differ in design These approaches differ in design paradigms and technical solutionsparadigms and technical solutions
Background (cont.)Background (cont.) Approaches used in interactive Approaches used in interactive
storytelling fall into the field of Narrative storytelling fall into the field of Narrative IntelligenceIntelligence
Narrative Intelligence commonly refers to Narrative Intelligence commonly refers to research on human narratives and research on human narratives and storytelling, and related technologiesstorytelling, and related technologies
Related research projects from this field:Related research projects from this field: CHOROS (N. M. CHOROS (N. M. Sgouros) Sgouros) Fuzzy Cognitive Maps (FCM) (E. Maffre)Fuzzy Cognitive Maps (FCM) (E. Maffre) HyperSoap (J. Dakss)HyperSoap (J. Dakss) Terminal Time (M. Mataes)Terminal Time (M. Mataes)
OverviewOverview
Introduction and BackgroundIntroduction and Background Motivation and Scenarios of UseMotivation and Scenarios of Use Architecture of the Interactive Architecture of the Interactive
Storytelling EngineStorytelling Engine Story GenerationStory Generation Encoding and Evaluating the Story Encoding and Evaluating the Story
ComponentsComponents Story ExampleStory Example Evaluation Criteria, Contributions Evaluation Criteria, Contributions
and Conclusionsand Conclusions
MotivationMotivation Each of the different approaches used in Each of the different approaches used in
interactive storytelling has its own interactive storytelling has its own strengths and weaknessesstrengths and weaknesses
A hybrid approach that can combine the A hybrid approach that can combine the strengths while reducing or eliminating the strengths while reducing or eliminating the weaknesses of several different approaches weaknesses of several different approaches could potentially be very importantcould potentially be very important
Certain characteristics of the approaches Certain characteristics of the approaches will also be discarded to keep complexity will also be discarded to keep complexity and understandability in checkand understandability in check
Motivation (cont.)Motivation (cont.)
Our hybrid approach proposes the use of an Our hybrid approach proposes the use of an evolutionary algorithm and story templates to evolutionary algorithm and story templates to construct story componentsconstruct story components
Current system states and interaction Current system states and interaction between fuzzy rules impose constraints, between fuzzy rules impose constraints, modifications and evaluations on the story modifications and evaluations on the story components generated by the evolutionary components generated by the evolutionary algorithmalgorithm
The end result is a template-based The end result is a template-based storytelling engine offering flexibility and storytelling engine offering flexibility and robustness in storytellingrobustness in storytelling
Scenarios of UseScenarios of Use
A murder mystery that can yield different A murder mystery that can yield different conclusions depending on the reader’s conclusions depending on the reader’s interaction with computer agents using interaction with computer agents using dynamically generated story componentsdynamically generated story components
Sub-module of a computer game engine Sub-module of a computer game engine that generates story scripts dynamicallythat generates story scripts dynamically
Interactive learning environment that Interactive learning environment that presents concepts in a narrative thread presents concepts in a narrative thread that includes recombination and that includes recombination and indeterminacyindeterminacy
Scenarios of Use in Scenarios of Use in EducationEducation
Multiple experiences (points of view) Multiple experiences (points of view) through changing plots, story lines, and through changing plots, story lines, and character relationships while adhering to character relationships while adhering to a common threada common thread
Student involvement in unfolding the Student involvement in unfolding the story through interactive exploration of story through interactive exploration of alternatives and role-playingalternatives and role-playing
Encourages personalization of the storyEncourages personalization of the story Children as authors of their own storiesChildren as authors of their own stories
OverviewOverview
Introduction and BackgroundIntroduction and Background Motivation and Scenarios of UseMotivation and Scenarios of Use Architecture of the Interactive Architecture of the Interactive
Storytelling EngineStorytelling Engine Story GenerationStory Generation Encoding and Evaluating the Story Encoding and Evaluating the Story
ComponentsComponents Story ExampleStory Example Evaluation Criteria, Contributions Evaluation Criteria, Contributions
and Conclusionsand Conclusions
System ArchitectureSystem Architecture
Evolutionary Programming Subsystem (EPS)
Knowledge Base
Fuzzy Decision Based Subsystem (FDBS)
Graphical/Object Library (GOL)
Integrated Authoring Environment (IAE)
evaluates/modifies parameters
processes/generates story components
monitors/modifies parameters of rules
user interactions
sends combined output
rendering instructions
authors/interactors
Interactive Storytelling EngineInteractive Storytelling Engine
Combines the global search and emergent Combines the global search and emergent characteristics of evolutionary programming characteristics of evolutionary programming and approximate reasoning of fuzzy logic to and approximate reasoning of fuzzy logic to generate new story components generate new story components
The solution can be any valid combination of The solution can be any valid combination of story components in accordance with the story components in accordance with the author’s rules, system states and templatesauthor’s rules, system states and templates
TemplatesTemplates are used to describe story are used to describe story components and the valid contextual sets of components and the valid contextual sets of actions, scripts, objects and agentsactions, scripts, objects and agents
Interactive Storytelling Engine Interactive Storytelling Engine (cont.)(cont.)
Story ComponentsStory Components are the representation of are the representation of a particular time step in the story. They a particular time step in the story. They facilitate the encoding process of the facilitate the encoding process of the Evolutionary Programming Subsystem (EPS)Evolutionary Programming Subsystem (EPS)
Contextual SetsContextual Sets are used to categorize are used to categorize various plot units, events, scripts, actions various plot units, events, scripts, actions and characters within the story componentsand characters within the story components
The The Evolutionary AlgorithmEvolutionary Algorithm explores the explores the problem space using solution instances of a problem space using solution instances of a particular story componentparticular story component
The The Fuzzy Decision Based Subsystem Fuzzy Decision Based Subsystem (FDBS) (FDBS) imposes constraints, modifications and imposes constraints, modifications and evaluations on the solution instances with evaluations on the solution instances with rules and variables in the knowledge base rules and variables in the knowledge base
Interactive Storytelling Engine Interactive Storytelling Engine (cont.)(cont.)
The The Knowledge BaseKnowledge Base stores user stores user interactions, events, fuzzy rules, discrete interactions, events, fuzzy rules, discrete rules and various variables to keep track rules and various variables to keep track of the story’s progressof the story’s progress
In order to ensure proper development In order to ensure proper development and conclusion of the story, underlying and conclusion of the story, underlying story threads must be created by the story threads must be created by the author to act as guidelines for the engineauthor to act as guidelines for the engine
Underlying story threads are coded as a Underlying story threads are coded as a collection of story components so that collection of story components so that solution instances generated by the EPS solution instances generated by the EPS can be evaluated by the FDBScan be evaluated by the FDBS
Integrated Authoring Integrated Authoring Environment (IAE)Environment (IAE)
A collection of tools and interfaces for A collection of tools and interfaces for authors to create interactive storiesauthors to create interactive stories
The collection of editing tools and library The collection of editing tools and library consists of:consists of: Editing ToolsEditing Tools
Agent CharactersAgent Characters Action ScriptsAction Scripts Story ComponentsStory Components The EPSThe EPS The FDBSThe FDBS
LibraryLibrary The Graphical/Object Library (GOL) The Graphical/Object Library (GOL)
OverviewOverview
Introduction and BackgroundIntroduction and Background Motivation and Scenarios of UseMotivation and Scenarios of Use Architecture of the Interactive Architecture of the Interactive
Storytelling EngineStorytelling Engine Story GenerationStory Generation Encoding and Evaluating the Story Encoding and Evaluating the Story
ComponentsComponents Story ExampleStory Example Evaluation Criteria, Contributions Evaluation Criteria, Contributions
and Conclusionsand Conclusions
Story Generation: Step 1Story Generation: Step 1
User interacts with the storyUser interacts with the story Knowledge base records user Knowledge base records user
interactionsinteractions
User
Integrated Authoring Environment (IAE)
Interacts
Knowledge Base
Modifies/Records user interactions
Story Generation: Step 2Story Generation: Step 2
The knowledge base stores parameters The knowledge base stores parameters from previous execution of the EPSfrom previous execution of the EPS
Evolutionary Programming Subsystem (EPS) Knowledge Base
Stores Parameters from previous execution
Story Generation: Step 3 Story Generation: Step 3
Knowledge Base Fuzzy Decision Based Subsystem (FDBS)
Provides rules/parameters for merging and checking
Knowledge base provides various Knowledge base provides various information and rules for the FDBSinformation and rules for the FDBS
Output of FDBS determines the firing Output of FDBS determines the firing strength of the various fuzzy rules strength of the various fuzzy rules
Story Generation: Step 4Story Generation: Step 4
Knowledge Base
Fuzzy Decision Based Subsystem (FDBS)
Evolutionary Programming Subsystem (EPS)
Provides parameters and information from rules to generate story components from templates
Provides templates and contextual sets used to populate story components
Information from the knowledge base Information from the knowledge base and FDBS are merged by the EPS to and FDBS are merged by the EPS to generate new story components generate new story components
Story Generation: Step 5Story Generation: Step 5
Knowledge BaseEvolutionary Programming
Subsystem (EPS)
Stores story component and other necessary data for historical record
State information about the system is State information about the system is stored in the knowledge base to allow stored in the knowledge base to allow the user to return to a particular scene the user to return to a particular scene of the storyof the story
Story Generation: Step 6Story Generation: Step 6
Knowledge Base
Provides story component used to generate rendering instructions
The resultant story component is sent The resultant story component is sent to GOL to be interpreted into a series of to GOL to be interpreted into a series of instructionsinstructions
Graphical/Object Library (GOL)
Story Generation: Step 7Story Generation: Step 7
Graphical/Object Library (GOL)
Integrated Authoring Environment (IAE)
Renders various graphical objects and agents
Various objects and agents are displayed Various objects and agents are displayed in the IAE based on the rendering in the IAE based on the rendering instructions generated in the previous stepinstructions generated in the previous step
These steps are repeated until the story These steps are repeated until the story terminatesterminates
OverviewOverview
Introduction and BackgroundIntroduction and Background Motivation and Scenarios of UseMotivation and Scenarios of Use Architecture of the Interactive Architecture of the Interactive
Storytelling EngineStorytelling Engine Story GenerationStory Generation Encoding and Evaluating the Story Encoding and Evaluating the Story
ComponentsComponents Story ExampleStory Example Evaluation Criteria, Contributions Evaluation Criteria, Contributions
and Conclusionsand Conclusions
Encoding Function Encoding Function Parameters in Floating Parameters in Floating
PointPoint
Each of the floating point numbers Each of the floating point numbers represent a gene in a chromosomerepresent a gene in a chromosome
They can encode the unknown variables They can encode the unknown variables in a function (z = v + w + x) or the in a function (z = v + w + x) or the partition sizes of a surface partition sizes of a surface
1.234 2.345 1.11 3.211 0.121 2.3222 0.982Floating point format
Encoding the Encoding the ChromosomesChromosomes
Chromosomes are created by Chromosomes are created by concatenating the floating point concatenating the floating point numbersnumbers
For the EPS, a chromosome For the EPS, a chromosome consists of a series of floating point consists of a series of floating point numbers numbers
Each floating point number Each floating point number represents a particular story represents a particular story sequence in the templatesequence in the template
1.234 2.345 1.11 3.211 0.121 2.3222 0.982Floating point format
Encoding the Story Encoding the Story ComponentsComponents
2 3 2 …
……………………………….……Valid sets of action and object instances
Gene in a chromosome
For example, a template sequence For example, a template sequence numbered 2, with two object instances numbered 2, with two object instances will be encoded as a floating point will be encoded as a floating point number 2.xx depending on the set being number 2.xx depending on the set being chosenchosen
CrossoverCrossover
Chromosome A
Chromosome B
Crossover operator allows chromosomes Crossover operator allows chromosomes to exchange genetic materialsto exchange genetic materials
Single-point and Multi-point crossovers Single-point and Multi-point crossovers are supported by the EPSare supported by the EPS
Chosen offset for crossover
Chromosome A*
Chromosome B*
MutationMutation
Chromosome A
Chromosome A*
Mutation operator randomly changes the gene in a chromosome
Introduces variations in the populationIntroduces variations in the population
Mutation will happen to this gene
After mutation
Fitness EvaluationFitness Evaluation
Chromosome in a population
Story thread given by the author which may indicate preferences or avoidance of certain actions or object instances
The story components are The story components are evaluated against the story evaluated against the story thread given by the authorthread given by the author
OverviewOverview
Introduction and BackgroundIntroduction and Background Motivation and Scenarios of UseMotivation and Scenarios of Use Architecture of the Interactive Architecture of the Interactive
Storytelling EngineStorytelling Engine Story GenerationStory Generation Encoding and Evaluating the Story Encoding and Evaluating the Story
ComponentsComponents Story ExampleStory Example Evaluation Criteria, Contributions Evaluation Criteria, Contributions
and Conclusionsand Conclusions
The Three Little Pigs Story The Three Little Pigs Story ExampleExample<storyline><storyline>
<timestep order="1" name="introduction" sequence="and"><timestep order="1" name="introduction" sequence="and"> <set type="introduction" pickatmost="1"> <set type="introduction" pickatmost="1"> <element name="act_10" addfitness="1"/><element name="act_10" addfitness="1"/> </set></set> </timestep></timestep> <timestep order="2" name="wolf threat" sequence="and"><timestep order="2" name="wolf threat" sequence="and"> <set type="wolf threat" pickatmost="1"/><set type="wolf threat" pickatmost="1"/> </timestep></timestep> <timestep order="3" name="wolf attack" sequence="and"><timestep order="3" name="wolf attack" sequence="and"> <set type="wolf attack" pickatmost="2" <set type="wolf attack" pickatmost="2" placement="remove"/>placement="remove"/> </timestep></timestep> <timestep order="4" name="wolf's plan" sequence="and" <timestep order="4" name="wolf's plan" sequence="and" loop="true" condition="{isPigStillAvailable}==true OR loop="true" condition="{isPigStillAvailable}==true OR {isWolfDead}==true">{isWolfDead}==true"> <set type="wolf's plan" pickatmost="1" <set type="wolf's plan" pickatmost="1" placement="noremove">placement="noremove"> <element name="act_13" addfitness="2"/><element name="act_13" addfitness="2"/> </set></set> </timestep></timestep></storyline></storyline>
The Three Little Pigs Example The Three Little Pigs Example (cont.)(cont.)
Introduction
Wolf’s Threat
Wolf’s Attack
Wolf’s Plan
While the wolf is still alive or a pig is available for the wolf, these template sets provide various alternatives for the wolf to attack the pigs and the pigs’ response to those attacks
Provides background description of the story to the reader. It also determines the template sets that will be used in the rest of the story by selecting a particular variant of the story
The wolf actor shows up. Depending on how the pigs react, certain action instances could dictate that the pigs are paralyzed with fear, and thus eaten by the wolf which can lead to an early conclusion to the story
Conclusion
The wolf may have been killed or cooked by one of the pigs, or all of the pigs were eaten by the wolf, yielding a conclusion
OverviewOverview
Introduction and BackgroundIntroduction and Background Motivation and Scenarios of UseMotivation and Scenarios of Use Architecture of the Interactive Architecture of the Interactive
Storytelling EngineStorytelling Engine Story GenerationStory Generation Encoding and Evaluating the Story Encoding and Evaluating the Story
ComponentsComponents Story ExampleStory Example Evaluation Criteria, Contributions Evaluation Criteria, Contributions
and Conclusionsand Conclusions
Evaluation CriteriaEvaluation Criteria
Quality of story is based on user’s Quality of story is based on user’s evaluation of the events, deviation evaluation of the events, deviation from the author’s story line, and from the author’s story line, and congruence of the transitions in the congruence of the transitions in the storystory
Responsiveness, performance and Responsiveness, performance and execution speed of the engineexecution speed of the engine
Stability and consistency in the Stability and consistency in the knowledge baseknowledge base
ContributionsContributions
Simple and integrated environment Simple and integrated environment for authors to create interactive and for authors to create interactive and recombinant storiesrecombinant stories
Materials used in one story can be Materials used in one story can be reused in a related story settingreused in a related story setting
A useful educational tool when rules A useful educational tool when rules are integrated into the engine to are integrated into the engine to reflect certain physical or social reflect certain physical or social phenomenaphenomena
ConclusionsConclusions
The hybrid approach used in The hybrid approach used in storytelling is not perfect since it lacks storytelling is not perfect since it lacks the ability to automatically generate the ability to automatically generate story templates out of a knowledge basestory templates out of a knowledge base
Gives authors the power to have full Gives authors the power to have full control of the cyborg authoring control of the cyborg authoring environment and provides tools to environment and provides tools to author various aspects of the interactive author various aspects of the interactive storiesstories