imaginary worlds and grounded (embodied) cognition john b. black teachers college columbia...
TRANSCRIPT
Imaginary Worlds and Imaginary Worlds and Grounded (Embodied) Grounded (Embodied)
CognitionCognition
John B. BlackJohn B. Black
Teachers CollegeTeachers College
Columbia UniversityColumbia University
Research with Gordon BowerResearch with Gordon Bower
Studied Students Reading and Studied Students Reading and Remembering StoriesRemembering StoriesFound That Switches in Point of View Found That Switches in Point of View Lead to Longer Reading Times and Lead to Longer Reading Times and Memory Errors – Spatial LayoutMemory Errors – Spatial LayoutFound That Actions That Changed The Found That Actions That Changed The Story World Were Remembered BetterStory World Were Remembered BetterSeemed to be Story World Separate From Seemed to be Story World Separate From Symbolic Content of Story (Mental Model)Symbolic Content of Story (Mental Model)
Symbolic Content vs Model WorldSymbolic Content vs Model World
Johnson-Laird Proposed Mental Models Johnson-Laird Proposed Mental Models That Involved Imagining Models of Logical That Involved Imagining Models of Logical ExpressionsExpressionsKintsch Proposed Situation Models of Kintsch Proposed Situation Models of Textual Content Separate from Textbase Textual Content Separate from Textbase of Propositional Networksof Propositional NetworksPhilosophers like Frege and Carnap Philosophers like Frege and Carnap Distinguish Between Sense and Distinguish Between Sense and Reference Reference
My Research at TCMy Research at TC
Shifted from Text to MultimediaShifted from Text to Multimedia
What is the Nature of These Mental What is the Nature of These Mental Referent or Imaginary Worlds?Referent or Imaginary Worlds?
How Does One Teach For Imaginary How Does One Teach For Imaginary World Level of Understanding?World Level of Understanding?
World of a CathedralWorld of a Cathedral
Multimedia System to Learn About St. John’s Multimedia System to Learn About St. John’s Cathedral (Van Esselstyn and Black)Cathedral (Van Esselstyn and Black)
Learned a lot from just Floor Plan (survey map Learned a lot from just Floor Plan (survey map knowledge) and Textknowledge) and Text
Providing Virtual Tour (dynamic route map Providing Virtual Tour (dynamic route map knowledge) only helped when given rationale for knowledge) only helped when given rationale for spatial layoutsspatial layouts
Constructing this Cathedral World needed Constructing this Cathedral World needed reasons for spatial layoutreasons for spatial layout
Research with Maggie ChanResearch with Maggie Chan
Crucial Element of Imaginary World Crucial Element of Imaginary World Reasoning is Functional RelationsReasoning is Functional RelationsFunctional Relations Describe How One Functional Relations Describe How One Entity Changes as a Function of Change Entity Changes as a Function of Change in Anotherin AnotherDeeper Lever Understanding Than Deeper Lever Understanding Than Structural Causal RelationsStructural Causal RelationsShowed Direct Manipulation Animation Showed Direct Manipulation Animation Good Way to Learn ThisGood Way to Learn This
Roller Coaster Game StudiesRoller Coaster Game Studies
Snapshots (like Comics) Better than Snapshots (like Comics) Better than System Controlled AnimationSystem Controlled Animation
Consistent with Many Studies of Animation Consistent with Many Studies of Animation in Tversky Survey in Tversky Survey
““Direct Manipulation” of one Factor with Direct Manipulation” of one Factor with Animation of Other Factors Gave Better Animation of Other Factors Gave Better Learning (Chan and Black)Learning (Chan and Black)
Experimental DesignExperimental DesignIV1: Presentation Formats:
text-only, text-plus-static diagrams, text-plus-direct-manipulation animation
IV2: System Complexity:
simple (Energy conversion in a playground swing),
moderate (Energy conversion in a roller coaster ride),
complicated (Energy conversion of pole vaulting)
Dependent Measures:
DV1: Verbal Recall (summary)
DV2: Visual Recall (drawing)
DV3: What-if Scenarios
DV4: What’s wrong here? Case study
DV5: Transfer
Direct Manipulation Direct Manipulation
AnimationAnimationSimple system --- Simple system --- a playground swing a playground swing
Direct Manipulation Direct Manipulation
AnimationAnimation
Complex system --- pole-vaultingComplex system --- pole-vaulting
Result Varies within Type of Result Varies within Type of Student and System ComplexityStudent and System Complexity
General form seems to be for simple General form seems to be for simple systems and capable students text is systems and capable students text is sufficientsufficient
Also for capable students and intermediateAlso for capable students and intermediate
systems text plus diagrams sufficientsystems text plus diagrams sufficient
But for complex systems for any students But for complex systems for any students and less capable students for any system and less capable students for any system need direct manipulation animationneed direct manipulation animation
Video Games for LearningVideo Games for Learning
There is a Lot of Hype About Video There is a Lot of Hype About Video Games as Learning EnvironmentsGames as Learning Environments
However, Almost No Studies That Show However, Almost No Studies That Show This – in fact, Some Show ReverseThis – in fact, Some Show Reverse
But We Have Found Video Games Good But We Have Found Video Games Good as Part of Learning Environment But Not as Part of Learning Environment But Not By ThemselvesBy Themselves
Video Games as Preparation for Video Games as Preparation for Future LearningFuture Learning
CivilizationCivilization
Preparation for Future Learning Preparation for Future Learning Video Game StudyVideo Game Study
Compared Experts in Civilization Compared Experts in Civilization
Formally Learning New MaterialFormally Learning New Material
Civilization Experts Were Better at Civilization Experts Were Better at Formally Learning New Material –Formally Learning New Material –
Particularly Systems RelatedParticularly Systems Related
Thus Video Games May Provide Thus Video Games May Provide
Experiences asExperiences as
Preparation for Future LearningPreparation for Future Learning
Reflective Questions with GamesReflective Questions with Games
Studied Business Student Learning From Studied Business Student Learning From Business Simulation Game (from HBS)Business Simulation Game (from HBS)Students Learned Much More If Game Students Learned Much More If Game Supplemented with Reflective Questions Supplemented with Reflective Questions About Strategies Used and Relation to About Strategies Used and Relation to Background ReadingBackground ReadingArmy Has Found Learning With America’s Army Has Found Learning With America’s Army Game Occurs in After Action Army Game Occurs in After Action Reports Which Provide ReflectionReports Which Provide Reflection
Grounded (Embodied) CognitionGrounded (Embodied) Cognition
Currently Think Imaginary Worlds Involves Currently Think Imaginary Worlds Involves Grounded or Embodied Cognition Grounded or Embodied Cognition
This is a Perceptual Simulation to This is a Perceptual Simulation to Supplement (Provide Mental Referents)Supplement (Provide Mental Referents)
Recent Neural Imaging Studies Show Recent Neural Imaging Studies Show Perceptual Areas of Brain Activated in Perceptual Areas of Brain Activated in Symbolic Tasks in Addition to Symbolic Symbolic Tasks in Addition to Symbolic AreasAreas
How To Investigate This?How To Investigate This?
Glenberg Text ComprehensionGlenberg Text Comprehension
ARIAARIA
REALREAL
??????
ARIA Project –ARIA Project –Associating Real and Imaginary Associating Real and Imaginary
AgentsAgents
ARIA Project Provides Embodiment by Having ARIA Project Provides Embodiment by Having Students Work With Robot Simulations on Students Work With Robot Simulations on Computer Screen Then Construct Robots to Do Computer Screen Then Construct Robots to Do It In Real WorldIt In Real World
Also Provides Embodiment By Having Students Also Provides Embodiment By Having Students Act Out Using There Own Bodies What they Act Out Using There Own Bodies What they Want Robots to DoWant Robots to Do
Being Done as After School Program at The Being Done as After School Program at The School at Columbia U.School at Columbia U.
REAL—Reflective Agent Learning REAL—Reflective Agent Learning environmentenvironment
Provides Embodiment by Allowing Learner Provides Embodiment by Allowing Learner to Input Knowledge Representations in to Input Knowledge Representations in Head of AgentsHead of Agents
Then See How Agent with KnowledgeThen See How Agent with Knowledge
Behaves in Virtual WorldBehaves in Virtual World
Various Depictions of Results ReflectsVarious Depictions of Results Reflects
What is Good and Bad About KnowledgeWhat is Good and Bad About Knowledge
Then Try AgainThen Try Again
Constructing Dynamic Imaginary Constructing Dynamic Imaginary WorldsWorlds
Virtual Worlds with Animated Agents can be Virtual Worlds with Animated Agents can be Effective for Learning ifEffective for Learning if
Spatially MeaningfulSpatially MeaningfulCausally TransparentCausally TransparentDirectly Manipulate Factors and SeeDirectly Manipulate Factors and See
Animated ResultsAnimated ResultsGets Internalized as Imaginary WorldGets Internalized as Imaginary World
Augmented Reality Promises to GiveAugmented Reality Promises to GiveSame Benefits Better Integrated withSame Benefits Better Integrated withReal WorldReal World