E19.2174 Cognitive Science and Educ. Technology I

New York University

Class # 4

Spring 20010

© 2010, Jan L. Plass 1

Cognitive Science

“Instructional media are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that delivers our groceries causes changes in our nutrition." (Clark, 1983, p. 445)

Introduction

Cognitive Science

Overview

• Cognitive Theory of Multimedia Learning • Integrated Model of Text & Picture Comprehension • Application to the Design of Multimedia Instruction

Next Week • Cognitive Load Theory • Draft Design Document Due

Cognitive Science

Information Processing

Cognitive Science

What is multimedia?

Multimedia Learning

Cognitive Science

Multimedia:

“presenting both words and pictures.”

Multimedia Learning

E19.2174 Cognitive Science and Educ. Technology I

New York University

Class # 4

Spring 20010

© 2010, Jan L. Plass 2

Cognitive Science

Multimedia Learning

Vitruvius, 1511

http://upload.wikimedia.org/wikipedia/commons/thumb/9/9e/VitruviusTenBooksMHMorgan1914p296.jpg/566px-VitruviusTenBooksMHMorgan1914p296.jpg

Cognitive Science

Multimedia Learning

Comenius, 1658

http://pauillac.inria.fr/~codognet/images/cesariano.jpg

Cognitive Science

Delivery Medium System used to present instruction (e.g., book, computer, voice)

Presentation Mode Format used to represent the instruction (e.g., words, pictures)

Sensory Modality Information processing channel used by learner (e.g., auditory, visual)

Multimedia Learning

Cognitive Science

Delivery Medium System used to present instruction (e.g., book, computer, voice)

Presentation Mode Format used to represent the instruction (e.g., words, pictures)

Sensory Modality Information processing channel used by learner (e.g., auditory, visual)

Multimedia Learning

Cognitive Science

What is multimedia learning?

Multimedia Learning

Cognitive Science

Multimedia learning

Mayer: Building mental representations from words and pictures.

Schnotz: Using external representations as information sources to construct internal representations of learning content, and storing these representations in long-term memory.

Multimedia Learning

E19.2174 Cognitive Science and Educ. Technology I

New York University

Class # 4

Spring 20010

© 2010, Jan L. Plass 3

Cognitive Science

What is the rationale for multimedia learning?

Multimedia Learning Scenario

Cognitive Science

Class Discussion

What does the multimedia principle predict? Discuss its implication for the design of multimedia learning environments. Consider: -the modalities/presentation modes used -the type of learning fostered -the learning outcomes achieved

Report a summary of your thoughts to the class.

Cognitive Science

Multimedia principle:

People learn more deeply from words and pictures than from words alone.

Desired Learning Outcomes

• Remembering (Retention)

• Understanding (Transfer)

Multimedia Learning

Cognitive Science

Metaphors

Multimedia Learning as Information Acquisition

Multimedia Learning as Knowledge Construction

What are the implications of each metaphor for the design of multimedia instruction?

Multimedia Learning

Cognitive Science

Multimedia Learning as Information Acquisition

• Teacher: Information provider

• Learner: Passive recipient

• Content: Information

• Goal: Add information to memory

• Multimedia: Delivery vehicle

Multimedia Learning

Cognitive Science

Multimedia Learning as Information Acquisition

• Teacher: Information provider

• Learner: Passive recipient

• Content: Information

• Goal: Add information to memory

• Multimedia: Delivery vehicle

Multimedia Learning

E19.2174 Cognitive Science and Educ. Technology I

New York University

Class # 4

Spring 20010

© 2010, Jan L. Plass 4

Cognitive Science

Multimedia Learning as Knowledge Construction

• Teacher: Information provider Cognitive guide

• Learner: Passive recipient Active sense-maker

• Content: Information Knowledge

• Goal: Add information Build coherent mental structure

• Multimedia: Delivery vehicle Provides cognitive guidance

Multimedia Learning

Cognitive Science

Multimedia Learning as Knowledge Construction

• Teacher: Information provider Cognitive guide

• Learner: Passive recipient Active sense-maker

• Content: Information Knowledge

• Goal: Add information Build coherent mental structure

• Multimedia: Delivery vehicle Provides cognitive guidance

Multimedia Learning

Cognitive Science

Assumptions • Dual Channels (Dual Coding, Paivio, 1986, Baddeley, 1992) • Limited Capacity (Cognitive Load, Sweller, 1999; Baddeley, 1992) • Active Processing (Wittrock, 1989)

Cognitive Theory of Multimedia Learning

Cognitive Science

Assumptions • Active Processing (Wittrock, 1989, 1992) To comprehend, the learner must actively create meaningful relations. -Relationships among instructional concepts -Relationships between instructional concepts and the learner’s prior knowledge and experience

Cognitive Theory of Multimedia Learning

Cognitive Science

Active Learning

Multimedia Learning

Cognitive Activity LOW HIGH

LOW ? ?

HIGH ? ?

Beh

avio

ral A

ctiv

ity

Cognitive Science

Active Learning

Multimedia Learning

Cognitive Activity LOW HIGH

LOW Does not foster meaningful learning

Fosters meaningful learning

HIGH Does not foster meaningful learning

Fosters meaningful learning

Beh

avio

ral A

ctiv

ity

E19.2174 Cognitive Science and Educ. Technology I

New York University

Class # 4

Spring 20010

© 2010, Jan L. Plass 5

Cognitive Science

Cognitive Theory of Multimedia Learning

Select Images

Select Words

Organize Images

Organize Words

Sounds

Images

Integrate

Verbal Model (Verbal Mental Representation)

Pictures

Words

Multimedia Presentation

Ears

Eyes

Sensory Memory

Pictorial Model (Visual Mental Representation)

Long-term Memory

Prior Knowledge

Working Memory

Cognitive Science

Select relevant information from what is presented

-words –> processed in verbal WM (text base) -images –> processed in visual WM (image base)

Cognitive Theory of Multimedia Learning

Cognitive Science

Organize the pieces of information into a coherent mental representation

-text base –> verbal mental model -image base –> pictorial mental model

-Associative processing: Associations within visual and within verbal system

Cognitive Theory of Multimedia Learning

Cognitive Science

Integrate the newly constructed representation with others

-Build referential connections between the two representations -Integrate with mental model

Cognitive Theory of Multimedia Learning

Cognitive Science

Integrated Text & Picture Comprehension Model

• Sensory Registers

• Working Memory

• Long term memory

• Perceptual Level: multiple sensory channels

• Cognitive Level: verbal and pictorial channels

Cognitive Science

Integrated Text & Picture Comprehension Model

Two Basic Forms of Representation:

Descriptive representations - consist of symbols Relationship to referent based on convention

Depictive representations - consist of icons Relationship to referent based on perceptual similarity

E19.2174 Cognitive Science and Educ. Technology I

New York University

Class # 4

Spring 20010

© 2010, Jan L. Plass 6

Cognitive Science

Integrated Text & Picture Comprehension Model

Forms of Internal Representation:

Verbal: Text-surface representation Propositional Model Descriptive

Pictorial: Perceptual representation Mental model Depictive

Cognitive Science

Integrated Text & Picture Comprehension Model

• Both verbal and pictorial information can enter working memory through different sensory channels

• Both verbal and pictorial information require prior knowledge that is stored in long-term memory

Cognitive Science

Integrated Text & Picture Comprehension Model

Cognitive Science

Integrated Text & Picture Comprehension Model

• Selection of information

• Organization of information

• Activation of prior knowledge

• Active coherence formation by integration of information from different sources

Cognitive Science

Integrated Text & Picture Comprehension Model

How does this model differ from the

Cognitive Theory of Multimedia Learning?

Cognitive Science

Media Effects Is one medium better than another?

Multimedia Effects Is multimedia instruction effective? Single vs. multiple representations

Interaction Effects For whom is multimedia effective?

(Mayer, 1997)

Multimedia Learning

E19.2174 Cognitive Science and Educ. Technology I

New York University

Class # 4

Spring 20010

© 2010, Jan L. Plass 7

Scenario

Cognitive Science

Group Activity

How can the Cognitive Theory of Multimedia Learning and the Integrated Model of Text & Picture Comprehension be used to inform the design of effective multimedia instruction? Use your own example or one of these scenarios for your answer. •  Introduce medical students to human anatomy. •  Improve freshmen students’ Arabic vocabulary and support the

process of reading a text in Arabic. •  Train experienced pilots on the instruments of a new type of

airplane. •  Introduce the history of the American civil war to high school

students. •  Teach college students about molecular structures in organic

chemistry.