aect 2016 multimedia

05/03/2023 1

DDL – Multimedia & Video:Empirical Effects of Multimedia Design on

Perception, Learning Effectiveness, and Cognitive Load

2

Agenda Introduction About ODUOnline Problem Statement Research Questions Methods Findings Conclusion Your thoughts? comments? revisions?

3

Old Dominion University: ODUOnline

4

General Problem

This experiment investigated the effectiveness of several video design techniques to help determine how these multimedia designs can be most effectively applied in authentic instructional design contexts.

5

Independent variable (multimedia presentation design) Instructor-only, Slides-only, Video-switching, Dual-windows, and Superimposed-slides

Research Questions

6

“Instructor-Only” multimedia design:

7

“Slides-Only” multimedia design:

8

“Video-Switching” multimedia design:

9

“Dual-Windows” multimedia design:

10

“Superimposed-Slides” multimedia design:

11

Credibility and Immediacy

Credibility The learner’s perception of the instructors expertise,

concern for students, & trustworthiness Immediacy

Smiling, friendly, gestures, tone, eye-contact Ability to effectively communicate

Both factors that enhance motivation and could impact learning

12

Research Questions

What is the effect of instructor-only, slides-only, video-switching, dual-windows, and superimposed-slides multimedia presentation designs on:

1. Learning effectiveness? (recall and comprehension pre & post-test) 2. Perceived instructor credibility? (McCroskey & Teven, 1999) 3. Nonverbal immediacy? (Richmond, Gorham, & McCroskey, 1997) 4. General immediacy? (Anderson, 1979) 5. Instructional environment design? (Jayasinghe, Morrison, & Ross, 1997)6. Cognitive load? (NASA, 1986)

13

Research Methods

14

Participants 450 started the study: 226 = NASA TLX, 211 = survey, and 171 =

post-test. ~30 participants randomly assigned into each treatment group 61.5% female and 30.5% male Average age = 27.1, with a median age = 22 years

Range from 17 to 66 120 participants reported using laptops, 62 = workstations or

desktops, 39 = phones, and 21 = tablets + mini tablets

15

Procedures Participant invitations

Student Announcements (4 weeks) Full page ad Mace & Crown, and online ad, (1 week) $5 Starbucks or Amazon gift card, drawing for $80 Kindle

Technical requirements Any Internet capable device

Time required Approximately 45 minutes

Tasks involved….

Participant Procedure

16

InitialData Collection

Website

1. Informed consent2. Demographics & 20-item Pre-test3. Assignment into treatment & watch video

Instructor-Only Video

Superimposed-Slides Video

TLX, Surveys,And Post-Test

TLX, Surveys,And Post-Test

OptionalGiftCard

4. NASA TLX5. Credibility6. Nonverbal Immediacy7. General Immediacy8. Design9. 20-item Post-test10. Gift card selection

Slides-Only Video

TLX, Surveys,And Post-Test

Video-Switching Video

TLX, Surveys,And Post-Test

Dual-Windows Video

TLX, Surveys,And Post-Test

17

Research Findings

18

Research Question 1: Learning Effectiveness No statistically significant difference

ANCOVA, F(4,151) = .56, p = .7. All five groups appeared to recall and comprehend the subject

matter pre-test scores (M = 10.02, SD = 3.50), and post-test scores (M = 13.00,

SD = 3.25), t(156) = 11.53, p < .01 All five groups performed similarly on the multiple-choice post-test

Design of each treatment too similar to differentiate? (include problem solving in future projects?)

19

Research Question 2: Instructor Credibility Inclusion of slides appeared to enhance instructor credibility

ANOVA was significant, F(4,203) = 2.47, p < .05

Dual-Windows Superimpossed-Slides Video-Switching Slides-Only Instructor-Only4.5

4.7

4.9

5.1

5.3

5.5

5.7

5.9

5.75

5.53 5.51 5.49

5.1

Source Credibility

20

Research Question 2: Instructor Credibility Inclusion of slides appeared to enhance instructor credibility

ANOVA was significant, F(4,203) = 2.47, p < .05

Dual-Windows Superimpossed-Slides Video-Switching Slides-Only Instructor-Only4.5

4.7

4.9

5.1

5.3

5.5

5.7

5.9

5.75

5.53 5.51 5.49

5.1

Source Credibility

21

Research Question 3: Nonverbal Immediacy Inclusion of instructor appeared to enhance immediacy

ANOVA indicated a significant difference between treatment groups, F(4,206) = 4.68, p <.01

Superimpossed-Slides Video-Switching Instructor-Only Dual-Windows Slides-Only3.5

3.7

3.9

4.1

4.3

4.5

4.37

4.29 4.27 4.24

3.83

Nonverbal Immediacy

22

Research Question 3: Nonverbal Immediacy Inclusion of instructor appeared to enhance immediacy

ANOVA indicated a significant difference between treatment groups, F(4,206) = 4.68, p <.01

Superimpossed-Slides Video-Switching Instructor-Only Dual-Windows Slides-Only3.5

3.7

3.9

4.1

4.3

4.5

4.37

4.29 4.27 4.24

3.83

Nonverbal Immediacy

23

Research Question 4: General Immediacy

No statistically significant difference superimposed-slides group perceived the general immediacy of the instructor

highest, though the results were not statistically significant, F(4,205) = .82, p = .51

Design of survey may have impacted results Participants first required to read a short paragraph that defines “immediacy” Survey items did not ‘fit’ in online survey tool format

Exclude in future research projects?

24

Research Question 5: Instructor Evaluation Measure Item 9: desire to continue in the course as presented:

ANOVA analysis indicated that there was a significant difference between treatments on this instrument, F (4,206) = 4.03, p < .01

Instructor-Only Slides-Only Video-Switching Superimposed-Slides Dual-Windows2.22.42.62.8

33.23.43.63.8

44.24.44.64.8

5

4.35

3.443.27

3 2.97

Desire to Drop Course

25

Research Question 5: Instructor Evaluation Measure Item 9: desire to continue in the course as presented:

ANOVA analysis indicated that there was a significant difference between treatments on this instrument, F (4,206) = 4.03, p < .01

Instructor-Only Slides-Only Video-Switching Superimposed-Slides Dual-Windows2.22.42.62.8

33.23.43.63.8

44.24.44.64.8

5

4.35

3.443.27

3 2.97

Desire to Drop Course

26

Research Question 6: Cognitive Load No statistically significant differences

(average TLX score), F(4,221) = .56, p = .69 HD video/slides + studio grade audio + user centered interface design =

low extraneous cognitive load? Subject matter not challenging enough? (low intrinsic cognitive load?) Instructional design not engaging enough? (low application of germane

resources?) Make design more interactive? (use different subject matter in future

projects?)

27

Other Findings

28

Media Comparison: No statistically significant difference

post-test, F=(3,167) = 1.19 p = .32

Desktop/Workstation Laptop Phone Tablet11

11.5

12

12.5

13

13.5

14

14.5

15

13.3

12.72 12.57

14.56

Device Type compared to Post-test

29

Conclusions

30

Key Findings All five designs appear applicable in terms of recall and

comprehension learning effectiveness HD studio produced audio and video Following our ID’s PowerPoint Guidelines

Instructor credibility and immediacy can be enhanced by including both slides/presentation and visuals of the instructor

Less experienced students require additional support and motivation The pedagogical design of the course is more important than the

technology/media used to deliver that course

31

Future Research Directions?

Subject matter? Audio only no visuals? Visuals only no audio? Text density? Graphics only? Let students choose? TelePresence? Interactive instructor? Problem solving post-test?

Standing/walking instructor? Shorten survey/post-test Transcript window? More than 20 minutes? Less than 20 minutes? Eye-tracking?

05/03/2023 32

Thank You!Dr. Miguel Ramlatchan, mramlatc@odu.edu

Dr. Ginger Watson, gswatson@odu.eduDr. John Baaki, jbaaki@odu.edu

Dr. Peter Baker, pbaker@odu.edu

33

ReferencesAnderson, J. F. (1979). Teacher immediacy as a predictor of teaching effectiveness. In D.

Nimmo (Ed.), Communication Yearbook 3. New Brunswick, NJ: Transaction Books.Jayasinghe, M. G., Morrison, G. R., & Ross, S. M. (1997). The effect of distance learning

classroom design on student perceptions. Educational Technology Research andDevelopment, 45(4), 5-19.

McCroskey, J. C., & Teven, J. J. (1999). Goodwill: A reexamination of the construct and itsmeasurement. Communication Monographs, 66, 90-103.

National Aeronautics and Space Administration, Human Performance Research Group. (1986).NASA task load index (TLX) v.1.0: Paper and pencil package. Retrieved from

http://humansystems.arc.nasa.gov/groups/tlx/paperpencil.htmlRichmond, V. P., Gorham, J. S., & McCroskey, J. C. (1987). The relationship between selected

immediacy behaviors and cognitive learning. Communication Yearbook, 10, 574-590.