aect 2016 multimedia
TRANSCRIPT
05/03/2023 1
DDL – Multimedia & Video:Empirical Effects of Multimedia Design on
Perception, Learning Effectiveness, and Cognitive Load
2
Agenda Introduction About ODUOnline Problem Statement Research Questions Methods Findings Conclusion Your thoughts? comments? revisions?
3
Old Dominion University: ODUOnline
4
General Problem
This experiment investigated the effectiveness of several video design techniques to help determine how these multimedia designs can be most effectively applied in authentic instructional design contexts.
5
Independent variable (multimedia presentation design) Instructor-only, Slides-only, Video-switching, Dual-windows, and Superimposed-slides
Research Questions
6
“Instructor-Only” multimedia design:
7
“Slides-Only” multimedia design:
8
“Video-Switching” multimedia design:
9
“Dual-Windows” multimedia design:
10
“Superimposed-Slides” multimedia design:
11
Credibility and Immediacy
Credibility The learner’s perception of the instructors expertise,
concern for students, & trustworthiness Immediacy
Smiling, friendly, gestures, tone, eye-contact Ability to effectively communicate
Both factors that enhance motivation and could impact learning
12
Research Questions
What is the effect of instructor-only, slides-only, video-switching, dual-windows, and superimposed-slides multimedia presentation designs on:
1. Learning effectiveness? (recall and comprehension pre & post-test) 2. Perceived instructor credibility? (McCroskey & Teven, 1999) 3. Nonverbal immediacy? (Richmond, Gorham, & McCroskey, 1997) 4. General immediacy? (Anderson, 1979) 5. Instructional environment design? (Jayasinghe, Morrison, & Ross, 1997)6. Cognitive load? (NASA, 1986)
13
Research Methods
14
Participants 450 started the study: 226 = NASA TLX, 211 = survey, and 171 =
post-test. ~30 participants randomly assigned into each treatment group 61.5% female and 30.5% male Average age = 27.1, with a median age = 22 years
Range from 17 to 66 120 participants reported using laptops, 62 = workstations or
desktops, 39 = phones, and 21 = tablets + mini tablets
15
Procedures Participant invitations
Student Announcements (4 weeks) Full page ad Mace & Crown, and online ad, (1 week) $5 Starbucks or Amazon gift card, drawing for $80 Kindle
Technical requirements Any Internet capable device
Time required Approximately 45 minutes
Tasks involved….
Participant Procedure
16
InitialData Collection
Website
1. Informed consent2. Demographics & 20-item Pre-test3. Assignment into treatment & watch video
Instructor-Only Video
Superimposed-Slides Video
TLX, Surveys,And Post-Test
TLX, Surveys,And Post-Test
OptionalGiftCard
4. NASA TLX5. Credibility6. Nonverbal Immediacy7. General Immediacy8. Design9. 20-item Post-test10. Gift card selection
Slides-Only Video
TLX, Surveys,And Post-Test
Video-Switching Video
TLX, Surveys,And Post-Test
Dual-Windows Video
TLX, Surveys,And Post-Test
17
Research Findings
18
Research Question 1: Learning Effectiveness No statistically significant difference
ANCOVA, F(4,151) = .56, p = .7. All five groups appeared to recall and comprehend the subject
matter pre-test scores (M = 10.02, SD = 3.50), and post-test scores (M = 13.00,
SD = 3.25), t(156) = 11.53, p < .01 All five groups performed similarly on the multiple-choice post-test
Design of each treatment too similar to differentiate? (include problem solving in future projects?)
19
Research Question 2: Instructor Credibility Inclusion of slides appeared to enhance instructor credibility
ANOVA was significant, F(4,203) = 2.47, p < .05
Dual-Windows Superimpossed-Slides Video-Switching Slides-Only Instructor-Only4.5
4.7
4.9
5.1
5.3
5.5
5.7
5.9
5.75
5.53 5.51 5.49
5.1
Source Credibility
20
Research Question 2: Instructor Credibility Inclusion of slides appeared to enhance instructor credibility
ANOVA was significant, F(4,203) = 2.47, p < .05
Dual-Windows Superimpossed-Slides Video-Switching Slides-Only Instructor-Only4.5
4.7
4.9
5.1
5.3
5.5
5.7
5.9
5.75
5.53 5.51 5.49
5.1
Source Credibility
21
Research Question 3: Nonverbal Immediacy Inclusion of instructor appeared to enhance immediacy
ANOVA indicated a significant difference between treatment groups, F(4,206) = 4.68, p <.01
Superimpossed-Slides Video-Switching Instructor-Only Dual-Windows Slides-Only3.5
3.7
3.9
4.1
4.3
4.5
4.37
4.29 4.27 4.24
3.83
Nonverbal Immediacy
22
Research Question 3: Nonverbal Immediacy Inclusion of instructor appeared to enhance immediacy
ANOVA indicated a significant difference between treatment groups, F(4,206) = 4.68, p <.01
Superimpossed-Slides Video-Switching Instructor-Only Dual-Windows Slides-Only3.5
3.7
3.9
4.1
4.3
4.5
4.37
4.29 4.27 4.24
3.83
Nonverbal Immediacy
23
Research Question 4: General Immediacy
No statistically significant difference superimposed-slides group perceived the general immediacy of the instructor
highest, though the results were not statistically significant, F(4,205) = .82, p = .51
Design of survey may have impacted results Participants first required to read a short paragraph that defines “immediacy” Survey items did not ‘fit’ in online survey tool format
Exclude in future research projects?
24
Research Question 5: Instructor Evaluation Measure Item 9: desire to continue in the course as presented:
ANOVA analysis indicated that there was a significant difference between treatments on this instrument, F (4,206) = 4.03, p < .01
Instructor-Only Slides-Only Video-Switching Superimposed-Slides Dual-Windows2.22.42.62.8
33.23.43.63.8
44.24.44.64.8
5
4.35
3.443.27
3 2.97
Desire to Drop Course
25
Research Question 5: Instructor Evaluation Measure Item 9: desire to continue in the course as presented:
ANOVA analysis indicated that there was a significant difference between treatments on this instrument, F (4,206) = 4.03, p < .01
Instructor-Only Slides-Only Video-Switching Superimposed-Slides Dual-Windows2.22.42.62.8
33.23.43.63.8
44.24.44.64.8
5
4.35
3.443.27
3 2.97
Desire to Drop Course
26
Research Question 6: Cognitive Load No statistically significant differences
(average TLX score), F(4,221) = .56, p = .69 HD video/slides + studio grade audio + user centered interface design =
low extraneous cognitive load? Subject matter not challenging enough? (low intrinsic cognitive load?) Instructional design not engaging enough? (low application of germane
resources?) Make design more interactive? (use different subject matter in future
projects?)
27
Other Findings
28
Media Comparison: No statistically significant difference
post-test, F=(3,167) = 1.19 p = .32
Desktop/Workstation Laptop Phone Tablet11
11.5
12
12.5
13
13.5
14
14.5
15
13.3
12.72 12.57
14.56
Device Type compared to Post-test
29
Conclusions
30
Key Findings All five designs appear applicable in terms of recall and
comprehension learning effectiveness HD studio produced audio and video Following our ID’s PowerPoint Guidelines
Instructor credibility and immediacy can be enhanced by including both slides/presentation and visuals of the instructor
Less experienced students require additional support and motivation The pedagogical design of the course is more important than the
technology/media used to deliver that course
31
Future Research Directions?
Subject matter? Audio only no visuals? Visuals only no audio? Text density? Graphics only? Let students choose? TelePresence? Interactive instructor? Problem solving post-test?
Standing/walking instructor? Shorten survey/post-test Transcript window? More than 20 minutes? Less than 20 minutes? Eye-tracking?
05/03/2023 32
Thank You!Dr. Miguel Ramlatchan, [email protected]
Dr. Ginger Watson, [email protected]. John Baaki, [email protected]
Dr. Peter Baker, [email protected]
33
ReferencesAnderson, J. F. (1979). Teacher immediacy as a predictor of teaching effectiveness. In D.
Nimmo (Ed.), Communication Yearbook 3. New Brunswick, NJ: Transaction Books.Jayasinghe, M. G., Morrison, G. R., & Ross, S. M. (1997). The effect of distance learning
classroom design on student perceptions. Educational Technology Research andDevelopment, 45(4), 5-19.
McCroskey, J. C., & Teven, J. J. (1999). Goodwill: A reexamination of the construct and itsmeasurement. Communication Monographs, 66, 90-103.
National Aeronautics and Space Administration, Human Performance Research Group. (1986).NASA task load index (TLX) v.1.0: Paper and pencil package. Retrieved from
http://humansystems.arc.nasa.gov/groups/tlx/paperpencil.htmlRichmond, V. P., Gorham, J. S., & McCroskey, J. C. (1987). The relationship between selected
immediacy behaviors and cognitive learning. Communication Yearbook, 10, 574-590.