Academic Students’ Satisfaction and Learning Outcomes in a HyFlex Course:

Do Delivery Modes Matter?

Sawsen Lakhal, Ph. D., Universityof SherbrookeHager Khechine, Ph. D., Laval University

Daniel Pascot, Ph. D., Laval University

E-LEARN 2014 - World Conference on E-Learning

New Orleans, Louisiana: October 27-30, 2014

Outline

• Introduction

• Study variables and research hypotheses

• Methodology

• Results

• Conclusion

Introduction

• The Hybrid-Flexible (HyFlex) instructional model is a course design model that combines Hybrid learning in a Flexible way, such that students can either attend face-to-face class sessions, participate online or do both (Abdelmalak, 2013; Kyei-Blankson & Godwyll, 2010; Educause, 2010).

• 3 course delivery modes: face-to-face, online (synchronous or asynchronous) and hybrid (alternate between face-to-face mode and online mode) within the same course.

• Within a HyFlex design model, faculty provide teaching and learning activities to respond to both face-to-face students and online students. These activities are not totally separated (Beatty, 2010).

Introduction –Advantages• It responds to student needs by offering flexibility in course attendance (Abdelmalak, 2013;

Beatty, 2007; Educause, 2010; Kyei-Blankson & Godwyll, 2010; Miller et al., 2013).

• It enables students to use the kind of technologies they use in their daily lives for learning purposes (Miller et al., 2013; Thompson, 2013).

• It enriches the teaching and learning environment (Abdelmalak, 2013; Beatty, 2007).

• It gives students equal opportunities to interact with other students and with faculty, regardless of the mode chosen (Miller et al., 2013).

• It allows faculty to differentiate instruction to meet different student learning styles, preferences, approaches and strategies (Abdelmalak, 2013).

• It promotes student engagement in their learning (Abdelmalak, 2013; Miller et al., 2013).

• It may represent a solution for higher education institutions with limited classroom space (Educause, 2010; Miller et al., 2013).

Introduction - Principles

• According to Beatty (2007), four principles are to be applied when designing a HyFlex course in order to ensure its quality: Learner Choice, Equivalency, Reusability, and Accessibility.

• Learner Choice means that faculty can “provide meaningful alternative participation modes and enable students to choose between participation modes weekly” (Beatty, 2010).

• Equivalency implies that faculty should “provide equivalent learning activities in all participation modes” (Beatty, 2010).

• Reusability signifies that faculty should allow students to “utilize artifacts from learning activities in each participation mode as learning objects for all students” (Beatty, 2010).

• Accessibility means that faculty should “equip students with technology skills and access to all participation modes” (Beatty, 2010).

Introduction –Aim of this study

• In light of these principles, students should have the choice of attending face-to-face sessions or online sessions without any learning deficits (Beatty, 2007). Student satisfaction and learning outcomes (i.e. academic performance) should be the same regardless of the mode they choose.

• The aim of this study is to investigate the effectiveness of HyFlex courses. The level of student satisfaction and student performance is explored within this learning environment. Academic performance is measured by three means: multiple choice test, written exam and continuous assessment scores. Moreover, these outcomes are explored by course delivery modes chosen by students.

• To our knowledge, very few studies have examined these issues (Kyei-Blankson & Godwyll,

2010; Miller et al., 2013). More broadly, to date, research regarding HyFlex course design is very limited (Abdelmalak, 2013; Kyei-Blankson & Godwyll, 2010).

Study variables and research hypotheses - Satisfaction• Hobbs and Osburn (1989) define student satisfaction as elements nurturing

the sense of well-being experienced by students in the course, from both the technological and the pedagogical perspectives.

• According to Beatty (2010), in a HyFlex course, faculty provides equivalent learning activities to students, regardless of the course delivery mode they choose. This principle of Equivalency suggests that students should be uniformly satisfied in different course delivery mode groups. We thus formulate the first study hypothesis as follows:

• H1: the level of satisfaction is the same, on average, for students who choose different delivery modes in HyFlex courses.

Study variables and research hypotheses –Academic Performance• Student academic performance is considered a key variable in studies

evaluating learning outcomes (Piccoli, Ahmad & Ives, 2001). In a study conducted among undergraduate students, Miller et al. (2013) reported that no significant differences were found between online students and face-to-face students on academic performance measure by homework grades, midterm scores and final course grades, suggesting that enrolled students in HyFlexcourse acquired equivalent learnings (n = 77). These results led us to believe that academic performance would be the same for students who choose different delivery modes in HyFlex courses. In the present study, academic performance is measured by three means: multiple choice test, written exam and continuous assessment. We put forward the following hypothesis:

• H2: the level of academic performance measured by three means (multiple choice test, written exam and continuous assessment) is the same, on average, for students who choose different delivery modes in HyFlex courses.

Methodology - Sample

• The study sample was made of students enrolled in a HyFlex undergraduate management information systems course. The faculty scheduled, for every week, a classroom session that was broadcasted live and recorded via Elluminate. Among the 439 students registered in the 2013 winter session, 376 filled the online questionnaire (response rate of 86%).

Age Gender TotalMale Female

20 years and younger 82 84 16621 – 25 years 99 79 17826 years and older 24 8 32Total 205 171 376

Methodology - Procedure

• At the beginning of the 2013 winter semester, one of the researchers visited the class and explained the purpose of the study and the potential involvement of the students. Online synchronous students were able to ask questions using Elluminate. These explanations were also recorded using Elluminate system and available for students who followed the course session online asynchronously. The researcher left her e-mail address and students were able to ask questions on the study whenever it suited them.

• The questionnaire used in this study was comprised of 20 items and required 10 minutes to be completed. It was put online during the final five weeks of the 2013 winter semester.

• To encourage students’ participation, four gift-certificates of 100$ were randomly drawn at the end of data collection. During the data collection period, a reminder message was sent to students by email to invite them to participate in the study.

Methodology - Measures

• Satisfaction

• Student satisfaction was measured using the adapted French version (Fillion, 2005;

Lakhal et al., 2007) of the survey composed of 15 items suggested by Hobbs and Osburn (1989). These items are rated on a seven-point Likert-type scale (from 0 = strongly disagree to 7 = strongly agree).

• Academic performance

• Academic performance was measured by the scores obtained by students on a multiple choice test, a written exam and continuous assessment in the HyFlexcourse. These scores were all converted in percentages.

Methodology - Measures

• Course delivery modes in the HyFlex course

• Fours groups: face-to-face, online synchronous, online asynchronous and hybrid.

• In order to classify students in each group, three items were used:

• (1) How many face-to-face sessions did you attend?

• (2) How many online course sessions did you attend, using Elluminate, in real time?

• (3) How many sessions of the online course did you attend using records of course sessions on Elluminate?

• Students were asked to rate on a scale ranging from 1 (1 class session) to 10 (10 class sessions) how many times he or she attended course sessions according to the three modalities described above. Note that the management information systems course was comprised of 10 course sessions.

Methodology - Measures

• Course delivery modes in the HyFlex course

• We used the definition of course delivery modes provided by Allen, Seaman and Garrett (2007) to classify students in the four groups.

Proportion of course sessions carried out online

Number of sessions in the study carried

out online

Course deliverymodes

Number of students in each group

0% to 29% 0 to 2 Face-to-face 7480% and more in real time 8 and more Online synchronous 6080% and more offline 8 and more Online asynchronous 18730% to 79% 3 to 7 Hybrid (Blended) 55

Results - Descriptive statistics and ANOVA tests on satisfaction items among the HyFlex course

M SD Agree%

Disagree%

Neutral % F p

1. I am satisfied with the course in general. 4.33 1.96 56.40 36.70 6.90 0.40 .752. I am satisfied with the quality of teaching. 4.97 1.83 66.80 22.60 40.00 0.68 .573. I am satisfied with the technical quality of the course. 4.89 1.89 64.60 24.70 10.60 1.21 .314. I am satisfied with the cost of the course as compared to other alternatives.

4.86 1.78 54.52 15.69 29.79 0.84 .47

5. I am satisfied with the level of difficulty of the course. 4.30 1.94 50.80 37.23 11.97 0.62 .606. I am satisfied with the supervision available to me in the course.

4.99 1.75 65.43 19.68 14.89 0.40 .76

7. I am satisfied with the course content. 4.03 2.06 48.14 43.35 8.51 0.55 .658. I am satisfied with the technical support access provided during the course.

5.24 1.59 65.69 11.17 23.14 1.95 .12

9. I am satisfied with the technological equipment used during the course.

5.42 1.62 72.61 11.44 15.96 1.62 .18

10. I am satisfied with the ease of use of the technological equipment during the course.

5.63 1.47 77.13 7.98 14.89 2.87 .04*

11. I am satisfied with the technical reliability of the technological equipment used during the course.

5.57 1.45 78.19 9.04 12.77 1.22 .30

12. I am satisfied with the support of the university for the course.

5.32 1.46 65.43 7.45 27.13 0.38 .77

13. I am satisfied with the inclusion of the course in my curriculum.

4.15 2.15 50.27 39.89 9.84 1.20 .31

14. I am satisfied with the amount of theoretical knowledge acquired in the course.

4.36 2.05 56.38 34.04 9.57 0.55 .65

15. I am satisfied with the amount of practical knowledge acquired in the course.

4.17 2.04 51.86 37.77 10.37 0.26 .85

Results –Hypotheses testing

• In order to examine the effectiveness of the HyFlex course, four one-way ANOVA tests were performed on the data (Field, 2013).

• H1, which stated that the level of satisfaction is the same, on average, for students who choose different delivery modes in HyFlex courses, is thus supported.

• H2, which stated that the level of academic performance is the same, on average, for students who choose different delivery modes in HyFlex courses, is thus partially confirmed.

Study variables ANOVA testsF p η2

Satisfaction 1.08 0.36 .01Multiple-choice test scores 1.04 0.37 .01Written exam scores 0.28 0.84 .00Continuous assessment scores 3.47 0.02* .03

Conclusion

• Previous studies considered two groups of students: those who attended face-to-face sessions and those who followed the course online (Miller et al., 2013). Within the scope of the present study, online students were divided into two groups: synchronous and asynchronous. This allowed us to refine our results and to test within subject groups. Indeed, if we had not considered these two groups, we would not have found any differences between the groups (face-to-face, online and hybrid), as in previous studies (Miller et al., 2013).

• Faculty should consider these two groups of students if he or she wishes to give all students equal opportunities to learn (Beatty, 2007), as online asynchronous students showed some learning deficits as compared to online synchronous students.

• Online synchronous students were found to be more satisfied with the ease of use of the technological equipment during the course than online asynchronous students. The principle of Accessibility (i.e. all students should be able to choose and participate in different delivery modes, without worrying about access to technology) is clearly not met. Other studies on technology acceptance should be conducted in this kind of courses.

Thank you!

Sawsen Lakhal, Ph. D.• Professor• Department of Pedagogy, PERFORMA• University of Sherbrooke• Sawsen.Lakhal@USherbrooke.ca

Hager Khechine, Ph. D.• Professor• Department of Management Information Systems• Laval University• Hager.Khechine@sio.ulaval.ca

Daniel Pascot, Ph. D.• Professor• Department of Management Information Systems• Laval University• Daniel.Pascot@sio.ulaval.ca