expert perceptions of the feasibility of moocs
DESCRIPTION
This slideshow highlights some findings from a modified e-Delphi studied about the feasibility of MOOCs.TRANSCRIPT
![Page 1: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/1.jpg)
Expert Perceptions of the Feasibility of MOOCsReporting on a Modified E-Delphi Study
Shalin Hai-JewPaper Forum2014 Big XII Teaching and Learning ConferenceOklahoma State UniversityStillwater, OklahomaAug. 4 – 5, 2014
![Page 2: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/2.jpg)
Expert Perceptions of the Feasibility of MOOCs
2 Presentation description
For any innovation to take, experts and other decision-makers in a field have to buy into the endeavor. This is likely so for the adoption of MOOCs. To determine the feasibility of MOOC adoption, a modified e-Delphi study was conducted recently to capture the thinking of an expert panel of individuals involved in e-learning. This presentation describes the processes of setting up the modified e-Delphi study, beginning with the extensive literature review undertaken for the development of the survey instrument (and the use of NVivo 10 to code the sources). The major findings from this qualitative and mixed methods research suggests critical design, policy, and other structural issues that need to be addressed individually and collectively in order to roll out successful MOOCs.
Caveats: This is a quick view, with various types of data reduction from the content analysis of the responses in the modified e-Delphi study.
![Page 3: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/3.jpg)
Expert Perceptions of the Feasibility of MOOCs
3 Welcome!
What are your interests re: this presentation?
Any experiences with modified e-Delphi studies?
What are your experiences with MOOCs? (Designing? Creating? Teaching? Administering? Supporting? Taking?)
![Page 4: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/4.jpg)
Expert Perceptions of the Feasibility of MOOCs
4 A modified e-Delphi study
A classic Delphi study (a structured group procedure):
Delphi study as a research method originating in the 1950s by the RAND Corporation (“Research ANd Development”); first described in research in Dalkey & Helmer, 1965
Seating an expert panel from the same field or widely disparate ones
Working with experts in a field (or in unrelated fields) on a difficult problem
Eliciting insights individually (and anonymously) in an unstructured way in an early exploration phase and then moving to structured elicitation of the experts
![Page 5: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/5.jpg)
Expert Perceptions of the Feasibility of MOOCs
5 A modified e-Delphi study (cont.)
A classic Delphi study (a structured group procedure) (cont.):
Sharing the expert insights and tacit knowledge; examining whether there is a move toward consensus or convergence on a difficult or complex problem
Evolving the process based on new information
Forecasting (~ Delphic oracle) and enhancing long-range human policy-making and decision-making
![Page 6: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/6.jpg)
Expert Perceptions of the Feasibility of MOOCs
6 A modified e-Delphi study (cont.)
A modified e-Delphi study:
May be one-stage or multi-stage; may manifest differently over time
Brings in various sampling technologies including through information and communication technology (ICT)
Enables geographically distributed expert panels
Uses a range of survey techniques and technologies (with a focus on experts and on forecasting and on policymaking)
Does not strive for consensus per se (researchers expect a wide range of responses, including dissensus)
![Page 7: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/7.jpg)
Expert Perceptions of the Feasibility of MOOCs
7 An overview of the research workflow
![Page 8: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/8.jpg)
Expert Perceptions of the Feasibility of MOOCs
8 The extant literature on MOOCs (in 2013)
Impetuses for MOOCs
Types of MOOCs
Teaching and learning platforms with digital content delivery, comment ecosystems, mass assessments, back-end tracking, and augmentation by social media
Teaching strategies
Application of automation
Research on MOOC learning experiences and learner “types” (such as completing, auditing, disengaging, and sampling)
Funding
![Page 9: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/9.jpg)
Expert Perceptions of the Feasibility of MOOCs
9 The extant literature on MOOCs (cont.)
Challenges with learner participation, retention, and completion
Big data research potentials
Types of courses and course contents
Credentialing and badging
![Page 10: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/10.jpg)
Expert Perceptions of the Feasibility of MOOCs
10The survey instrument For the modified e-Delphi study
![Page 11: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/11.jpg)
Expert Perceptions of the Feasibility of MOOCs
11 The open-ended survey instrument
About You
Part 1: Impressions of MOOCs (open-ended questions with text-based responses for all sections with a few exceptions)
Experiences
Impressions
Sentiments
Part 2: Local Adoptions of MOOCs as Course Offerings
Benefits of MOOCs (for stakeholder groups, for reputation, for budgets, for resources)
Pre-requisites for adoption
![Page 12: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/12.jpg)
Expert Perceptions of the Feasibility of MOOCs
12 The open-ended survey instrument (cont.)
Part 3: Ensuring MOOC Quality
Strategies for managing learner “massiveness”
Definition of “quality learning” in MOOCs
Student identity in MOOCs
Criteria for course topics selection for MOOCs
Preferred instructor characteristics
Part 4: MOOCs in the Future
MOOC platforms as disruptive technologies or not
Open-courseware movement? Open-source movement? As critical parts of MOOCs?
Informal learning and certification or badging? Formal learning?
Long-term potentials?
![Page 13: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/13.jpg)
Expert Perceptions of the Feasibility of MOOCs
13Sliders Indicating Future Likelihood for the “Potential Scenarios” (last) Question
![Page 14: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/14.jpg)
Expert Perceptions of the Feasibility of MOOCs
14 The open-ended survey instrument (cont.)
Potential Scenarios
Near-term and midterm scenario for MOOCs
Popularity
Use by universities for formal learning and college credit? Use by corporations for training?
Star professors or common professors
Learner-taught MOOCs
Used for development in developing countries
Cobbled MOOC platforms using Web 2.0 technologies
Copyright or open-source
Big data advancements and insights
![Page 15: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/15.jpg)
Expert Perceptions of the Feasibility of MOOCs
15 Seating the e-Delphi expert panel
Announcements through professional electronic mailing lists; announcements in the microblogging site Twitter (social network connectivity), and others
Self-identified and self-selected experts
Anonymous respondents
A fairly small dataset (may range from a few to a thousand or so traditionally)
Some 103 total respondents but not responses on all questions
![Page 16: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/16.jpg)
Expert Perceptions of the Feasibility of MOOCs
16 Panel: Years spent in online learning in any capacity
![Page 17: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/17.jpg)
17 Panel: Professional roles
![Page 18: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/18.jpg)
18Panel: Direct experiences with MOOCs
![Page 19: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/19.jpg)
19 Panel: Types of workplaces
![Page 20: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/20.jpg)
Expert Perceptions of the Feasibility of MOOCs
20 Panel: Years in the current professional position
9.5 years average
Median: 10 years
Range from 1 – 30 years in the current professional position
![Page 21: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/21.jpg)
21 Panel: Direct experiences w/ MOOC courses
![Page 22: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/22.jpg)
22 Panel: Ways respondents learned about MOOCs (check all that apply)
![Page 23: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/23.jpg)
23 Panel: Familiarity measures with MOOCs
![Page 24: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/24.jpg)
24 Panel: Familiarity measures with MOOCs (on a bell curve)
![Page 25: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/25.jpg)
Expert Perceptions of the Feasibility of MOOCs
25Some findings…
![Page 26: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/26.jpg)
26 Potential positive social implications of MOOCs in a globalist context
![Page 27: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/27.jpg)
27 Potential negative social implications of MOOCs in a globalist context
![Page 28: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/28.jpg)
28 Possible benefits of MOOCs to the local university
![Page 29: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/29.jpg)
29 Anticipated pre-requisites for launching MOOCs
![Page 30: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/30.jpg)
30 Understood strategies for managing massive numbers of students
![Page 31: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/31.jpg)
Expert Perceptions of the Feasibility of MOOCs
31 A Wordle™ of text responses for the definition of quality in MOOCs
![Page 32: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/32.jpg)
32 Identity verification strategies for MOOC learners
![Page 33: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/33.jpg)
Expert Perceptions of the Feasibility of MOOCs
33 Strategies for the selection of MOOC course topics
Current areas of research to spark innovations
General areas
University specialists
Learning groups of non-traditional and mid-career professionals to meet employer needs
Long-tail niche courses
Topics that other MOOCs haven’t covered
Future focused topics
“Why not anything?”
![Page 34: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/34.jpg)
34 Criteria for the selection of faculty to lead MOOCs
![Page 35: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/35.jpg)
35 MOOC platforms disruptive or evolutionary?
![Page 36: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/36.jpg)
36 Intellectual property (IP) and MOOCs
![Page 37: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/37.jpg)
37 MOOC suitability for various types of learning
![Page 38: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/38.jpg)
38 Long-term potentials for MOOCs
![Page 39: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/39.jpg)
Expert Perceptions of the Feasibility of MOOCs
39MOOC scenariosNear- and mid- term
![Page 40: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/40.jpg)
Expert Perceptions of the Feasibility of MOOCs
40 Summary of MOOC predictions
Little chance MOOCs will disappear in the near-term because of a lack of clear funding models
A fairly high probability MOOCs will be popularized beyond well-funded universities
A fairly high likelihood that MOOCs would be used for training by corporations
A low likelihood that MOOCs would be used as formal college courses
A high likelihood that MOOCs would be used as short courses, tutorials, and automated trainings
A low probability that star professors will be the main ones teaching MOOCs
![Page 41: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/41.jpg)
Expert Perceptions of the Feasibility of MOOCs
41 Summary of MOOC predictions (cont.)
A better than chance likelihood that non-star professors will lead MOOCs
A 50% chance likelihood that MOOCs will be taught by the learners in that MOOC
A low probability that MOOCs will only be developed in advanced or well developed countries (a high probability that MOOCs will be adopted and developed in developing countries)
A high likelihood that MOOC platforms will be developed and maintained commercially
A high likelihood that MOOC platforms will be created by cobbling Web 2.0 technologies
![Page 42: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/42.jpg)
Expert Perceptions of the Feasibility of MOOCs
42 Summary of MOOC predictions (cont.)
A high likelihood that MOOC learners will be separated into an elite tuition-paying core and a non-paying general public periphery
An equal chance that learners will be treated equally (without a paying elite vs. a general non-paying public)
A fairly high probability that MOOC courses would fall under copyright protection (and other intellectual property laws); also a fairly high probability that MOOCs will be released in an open-source way
A high probability of new insights of human learners and their needs from “big data” resulting from MOOCs; respondents also suggested that there is a low probability of not finding out new insights from the big data related to MOOCs (in a conversely-phrased “checking” question)
![Page 43: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/43.jpg)
Expert Perceptions of the Feasibility of MOOCs
43 Iff and other conditionals
Actual and documented learning
Effective design
Respect for the individual learner and his /her personhood
Faculty voice and choices
Egalitarianism in terms of faculty leading
A clear (and fair) funding model
Useful research from the big data
Proper identification of learners
Proper crediting of learning (in the various contexts)…
![Page 44: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/44.jpg)
Expert Perceptions of the Feasibility of MOOCs
44 Real-world test of the findings?
In the year since this study was conducted (June 2013), what predictions do you see as having been borne out? (Is this a valid test of the “predictions” of the modified e-Delphi study results? Are there extant trends that were not addressed here?
Any core changes to the status quo in regards to MOOCs?
![Page 45: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/45.jpg)
Expert Perceptions of the Feasibility of MOOCs
45 Implications? Questions? Comments?
Are some of these findings intuitive? Others counter-intuitive? Elaborate.
How would you answer some of these survey questions? Why?
What other questions would you have asked in this instrument? Why?
In terms of variations on MOOCs (MCs, MOCs, MOCs, OOCs, OCs, OCs), which do you see as having the most potential? Why? (A recent suggestion involved the creation of LOOCs or “local open online courses”.)
What do you think is next?
![Page 46: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/46.jpg)
Expert Perceptions of the Feasibility of MOOCs
46 References
Hai-Jew, S. (2014). Iff and other conditionals: Expert perceptions of the feasibility of massive open online courses (MOOCs)—A modified e-Delphi study. Ch. 13. In Hai-Jew’s Remote Workforce Training: Effective Technologies and Strategies. IGI-Global.
![Page 47: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/47.jpg)
Expert Perceptions of the Feasibility of MOOCs
47
An addendum re: the research method
![Page 48: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/48.jpg)
Expert Perceptions of the Feasibility of MOOCs
48 Some thoughts on modified e-Delphi studies
Overall
Serves as an effective research methodology that uses survey system technologies well (particularly those that incorporate multimedia elements like videos, audio files, slideshows, and images)
Works well as an open-ended method with plenty of designed open-ended questions (without any forced consensus or forced dissensus)
Seating Expert Panels
May be benefitted by the far-distance and near-in access to various experts
May be weakened by self-assertions of expertise by anonymous participants
![Page 49: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/49.jpg)
Expert Perceptions of the Feasibility of MOOCs
49 Some thoughts on modified e-Delphi studies (cont.)
Creating Related Survey Instruments
Requires in-depth and thorough secondary research about the topic to create depth and breadth
Requires plenty of vetting of test instruments
Single or Multi-Staging?
May be more effective as a multi-stage process particularly for more complex issues
![Page 50: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/50.jpg)
Expert Perceptions of the Feasibility of MOOCs
50 Some thoughts on modified e-Delphi studies (cont.)
Light Use
May work as a near-term and mid-term forecasting method (a kind of predictive analytics), if held lightly
If something has to be put at risk based on the results of the modified e-Delphi study, it would be good to conduct much more research
Additional Research Necessary about Effective Modified e-Delphi Study Methods
Would benefit from more research in terms of efficacy
![Page 51: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/51.jpg)
Expert Perceptions of the Feasibility of MOOCs
51 The benefits of maintaining a research journal
Structured reflective practice around the research
A documented venue for questioning at every stage of the research work
Not released with the work at publication
![Page 52: Expert Perceptions of the Feasibility of MOOCs](https://reader035.vdocuments.site/reader035/viewer/2022070313/5549a52cb4c9050c738b525a/html5/thumbnails/52.jpg)
52 Conclusion and contact
Dr. Shalin Hai-Jew
Instructional Designer
Information Technology Assistance Center (iTAC)
Kansas State University
212 Hale Library
Manhattan, KS 66506-1200
785-532-5262 (work phone)
A special thanks to the attendees at the SIDLIT Special Interest Group (SIG) lunch session on MOOCs who helped catch a few problematic charts, which have since been re-drawn and labeled. :P It’s always good to have many eyes on a work, especially before going public!