understanding online learning: cognitive prensence and the solo taxonomy
DESCRIPTION
Presentation at 15th Annual SLOAN-C Conference for Online Learningby Peter Shea, Jason Vickers, Suzanne HayesUniversity at AlbanyTRANSCRIPT
Peter Shea
Jason VickersSuzanne Hayes
University at Albany
Revisiting the Community of Inquiry Framework of Online Learning
Outline
Introduction Status of CoI Research Overview of project phases and design Findings and Implications
Changes in Higher Education
Shift underway in undergraduate education from providing classroom instruction to producing learning
With growth in online learning raises parallel question “What matters in online learning?”
CoI is the widely accepted framework for explaining and predicting how people learn in online environments in the absence of face to face instruction
Community of Inquiry Framework
Social PresenceThe ability of participantsto identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter-personal relationships by way of projecting their individual personalities.
Cognitive PresenceThe extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry.
Teaching PresenceThe design, facilitation and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes
Current CoI Research
Focused in two areas: Large scale surveys of student perceptions
of SP, TP and CP Quantitative content analysis of online
discussions
What is QCA?
Research methodology used to examine transcripts of online interactions to see what participants say to each other
Use SP, TP and CP codes to classify these statements
Results used to look for patterns within course
How has QCA been used?
Have focused primarily on threaded discussions
Has constrained us from seeing the broader picture of activity within a course
TP in threaded discussions only accounts for 20-30% of TP activity
Some limitations of past research using QCA in CoI
Early studies relied on samples of threaded discussions
No studies examining an entire online course Most studies focus on one form of presence at a
time Many studies based on a single course Early studies did not fully document IRR
measures
Our purpose
Address prior methodological shortcomings Use a holistic approach Explore expansion of CoI framework to make it
more encompassing and descriptive of all online course activities
Determine if previous research results can be verified, refined or extended
Improve and enhance CoI model and its constructs
Research Design
Two identical courses Based on template developed by content experts
and instructional designers Instructors were not the developers Five modules with discussions, mini lectures,
case studies, scaffolded research project submitted in stages
Upper level business management course from a state college in northeast
Discussion: 25% of the final grade
Debates: 10%
Other Course Activities: 65%
Purposive Sample
Four sections offered in Fall 2007 Two selected based initial assessment to
identify instructors with different approaches Classes comparable in size
Instructor A Instructor B
Students enrolled at start of the term 19 20
Students completing course 17 16
Data Analyzed in Multiple Phases
Phase One: May 08 – March 09 Online discussions in each of the 5 main course
modules in each course (SP, TP, CP)
Phase Two: Jan – June 09 Examined student case studies for CP & SOLO
taxonomy (Biggs and Collis, 1982) to measure high order thinking
Phase Three: Aug 09 + Coding for SP and TP* that take place in course
communications (outside of online discussions) Coding of all course documents for SP and TP*
* CP coding not meaningful in this context
P1
P2
P3
CP SOLO
Case Study Mod 1 X X
Case Study Mod 2 X
Case Study 3 X X
Case Study 4 X X
Description of Term Project
Research Paper Outline
Final Research Paper
Discussions SP TP CP
Module 1 X X X
Module 2 X X X
Module 2 Debate** Coding Underway
Module 3 X X X
Module 4 X X X
Module 5 X X X
SP TP CP
Orientation & Syllabus X X X
Introduction Module (w/ Icebreaker) X X
Module 1 X X
Module 2 X X
Module 3 X X
Module 4 X
Module 5
Course Instructions & Documents
Course Material CodedP1
Communicative DocumentsP3
P3
P2Course AssignmentsDiscussions
Coding Process
Paired coders Coded random module for practice Established and documented
guidelines to assure consistency Coded target modules Met to negotiate, identify and resolve
disagreements, when possible Recorded IRR prior to and following
negotiation
Coding Process*
Message as Unit of Analysis
Recorded presence/absence of category e.g. for SP we marked AF, OC, CH or NC
Did not count occurrences of each indicator within each message
Discussions & Course Communications
Whole paper as Unit of Analysis in SOLO
Paragraph as Unit of Analysis in CoI course documents coding(presence/absence of category e.g. for SP we marked AF, OC, CH or NC)
Did not count occurrences of each indicator within each message
Assignments &Course Documents
*Coding for only CoI and SOLO in discussions and case studiesrequired more than 6300 individual decisions.
Cognitive Presence Coding Sheet
Phase Code Indicator Socio-Cognitive Process
Triggering Event CP-TE-1 Recognize problemPresenting background information that may culminate in a question or presents a problem/issues
CP-TE-2 Sense of puzzlementAsking questions or Messages that take discussion in a new direction
Exploration CP-EX-1Exploration within the online community
Unsubstantiated agreement or disagreement/contradiction of previous ideas
CP-EX-2 Exploration within a single message Many different ideas/themes presented in one message
CP-EX-3 Information exchange Personal narratives or description
CP-EX-4 Suggestions for consideration Author explicitly characterizes message as exploration
CP-EX-5 Leaps to conclusions Offers unsupported opinions
Integration CP-IN-1Integration among groups members; Building on, adding to others' ideas
Reference to previous message followed by substantiated agreement or disagreement (I agree/disagree because…)
CP-IN-2Integration within a single message (response to prompt) Justified, developed, defensible, yet tentative hypotheses
CP-IN-3 Connecting ideas, synthesisIntegrating information from one or more sources - textbook, articles, personal experience, other posts or peer contributions.
CP-IN-4 Creating solutions Explicit characterization of message as a solution by participant
Resolution/Application CP-RE-1
Vicarious application to real world testing solutions; Providing examples of how problems were solved
CP-RE-2 Defending solutions Defending why a problem was solved in a specific manner
Solo Taxonomy Coding
0 = no response
1= prestructural: misses the point
2 = unistructural: minimal response
3 = multistructural: source facts only
4 = relational: substantiated positions
5 = extended abstract: higher level of abstraction
Used low and high indicators for levels 2-4.
Inter-Rater Reliability Reported Two Ways
Cohen’s kappa Corrects for chance or
random agreement between 2 coders
Conservative May be artificially low
(even with high level of agreement)
1.0 perfect agreement; .75+ excellent; .40 poor
Measures the extent to which two or more coders agree
Holsti’s Coefficient Expressed as a
percent Less Conservative Minimum of 80%
required 90% is excellent
Findings: Locus of TP
Not just limited to threaded discussion Majority of TP found outside of discussion in
communicative processes such as: Course e-mail Private folders Comments on written assignments
Distribution of Teaching Presence within Each Course
Findings: CP Measures in Discussions Consistent with prior research Majority of student postings are at exploration
stage Low levels of integration and resolution
Findings: SOLO Measures in Case Studies
SOLO: used because it measures outcomes instead of process
Students did not "excel" on these measures (i.e. reach 4s and 5s)
Student did "average" or slightly below (2s-3s)
Mismatch between culture of grading and the ideals of SOLO framework
Findings: SOLO Measures in Case Studies cont.
However, SOLO scores combined with CoI metrics do explain a majority of variance in teacher assigned grades
The two models combined therefore show promise in describing and explaining learning in online environments - prime goals of a theoretical framework
More research using this model is needed
Alternate models for measuring outcomes should be considered
Implications
Design of course learning activities needs to be well integrated to encourage: integration of public discourse in discussion AND documentation of student’s learning in their
private written artifacts When this does not occur there is limited
opportunities for students to draw associations or reinforce course concepts to learning activities
Findings
Disconnect between instructor’s grades and student learning outcomes based on SOLO and CP
Failure of meaningful learning or difference Who’s wrong: The instructor or the
framework?
Implications
Failure of meaningful learning? Difference in cultures of measurement
between researchers and instructors?
Thank you!
Peter SheaJason Vickers
Suzanne HayesUniversity at Albany