drs. steven c. myers & michael a. nelson myers@uakron
DESCRIPTION
Do On-Line Students Analyze, Synthesize And Evaluate Better Than Face-To-Face Students? Preliminary Evidence. Drs. Steven C. Myers & Michael A. Nelson [email protected] Department of Economics - The University of Akron Presented to the Windows on the Future 2003 Conference OLN and ITEC-Ohio - PowerPoint PPT PresentationTRANSCRIPT
Do On-Line Students Analyze, Synthesize And Evaluate Better Than Face-To-Face Students?
Preliminary Evidence Drs. Steven C. Myers & Michael A. Nelson
Department of Economics - The University of Akron
Presented to the
Windows on the Future 2003 Conference
OLN and ITEC-Ohio
March 2, 2003
Online at http://gozips.uakron.edu/~myers/online
Goals for Presentation Talk about our ongoing research
Research Questions Modeling process Results
Talk about the Instructional Design of the Online course
Mastery and Competency Based Learning
Encourage you to contact us
Research Questions(1) Does the mode of delivery (face-to-face or web-
based) have an influence on learning outcomes? (2) Are students in an online environment as likely to
do as well as in face-to-face classes? Will they be able to equal the complex problem solving of the face-to-face students?
(3) Will web-based students develop more favorable attitudes towards economics than the attitudes developed by students in the face-to-face class?
(4) Do student myths about economics affect their learning outcomes and attitudes towards economics?
Acknowledgments Grant from
Carnegie Teaching Academy
Scholarship of Teaching, Assessment and Learning Funds
Institute for Teaching and Learning
The University of Akron
Data collected under signed informed consent from our students subject to the Institutional Research Board for the Protection of Human Subjects at The University of Akron.
Very few failed to give informed consent
Data collection instruments validated by a number of independent reviewers and created by MA grad, Michael Lovette.
Course under study Introduction to Economic Analysis
One-semester principles of economics 3 cr. General education course Required for engineering majors Both face-to-face and on line The online course is similar to a
graduate course offered since Fall 2001
Comparison Face-to-face
Offered by Michael Nelson
Regular class times Lecture and active
learning collaborative techniques
Better than the norm of 83% “chalk and talk” (Becker and Watts)
Some minor web-enhancements such as online gradebook
Online Offered by
Myers (F02) / Nelson (S03)
No face-to-face meetings Mastery & competency
based learning High professor-student
interaction No student to student
interaction Full use of the WebCT
environment
Student Comments / online “The fact that this course was completely
internet based, had no bearing on the level of knowledge I gained from the course. I think I learned as much, or even more, than I would have in a traditional classroom. I contribute this most to the way the instructor organized the course.”
Student Comments / online “I believe that the benefits of web-based
courses … far outweigh any disadvantages such as lack of face-to-face interaction--at least in this particular course. Dr. Myers' course was, of the four I took this semester over the web, truthfully the best organized, most well-adapted to the web (by his efforts) of them all.”
Research design
Student’s success
=f( student’s
initial endowments,
awareness of the economy,
attitudes about economics,
course modality,
student characteristics)
Research design
Student’s success
=f( student’s
initial endowments,
awareness of the economy,
attitudes about economics,
course modality,
student characteristics)
Pre-test scores
total
on recall and recognition
on simple application problems
on complex problems
GPA
Previous attempt of course
Research design
Student’s success
=f( student’s
initial endowments,
awareness of the economy,
attitudes about economics,
course modality,
student characteristics)
Survey of economic facts
20 questions…on trends and facts in the economy.
2 questions:
How does the student collect business and economic news.
How many hours per week does the student spend collecting news
Research design
Student’s success
=f( student’s
initial endowments,
awareness of the economy,
attitudes about economics,
course modality,
student characteristics)
Three Survey Questions:Level of student report of being interested in economics.
Level of student report of liking this economics course.
Level of student report about likelihood of taking more economics courses.
5=strongly agree
4=agree
3=indifferent/no opinion
2=disagree
1=strongly disagree
Research design
Student’s success
=f( student’s
initial endowments,
awareness of the economy,
attitudes about economics,
course modality,
student characteristics)
Online or face-to-face
Major
Level, e.g., freshman
Credits completed
Age
Gender
Ethnicity
Dependent Variable: Student Success Outcome Measures examscore on first 6 chapters
In class exam for face-to-face class Average of first three module quizzes for online class
Final examination (FE, FE_recog, FE_apply, FE_complex) First 3 or 4 levels of Bloom’s Taxonomy
Writing assignment writing2 4th to 6th level of Bloom
Regular Grades in class
Bloom’s Taxonomy (from Carla Lane)
1. Knowledge: arrange, define, duplicate, label, list, memorize, name, order, recognize, relate, recall, repeat, reproduce state.
2. Comprehension: classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, restate, review, select, translate,
3. Application: apply, choose, demonstrate, dramatize, employ, illustrate, interpret, operate, practice, schedule, sketch, solve, use, write.
4. Analysis: analyze, appraise, calculate, categorize, compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test.
5. Synthesis: arrange, assemble, collect, compose, construct, create, design, develop, formulate, manage, organize, plan, prepare, propose, set up, write.
6. Evaluation: appraise, argue, assess, attach, choose compare, defend estimate, judge, predict, rate, core, select, support, value, evaluate.
Student Characteristics Online Students
More likely to be To be older To be non-white To be female To have taken more
credits To be decided in their
major More favorably
disposed to economics
Face-to-face StudentsMore likely To be male To be a freshman To be undecided in
their major To spend less time
gathering business and economic news.
The online course design has influenced student success
Grades Distribution
Modules Total Recognition
Conceptual
Analytical
%C&A
2-8 Micro 331 29% 45% 26% 71%
9-15 Macro
563 32% 54% 15% 68%
2-15 Total
894 31% 51% 19% 69%
MC Learning Assessment Breakdown / Online
A B C/D F
Online Course 33% 39% 0% 10%
Face-to-face Course 8% 37% 40% 10%
Results
Students’ Initial Endowments All prior expectations are met Higher pretest and higher GPA are
positive on examscore, writing2 and less so on FE.
Higher complex pretest scores and higher GPA are positive for writing2 and FE_Complex
ResultsEconomic Awareness & Attitudes
“Survey” is weakly positive for examscore and writing2, but has no effect on FE.
No combination of “News gathering” is ever significant.
“Attitudes” are strongly significant for examscore and writing2, but no effect on FE
When attitudes matter take_more econ > interest in econ > will like econ
Results
Student characteristics: Those with an undeclared or undecided major
do much worse than engineering majors for examscore and FE.
Older students do worse on examscore, no effect on writing2 and strongly positive on FE
Females do better in writing2 (strongly significance), but do worse on FE (almost significant)
ResultsOnline Students: Do much better on examscore Show no significance difference in scoring on analytical
and complex reasoning tasks. Insignificant coefficients in writing2 Insignificant and small coefficients in
FE FE_recog FE_Apply FE_complex
There is some evidence of a possible interaction with females
Weak suggestion in the data that Men in online classes do better and Women in online classes do worse all things equal
Design of the online course
Built in modules Modules completed in order Competency based testing Use student feedback
For student learning enhancement For modifying & improving the course
CourseDesign
DL requires planning for contingencies
14 Content Modules
Active Learning vs. Passive Learning
Pre-class and Module 1: Preparing Students to Learn Email me! http://gozips.uakron.edu/~myers/online/
Is Distance Learning for Me? VARK – testing learning styles Orientation Module—‘ How to logon to
WebCT’ Orientation Module—‘ How to Use WebCT’
Syllabus Graduate course Undergraduate course
Pre-class and Module 1:Additional Orientation How to Communicate with Dr.
Myers How to access your online text
http://www.economicsplace.com/econ5e/
Rules of the game Building a relationship –
Breakdown the anonymity Survey “Tell me about yourself”
Content Modules 2-15
Module Introduction & Objectives Chapter Introductions Content Supportive Materials Assessment of Learning - Quizzes
on objectives with multiple trials Evaluation
2 Research & Writing Assignments – some objectives
Practice and experience in reflecting on a topic in the current economy.
Practice in analysis of economic trends.
Gaining of confidence about talking about the economy.
Ability to know and use the resources of economic commentary, prior analysis and data.
Introductions& Content Mostly Passive Learner
Centered Students
progress without intervention
Micro – Modules 2-8
Macro –Modules 9-15
GradedAssessment
Competency based
Everyone strives to get a perfect 10 (Mastery)
Three attempts, 15 min. time limit
Questions a mix of (1) Recognition, (2) Conceptual, &(3) Analytic~70% C&A
Random intervention by Professor
Role of Module Evaluation Planned intervention Forced contact Focus on the
learning Professorial
encouragement Decreases dropout
rates Process repeats
Module EvaluationA Classroom Assessment Technique from Angelo and Cross (1993); tested by Chizmar and Ostrosky (1998)
What comments do you have on this module and your experience in completing it?
What main point have you learned that you did not fully understand before?
What questions … Include any points that still remain muddy or unclear. Do consider posing the muddy points to your fellow students in the discussions.
What recommendations do you have for us as we continue to change and enhance the course?
Do On-Line Students Analyze, Synthesize And Evaluate Better Than Face-To-Face Students?
The answer is NO
and they don’t do any worse either.
Contact: Steve Myers [email protected]
The University of Akron
Or get this paper and presentation
Online at http://gozips.uakron.edu/~myers/