the new teaching institute program evaluation report lin chengyan eric whitmer anastasia kalmykova
TRANSCRIPT
![Page 1: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/1.jpg)
The New Teaching Institute ProgramEvaluation Report
Lin ChengyanEric Whitmer
Anastasia Kalmykova
![Page 2: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/2.jpg)
Agenda
• Introduction• Evaluation plan description• Description of Materials Evaluated• Data collection tools• Results
![Page 3: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/3.jpg)
Introduction
Name of the Program: The New Teaching Institute.
Time range: November,2014 -- March,2015.
Purpose of the course: Incorporate the new technology into their teaching methods.
Participants: 11 Faculty & Professors.
Motivation for participants:
– Upgrade Teaching Skills and Methods
– Additional Reward.
![Page 4: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/4.jpg)
Evaluation plan description
The objectives of the course:
• Generate a fully functional hybrid course shell including a variety of course materials designed to engage students in achieving the course goals
• Generate concise learning goals and objectives appropriate for a hybrid course
• Generate assessments appropriate for evaluating the stated course outcomes
• Explain the ADDIE model of instructional design and its applicability in a hybrid course
• Evaluate the effectiveness of other instructors’ hybrid courses
![Page 5: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/5.jpg)
Evaluation plan description
Effectiveness of Course
Efficiency of Course
Usability of Course
Appeal of Course
Four main factors to evaluate the module
![Page 6: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/6.jpg)
Description of Materials Evaluated
Accessed via Blackboard Start Here page
Blocks provide sense of community and social presence for learners
Blocks provide important information of the course
• Contact Information of the Course Facilitator
• Introductions • Cyber Café
• Navigating the Course• Syllabus• Course Overview• Netiquette • UT Hybrid Course Rubric
![Page 7: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/7.jpg)
Description of Materials Evaluated
The module is organized in hierarchical structure represented in a form of tree. There are main sections menu of the module available on each page. Each section contains a set of topics with specific contents.
![Page 8: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/8.jpg)
Data collection tools
• Entry Skills survey
• Usability survey
• Attitudes and Acceptance survey
• Course Review Rubric data
Tool Administration
• Surveymonkey
Data collection tools
![Page 9: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/9.jpg)
Results of Evaluation of New Teaching Institute Course
![Page 10: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/10.jpg)
How Effectiveness of NTI Module was Evaluated?
Conducted a paired samples t-test Mean N
Entry skills testCourse review data
2.362.91
77
Course review data Entry skills test
3
2
1 Developed questions of the entry skills test based on the course objectives and course review rubrics.
![Page 11: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/11.jpg)
What is the Result of NTI Module Effectiveness?
The course achieved its objectives with the intended audience
The objective “Evaluate the effectiveness of other instructors’ hybrid courses” was measured by indirect observation of work performance
![Page 12: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/12.jpg)
How Efficiency of NTI Module was Evaluated?
Course review data
Calendar
![Page 13: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/13.jpg)
What is the Result of NTI Module Efficiency?
![Page 14: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/14.jpg)
How Usability of NTI Module was Evaluated?
• Developed survey questions• Conduct the survey• Analyze the results
![Page 15: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/15.jpg)
What is the Result of NTI Module Usability?
Question Unsatisfactory Acceptable Excellent1. Comfortable to Use X XX X
2. Navigation works, easy to use, & consistent
XXX X
3. Clear Organization XXX X
4. Practice Opportunities Appropriate XXX X
5. Expected Functions & Capabilities X XX X
6. Easily Accessible in the event accidental termination
XX X
7. Overall Satisfaction X XX X
![Page 16: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/16.jpg)
How Appeal of NTI Module was Evaluated?
• Developed survey questions• Conduct the survey• Analyze the results
NTI Attitudes & Acceptance
Welcome to the NTI Attitudes & Acceptance Survey
![Page 17: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/17.jpg)
What is the Result of NTI Module Appeal?
Question Strongly Disagree
Disagree Undecided Agree Strongly Agree
1. Maintains Interest X XX X
2. Helps Generate Fully Functional Hybrid Course
XX XX
3. Assignments Effective & Assist in Learning
X XXX
4. Course Follows Objectives
X XX X
5. Helps to Evaluate Other Instructors' Hybrid Courses
X XXX
Part1
![Page 18: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/18.jpg)
What is the Result of NTI Module Appeal?
Question Strongly Disagree
Disagree Undecided Agree Strongly Agree
6. Provided Proper Tools to Generate Assessments for Evaluating Course Outcomes
XX XX
7. Learner was able to Generate Concise Learning Goals & Objectives for a Hybrid Course
X X X X
8. ADDIE Model Effective in Delivering Content
XX X X
9. Course has been Beneficial XX X X
10. Course is Challenging X X X X
Part2
![Page 19: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/19.jpg)
Pattern of Like or Dislike
Expert experience
Boring
Entry skills survey Appeal survey
No experience Interesting
![Page 20: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/20.jpg)
Recommendations
• Conduct the regular and systematic evaluation– Develop and conduct pre/post tests– Conduct entry skills survey– Conduct usability survey– Conduct attitudes and acceptance survey
• Make subsequent changes based on the resultsif following evaluation will indicate that the majority of learners have high level of entry skills we recommend:
– Add more challenging practice items– Reduce time to complete courseif following evaluation will indicate that the majority of learners have problem finding information we recommended:
– Make navigation simpler
![Page 21: The New Teaching Institute Program Evaluation Report Lin Chengyan Eric Whitmer Anastasia Kalmykova](https://reader035.vdocuments.site/reader035/viewer/2022070407/56649e185503460f94b045a5/html5/thumbnails/21.jpg)
Thank you.
Questions?