2009 professional development day
DESCRIPTION
Student Survey of Instruction (SSI): Increasing Response Rates and Accessing Results. 2009 Professional Development Day. October 2009. Student Survey of Instruction. Presenters Jesse Coraggio , Director, Academic Effectiveness - PowerPoint PPT PresentationTRANSCRIPT
2009 Professional Development Day
October 2009
Student Survey of Instruction (SSI): Increasing Response Rates and
Accessing Results
SSI: Increasing Response Rates and Accessing Results October 2009
Student Survey of Instruction
Presenters Jesse Coraggio, Director, Academic
Effectiveness Deb Montalvo, SSI Coordinator and
Coordinator, Placement & Career Services
October 23, 2009 Academic Effectiveness 2
SSI: Increasing Response Rates and Accessing Results October 2009
Purpose
The SSI has undergone a number of changes during the last year, including a move to an online environment and improved access to results. Learn about these recent changes and best practices for increasing your student participation, as well as how to access your own SSI results online.
October 23, 2009 Academic Effectiveness 3
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 4
Original SSI Items
Original SSI forms developed by committee Items reviewed for content and lecture form
aggregated into four categories (e.g., faculty student interaction, organization, presentation, and evaluation)
There are four forms (Scantron): Lecture, Non-lecture, Clinical A*, and Clinical B*
Results scanned by PCSB and entered into PeopleSoft
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 5
eCampus SSI Items
Online SSI form was developed by eCampus to assess online courses
Form contained exact or slightly modified items as in Lecture SSI form with same four categories
Original online survey tool created to capture summative information in database
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 6
Issues with prior process
Expenses related to Scantron forms, and numerous hours of sorting, organizing, and transporting forms.
Time consuming process of transcribing open-ended items.
Not all Scantrons were scanned and processed (scanning issues, data integrity issues, etc.)
Scantron results and online results captured separately and reported separately
Process conducted by multiple departments with no central ownership
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 7
Performance Improvement
From Compliance to Performance Improvement
Data Information ImprovementKnowledge
Improved OutcomesSPCReportsAssessment Data
CommunityCollege LeadershipProgram Directors
FacultyStudents
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 8
New Online Survey Process
New SSI process designed to: Provide stakeholders (Faculty, Program
Administrators, etc.) information to assist in making decisions that will result in performance improvement
Provide more reliable and valid course evaluation information
Create centralized coordination of the process Save the College time and money (est. $45,000 per
year) Provide more course time for instruction
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 9
SSI Online Process Timeline
Fall 2007 Established SSI Team Factor analysis determined that lecture and online SSI forms
have three factors (e.g., faculty engagement, preparation and organization, and course instruction)
Spring 2008 100 section pilot of FTF courses conducted
Fall 2008 First collegewide administration (All ANGEL shells enabled) Created an online reporting tool (PAs only) Revised older survey tool
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 10
SSI Online Process Timeline
Spring 2009 Created customized usage reporting tools Created new survey interface with omit
capability Added Faculty Representative to SSI Team Created online reporting tool access for
Faculty (No more paper)
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 11
SSI Online Process Timeline
Summer 2009 First collegewide summer administration Modified faculty email notices to include link to
real-time response data. Began working with HEC on Clinical Solutions Evaluated current SSI items and forms (SSI
Revision Committee) Fall 2009
Implement self-service module so faculty can revise administration dates
SSI: Increasing Response Rates and Accessing Results October 2009
Information about the Process
October 23, 2009 Academic Effectiveness 12
http://www.spcollege.edu/central/ssi/
SSI: Increasing Response Rates and Accessing Results October 2009
Information about the Process
October 23, 2009 Academic Effectiveness 13
http://www.spcollege.edu/central/ssi/
SSI: Increasing Response Rates and Accessing Results October 2009
Accessing Results
All SSI survey reports are now available on the online SSI Administration site: http://it.spcollege.edu/ssi/
Login by entering your SPC network username and password and selecting the Login button
October 23, 2009 Academic Effectiveness 14
SSI: Increasing Response Rates and Accessing Results October 2009
Accessing Results
October 23, 2009 Academic Effectiveness 15
SSI: Increasing Response Rates and Accessing Results October 2009
Accessing Results
Select Term from the drop-down menu (i.e. 0405) Select how you would like to view the results
Each individual section with its own results OR Combined sections by instructor Select the Search button
October 23, 2009 Academic Effectiveness 16
SSI: Increasing Response Rates and Accessing Results October 2009
Accessing Results
After selecting a term and indicating ‘Each Individual Section with its own results’…
View each individual section with its own results, as selected on the previous screen for Term: 0405.
Select any of the courses to view the SSI report for that course.
October 23, 2009 Academic Effectiveness 17
SSI: Increasing Response Rates and Accessing Results October 2009
Accessing Results
View the SSI results for the course selected Select the “+” next to your Acad. Org to view averages for similar SSI survey forms
(2-Year Online, Lecture, Non-lecture, etc.) within that academic org (listed as SOCSCI-LD in this example).
October 23, 2009 Academic Effectiveness 18
SSI: Increasing Response Rates and Accessing Results October 2009
Accessing Results
After selecting a term and ‘Combined sections by Instructor’… View combined sections with results, as selected on the term selection
screen (Term: 0405 in this example). Select any of the instructor names to view their combined SSI report for that
course. Note: multiple reports for a single instructor may be listed if the instructor taught courses with either different SSI survey forms, different academic orgs, or on different campuses.
October 23, 2009 Academic Effectiveness 19
SSI: Increasing Response Rates and Accessing Results October 2009
Accessing Results
October 23, 2009 Academic Effectiveness 20
SSI: Increasing Response Rates and Accessing Results October 2009
Accessing Results
Select ‘View in Adobe Reader’ for an electronic version of the report (pdf) ‘Each Individual Section with its own results’ report
produces individual SSI results, three comparative measures, graph, and student comments.
‘Combined sections by Instructor’ report produces an instructor’s combined sections SSI results with three comparative measures, graph, and student comments by course. This report is somewhat similar to the SSI reports generated by AIS in the past.
October 23, 2009 Academic Effectiveness 21
SSI: Increasing Response Rates and Accessing Results October 2009
Changing Administration Dates
SSI is intended to be an evaluation of the instruction provided during the course.
It is administered after the students have participated in the majority of the course (no sooner than the 60% mark of the course) in order to provide the student an opportunity to experience the course instruction.
Length of the administration period is approximately the last 20%-25% of the course.
Faculty are provided default SSI administration start and end dates, and are permitted to make minor adjustments to these dates within a predetermined timeframe. This ensures a more standardized and consistent data collection process.
October 23, 2009 Academic Effectiveness 22
SSI: Increasing Response Rates and Accessing Results October 2009
Changing Administration Dates
Locate the course you would like to change… Select the Calendar icon
October 23, 2009 Academic Effectiveness 23
SSI: Increasing Response Rates and Accessing Results October 2009
Changing Administration Dates
Click on “Next Month” on the upper right hand side of the calendar to bring you to December. The sample calendar above shows the course ends on 12/18 and the SSI closes on 12/11.
October 23, 2009 Academic Effectiveness 24
SSI: Increasing Response Rates and Accessing Results October 2009
Changing Administration Dates
To extend the SSI to remain open until 12/18… Select 12/18. It will turn blue through 12/18. This changes in real-time. Should you make a mistake, simply click back on the original survey end
date of 12/11 or select another. The latest you can extend the survey to is 12/24.
October 23, 2009 Academic Effectiveness 25
SSI: Increasing Response Rates and Accessing Results October 2009
Response Rates
October 23, 2009 Academic Effectiveness 26
Submission Entries by Campus for
0400 Term Submission Entries by Campus for
0405 Term SSI Submissions Split by Campus
0410 TermCampus Roster Entries Opt-outs* Average Roster Entries Opt-outs* Average Roster Entries Opt-outs AverageAllstate Center 1811 1388 0 76.6% 2540 1537 23 60.5% 1226 624 25 50.9%Clearwater Campus 16001 7194 0 45.0% 15292 5469 48 35.8% 6527 2209 35 33.8%Electronic Campus 9563 5937 0 62.1% 10658 4785 100 44.9% 5679 2674 57 47.1%EpiCenter 908 451 0 49.7% 1022 295 1 28.9% 271 113 2 41.7%
Health Education Center 8484 5121 0 60.4% 9271 4438 74 47.9% 4242 2091 97 49.3%Open Campus 654 151 0 23.1% 556 132 3 23.7% 52 24 0 46.2%Seminole Campus 8461 4911 0 58.0% 8101 3610 58 44.6% 4226 1942 41 46.0%St Pete/Gibbs Campus 20991 9332 0 44.5% 20706 7786 71 37.6% 8336 2739 23 32.9%Tarpon Springs Campus 15736 7034 0 44.7% 12634 6006 51 47.5% 4691 2310 48 49.2%Totals 82609 41519 0 50.3% 80780 34058 429 42.2% 35250 14726 328 41.8%*Opt-outs are included in the number of Entries and were not available during Fall Term
SSI: Increasing Response Rates and Accessing Results October 2009
Response Rate Comparison
Palm Beach Community College Online Course Evaluation Palm Beach Community College debuted its online course
evaluation system in Fall 2007. Until the Spring 2007 term, PBCC relied on a paper-based course evaluation system for its traditional classroom courses.
Response rates:Pilot testSpring 2007: 30% classroom, 26% distance learningCollegewideFall 2007: 20% classroom, 26% distance learningSpring 2008: 24% classroom, 29% distance learning
October 23, 2009 Academic Effectiveness 27
SSI: Increasing Response Rates and Accessing Results October 2009
Increasing Response Rates
While not the primary focus of the SSI Revision Committee (Summer 2009), there was considerable interest in identifying and implementing ideas and processes to increase the response rate on the SSI.
Through creative brainstorming at the meetings and online, a list of “best practices” was crafted.
October 23, 2009 Academic Effectiveness 28
SSI: Increasing Response Rates and Accessing Results October 2009
Increasing Response Rates
SSI Revision Committee Recommendations1. Send Constant Reminders2. Stress Confidentiality3. Imbed link in announcement page -- “inescapable messaging”4. Move every week to top of unit5. Link to “content” – graded or non-graded6. Bring students to lab /use of lab7. Give extra credit8. Genuinely convey the message that the information will be used to
make this class better9. Hold activities on campus, e.g., a table highlighting importance for
accreditation
October 23, 2009 Academic Effectiveness 29
SSI: Increasing Response Rates and Accessing Results October 2009
Increasing Response Rates
There was some concern that item #5 (linking the SSI to accessing exams or other course materials) was not acceptable from a legal standpoint.
The question was posed to the Office of General Counsel, and, in brief, the answer was ‘yes’, the faculty could block access to course materials assuming that the content “blocked” is just the next module or assignment the student would see, not specifically an exam.
The Office of General Counsel also advised against giving extra credit for taking the SSI.
October 23, 2009 Academic Effectiveness 30
SSI: Increasing Response Rates and Accessing Results October 2009
SSI Resources
‘Help with the SSI’ Section of website SSI Frequently Asked Questions (FAQs) document For questions or comments about the SSI process -
contact your dean, program director, or academic chair For technical assistance - contact your campus
instructional technologist SSI Administrator -
mailto:[email protected] Troubleshooting SSI links COMING SOON!
October 23, 2009 Academic Effectiveness 31
SSI: Increasing Response Rates and Accessing Results October 2009
October 23, 2009 Academic Effectiveness 32
Questions
2009 Professional Development Day
October 2009
Student Survey of Instruction (SSI): Increasing Response Rates and
Accessing Results