1.Enterprise Surveys and Course Evaluations: Features and Considerations for Institutional Adoption JASON RHODE Director of Distributed Learning Northern Illinois University #bbesce

2. ABOUT PRESENTER JASON RHODE Director of Distributed Learning Northern Illinois University jrhode@niu.edu @jrhode I have used Blackboard for 13 years. 3. SLIDES AND CONVERSATIONS slidesha.re/niubbesce #bbesce @jrhode 4. BACKCHANNEL todaysmeet.com/bbesce 5. AGENDA Background Features Exploration Roadmap Discussion 6. INSTITUTIONAL BACKGROUND Background Features Exploration Roadmap Discussion 7. NORTHERN ILLINOIS UNIVERSITY 8. CENTER STAFF 9. ABOUT NIU Main campus in DeKalb, IL located 65 miles West of Chicago Total enrollment: 21,869 (Fall 12) 1,185 instructional faculty Student to faculty ratio: 18:1 39 academic departments; 57 undergraduate majors; 80 graduate programs 95% of undergrads and 79% of grad students are from Illinois 685 international students from 116 countries 10. BLACKBOARD USAGE AT NIU 98% of students 82% of instructional Faculty/Staff/TAs 53% of course sections 4 course sections per student on Blackboard As of Fall 2012: 11. FEATURES Background Features Exploration Roadmap Discussion 12. ABOUT THE ESCE TOOL Introduced in Blackboard Learn 9.1 Service Pack 10, the new "Enterprise Surveys and Course Evaluations" tool adds a robust and flexible set of new capabilities tightly integrated within Blackboard for conducting surveys and course evaluations on a wide scale. 13. QUICK HIT INTRODUCING THE ESCE TOOL 14. FEATURES Collect feedback data anonymously Collect feedback from specific groups of users Integrated in Blackboard, but can be used for non-Blackboard courses Delivered at the system level (doesnt replace course surveys available to instructors) Notifications received in course, by email, or My Blackboard 15. QUESTION TYPES Either/Or Likert Scale Matrix Multiple Choice Open entry 16. SURVEY RECIPIENTS role in the system enrollment in courses membership in a department or business unit if your school licenses community engagement *NOTE - You can also upload a list of recipient email addresses to target community members, business leaders, alumni, parents and other stakeholders 17. RESPONSE PERIODS sets of recipients and a time frame that you specify 18. COMPARISON OPTIONS You can compare by enrollments, membership, demographics, or by a specific survey question. 19. ACCESS AND ACCOUNT ADMINISTRATION The role of Survey Administrator must be assigned to a user in order to access the enterprise survey tool. Once the role is assigned, from the My Institution tab, select Enterprise Surveys on the Tools panel. 20. QUICK DEMO 21. EXPLORATION BY NIU Background Features Exploration Roadmap Discussion 22. INITIAL IMPRESSIONS Seamless integration with existing Blackboard tools Very user-friendly for administration; no advanced technical skills needed Many possible applications exist beyond just course evaluations Report formatting options are very limited Report generation is sluggish 23. SIMPLIFIED STATISTICAL OUTPUT For example, using 5-point scale, for a sample set of 10 responses: 5, 5, 5, 4, 5, 3, 2, 1, 1, 1. The average would be 42/50 points or 84%, which is mostly a positive question. 24. PREFERRED STATISTICAL OUTPUT 25. SIMPLIFIED SUMMARY REPORT NEEDED 26. IN-COURSE EVALUATION BUTTON 27. NEEDED INSTRUCTOR ACCESS TO REPORTS FROM LOGIN MODULE 28. ENVISIONED IMPLEMENTATION Department designates Survey Author who has access to create and manage course evaluations. Evaluations can be standardized and reused at either college or dept level. Colleges need not necessarily coordinate evaluations but can choose to do so. Each college/dept can choose whether or not to allow faculty to add their own questions to their evals. (Additional steps not shown are needed if allowing faculty to add questions). Academic Unit Dept. Coordinator Academic Unit Academic Unit Academic Unit Dept. Coordinator Dept. Coordinator Dept. Coordinator 1) Design evaluation instrument 29. ENVISIONED IMPLEMENTATION 2) Create response period in Bb Academic Unit Dept. Coordinator Academic Unit Academic Unit Academic Unit Dept. Coordinator Dept. Coordinator Dept. Coordinator Fall 2013 Fall 2013 Fall 2013 Fall 2013 Timeline for evals can be customized by each department. Evals may be deployed to students at any time. 30. ENVISIONED IMPLEMENTATION 3) Dept Coordinator deploys evals to selected courses. Bb automatically sends email during the eval time window and each student can login to Bb and complete the eval Dept. Coordinator Student Student Student Student Bb can send automated email reminders and chosen intervals prior to the deadline for completion. Dept. Coordinator can specify such settings when configuring response period. 31. ENVISIONED IMPLEMENTATION 4) After grades are posted, faculty can login to their Bb course to view eval summary and can also run comparison reports. Dept. Coordinator can email faculty a reminder. BlackboardFaculty Faculty Faculty Faculty can run reports in Bb to compare responses across all their courses in a given response period (ie: term). Dept. Coordinator can run reports comparing select courses, or all courses, for select terms. 32. INSTITUTIONAL ADOPTION ROADMAP Background Features Exploration Roadmap Discussion 33. NECESSARY INSTITUTIONAL IMPLEMENTATION STEPS Setup institutional hierarchy in Blackboard to match academic organizational structure of institution Assign courses to terms Choose who will be survey administrator for each department 34. SHOW-STOPPERS 1) Statistics option added for course evaluation reports 2) "Take Survey" notification button in courses customizable, or changed to "Complete Evaluation" for deployed course evaluations 3) Report version summary without graphics (more useful for tenure review applications, personnel committees, etc.) 4) View results for individual courses available after login, without needed to enter each individual course 35. NIUS NEXT STEPS 1) Setup institutional hierarchy and terms in Blackboard (Summer 13) 2) Once show stopper feature gaps are resolved, conduct limited pilot with single academic department (hopefully, Fall 13) and identify any needed tweaks to processes or administrative workflows 3) Conduct widespread pilot (Spring 14) 4) Campus-wide production (Summer / Fall 14) 36. DISCUSSION Background Features Exploration Roadmap Discussion todaysmeet.com/bbesce 37. THANK YOU! JASON RHODE Director of Distributed Learning Northern Illinois University jrhode@niu.edu @jrhode If you would like to provide feedback for this session please email: BbWorldFeedback@blackboard.com The title of this session is: Enterprise Surveys and Course Evaluations