Welcome toTeaching + Learning Tuesdays

July 16, 2013 I 2:30PM to 4:30 PM

Title:Innovative Results Reporting: Student Assessment of Instruction

Presenter: Julian Hutcheson, Statistical & Research Analyst III, Aiken Technical College

3

Innovative Results Reporting:

Student Assessment of Instruction

4

Have a “SAI” in your education!”

5

In-house report development for student course evaluation results

Translating this to our vendor’s online system

Features of our online reports

Additional recent innovations

This is the Story of……

6

Fall Enrollment

around 3,000

Approx. 60 FT faculty and

100 PT faculty

Online course evaluation

began in Fall 2009

Vendor was Campus Cruiser,

Cruiser Eval product

Fall 2011 began using

IOTA Solutions

About Aiken Technical College

7

Part of FPMS, Evaluation of Instructors

Student Satisfaction

rates

Trends in courses,

departments, and programs

Collect assessment data on our QEP critical

thinking learning

outcomes

QEP Awareness

Withdrawn students feedback

Comparison of Online vs

Regular classroom

How ATC Uses the SAI Survey

8

16 core questions:

Rating Very Satisfied to

Very Dissatisfied

Converted to

numerical 4-point scale

2 QEP Awareness questions

for all

6 QEP Critical thinking

outcomes for

implementer courses

Faculty required to

keep an average of

3.0

About our SAI Survey (through Fall 2012)

9

Does your online system have standard, built-in reports that meet your needs?

Reporting Results

CourseID Section concat form Course & Section Instructor_Last_Name Instructor_First_Name UNIQUE_KEY Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14PSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 18881610 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 18970174 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 31067426 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 9723702 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 16188092 VS VS VS VS VS VS VS VS VS VS VS VS VS VSPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 13940057 S S D S S S S VS S S S D S SPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 13499398 S VD D S D D VD D VD VD D VD D VDPSY-201 057 PSY-201-57 PSY-201-57 Instructor_Last_Name Instructor_First_Name 22882508 VS VS VS VS VS VS VS VS VS VS VS VS VS VSSUR-102 140 SUR-102-140 SUR-102-140 Instructor_Last_Name Instructor_First_Name 16245664 VS VS VS VS VS VS VS VS VS VS VS VS VS VSSUR-102 140 SUR-102-140 SUR-102-140 Instructor_Last_Name Instructor_First_Name 8477022 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 045 BIO-210-45 BIO-210-45 Instructor_Last_Name Instructor_First_Name 1198689 VS D S S VS S S D S S VS S S SBIO-210 045 BIO-210-45 BIO-210-45 Instructor_Last_Name Instructor_First_Name 13710771 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 045 BIO-210-45 BIO-210-45 Instructor_Last_Name Instructor_First_Name 18652426 VS S S VS S VS VS S S VS VS VS VS VSBIO-210 045 BIO-210-45 BIO-210-45 Instructor_Last_Name Instructor_First_Name 25827011 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 14455382 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 17531362 S S S S S S S D S S S S S VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 8147534 VS VS VS VS VS VS VS S S S VS VS VS SBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 33536410 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 32656969 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 15839791 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 32074949 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 31723791 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 1838195 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 24478282 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 29149708 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 24438272 S VS S S S S S D S D D S D DBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 27803933 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 21691307 VS VS VS VS VS VS VS VS VS VS VS VS VS VSBIO-210 057 BIO-210-57 BIO-210-57 Instructor_Last_Name Instructor_First_Name 30463164 VS VS VS VS VS VS VS VS VS VS VS VS VS VSRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 20885947 VS VS VS VS VS VS VS VS VS VS VS VS VS VSRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 17212158 VS VS VS VS VS VS VS VS VS VS VS VS VS VSRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 11993973 VS VS VS S VS VS VS VS S S S S VS SRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 21247300 VS VS VS VS VS VS VS VS VS VS VS VS VS VSRDG-032 002 RDG-032-2 RDG-032-2 Instructor_Last_Name Instructor_First_Name 17106447 VS VS VS VS VS VS VS VS VS VS VS VS VS VS

Raw

11

Diligence

16

TermCourse ID Section

Instructor Last Name

Instructor First Name

Q1: My instructor explains and clarifies course learning outcomes, grading and requirements.

2010FA1 BUS-140 011 Louisa Delay 4.00

2010FA1 ECD-135 003 Mallory Brassell 4.00

2010FA1 MTT-112 040 Fernando Desanto 4.00

2010FA1 IMT-151 041 Kurt Prettyman 4.00

2010FA1 CPT-101 151 Codi Smith 3.92

2010FA1 IMT-113 041 Ted Wagoner 3.91

2010FA1 WLD-102 018 Louisa Garabedian 3.91

2010FA1 ACR-102 040 Karina Sobel 3.91

2010FA1 MKT-240 151 Tameka Valderman 3.91

2010FA1 WLD-132 069 Eric Sturtz 3.90

2010FA1 CPT-209 029 Lakisha Turnbough 3.89

2010FA1 WLD-103 040 Jamie Quisenberry 3.89

2010FA1 CRJ-202 069 John Schieber 3.86

2010FA1 CPT-101 451 Lance Vanders 3.86

2010FA1 WLD-111 040 Clayton Shira 3.84

2010FA1 WLD-102 040 Selena Dutil 2.83

2010FA1 IMT-151 040 Hillary Filip 3.84

2010FA1 CPT-162 057 Christian Tiano 3.83

2010FA1 RPT-223 058 Louisa Delay 3.82

2010FA1 MGT-101 060 Mallory Brassell 3.82

2010FA1 CPT-101 013 Fernando Desanto 3.82

2010FA1 IST-201 102 Kurt Prettyman 2.80

2010FA1 BAF-230 063 Codi Smith 3.81

2010FA1 CPT-257 103 Ted Wagoner 3.80

2010FA1 WLD-101 040 Louisa Garabedian 3.80

2010FA1 ECD-101 064 Karina Sobel 3.73

2010FA1 EET-251 040 Tameka Valderman 3.73

17

Term CourseID Section First_Name Instructor_Last_Name Instructor Overall Average with Course Section Averages

2010FA1 CPT-162 057 Louisa Delay 3.82

2010FA1 CPT-168 059 Louisa Delay 3.58

        Delay Average 3.70

2010FA1 WLD-132 069 Fernando Desanto 3.20

2010FA1 WLD-170 040 Fernando Desanto 3.20

        Desanto Average 3.20

2010FA1 RPT-243 069 Codi Smith 2.98

        Smith Average 2.98

2010FA1 ECD-102 002 Ted Wagoner 3.71

2010FA1 ECD-237 040 Ted Wagoner 2.95

2010FA1 ECD-131 015 Ted Wagoner 2.88

2010FA1 ECD-133 040 Ted Wagoner 2.68

        Wagoner Average 3.06

2010FA1 CPT-162 015 Louisa Garabedian 3.69

2010FA1 CPT-212 003 Louisa Garabedian 3.49

2010FA1 CPT-162 013 Louisa Garabedian 3.39

        Garabedian Average 3.52

2010FA1 MKT-240 151 Karina Sobel 4.00

2010FA1 MKT-240 014 Karina Sobel 3.70

2010FA1 MKT-101 058 Karina Sobel 3.62

2010FA1 MGT-120 013 Karina Sobel 3.07

        Sobel Average 3.60

2010FA1 EGT-252 069 Tameka Valderman 3.00

        Valderman Average 3.00

18

19

Positives

User friendly mostly

Overall averagesSorted by course and by instructor

PDF with Navigating tabs

Fall 2010: Good but Lengthy

20

21

22

23

Compact

Seeing results for

all questions on single

page

12 pages versus

116 pages

Can see areas of weakness in

student satisfaction:

Referring to the survey

questions

Patterns of red areas

Online courses and QEP

Implementer courses identified

by color coding

Sorted by course and by Instructor,

5 pages each

Spring 2011 Report

24

Encouraged customization

Very different from current

vendor

Can you do this? Usually “Yes!”

Site graphics made course evaluations

look like ATC

Enter IOTA Solutions, Inc

25

Even standard reports were a

significant step up

from previous vendor

Faculty views and

Dept. Chair views

impressive

What is standard and what

is custom?

Implemented Fall

2011

Overall went well,

but not all rosy

IOTA, continued

26

http://atc.iota.cc

Let’s go!

27

28

29

• All the information of the original report, plus response rates & comments link

• Automatically generated• Numerous interactive sorting, ranking, and

filtering features• Instantly look at the data in different ways• Link to comments instantly• Printable• QEP Reports

IOTA Custom Report

30

We are in the middle of a

journey

Going to a much more customized

reporting tool has its

downsides and risks

Bugs, Failures, Beta

Customer service may be

impacted by features not

working at all times

Customization

31

• Affordable• Prompt attentive customer service to correct problems

• Automated, customized reports with features already described

Positive Side

32

• Benefits clearly outweigh the costs

• For customized, automatic reporting features, more tolerance and patience are warranted to allow the development process to occur

Final Analysis:

33

• Moved from 16 questions on instruction to:• 5 Instructor questions• 4 Course questions, and a combination of

the following, based on delivery method• Lecture, Laboratory, Clinical Site, Online

Course, Hybrid Courses, Two-way Video Courses, Flexible / Open Lab, and Field Work

• 2 to 4 questions for each• Only the 5 instructor questions feed into

the FPMS and the Custom Report

New Development Spring 2013Questions Based on Delivery Method

34

Discussion

Discussion

35

Julian HutchesonAiken Technical CollegeOffice of Planning & Research803-508-7418hutchesj@atc.edu

Contact:

Join us every Third Tuesday for more Professional Development opportunities.Next Session: August 20, 2013TIME 2:30pm – 4:30pmTITLE Preparing for the Fifth-Year Report and ReaffirmationPRESENTERS Dr. Claudette Williams, SACS-COC