1ExamSoft Assessment Conference | #ASSESS2015

ExamSoft Assessment Conference

#ASSESS2015

2ExamSoft Assessment Conference | #ASSESS2015

Building A Better Test Taker

• Brad Marcum, Director, • Academic Data Services, KYCOM

• Meg Sidle, Director, Institutional Research • and Effectiveness, UPIKE

3ExamSoft Assessment Conference | #ASSESS2015

A Little Background

•Medical School Mission•Computer-based testing began in Fall 2012

•Provided immediate formative assessment results to improve student learning within individual courses

4ExamSoft Assessment Conference | #ASSESS2015

Unaddressed Issue

First-time pass rates of COMLEX Level 1 remained low

Since 2004, continued to examine relationships with admissions criteriaindividual course performance

Not making a difference

5ExamSoft Assessment Conference | #ASSESS2015

A Fresh Approach

• Comprehensive Osteopathic Medical Self-Assessment Examination (COMSAE)

• Strong correlation (r = .80) between KYCOM student performance on this assessment and their actual COMLEX Level 1 scores

• Find predictors for performance on the COMSAE:Regression indicated that seven predictors

explained 57.2% of the variance (R2 =.59, F(7,177)=36.07, ρ<.01).

6ExamSoft Assessment Conference | #ASSESS2015

Common Faculty Reaction to Statistics

7ExamSoft Assessment Conference | #ASSESS2015

Summative Assessment

• See if data from the computer-based testing on the item content from the four courses in the model could prove useful

• Major issues with categories• Thirty-eight (38) specific topics from these

courses were significantly related to the students’ COMSAE scores. (ρ<.01) Not necessarily strong relationships

8ExamSoft Assessment Conference | #ASSESS2015

Where Do We Go From Here

• Faculty need to question why these particular topics were significant out of the more that 150 topics in the four courses

• Faculty should examine the range of topics and specific lessons that they taught in the respective courses on these topics

• Faculty development workshops how to write criteria

9ExamSoft Assessment Conference | #ASSESS2015

THANK YOU!

• Brad Marcum, Director, • Academic Data Services, KYCOM

• Meg Sidle, Director, Institutional Research • and Effectiveness, UPIKE