2012 abe test assessment

7/31/2019 2012 ABE Test Assessment

1/29

The Test Assessment Questionnaire

Katherine M. Sauer William MertensMetropolitan State College of Denver University of Colorado at Boulder

[email protected] [email protected]

March 29, 2012 MBAA ABE Annual Conference Chicago, IL


2/29


3/29

Students are Overconfident

Walstad (2001) - suggests concepts from behavioral

economics (e.g. overconfidence) could be used to explainstudent behavior

Falchikov and Boud (1989) - students have grade

expectations that are higher than the typical distributionfor the course

Grimes (2002) - has principles students predict examscores and finds overconfidence in their understanding ofmaterial

Nowell and Alston (2007) - instructor grading practicescan influence the degree of overconfidence


4/29

What Happens When Students are Overconfident?

Students may start toblame the instructorfor thedisconnect between (perceived) effort and desired

grade.

Students may label the instructor as unfair or thecourse simply too hard.


5/29

Grimes, Millea, and Woodruff (2004) - degree to whichstudents accept personal responsibility for performanceaffects their evaluation of teaching effectiveness and

course satisfaction

Carrell and West (2010) - in course evaluations, studentsreward professors who increase achievement in the

contemporaneous course, not those who facilitated deeplearning for subsequent courses

Millea and Grimes (2002) - instructors need not water-

down courses in order to receive favorable courseevaluations they can positively influence evaluations byaddressing negative student attitudes about forthcomingcoursework


6/29

How can I help my students become

more self-aware in their learning?

Get them to focus on the type of mistake they are

making instead of the number of mistakes they aremaking.

Get them thinking about the link between their study

activities and their exam performance.


7/29

The Test Assessment Questionnaire

This instrument guides students through an analysis oftheir midterm exam mistakes.

questions about exam preparation activities

studying activities for the course in general

formulate a study strategy for the rest of the course


8/29

The Version Im Currently Using


9/29


10/29


11/29


12/29

Instructors can tailorthe first set of questions to theircourse.


13/29


14/29

Three Benefits

1) it guides students toward more self-awareness in theirstudies

2) it can be used as a part ofassessment and assurance oflearning efforts

3) may improve students final exam scores over themidterm


15/29

1) self-awareness in studies

On the course evaluations: Please comment on the

experience of completing the Test Assessment and/ormeeting with me to discuss it.

- It made me improve my studying skills.

-Good, it gave me one-on-one time with you to figure outhow I could improve.

-Helpful in learning how I make mistakes and how to fixthem better.


16/29

- It helped me to personally analyze my test taking andstudy habits. Very helpful! I have never had a teacher whocared enough to take the time to help me improve in thisaspect.

-Your tips helped me in other classes and got me runningagain and I dont feel as stressed. My mom thanks you.

There was no negative feedback on this question.


17/29


18/29


19/29

3) improve students final exam scores

Experiment:

Fall 2011 University of Colorado

All Students in Principles of Microeconomics10 course sections, 1713 students

62 recitation sections


20/29

Design:

Course instructors do not modify their class time orteaching activities in any way.

Each TA that teaches two or more recitation sections has

a control and experiment section assigned to them.

The experiment will take place during the recitationclass in which the midterm is handed back.


21/29

Control Sections: TAs spend the class time going overthe exam as they normally would and will offer somegeneral study strategies.

Experiment Sections: Students will be given the correct

answers and will spend the class time going over theexam by themselves while completing the TAQ.

After the TAQs are completed, the TA discusses study

strategies that complement each of the common mistakecategories.


22/29

All students complete Duttweilers(1984) InternalControl Index.

- many such instruments

- argue this one is most reliable and validone for adults

- normed on college students

- 28 questions, Likert-type scale


23/29


24/29

Half of the items are worded so that high internallyoriented respondents are expected to answer at the

"usually" end of the scale and the other half at the"rarely" end.The "rarely" response is scored as 5 points on items

1,2,8,11,14,17,19,22,23,24,26,27.

This produces a possible range of scores from 28 to 140with higher scores reflecting higher internal locus ofcontrol.

For college students: MEAN: 99.3- 120.8


25/29

Data:

Two course instructors opted out of the experiment.n = 1337

27 treatment recitations, n = 66727 control recitations, n = 670

But not all sections were full.


26/29

We received 522 first-round ICI surveys from 28different recitations.

We received 488 second-round ICI surveys also from 28different recitations (but not exactly the same recitationsas those for the first-round surveys).

We received 234 TAQs from 13 different recitations.


27/29

Treatment groupsWe received data that included both surveys and the

TAQ for six recitations involving a maximum of88students having all 3 sheets (if they match).

We received data that included one survey and a TAQ

from seven recitations involving a maximum of104students having both sheets (if they match).Control groups

We received data that includedboth surveys for tenrecitations involving a maximum of175 students having

both sheets (if they match).


28/29

The students TAQs and ICIs are being scanned and thedata entered into a spreadsheet.

Control data is being obtained from InstitutionalResearch.


29/29

Comments? Questions?

Katherine M. SauerMetropolitan State College of Denver

[email protected]

March 29, 2012 MBAA ABE Annual Conference Chicago, IL

2012 abe test assessment

Documents