Denise Clarkson, Willem Fourie, Dobrila

Lopez,

Mike Lopez, Kirsten Marais

Manukau

Institute of Technology

Paper presented at the Symposium on Tertiary Assessment and Higher Education Student Outcomes: Policy, Practice, and Research, Rutherford House, Wellington, New Zealand, 17-19 November 2008.

ContextResearch questionsCourses and sampleMethodResultsConclusionLecturer perspectiveQuestions

HistoricalModernPractitioner

Temple of Apollo at Delphi from belowPhotograph from the Wikimedia

Commons

Gnothi seautonKnow yourself

Socrates?Heraclitus?Pythagoras?From heaven?

Temple of Apollo at Delphi

Photograph from the Wikimedia

Commons

“The maxim “Know thyself” comes down to us from the heavens; it should be imprinted in the heart, and stored in the memory, whether you are looking for a wife, or wishing for a seat in the sacred Senate.”

Juvenal (Decimus lunius luvenalis) - circa 100AD The Satires of Juvenal, Satire 11, Dinner and a moral.

Gnothi seautonKnow yourself

Socrates?Heraclitus?Pythagoras?From heaven?

Everyone knows

Hi, my name is Mike and I’m an Alcoholic

Hi, my name is Mike and I’m a MathematicianAcknowledging where we are is the start of any journey of learningWe need to avoid judging the person

Issues for summative assessment?

We believe that self-assessment is a foundation capability for lifelong learning.This capability should be built progressively throughout a learner’s study.Self-assessment should be used for its believed contribution to learning, not as a means of generating marks.Marks (if any) should be awarded for the quality and perceptiveness of judgementsmade.

Do students value self-assessment?Do students believe professionals need to be able to judge their own work?

Does it help learning?Is it valid?Do students have the capability to assess?

sufficient subject knowledge and understanding of the marking criteria

Are students concerned about the workload involved?

We looked at self-assessment in three domains:Foundation studiesNursingComputing

218 participants66% female53% English as first language26% in first semester of study

Non-experimental, post test only, designQuantitative studyConversion of ordinal data to interval level measurement.Use of scales to reduce uncertaintyCompare scale means by t-testsNon parametric tests of raw responsesRegression used for background variable associations

Do students value self-assessment?t(217)=14.3691; p<.0001CI.99 = (6.5670 ≤ Valued ≤ 7.2557).

D01 The ability to assess my own performance is important 98%D03 The idea of self-assessment is a good one 93%

D16 A similar (but improved) scheme should be used in other courses

82%

D23 We should have more opportunities for self assessment 76%

Do students believe professionals need to be able to judge their own work?Supported (p<.0001)

D24 Most professionals need the ability to judge their own work.

90%

Does self-assessment help learning?t(217)=13.2929; p<.0001CI.99 = (6.6432 ≤ Learning ≤ 7.4380)

D07 I found assessing my own work to be valuable 90%D09 This exercise helped me make a realistic assessment of my own abilities 95%D10 I had to consider my own learning progress more closely than usual 96%D11 Having to justify my mark helped me focus on the value of the work

done96%

D12 This exercise made me more aware of what I need to know in this subject 96%

D02 Self-assessment is more demanding than having someone else assess me 81%

Is self-assessment seen as valid?t(217)=7.0193; p<.0001CI.99 = (5.5611 ≤ Valid ≤ 6.2167)

Negative codingD14 I found it difficult to follow the marking scheme 26%D18 The assessment scheme was unfair 13%D19 The whole exercise of self and peer marking was a waste of time 10%D21 I did not feel comfortable with the responsibility of marking my

own work

30%

Negative codingD22 It’s the lecturer’s job to mark my work. 53%D25 I don’t know enough to be able to judge whether work is good. 51%

Do students have sufficient subject knowledge and understanding of the marking criteria?t(217)=3.1983; p=0.0018CI.99 = (5.1159 ≤ Pknow ≤ 6.1004)

Negative coding (sub-scale of valid)D14 I found it difficult to follow the marking scheme 26%D25 I don’t know enough to be able to judge whether work is good. 51%

Are students concerned about the workload involved?Not significant (t(217)=1.7325; p=0.0807)

Negative codingD04 Self-assessment would be a good idea if we didn’t have such a high

workload76%

D15 I would like to see some changes in the procedure 46%D17 I don’t think the rewards were sufficient for the amount of time I spent 49%

Is self-assessment valued? Yes - strongProfessionals need to judge? Yes - strongDoes it help learning? Yes - strongIs it valid? Yes - strongDo students have capability? Yes - weakConcerned about workload? Not significant

But …

12% of participants agreed (or strongly agreed) that self-assessment was either:

A waste of time ORUnfair

Those who were working in the six months prior to the course were less likely to be opposed. (Beta = 0.2358; R2 = 6%; p< .01)Those who expressed a preference for a surface approach to studying were more likely to be opposed. (Beta = 0.1967; R2 = 4%; p< .01)

Others, although not strongly opposed, had some reservations about self-assessment.

Will this change over time if self-assessment is used progressively in more courses?

D22 It’s the lecturer’s job to mark my work. 53%D25 I don’t know enough to be able to judge whether work is good. 51%D14 I found it difficult to follow the marking scheme 26%D17 I don’t think the rewards were sufficient for the amount of time I spent. 49%

Criterion Predictor(s) Beta R2 Sig.

Opposed Surface approach 0.200 4% .0035

Capability Surface approach ‐0.299 9% <.0001

Valid Surface approach ‐0.187 3% .0057

Workload Surface approach 0.295 9% <.0001

Demanding Deep approach 0.230 5% .0008

Valued Strategic approach 0.323 10% <.0001

Learning Strategic approach 0.501 25% <.0001

Needed Strategic approach 0.209 4% 0.0021

Criterion Predictor(s) Key contrasts Beta R2 Sig.

Valid Gender  Female = more valid 0.287 8% p<.0001

Capability Domain  Computing = less 0.251 6% p=.0003

Valued Level  Foundation = more 0.193 4% p=.0043

Learning Level  Foundation = more 0.326 11% p<.0001

Workload Level  Foundation = more 0.291 8% p<.0001

Demanding LevelAge

Marginally significant even with 2 predictors

0.1520.149 4% p=.0097

Most demanding:Level 5 Over 25

Students strongly endorsed the value and validity of self-assessment and the learning that resulted from it.They endorsed the belief that they had the capacity to do it, but this was less strong.They were neutral on workload concernsSome (12%) were strongly opposed to its use.

Denise ClarksonSchool of Foundation Studies

Engagement

Students became their own teachers

They could see purpose and that effort counted

Test conditions

They wrote their own feedback

Fostered discussion

Evaluation

Students practised critical thinking

Exam TechniqueBecame intimate with the marking schedule

Exposure Repeated exposure created improvement

Questions