Purpose of Assessment and Feedback Combined approach to improve

teaching and learning whilst informing reporting and accountability

Lenore AdieAssociate Professor in Teacher Education and Assessment

Rowntree, 1987

How shall we know them?

Assessment Purpose

FormativeProgressing learning during teaching

Next-step learningContent + Process Learning goals

FeedbackRange of variables

Dialogic

SummativeReporting

Next-step learningContent + Process Learning goals

Feedback

Reporting

Range of variables

Why are you assessing?

How do you want them to know it?

Exactly what knowledge do

you want students to

have?

What task(s) will the students perform to

communicate their knowledge?

How will you analyse and interpret evidence?

What will you accept as evidence that a

student has the desired knowledge?

Assessment purpose - Assessment design

Adapted from Pellegrino, J.W., DiBello, L.V. & Goldman, S.R. (2016)

Educational assessment is at heart an exercise in evidentiary reasoning. (Mislevy & Riconscente, 2005, p. iv)

design of formative

assessment opportunities

high quality, multiple data

sources

attend to descriptive features of

student work

connect the data with instructional

plans -differentiation of

instruction

Appraise the information for validity

and relevance

formative feedback is a

finer grain size

Evidentiary decision-making

The indispensable conditions for improvement are that the student comes to hold a concept of quality roughly similar to that held by the teacher, is able to monitor continuously the quality of what is being produced during the act of production itself, and has a repertoire of alternative moves or strategies from which to draw at any given point. In other words, students have to be able to judge the quality of what they are producing and be able to regulate what they are doing during the doing of it.

Sadler (1989. p. 121)

Transparency

Fairness

Fine-grained details

Engendering trust Motivation

Right to privacy

Sadler (1985)

3 difficulties in defining criteria

Lack of distinct boundaries

Role of experience on interpretation

Unanticipated qualities

How to enable students to be better learners through knowledge of

criteria and developing expertise in their use?

reduce trial and error attempts in students’ efforts to produce ‘good work’

promote practices and provide artefacts that develop evaluative experience and expertise

4 enabling conditions

for including

students in assessment

practices

Teachers’ assessment identities

Students’ assessment identities

Social moderation

The role of artefacts

Teacher assessment identityKnowledge and skills

Beliefs, confidence, role, feelings

Development of expertiseProfessional standards

Evidentiary decision makingUse of data and evidence; artefacts.

Moderation Looney,Cumming,vanderKleij &Harris(2017)

(Adie, 2016)

Gra

du

ate

(G

)P

roficie

nt

(P)

Hig

hly

A

ccom

plish

ed

(HA

)

Lea

d

(L)

5.2 Provide Feedback to Students on Learning S

tud

en

tsS

cho

ol

PurposeContentAction

Effective Ran

gePurpose

Info

rmed

, Tim

ely

Judg

emen

ts

ModelL:

InitiateL:

About Current Needs in order to Progress Learning

HA:SelectHA:

ProvideP:

Demonstrate an Understanding

G:

ProgramsL:

Exemplary PracticeL: Support ColleaguesL:

About Achievement Relative to Learning Goals

P:

About LearningG:

+ Targeted+ EffectiveTimely, Appropriate Feedback

HA:P:G:

(Adie, Stobart & Cumming, forthcoming)

ArtefactsExemplar Cognitive commentary

Overall this student’s work sample is best matched with a ‘Developing +’ standard. This student generally expressed a point of view, used the exposition framework, provided some supporting evidence for the viewpoint and included some relevant details to provide information about life in the Gold Fields which are matched with the ‘Consolidating’ standard. However, the student needed to have written a final statement that referred to the viewpoint, explained the reason for her point of view in the thesis and made use of more specific conjunctions to strengthen the argument to achieve an overall ‘Consolidating’ standard. Because this student did not offer a conclusion but demonstrated a ‘Consolidating’ standard against 4 criteria, on balance, this work has been rated at no more or no less than a Developing +’ standard.

From the M eeting in the M iddle project (W yatt-Sm ith & B ridges, 2007, p . 8 )

Adie & W illis (2016).

Moderation insights

… you never really understand what they’re [criteria and standards] about until you are grading or you are using them. So, until you see them in operation it’s hard to know, but there is a danger of being too detailed and almost verbose with what you’re trying to do.

(Klenowski & Wyatt-Smith, 2010, p. 34)

Teacher 1: …what I have found is that it is impossible to design criteria that work all the time and what you need to do is that you need to use them more as a …

Teacher 2: …a guideline?

Teacher 1: …a rough guideline and then you need to see.

(Wyatt-Smith, Klenowski, & Gunn, 2010, p. 68)

Student assessment identityKnowledge and skills

Beliefs, confidence, role, feelings

Development of expertiseSelf-assessment; Peer assessment

Evidentiary decision makingUse of data and evidence; artefacts.

Feedback as a dialogic process

Van der Kleij, Adie, Cumming, 2016

Artefacts Criteria

Self-assessment

Panadero, Alonso-Tapia, and Huertas, 2012;

Fletcher, 2016

Chang, Liang, & Chen, 2013;

Wyatt-Smith & Bridges, 2007

higher-performing students may be more inaccurate in SA

(Hosein & Harle 2018)

A

Define the criteria by

which students assess

their work

B

Teach students how

to apply the criteria

CGive students

feedback on their

self-assessments

DGive students help in

using self-assessment

data to improve

performance

EProvide sufficient

time for revision

after self-

assessment

F

Do not turn self-

assessment into

self-evaluation by

counting it toward

a grade

Se

lf-asse

ssm

en

t: Imp

lica

tion

s fo

r p

rac

tice

Panadero, Jonsson, & Strijbos, 2016, p. 306

Peer assessment

Guiding questions, prompts Rotsaert, Panadero, Schellens, & Raes(in press); Gan & Hattie (2014)

Rubrics, Exemplars -Knowledge of standards and criteriaGan & Hattie (2014); Rotsaert, Panadero, Schellens, & Raes (in press)

Co-constructing and deconstructing criteriaLeenknecht & Prins (in press)

Psychologically safe conditions — TrustHarris & Brown (2013); Panadero, (2016)

Guidance and instructionTsivitanidou, Zacharia & Hovardas, (2011); Hovardas et al. (2014)

Practice

(Adie, Stobart & Cumming, forthcoming)

Site validity

Localisedknowledge,

situated contexts

System validity

Political accountability / improvement agendas, generalisable across sites

(Freebody & Wyatt-Smith, 2004; Adie & Wyatt-Smith, 2018 in-press)

Rowntree, 1987

How shall we know them?

Theoretical frameworkRobinson & Taylor (2007, p.8) four core values of student voice

A conception of communication as

dialogue

Dialogue enables

development of shared

understanding

The requirement for participation and democratic

inclusivity

Participation and democratic

inclusivity require all

voices to be listened to; and acceptance of

diversity

The recognition that power

relations are unequal and problematic

Equal power means

equitable opportunities

to actively express ideas and to shape

consequences

The possibility for change and

transformation

Transformation actively seeks

(student) voice, takes it

seriously and uses it to

creatively solve problems

Theoretical ways forward -Student Voice Lundy (2007)

Space to express a view

Coaching to communicate their learning

Action on views expressed

Audience to hear views

Reconceptualising reporting

teacher ownership of

data reinterpretation throughdialogue

participation and

inclusiveness mutual

partnership

expertise of teacher,

student, and family

reporting

progressing learningassessment

communicating

Combined approach to improve teaching and learning whilst informing reporting and accountability

Dialogic feedback

System and site validity

Reporting: Student voice

Student self- and peer assessment

How shall we know them?

Assessment designStandards-referenced assessment as an interconnected system

Assessment as shared inquiry

Assessment identity

ReferencesAdie, L. (2016). Working in a system of standards-referenced assessment: Traversing the intersections. In H. Fehring (Ed.). Assessment into Practice. Primary English Teaching Association Australia (PETAA).

Adie, L. & Willis, J. (2016). Making meaning of assessment policy through teacher assessment conversations. In D. Laveault and L. Allal (Eds.). Assessment for Learning: Overcoming the Challenges of Implementation. (pp. 35-53). Dordrecht, the Netherlands: Springer.

Adie, L., & Wyatt-Smith, C. (in-press 2018). What is the potential of standards, validation and moderation for demonstrating quality in initial teacher education? In C. Wyatt-Smith & L. Adie (Eds.), Teacher education, learning innovation and accountability. Springer.

Chang, C.-C., Liang, C., & Chen, Y.-H. (2013). Is learner self-assessment reliable and valid in a web-based portfolio environment for high school students? Computers & Education, 60(1), 325–334.

Fletcher, A. K. (2016). Exceeding expectations: Scaffolding agentic engagement through assessment as learning. Educational Research, 58(4), 400–419.

Freebody, P., & Wyatt-Smith, C. (2004). The assessment of literacy: Working the zone between system and site validity. Journal of Educational Enquiry, 5(2), 30-49.Gan, M. S., & Hattie, J. (2014). Prompting secondary students’ use of criteria, feedback specificity and feedback levels during an investigative task. Instructional Science, 42(6), 861–878.

Harris, L. R., & Brown, G. T. L. (2013). Opportunities and obstacles to consider when using peer- and self-assessment to improve student learning: Case studies into teachers' implementation. Teaching and Teacher Education, 36(0), 101-111.

Hovardas, T., Tsivitanidou, O. E., & Zacharia, Z. C. (2014). Peer versus expert feedback: An investigation of the quality of peer feedback among secondary school students. Computers & Education, 71(Supplement C), 133–152.

Klenowski, V. & Wyatt-Smith, C. (2010). Standards, teacher judgement and moderation in contexts of national curriculum and assessment reform. Assessment Matters, 2, 107-131.

Leenknecht, M. J. M., & Prins, F. J. (in press). Formative peer assessment in primary school: The effects of involving pupils in setting assessment criteria on their appraisal and feedback style. European Journal of Psychology of Education, 1–16. doi:10.1007/s10212-017-0340-2

Looney, A., Cumming, J., Van Der Kleij, F., & Harris, K. (2017). Reconceptualising the role of teachers as assessors: Teacher assessment identity. Assessment in Education: Principles, Policy & Practice, 1–26. https://doi.org/10.1080/0969594X.2016.1268090

Lundy, L. (2007). ‘Voice’ is not enough: conceptualising Article 12 of the United Nations Convention on the Rights of the Child. British Educational Research Journal, 33(6), 927–942. Mislevy, R. J., & Riconscente, M. M. (2005). Evidence-centered assessment design: Layers, structures, and terminology. Menlo Park, CA: SRI International.

Panadero, E. (2016). Is it safe? Social, interpersonal, and human effects of peer assessment: A review and future directions. In G. T. L. Brown & L. R. Harris (Eds.), Handbook of human and social conditions in assessment (pp. 247–266). New York, NY: Routledge.

Panadero, E., Alonso-Tapia, J., & Huertas, J. A. (2012). Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806–813.

Panadero, E., Jonsson, A., & Strijbos, J.-W. (2016). Scaffolding self-regulated learning through self-assessment and peer assessment: Guidelines for classroom implementation. In D. Laveault & L. Allal (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 311–326): Cham, Switzerland: Springer International.

Pellegrino, J.W., DiBello, L.V. & Goldman, S.R. (2016). A framework for conceptualizing and evaluating the validity of instructionally relevant assessments. Educational Psychologist, 51(1), 59-81.

Robinson, C., & Taylor, C. (2007). Theorizing student voice: values and perspectives. Improving Schools, 10(1), 5–17. Rotsaert, T., Panadero, E., Schellens, T., & Raes, A. (2017). “Now you know what you’re doing right and wrong!” Peer feedback quality in synchronous peer assessment in secondary education.

European Journal of Psychology of Education, 1–21. doi:10.1007/s10212-017-0329-xRowntree, D. (1987). Assessing students: How shall we know them? (Rev. ed.). London: New York: Kogan Page; Nichols Pub.

Sadler, D. R. (1985). The origins and functions of evaluative criteria. Educational Theory, 35(3), 285–297.

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144.

Tsivitanidou, O. E., Zacharia, Z. C., & Hovardas, T. (2011). Investigating secondary school students’ unmediated peer assessment skills. Learning and Instruction, 21(4), 506–519. Van der Kleij, F., Adie, L., & Cumming, J. (2017). Using video technology to enable student voice in assessment feedback. British Journal of Educational Technology, 48(5), 1092-1105.

Wyatt-Smith, C. M., & Bridges, S. (2007). Meeting in the middle – assessment, pedagogy, learning and educational disadvantage. Evaluation Study for the Department of Education, Science and Training on Literacy and Numeracy in the Middle years of Schooling Initiative Strand A, Queensland Project Report.

Wyatt-Smith, C., Klenowski, V., & Gunn, S. (2010). The centrality of teachers’ judgment practice in assessment: A study of standards in moderation. Assessment in Education: Principles, Policy & Practice, 17(1), 59–75.

Associate Professor Lenore Adie

Lenore.adie@acu.eduau