EDF 5807 week 5

The many facets of assessment

Assessment is a very complex and problematic issue, far more complex and problematic than is often realized. There is a lot of 'science' about assessment.

Roles of Assessment• Summative

– Occurs at the end of the unit– To produce a final score or grade

• Formative– Occurs during the unit– Teacher and students can check ongoing progress and improve

identified weaknesses• Diagnostic

– Occurs before the unit– Determine prior knowledge/skills– Determine range of students’ ideas and beliefs

Shifts in thinking about assessment

• Assessment of learning• Assessment for learning• Assessment as learning

Behaviourism

• Dominant paradigm prior to the 1980s• Curricula organised into specific skills• Each skill was expressed as a behavioural

objective• Precise measurement was used to ensure

each skill was assessed

Assessment OF learning

– from a measurement paradigm that is fairly behaviourist

– largely summative, diagnostic assessment of entry skills is also assessment of learning

– Still frequently needed/appropriate

Assessment FOR learning

Emerged from the learning paradigm– the assessment process will allow the student to

improve their understandings/skills– largely formative assessments, including self/peer

assessment– Collecting the range of students ideas and beliefs

is assessment for learning– involves consequential validity - by doing the

tasks, students learn

Assessment AS learning

Emerged from the authentic curriculum learning paradigm– New learning is expected as (not after) the

‘assessment’ task is undertaken– Problem Based Learning (PBL) projects and

AUSVELS Interdisciplinary Tasks are examples

Trustworthiness in evidence of learning

– Validity (accuracy & fairness)

– Reliability (reproducibility)

• Assessment is far less valid and far less reliable than is generally realized.

• There are often serious problems of validity in large scale standardised tests that cause concerns about the political influence of their results.

• The designers of such tests are not incompetent; as assessment moves from 1:1, to whole class, whole year level, state wide, nation wide, and international (and hence cross cultural) it commonly becomes much narrower (often solely based on tests) and increasingly difficult to assess validly.

Validity

An assessment is valid if …– what it assesses is appropriate.– it provides information which is useful for some

valuable purpose.– it is assessed with sufficient accuracy.

Construct validity Are you measuring what you say you have

been trying to teach?You don’t:

- measure the ability to design an effective web page by testing using individual bits of the software design package

- measure ability to play a piano by a written test on music theory

- measure the ability to analyse an argumentative piece of writing by asking students to reproduce your analysis

- measure understanding of a complex idea like energy by calling for a ‘correct’ definition

Reliability

• The reliability of an assessment task is the

consistency with which it assesses whatever it assesses.

• A task could be very reliable, but not valid I could reliably measure the heights of a netball team, but this would not be a valid measure of their value to the team.

Reliability -student

How confident can you be that judgements that you have made are not significantly affected by chance factors such as:– how the student was feeling on the day?– luck in being assessed on some things rather than

others?– luck in that the mode of assessment suited the

student particularly well (or badly?)

Reliability -assessor– If another assessor judged the assessment task,

would the student be awarded the same result? (inter-rater reliability)

– If the same assessor judged the work on another day would the result be the same? (intra-rater reliability).

– This type of reliability is particularly important with criterion-referenced assessment.

The big picture

• Assessment, even at the classroom level requires careful thinking about what you have been trying to teach and (multiple) ways that you could assess for learning. There are a range of issues associated with both setting and interpreting the outcomes of assessment.

• Skilful assessment is a crucial feature of good classrooms

Assessment and learning

1. Assessment has a major influence on how students learn

2. Therefore it is important to use tasks that stimulate/require quality learning

Returning to the discussionFour useful statementsFor each of these statements, explain why Vera or Joanne

said it and then discuss to what extent you now agree or disagree with it and why

1. Vera said Sugar is bad for you2. Jo-Anne said No, it is not, it gives you energy3. Jo-Anne later said That’s why we eat food [to get

energy]4. Vera then said If that was true then you should be able

to live on water and sugar alone, and I don’t think you could

Four aspects of quality learning

• Checks personal understanding• Links school to outside life• Retrieves and links schoolwork to relevant

prior views • Restructures existing ideas as neededSean in Using Concept maps as a before and

after

Metacognition –a key aspect of quality learning

• Cognition is knowing, metacognition is knowing about your own knowing

• Metacognitive learners have : -knowledge of what good learning involves -awareness of their own learning and so can -control their own learning

The Non metacognitive learner

• Sees school as a series of unrelated tasks• Does not recognise that there are key ideas

and skills that provide purposes for tasks or that it is worth thinking about this

• Sees assessment as something teachers do to students

• Hence the need to Move assessment into the process of learning

Using summative assessment formatively

This sounds straightforward, but is often one

of the most difficult lessons to try and run: you are asking students to revisit and wallow in what they see as pain and shame

You need students having a metacognitive understanding of how this can be useful

Aspects of quality learning that can be stimulated or required by assessment

Processing• Looks for key ideas• Checks personal understanding• Builds the richest possible meaning for a piece of content

Aspects of quality learning that can be stimulated or required by assessment

Planning/Preparing• Plans or reflects on the overall strategy –does not just dive in• Checks the purpose of the task• Considers the reasons behind suggested actions• Considers alternative approaches

Procedure F25 Preliminary thinking sheets

Variety matters; do not ask students the same sorts of questions each time. Look for opportunities to get students to work out part or all of the method, what they need to measure accurately, what are possible results, how data will be treated, links to relevant theory, to earlier practical activities or to outside life, what conclusions could be drawn from particular results, however do not ask all of these every time.

Aspects of quality learning that can be stimulated or required by assessment

Decision making• About what to do or explore including the appropriate level of

challenge• About how to manage their time• About how to present work

A smorgasbord

A one point question

A three point question

Aspects of quality learning that can be stimulated or required by assessment

Risk taking• Takes risks in creative tasks• Makes a prediction or informed guessWorking collaboratively• Engages in collaborative planning and working in

small groups

Creative writing in sciencePaula Protein, Vera Vitamin, Carol Carbohydrate, Fanny Fat and Fred Fibre are all in a ham sandwich that you eat. Describe what happens to each of these characters in a creative story that moves from the mouth to the anus• Splash! The gastric juices came squirting in. Paula started

screaming in pain - they were acid and were attacking and dissolving her. The others watched in horror, they didn’t seem so affected but Paula was breaking up into tiny pieces before their eyes. ...

• ...Fred lay alone, bruised and battered on the floor of the intestine - oh no! It was happening again! The walls closed in on him, squeezing and crushing him and pushing him further along. "I hate peristalsis," he groaned.

Big Ideas

1. Assessment has a major influence on how students learn

2. Therefore it is important to use tasks that stimulate/require quality learning

A few comments on skills of assessment

One common flaw

The task can be done/item responded to in ways different from what you intended/wanted but which are completely defensible in terms of the way the task is worded

Aboriginals are ___________ [nomads]

Write brief notes on…-The synthesis of ammonia-Mendeleev and the development of the periodic

table

Better

The synthesis of ammonia involves a trade off between two chemical principles. Discuss

Better

How would the design of the industrial production of ammonia change if the reaction between nitrogen and hydrogen was endothermic [this reverses one of the two competing principles so both now operate in the same direction]

Spot the error:

Who were the first explorers of Australia?A. the DutchB. DampierC. the FrenchD. British

Item with more than one correct answer

The Prime Minister of Australia isA. elected directly by the electorate.B. elected by the Members of Parliament in the governing

party or parties.C. elected by the Parliament.D. appointed by the Governor-General.

Misconceptions about multiple-choice tests

MisconceptionMultiple-choice tests can only measure low-level outcomes such as knowledge of facts, names and places

FactIn the hands of a skilled writer, multiple-choice tests can assess complex reasoning efficiently and well

Bloom application level

Bloom analysis level

Big idea

There is a rich range of skills in designing assessment that is valid and not technically flawed