Download - The Formative Use Of E Assessment
![Page 1: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/1.jpg)
The formative use of e-assessment
Some early implementations, and
Suggestions for how we might move on
Andrew Boyle
![Page 2: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/2.jpg)
Initial stuff – rationale, definitions, etc.
![Page 3: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/3.jpg)
Introduction
• Early predictions re e-assessment– e-assessment will lower barriers between formative and summative
assessment– Early years of e-assessment coincided with renewed interest in
formative assessment (FA)– Now e-assessment needs to move to ‘mature’ phase
• Literature review– Important role in FA research– Attempt to summarise where we have got to with formative e-
assessment (eFA)– Group implementations to establish coherence– Move towards critique
![Page 4: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/4.jpg)
Definitions• FA
– All those activities undertaken by teachers, and/or by students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged
– Contrasted with summative assessment Summarises learning Used for recording and reporting the amount of learning but not for feeding back
into learning– Important strand in both school and HE assessment research
Slightly different emphases
• e-assessment includes– Tests that are delivered on-screen– e-portfolios– Electronic discussion boards, forums and so on
![Page 5: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/5.jpg)
Research aims
• Identify types of implementations used frequently by researchers and developers working in the field
• Suggest ways in which the body of research evidence might be expanded
• Underlying aim to delineate those areas where eFA provides a distinctive input
• Balance ‘e-assessment as transformative’ and more sceptical views
![Page 6: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/6.jpg)
Method and scope
• Review is inclusive, rather than excluding– ‘Best evidence synthesis’– Seeks authentic, faithful and convincing results– Does not comply with one or more ‘objective’ criteria
• A thorough review of eFA literature– Backed up by a selective review of FA literature– Not a general review of e-assessment
![Page 7: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/7.jpg)
Background findings re ‘plain’ FA
![Page 8: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/8.jpg)
FA research is about several things
• Interaction– Teacher-learner; learner-learner
• Feedback is central– Speech – questions and answers– Comment-only marking on written work
Focus on features of the work, not the individual Permit learner to ‘close the gap’ between current and desired performance
• Features of FA items and tests relatively little studied– Can have more (or less) than one correct answer– Distractors explicitly connected to incorrect or incomplete conceptions– Item responses provide clues to effective action
![Page 9: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/9.jpg)
Claims about FA
• Associated with learning gains– Claimed to be one of the strongest associations
between educational intervention and learning gains– Some question about whether all reported gains are
caused by the FA intervention
• Disseminating messages to practitioners key– Evidence that changes in practice have been patchy
![Page 10: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/10.jpg)
Organisation of eFA findings
![Page 11: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/11.jpg)
FindingsAreas for
further work Key issues
![Page 12: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/12.jpg)
Finding 1
Electronic technologies provide a range of new tools that classroom teachers can use to create formative assessments to suit their and their students’ needs.
• Variations on the theme of MCQ• Sophisticated tasks – rich in interactivity and multimedia• Test designs specific to e-assessment• e-portfolios to closely integrate formative and summative
assessment• Communications tools such as electronic discussion boards
and forums for self- and peer feedback in e-learning courses
![Page 13: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/13.jpg)
Characteristics of formative & summative items
• eFA implementations take item and task types from summative assessment– Cf. Wiliam’s characteristics of good formative items
• Need to build on research into FA item types• Establish distinctive characteristics of ‘good’ eFA
items/tasks
![Page 14: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/14.jpg)
eFA as exam revision
• FA research emphasises several facets• Many eFA implementations equate FA with
revision or practice testing– Revision not a bad thing per se
Can reduce test anxiety Support distance learners
– But is a reduced concept compared to totality of FA
![Page 15: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/15.jpg)
Need to write own test questions
• Early implementations of eFA often involve innovators developing own questions– Realistic for all teachers to write their own questions?– If teachers select from pre-written banks of questions,
is anything lost? Would they be able to tailor the formative
assessment to their learners’ needs? What sort of skill is selecting balanced formative
tests from an item bank?
![Page 16: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/16.jpg)
Finding 2
e-assessment functionality permits formative feedback to be given in variety of ways not possible in plain FA.
• Variety of feedback methods in eFA test applications• Advantages of e-portfolios for providing feedback
– Encourages teachers and learners to interact about drafts– Process of generating work forefronted, rather than merely concentrating on the final
product– Communications tools can give feedback varied by: channel, recipient, formality, etc.
Can target learners of different styles
• E-learning technologies can facilitate giving of feedback to distance learners– Contribute to online discussions (peer feedback)– Reflective journal (self-assessment)– Evaluate peers’ work (either formally or informally)
![Page 17: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/17.jpg)
Principles for feedback in e-tests
• Need to systematise understanding of e-test feedback– Does ‘comments not grades’ apply to eFA?
If so, one of the most obvious benefits of e-assessment (rapid right/wrong info) is removed.
– Is engagement with rich media and interactivity synonymous with deep learning? Could learners be distracted – clicking through
sites without really truly processing content?– Could we relate feedback to students’ learning styles?
![Page 18: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/18.jpg)
Practicality of e-portfolio feedback
• Need logistical studies that investigate practicality of using e-portfolios– Ensure that teachers can provide high-quality
feedback and potential is not lost– ICT elements of portfolios should reduce burden
![Page 19: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/19.jpg)
Impact of cultural factors in e-communication
• Students giving and receiving feedback need to understand cultural norms– Students working at distance may bring different prior
assumptions to feedback– Culture can also mean academic culture
Concern if students have not internalised academic norms Requirement to style switch between varying e-discourses
could make it harder for some to internalise ways of writing
![Page 20: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/20.jpg)
Finding 3
eFA applications can be used remotely. This provides a resource which is not easily replicated via pencil-and-paper materials.
• Advantages of asynchronicity for undergraduates:– Reduced examination stress– Popular and motivating– Free up teacher time– Allow students to increase self-regulation for tertiary study– Asynchronous online discussions facilitates enhanced reflection
• Areas needing further clarification– Effect of learning styles and motivation of usage
Already self-regulating students benefit more Usage differs between intrinsically interested & ‘pragmatists’
– Clarify most common usage patterns
![Page 21: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/21.jpg)
Better studies of learning gains
• Many papers assert attainment benefits of eFA– Esp. for self-access use
• Claims undermined by research design– Small cohorts– Test difficulty not equated– Variables confounded
• Does eFA provide attainment benefits over and above ‘plain’ FA?– It might – there are also claims that use of ICT in schooling is
associated with improved attainment
![Page 22: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/22.jpg)
Self-access for primary and secondary students?
• eFA as self-access strongly associated with tertiary• What issues for self-access use by school pupils?
– Greater obligation on teachers to moderate feedback– Role of parents in online learning– Effective use to support ‘personalised learning’?
![Page 23: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/23.jpg)
Superordinate ‘key issues’
![Page 24: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/24.jpg)
Define how the ‘e’ goes beyond plain FA, AND
Define genuine distinctiveness of eFA as opposed to summative e-assessment
•Use of instruments by practitioners
•Provision of feedback
•Need to demonstrate learning gains
eFA must not be a reduced version of ‘plain’ FA
•eFA as exam revision
•Provision of feedback
Beware of how eFA may impose novel burdens on teachers and students•Requirement of teachers to write own test questions•Need to provide manageable systems for feeding back via e-portfolios
Need to understand how teachers and students adapt to novel roles that eFA requires
•Cultural demands of using e-communication tools
•Strong element of independent working and self-assessment
![Page 25: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/25.jpg)
•Thanks for listening.
•Questions/comments …
![Page 26: The Formative Use Of E Assessment](https://reader035.vdocuments.site/reader035/viewer/2022062708/558a30fad8b42a32138b45a7/html5/thumbnails/26.jpg)
Features of FA items and tests• Can have more (or less) than one correct
answer• Distractors explicitly connected to incorrect or
incomplete conceptions• Item responses provide clues to effective action