using comparative judgement to assess student e-portfolios
DESCRIPTION
Presentation given by Matt Wingfield (TAG) and Susan McLaren (University of Edinburgh) at eAssessment Scotland 2010TRANSCRIPT
© 2010 BLI Education Ltd.
Using Comparative Judgement to Assess Student e-Portfolios
Matt WingfieldManaging Director – TAG Developments
Susan McLarenSenior Lecturer - University of Edinburgh
© 2010 BLI Education Ltd.
• Portfolio Builder (1997) – developed by TAG in partnership with TERU specifically for The London Borough of Newham and Apple;
• Red Pen Tool (2010) – on screen annotation/marking of any digital file;
The evolution of TAG’s e-portfolio solutions
• e-scape (2004 - present) – Formative assessment or project based learning and delivery controlled qualification coursework assessment;
• MAPS (2001 - present) – Multi award-winning National Formative and Summative Assessment ePortfolio for curriculum and qualification work;
• LiveAssess (2010) – Dynamically capture 21st Century Skills.
© 2010 BLI Education Ltd.
Utilised MAPS e-portfolio - enabling teachers to create, deliver, manage controlled assessments of student work (primary, secondary and H.E.); Each student is given a portable device (PDA, Fizzbook Spin);
These devices are connected wirelessly to either a web-based or locally hosted e-portfolio system;
The students receive instructions on what they should do for each activity directly to their device;
As each activity is completed, the work, comments and evidence is dynamically captured by the e-portfolio to be viewed by the teacher.
Pilot project, run by Goldsmiths, University of London in England, University of Edinburgh in Scotland, and Limerick University in Ireland;
Capturing Project Based Learning
Introducing Project e-Scape
© 2010 BLI Education Ltd.
How does it work:
An activity authoring tool is used to define the content and number of stages within the assessment;
The students access the activity through a web-based client;
Evidence is captured on the fly using in-built tools, including:
text & drawing capture;
photo & video capture;
audio capture; and
mind maps.
The Evidence is dynamically sent back to the storyboard portfolio for review.
Capturing Project Based LearningIntroducing Project e-Scape
© 2010 BLI Education Ltd.
Real-time capture of student evidence – progressive storyboard format;
Controlled assessment – structured, timed, no need for other software;
Ideal for supporting the assessment of 21st Century Skills;
BUT – how do we assess creativity, team work or problem solving?
Capturing Project Based LearningIntroducing Project e-Scape
© 2010 BLI Education Ltd.
What is Comparative Pairs?
Measuring without scales;
Based on Rasch/Thurstone - Law of Comparative Judgement;
More recent work from Alastair Pollitt at the University of Cambridge;
Let’s try Comparative Pairs…
Assessment vs. EvaluationComparative Pairs Assessment
© 2010 BLI Education Ltd.
portfolio A
portfolio B
TAG has created a web-based comparative pairs engine, that facilitates highly accurate paired assessment of e-portfolios for the first time on a national scale!
Assessment vs. EvaluationComparative Pairs Assessment
© 2010 BLI Education Ltd.
The Comparative Pairs Engine dynamically refines the ranking of the student work through a series of judging rounds.
What emerges is a collective professional consensus of an expert group of judges.
Produces a reliability coefficient of 0.95
Assessment vs. EvaluationComparative Pairs Assessment
© 2010 BLI Education Ltd.
UK:
Continued research by Goldsmiths (Apprenticeships, Diplomas, SLTs)
Bath Spa University & AstraZeneca Science Teaching Trust to support science & technology in Primary schools
Ireland:
Limerick University to support Degree courses
Israel:
ORT - pilot of school-based examinations – esp. practical science
Western Australia:
Curriculum Council piloting the school leaving exams in Engineering/LOTE/IT and Physical Education
Comparative Pairs Assessment of e-portfolios: where is it being used
© 2010 BLI Education Ltd.
Matt Wingfield
Managing Director – TAG Developments
Tel: 07850 488 502
Susan McLaren
Senior Lecturer – University of Edinburgh
0131 651 6615
http://www.mapsassessment.com
http://www.tagdevelopments.com
Contact Information & Useful Links