eassessment for 21 century learning and skills - is unit...
TRANSCRIPT
eAssessment for 21st Century Learning and Skills
Anusca Ferrari Christine Redecker & Yves Punie JRC-IPTS
IPTS: Part of Joint Research Centre
of the EC: 7 Research Institutes
across Europe
Mission: “to provide customer-
driven support to the EU policy-
making process by developing
science-based responses to policy
challenges that have both a socio-
economic as well as a
scientific/technological dimension”
IPTS- Who we are
Future Learning Spaces in 2020
2005 2006 2007 2008 2009 2010 2011 2012
Learning 2.0 in formal E&T
Learning 2.0 in informal settings
Creativity & Innovation in
schools in EU
eLearning in the EU10 MS
- Digital Competence
- Active Ageing & Learning
Tell-Net Teacher Networking
Review ICT
Impact
IPTS Studies on ICT for Learning
The Future of Learning
ICT for the Assessment of
Key Competences
Digital Competence
Where we stand
Multiple Choice
Short Answer
Generation 1
Automated administration
and/or scoring of
conventional tests
Adaptive Testing
Generation 2 Generation 4
Intelligent Tutors
Automated Feedback
Personalised
feedback and
tutoring
Learning 2.0
Generation 3
Continuous
integrated
assessment
Data Mining & Analysis
Behavioural tracking
Generation Re-invention Transformative strategy: use technology
to change test formats
Peer assessment
Collaborative multimedia learning
environments
Online collaboration Virtual laboratories
Games
Simulations
Virtual Worlds
Replication of complex
real life situations within
a test
Bund
erso
n,
Ino
uye
& O
lsen
(1
98
9).
The
four
gen
erat
ions
of
com
pute
rize
d e
duca
tio
nal
mea
sure
men
t. I
n R
. L
.
Lin
n (
Ed
.),
Ed
uca
tio
na
l m
easu
rem
ent
(Thir
d e
d.,
pp
. 3
67
-40
7).
New
Yo
rk:
Mac
mil
lan
Ben
net
t, R
. E
. (2
01
0).
Tec
hno
logy f
or
Lar
ge-S
cale
Ass
essm
ent.
In P
. P
eter
son,
E. B
aker
& B
. M
cGaw
(E
ds.
),
Inte
rna
tio
na
l E
ncy
clo
ped
ia o
f E
du
cati
on
(3
rd e
d.,
Vo
l. 8
, p
p. 4
8-5
5).
fo
rd:
Els
evie
r.
1990 1995 2000 2005 2010 2015 2020 Efficient testing Personalised learning
Learning Analytics
Multiple Choice
Short Answer
Generation 1
Automated administration
and/or scoring of
conventional tests
Adaptive Testing
Generation 2
Generation 4
Intelligent Tutors
Automated Feedback Personalised
feedback and
tutoring
1990 1995 2000 2005 2010 2015 2020 Efficient testing Personalised learning
Learning 2.0
Generation 3
Continuous
integrated
assessment
Data Mining & Analysis
Behavioural tracking
Generation Re-invention Transformative strategy: use technology
to change test formats
Computer-Based Assessment (CBA) Technology-enhanced learning
Peer assessment
Collaborative multimedia learning
environments
Online collaboration Virtual laboratories
Games
Simulations
Virtual Worlds Replication of complex
real life situations within
a test
Bund
erso
n,
Ino
uye
& O
lsen
(1
98
9).
The
four
gen
erat
ions
of
com
pute
rize
d e
duca
tio
nal
mea
sure
men
t. I
n R
. L
.
Lin
n (
Ed
.),
Ed
uca
tio
na
l m
easu
rem
ent
(Thir
d e
d.,
pp
. 3
67
-40
7).
New
Yo
rk:
Mac
mil
lan
Ben
net
t, R
. E
. (2
01
0).
Tec
hno
logy f
or
Lar
ge-S
cale
Ass
essm
ent.
In P
. P
eter
son,
E. B
aker
& B
. M
cGaw
(E
ds.
),
Inte
rna
tio
na
l E
ncy
clo
ped
ia o
f E
du
cati
on
(3
rd e
d.,
Vo
l. 8
, p
p. 4
8-5
5).
fo
rd:
Els
evie
r.
Learning Analytics
Where we stand
1st Generation CBA: Problem-Solving
http://www.worldclassarena.org/v5/flash/0
9_year_old/backtofront.swf
World Class Arena Sample Test: Problem
Solving (9 years)
Examples
http://www.worldclassarena.org/v5/flash/
13_year_old/pyramids.swf
Problem Solving 13 years
Generation Re-Invention: Problem-Solving
Examples
1st Generation CBA: Situational Judgement
EPSO sample situational judgement test questions; http://europa.eu/epso/discover/prepa_test/sample_test/index_en.htm
Examples
Generation Re-Invention: e-tray exercise
EP
SO
sam
ple
ass
essm
ent
cen
tre
e-tr
ay e
xer
cise
htt
p:/
/eu
rop
a.eu
/ep
so/d
isco
ver
/pre
pa_
test
/sam
ple
_te
st/i
nd
ex_en
.htm
Some Research findings on CBA
• Advantages of CBA: • Standardization of test administration; • Machine-scorable responses; • Opportunities for more interactive question types • Automatic scoring enables teachers to focus on analysis and interpretation
Automatically generated reports supporting formative assessment
• Evidence: • Computer-based tests have (arguably) a positive effect on students' motivation,
concentration and performance; • Evidence on the comparability of computer- and paper-based testing
inconclusive; • Reliability and validity are continuously being improved; • CBA also useful for formative assessment.
• Automated Scoring • For short-answer free text responses automatic scoring is at least as good as
human scoring; • For essays, automatic and human scoring are highly correlated; discrepancies
arise at the top end of scores; • Automatic scoring of highly predictable speech (e.g. one sentence answers)
matches with human scoring.
Future eAssessment Strategies
Multiple Choice
Short Answer
Generation 1
Automated administration
and/or scoring of
conventional tests
Adaptive Testing
Generation 2
Generation 4
Intelligent Tutors
Automated Feedback Personalised
feedback and
tutoring
1990 1995 2000 2005 2010 2015 2020 Efficient testing Personalised learning
Learning 2.0
Generation 3
Continuous
integrated
assessment
Data Mining & Analysis
Behavioural tracking
Generation Re-invention Transformative strategy: use technology
to change test formats
Computer-Based Assessment (CBA) Technology-enhanced learning
Peer assessment
Collaborative multimedia learning
environments
Online collaboration Virtual laboratories
Games
Simulations
Virtual Worlds Replication of complex
real life situations within
a test
Bund
erso
n,
Ino
uye
& O
lsen
(1
98
9).
The
four
gen
erat
ions
of
com
pute
rize
d e
duca
tio
nal
mea
sure
men
t. I
n R
. L
.
Lin
n (
Ed
.),
Ed
uca
tio
na
l m
easu
rem
ent
(Thir
d e
d.,
pp
. 3
67
-40
7).
New
Yo
rk:
Mac
mil
lan
Ben
net
t, R
. E
. (2
01
0).
Tec
hno
logy f
or
Lar
ge-S
cale
Ass
essm
ent.
In P
. P
eter
son,
E. B
aker
& B
. M
cGaw
(E
ds.
),
Inte
rna
tio
na
l E
ncy
clo
ped
ia o
f E
du
cati
on
(3
rd e
d.,
Vo
l. 8
, p
p. 4
8-5
5).
fo
rd:
Els
evie
r.
Learning Analytics
Towards Generation 3: Datamining
Signals http://www.itap.purdue.edu/tlt/
signals/signals_final/index.htm
Similar: GPS at Northern
Arizona University
http://resourceconnect.nau.edu/
grade-performance-status.asp
Examples
Towards Generation 3: Behaviour Tracking
Example: SNAPP
SNAPP is a software tool
that allows users to
visualize the network of
interactions resulting from
discussion forum posts and
replies. Teachers can
rapidly identify patterns of
user behaviour. SNAPP
can extract all user
interactions from various
commercial and open
source learning
management systems, such
as BlackBoard and
Moodle.
Examples
Towards Gen 4: Intelligent Tutoring Systems
GnuTutor (http://gnututor.com/)
OpenSource version of AutoTutor https://www.youtube.com/watch?v=k6HDEEnYuIA
&feature=player_embedded
AutoTutor https://www.youtube.com/watch?v=k6HDEEnYuIA
&feature=player_embedded
Examples
Learning 2.0: Virtual Laboratories
Example from Molecular Workbench http://workbench.concord.org/database/activities/19.html
http://workbench.concord.org/database/
Virtual Frog Dissection Kit http://froggy.lbl.gov/cgi-bin/dissect
Examples
Virtual Laboratories
BioLogica http://biologica.concord.org/
Ingredients:
• enjoyable and engaging
learning activities
• monitoring progress
• electronic tutoring
• portfolio building
Examples
Immersive Games
Quest Alantis Quest Atlantis (QA) is an international learning and teaching project that uses a 3D multi-user environment to immerse children, ages 9-16, in educational tasks. QA combines strategies used in the commercial gaming environment with lessons from educational research on learning and motivation.
Examples
Educational game,
designed to help teach
concepts in citizenship,
geography, and media.
Developed by Serious
Games International, it
has detailed lesson plans
and assignments for
students.
Example: Child labour in Bangladesh:
http://www.youtube.com/watch?v=mf9Ju86IVYw
Global Conflicts http://www.globalconflicts.eu/
Immersive Games
Examples
Conclusions
Generation 1 Generation 2
Generation 4
1990 1995 2000 2005 2010 2015 2020 Efficient testing Personalised learning
Learning 2.0
Generation 3
Generation Re-invention
Computer-Based Assessment (CBA) Technology-enhanced learning
Bund
erso
n,
Ino
uye
& O
lsen
(1
98
9).
The
four
gen
erat
ions
of
com
pute
rize
d e
duca
tio
nal
mea
sure
men
t. I
n R
. L
.
Lin
n (
Ed
.),
Ed
uca
tio
na
l m
easu
rem
ent
(Thir
d e
d.,
pp
. 3
67
-40
7).
New
Yo
rk:
Mac
mil
lan
Ben
net
t, R
. E
. (2
01
0).
Tec
hno
logy f
or
Lar
ge-S
cale
Ass
essm
ent.
In P
. P
eter
son,
E. B
aker
& B
. M
cGaw
(E
ds.
),
Inte
rna
tio
na
l E
ncy
clo
ped
ia o
f E
du
cati
on
(3
rd e
d.,
Vo
l. 8
, p
p. 4
8-5
5).
fo
rd:
Els
evie
r.
- mainstream use for high stakes testing
- further development of innovative test formats
- increased ease of use at school level
- trend towards automated scoring
→ cultural change:
increased acceptance of CBA
- Multimedia collaborative
learning environments support
21st century skills
-Data mining and analysis will
make integrated assessment and
tutoring possible in the future.
We are in a transitional period: Moving from the “Testing
Paradigm” to the (future) “Tutoring Paradigm” in which
explicit testing becomes obsolete
The Paradigm Shift
Technology
1990 1995 2000 2005 2010 2015 2020
Focus on: Efficient testing Focus on: Personalised learning
Era of Computer-Based Assessment (CBA) Era of Technology-enhanced learning
Pedagogy
Efficient
Testing
Validity of
Scores
Quick
Feedback
Authentic
Contexts
CBA
(Gen.1)
Adaptive
Tests
(Gen.2)
Automatic
Scoring
Gen. Re-
Invention
Learning
2.0
Immersive
Learning
Learning
Analytics
Intelligent
Tutors
? ? ? ?
→ Primacy of Pedagogy:
Pedagogical guidelines for intergrated assessment
Transparency & teacher mediation
• Thank you very much for your attention!
• Anusca Ferrari • Christine Redecker
• Yves Punie
• [email protected] • [email protected]
• http://is.jrc.ec.europa.eu/pages/EAP/eLearning.html
further information ……
http://is.jrc.ec.europa.eu
Newsletters: http://is.jrc.es/pages/newsletter.html
Publications: http://is.jrc.ec.europa.eu/pages/Publications.html
Contact: [email protected]