What does the future hold for e-assessment?

Martin RipleyWorld Class Arena Ltd

www.worldclassarena.net

Development and adoption of project

• Cisco, Intel, Microsoft joint contribution to improve education• Joint company taskforce – 9 members• Dr Robert Kozma as consultant

• Aim to improve 21st century skills• Define them clearly• Make them measurable• Connect with the classroom

• Cisco, Intel Microsoft now supporting an international team

Existing models of assessment are typically at odds with the skills, knowledge, attitudes and

characteristics of self-directed and collaborative learning that are increasingly important for

our global economy and fast changing world.

New assessments are needed that measure these skills and provide information that is

needed by students, teachers, parents, administrators, and policymakers to catalyze and

support systemic education reform.

These assessments should engage students in the use of technology and digital resources

and the application of a deep understanding of subject knowledge to solve complex, real

world tasks and create new ideas, content, and knowledge.

Intel, Microsoft, and Cisco Education TaskforceTransforming Education:

Assessing and Teaching the Skills Needed in the 21st CenturyA Call to Action

Assessment in 21st Century

Engagement of countries and other companies

Founder Countries• Australia, Finland, Portugal, Singapore, UK, USA

Other countries can join• Collaborative electronic space

Other companies can fund work• If it fits the project’s program• If the company has relevant expertise• If the company agrees that

all results will be in the public domain, asCisco, Intel and Microsoft have.

White papers developed in 2009 by working groups

• Defining 21st Century Skills• Ms Senta Raizen, WestEd

• Methodological Issues• Dr Mark Wilson, University of California, Berkeley

• Technological Issues• Dr Beno Csapo, University of Szeged, Hungary

• Classrooms and Formative Evaluation. • Dr John Bransford, University of Washington

Dr Marlene Scardamalia, University of Toronto• Policy Frameworks for New Assessments

• Dr Linda Darling-Hammond, Stanford University

Country/ Region Document(s)European Union Key Competencies for Lifelong Learning – A European Reference

Framework, November 2004

Recommendation of the European Parliament and of the Council of 18 December 2006 on key competences for lifelong learning.

Implementation of “Education and Training 2010” Work Programme.

SOURCE: ec.europa.eu/dgs/education_culture/.../ll-learning/keycomp_en.pdf

European Union New Millennium Learners Project: Challenging our Views on ICT and LearningSOURCE:http://www.oecd.org/document/10/0,3343,en_2649_35845581_38358154_1_1_1_1,00.html

USA (Partnership for 21st Century Skills)

P21 Framework Definitions

P21 Framework Flyer

SOURCE:http://www.21stcenturyskills.org/documents/p21_framework_definitions_052909.pdf

Defining 21st Century Skills

Japan Center for Research on Educational Testing (CRET)

Australia Melbourne Declaration on Educational Goals for Young Australians

SOURCE:http://www.curriculum.edu.au/verve/_resources/National_Declaration_on_the_Educational_Goals_for_Young_Australians.pdf

Scotland A Curriculum for Excellence – the Four Capabilities

SOURCE:http://www.ltscotland.org.uk/curriculumforexcellence/index.asp

England The Learning Journey

England Personal Learning & Thinking Skills – The National Curriculum for EnglandSOURCE:http://curriculum.qca.org.uk/uploads/Personal,%20learning%20and%20thinking%20skills%20leaflet_tcm8-12831.pdf

Defining 21st Century Skills

Northern Ireland Assessing the cross curricular skills

SOURCE:http://www.nicurriculum.org.uk/key_stage_3/assessment_and_reporting/cross_curricular_skills.asp

ISTE National Educational Technology Standards for Students, Second Edition, Global Learning in the Digital Age

USA. National Academies, science for the 21st Century

Exploring the Intersection of Science Education and the Development of 21st Century Skills

USA, Department of Labor

Competency Models:

A Review of the Literature and

The Role of the Employment and Training Administration (ETA), Michelle R. Ennis

Defining 21st Century Skills

Defining 21st Century Skills• Conceptual structure

• Ways of thinking• Creativity and innovation• Critical thinking, problem solving• Learning to learn, metacognition

• Ways of working• Communication• Collaboration (teamwork)

• Tools for working• Information literacy• ICT literacy

• Living in the world• Citizenship – local and global• Life and career• Personal, social responsibility

A framework for 21st century skills

Assessments signal priorities for curriculum and instruction Teachers model the pedagogical approachCurriculum developers respond

Schools and teachers tend to focus on what is tested rather than underlying standards or learning goals

May encourage a one-time performance orientation and transmission-type teaching

Instructional/teaching time is diverted to specific test preparation activities

WCAL research

Collaborative problem solving

Collaborative Problem Solving

Social Skills

Participation Perspective taking

Social regulation

Cognitive Skills

Task regulation

Knowledge building

Participation skills

Low• Peripheral participation• Low subjective responsibility for outcomes of collaboration, leading to lurking behaviour• Simple epistemological beliefs (knowledge is perceived as fixed and to be transmitted from

teacher/textbook to learner)

Middle• Activity in scaffolded environments • Responding to cues in communication• Medium subjective responsibility for outcomes of collaboration• Developed epistemological beliefs (knowledge is perceived as fixed, but can be elaborated

through communication and collaboration)

High• Initiating and promoting interaction• Activating and scaffolding others in participation• Ensuring equal participation rates among group members• High subjective responsibility for outcomes of collaboration• Sophisticated epistemological beliefs (knowledge is perceived as fluid, constructed, and

inherently social/collaborative in nature)

Perspective taking skills

Low• Low levels of empathy• High egocentric bias• Social projection (expectation of others as highly similar to oneself)• Ignoring contributions from others• Contributions are not tailored to participants

Middle• Medium levels of empathy• Medium level of egocentric bias• Receptive ability (being able to understand what others want to convey, e.g. from

overhearing)• Contributions from others are taken into account• Contributions are moderately tailored to recipients

High• High levels of empathy• Low or no egocentric bias• Contributions from others are embraced and contextualized with respect to

collaborators’ opinions and skills• Eliciting contributions from others (e.g. through questions)• Contributions are tailored to recipients (audience design)

Task regulation skills

Low• Trial and error hypothesis testing• Unorganized sequence of solution attempts• Little or no goal setting• Variety of taskwork mental models will be ignored

Middle• Forward search through a problem space• Organized sequence of solution attempts• Setting of unspecific goals• Variety of taskwork mental models will be taken into account

High• Reflective regulation• Forward and backward search through a problem space• Strategic oversight over collaborative strategy• Setting of specific goals• Variety of taskwork mental models will be harnessed productively

Knowledge building skills

Low• Knowledge telling• Sharing of information• Isolated contributions• Lack of argumentation patterns

Middle• Critical analysis of information• Building on input from others• Adding information/data• Forming of incomplete arguments

High• Knowledge transforming• Integration and synthesis of multiple artefacts • Forming of complete, proper arguments (explanatory coherence)

Social regulation skills

Low• Low tolerance for ambiguity• Competitive or individualistic social value orientation• Low readiness to negotiate joint understanding• Tendency to withdraw after conflict arises

Middle• Cooperative social value orientation• Attempts to negotiate joint understanding• Conflicts will be avoided• Initiation of compromises

High• Pro-social attitudes• Strategies for conflict resolution• Conflicts are regarded as productive tensions• Initiation of successful compromises

WCAL research

Collaborative problem solving

A simple task (i.e. one we don’t want)

WCAL research

More complex problem solving model

WCAL research

Collaborative problem solvingA complex task (parallel rather than serial processes)

Sample task

WCAL approach

In teams

• Explore environment• Try out ideas• Record interactions• Make sure all team members understand

In teams

• Try out new understanding in slightly different context• Record interactions• Make sure all team members understand

Individually

• Working individually (without help) solve a similar problem which draws on understanding developed in parts 1 and 2

• To be successful as a team all members must complete this task

Sample task

Sample task