Applying CEFR Applying CEFR principles principles

to testing and to testing and assessment in practiceassessment in practice

ACERT ConferenceACERT Conference1515thth March March 2014, Prague2014, Prague

Mila AngelovaMila Angelova

Aims of the sessionAims of the session

• To give you a short update on To give you a short update on EAQUALS operation and activitiesEAQUALS operation and activities

• To introduce you to the EAQUALS To introduce you to the EAQUALS Certification SchemeCertification Scheme

• To (re)-introduce you to the most To (re)-introduce you to the most salient features of the six global salient features of the six global levels according to CEFRlevels according to CEFR

• To assess two oral performances for To assess two oral performances for standardisation purposesstandardisation purposes

The AssociationThe Association

• Founded in 1991Founded in 1991• A company registered in UK as A company registered in UK as

charity charity • International NGO with participatory International NGO with participatory

status at the Council of Europe status at the Council of Europe (2006)(2006)

EAQUALS MissionEAQUALS Mission

The mission of EAQUALS isThe mission of EAQUALS is:: • to contribute to the development of to contribute to the development of

excellenceexcellence in in language education language education• to be the internationally recognised to be the internationally recognised

authority for quality standards and authority for quality standards and quality assurance in language quality assurance in language education and training education and training

EAQUALS ValuesEAQUALS Values

• PlurilingualismPlurilingualism• Intercultural understandingIntercultural understanding• International cooperationInternational cooperation• Lifelong learningLifelong learning

EAQUALS EAQUALS MembersMembersA network connecting 35 countries

Accredited members x 11225 languages, private and state sector

Associate members x 27British Council; Instituto Cervantes; CIEP France; Goethe-Institut; Cambridge Assessment; Trinity London; g.a.s.t. Germany; 12 national quality associations

Project partners x 5Centro di Ricerca e di Servizio + Universita per Stranieri di Perugia School of Languages, Sabanci, Turkey

Individual membership Individual membership (2011)(2011)

EAQUALS EEAQUALS Expertisexpertise• Quality management: - design of quality accreditation schemes

- training and standardisation - capacity building at local level

• The Common European Framework of Reference for Languages (CEFR):

- descriptors, curriculum, assessment, - portfolio, certification

• Teacher management and professional development:

- teacher competences (profiling grid) - teacher training and development

- academic management

International accreditation of language International accreditation of language educationeducation- standards, inspections; recommendations- standards, inspections; recommendations

Support and pathways for institutional Support and pathways for institutional developmentdevelopment- consultancy; self-assessment; training resources- consultancy; self-assessment; training resources

An international professional networkAn international professional network- regular conferences; involvement in funded projects- regular conferences; involvement in funded projects

Certificates of achievement – CEFR levelsCertificates of achievement – CEFR levels- training support, numbered certificates, website - training support, numbered certificates, website archivearchive

EAQUALS EAQUALS ActivitiesActivities

EAQUALS EAQUALS ProjectProjectss

• QQuality managementuality management - ISO- ISO: : standardstandard for providers of language learning for providers of language learning - - Council of Europe: code of practice for integration Council of Europe: code of practice for integration

of adult migrantsof adult migrants - Leonardo: Quality evaluation of on-line teacher - Leonardo: Quality evaluation of on-line teacher

training training, Quality Guidelines , Quality Guidelines

• CEFRCEFR - Core Inventories (English; French)- Core Inventories (English; French)

- Listening and Reading - Listening and Reading scenarios (Eng, French)scenarios (Eng, French) - - EAQUALS Certificate of achievementEAQUALS Certificate of achievement

• Teacher management & developmentTeacher management & development - - profiling grid for teachers (EPG Project)profiling grid for teachers (EPG Project) - - teacher development framework teacher development framework

EAQUALS AccreditationEAQUALS Accreditation

• iinternationalnternational and and plurilingualplurilingual• across educational sectorsacross educational sectors• cconsultativeonsultative and developmental and developmental• based on based on EAQUALS EAQUALS CharteCharterrs

EAQUALS EAQUALS SStandardstandards

• designed for language education designed for language education servicesservices

• 12 main areas of assessment12 main areas of assessment• specific indicators of quality for each specific indicators of quality for each

areaarea• 50 key assessment criteria: 50 key assessment criteria:

- clear and transparentclear and transparent- derived from the Chartersderived from the Charters

Becoming a MemberBecoming a Member

Attend an EAQUALS ConferenceAttend an EAQUALS Conferenceand get to know EAQUALS: details on www.eaquals.organd get to know EAQUALS: details on www.eaquals.org

Complete a Self-assessmentComplete a Self-assessmentand identify areas that may need improvementand identify areas that may need improvement

Have a One-day Advisory VisitHave a One-day Advisory Visitand receive consultancy on achieving EAQUALS and receive consultancy on achieving EAQUALS

standardstandard

Arrange your First InspectionArrange your First Inspectionafter working through the points raisedafter working through the points raised

The EAQUALS The EAQUALS Certificate of AchievementCertificate of Achievement

A Guarantee for Quality A Guarantee for Quality AssessmentAssessment

Brief introduction to the schemeBrief introduction to the scheme

MMain aims of the ain aims of the schemescheme

• To increase EAQUALS’ profile: the first To increase EAQUALS’ profile: the first organisation to award CEFR certificates of organisation to award CEFR certificates of achievement across different languages achievement across different languages Europe-wideEurope-wide

• To help members differentiate themselves To help members differentiate themselves from their competitors and demonstrate from their competitors and demonstrate their competencetheir competence

• To open the possibility, at a second stage, To open the possibility, at a second stage, of accrediting CEFR certificates awarded of accrediting CEFR certificates awarded by schools that are not yet accredited by schools that are not yet accredited members of EAQUALSmembers of EAQUALS

Core Core messagemessage

• A guarantee for quality assessment in the accredited institutions by thoroughly screening all assessment processes

• CEFR-implemented approach to syllabus and curriculum design

• Feasibility of standardisation and moderation techniques used by the institution

EAQUALS EAQUALS Certificate oCertificate of f AchievementAchievement •Phase 1: voluntary – Accredited voluntary – Accredited Members only: accreditation carried Members only: accreditation carried out as a “top up” to the EAQUALS out as a “top up” to the EAQUALS inspectioninspection

• Phase 2:Phase 2: compulsory for Accredited compulsory for Accredited Members; voluntary for schools which Members; voluntary for schools which are members of Associate Members: are members of Associate Members: the accreditation process is fully the accreditation process is fully integrated into the EAQUALS inspection integrated into the EAQUALS inspection and carried out as a top up to and carried out as a top up to inspections of Associate Membersinspections of Associate Members

Why EAQUALS Why EAQUALS Certificate?Certificate?

Benefits for:Benefits for:• the institutionthe institution• the course participantsthe course participants• the stake holdersthe stake holders• EAQUALS profileEAQUALS profile

MMain prerequisites to ain prerequisites to apply for the Certification apply for the Certification SchemeScheme

A) CEFR-based curriculumA) CEFR-based curriculum

B) Standardisation training B) Standardisation training

C) Moderation techniquesC) Moderation techniques

CEFR-based CEFR-based curriculumcurriculum

1.1. CEFR-related level descriptions and CEFR-related level descriptions and syllabus documentssyllabus documentso the levels referenced to CEFR (CEFR criteria

grids adopted or adapted to local circumstances)

o learning aims specified by „can-do” statements - based on ELP or CEFR scales and adapted to the institution’s own needs

2. 2. WWritten guidelines for teachers on ritten guidelines for teachers on levels, teaching and assessmentlevels, teaching and assessment

3. Application of CEFR in teaching3. Application of CEFR in teaching o coherence between curriculum and syllabus coherence between curriculum and syllabus

aims and classroom practiceaims and classroom practice

Implementing CEFR-based Implementing CEFR-based curriculum and syllabusescurriculum and syllabuses• Core of the syllabuses is the practice-Core of the syllabuses is the practice-

oriented background of the CEFRoriented background of the CEFR

• Can-do-statements developed for Can-do-statements developed for each type of course and level / sub-each type of course and level / sub-levellevel

• Can-do-statements serve as a basis Can-do-statements serve as a basis for tests developmentfor tests development

• Can-do-statements used as a self-Can-do-statements used as a self-assessment instrumentassessment instrument

Standardisation Standardisation trainingtraining - - requirementsrequirements• Familiarisation with CEFR levels Familiarisation with CEFR levels

and descriptorsand descriptors• Systematic standarisation Systematic standarisation

training with CEFR illustrative training with CEFR illustrative samplessamples (DVDs or scripts: the EAQUALS Standardisation packs – for English, French and German)

• Procedures for CEFR-novice and Procedures for CEFR-novice and CEFR-experienced teachersCEFR-experienced teachers

Examples Examples of standardisation trainingof standardisation training

• Means of writing standardisation Means of writing standardisation (making use of local samples)(making use of local samples)

• Means of speaking standardisation Means of speaking standardisation (with official samples)(with official samples)

• Standardisation of the placement Standardisation of the placement testing (with standardisation videos testing (with standardisation videos and other CEFR-related tasks)and other CEFR-related tasks)

ModerationModeration

Systematic reduction of Systematic reduction of subjectivity in the assessmentsubjectivity in the assessment.

Moderation Moderation techniquestechniques - - requirementsrequirements• Collective assessment techniquesCollective assessment techniques

o grading by two assessors (all performances, a structured sample or with less-experienced staff), oro assessing initially as a panel

• Quality controlQuality controlo liaison between teacherso systematic monitoring by academic managers

Examples of moderation Examples of moderation techniquestechniques

• Doing buzz observations during Doing buzz observations during testing periods to monitor application testing periods to monitor application of assessment standardsof assessment standards

• Monitoring inexperienced teachers’ Monitoring inexperienced teachers’ assessment of spoken production assessment of spoken production and interaction / written workand interaction / written work

Assessment procedures Assessment procedures should include at least two should include at least two ofof::

a) teacher judgements through continuous teacher judgements through continuous assessment in relation to CEFR/ELP assessment in relation to CEFR/ELP descriptorsdescriptors

b) b) performances in specific spoken/writen performances in specific spoken/writen ttaasks assessed with CEFR-related criteriasks assessed with CEFR-related criteria

c) c) use use of tests (progress test giving an of tests (progress test giving an indication of good performance - indication of good performance - e.g.15/20), or an independent, validated e.g.15/20), or an independent, validated “CEFR level“CEFR level test” reporting a CEFR leveltest” reporting a CEFR level

Procedure to accredit Procedure to accredit assessment processesassessment processes

Step 1: Introduction of a CEFR-based assessment approach

»Standardisation training»Assessment procedures»Moderation techniques

Step 2: A formal applicationStep 3: Evaluation by the

EAQUALS Assessment Panel

IntroductionIntroduction

Please, discuss the following questions Please, discuss the following questions with the person sitting next to you:with the person sitting next to you:

o How often do you do standardisation How often do you do standardisation training at your institutions? Is it enough?training at your institutions? Is it enough?

o Do all teachers have to take part in it? If Do all teachers have to take part in it? If not, why?not, why?

o What problems do you face when you do What problems do you face when you do standardisation training?standardisation training?

WarmerWarmer

• Look at the descriptors for spoken Look at the descriptors for spoken production for B1 level. Fill in the production for B1 level. Fill in the gaps with the missing salient gaps with the missing salient features for the level.features for the level.

• Compare your ideas with the Compare your ideas with the person sitting next to you.person sitting next to you.

CEFR Speaking Skills CEFR Speaking Skills overviewoverview

• Arrange the descriptors for the six Arrange the descriptors for the six basic levels in the correct order.basic levels in the correct order.

• Compare with your answers from Compare with your answers from the previous stage with the actual the previous stage with the actual descriptor for B1descriptor for B1

Oral sampleOral sample

• Watch a short performance for 3-4 min. Watch a short performance for 3-4 min. Decide on the speakers’ level.Decide on the speakers’ level.

Bell Krakow_Anna & Magda.mpeg• Write a short description of the speaking Write a short description of the speaking

abilities of the students in the video.abilities of the students in the video.• Compare your answers with the CEFR Compare your answers with the CEFR

descriptor.descriptor.• Provide a rationale for your decision by Provide a rationale for your decision by

referring to the most salient features of referring to the most salient features of the descriptors.the descriptors.

Criteria for speaking Criteria for speaking assessmentassessment

• List the basic criteria that you use List the basic criteria that you use for speaking assessment.for speaking assessment.

• Compare your ideas with a Compare your ideas with a suggested set of criteria, used at suggested set of criteria, used at one EAQUALS institution.one EAQUALS institution.

Sample mark sheet Criteria Candidate

A Candidate B

Discourse management Can manage very short, isolated, mainly pre-packaged utterances, with much pausing to search for expressions, to articulate less familiar words, and to repair communication. Can link words or groups of words with very basic linear connectors like “and” or “then”

1 2 3 4 5 1 2 3 4 5

Pronunciation: Can produce a limited number of words / phrases in a relatively recognizable manner and make her/himself understood in simple everyday situations but there is very noticeable interference from mother tongue in terms of individual sounds, word and sentence stress

1 2 3 4 5 1 2 3 4 5

Interaction: Can ask and answer questions about personal details.Can interact in a simple way but communication is totally dependent on repetition, rephrasing and repair

1 2 3 4 5 1 2 3 4 5

Grammar and vocabulary: Has a very basic repertoire of words and simple phrases related to personal details and particular concrete situations. Shows only limited control of a few simple grammatical structures and sentence patterns in a memorized repertoire.

1 2 3 4 5 1 2 3 4 5

Global achievement: Overall impression mark 1 2 3 4 5 1 2 3 4 5

Initial Initial standardisationstandardisation

Task 1:Task 1: • Watch the standardisation video Watch the standardisation video

again and assess the speakers’ again and assess the speakers’ performance using the mark sheet performance using the mark sheet

• Discuss your grades with a partner, Discuss your grades with a partner, then compare ideas with another pairthen compare ideas with another pair

Thank you!

www.eaquals.org