www.gu.se

SQA SeminarGlasgow, 8 April 2009

Sweden’s experiences in curriculum and assessment development

Gudrun Erickson

University of Gothenburg, SwedenDepartment of Education

www.gu.se

Outline

• The Swedish school system

• Curriculum: current system – discussions – (proposed) changes

• Assessment: current system – discussions – (proposed) changes

• Collaborative approaches to national assessment

• National and international experiences

• Concluding remarks – Common concerns

• DISCUSSION

www.gu.se

The Swedish school systemhttp://www.skolverket.se/sb/d/190

• Highly decentralized; main responsibilities at local level

• National level (e.g.): Education Act; Curricula; Syllabi for subjects, including grading criteria;

National tests

• A goal and criterion referenced system

• Preschool; 9-year compulsory school; upper secondary education; adultadult education; universities… (http://skolnet.skolverket.se/polopoly/utbsys-eng/)

• > 98% of students continue to three-year upper secondary education

• Development dialogues; individual development plans; written reports; formalgrading (3/4 point scale), from school year 8

• Teachers responsible for grading

• Extensive system of advisory national testing and assessment

www.gu.se

The Swedish National Agency for Education

Director-generalDeputy director general

National development

National evaluation

Analysis and research

Finance

Human resources

ICT

Quality development.

Advisory council

Education Administration

School improvement

Government funding

Education statistics

Result analysis

Adult education

Upper secondary school

Preschool and Compulsory school

Assessment and testing

Director-general´s office

Legal secretariatInformation service

Information

Internal audit

www.gu.se

Syllabi and local planning

Nationallevel

Locallevel

Whythis subject?

Whatdirection?

How?(Very limited

guidance)

When?(Limited

guidance)

Why this to reachgoals?

What (exact)subject matter?

How will we work?

When to teach what?

www.gu.se

Balancing act

Equity / EqualityLocal independenceFlexibility Quality assurance

www.gu.se

Discussions & (Proposed) Changes• Increasing clarity of curricula and syllabi, including indications of

essential content areas

• Increased quality control

• More grade levels (from 3/4 to 6 + “no basis for grading”)

• Earlier start of formal grading (from school year 8 down to 6)

• Earlier national tests; national tests in a wider range of subjects

• Individual written documentation from school year one

Ongoing revision of national curricula and syllabi

www.gu.se

Aims of the current national testing/assessment system

• Enhance individual educational achievement;

• Clarify curricular subject goals and indicate clearly strengths and weaknesses in individual learner profiles;

• Concretize curricular subject goals and grading criteria;

• Enhance equity and fairness in assessment and grading;

• Provide statistics for local and national analyses of educational achievement

Advisory function

www.gu.se

The national assessment systemshould not:

• Determine the precise choice of content and methodsTo be done in collaboration between teachers and students

• Function as examinationsTeachers to award grades based on the assessment of students’ accumulated work;Consequently, the national tests have an advisory function; however, to what extent is not defined

www.gu.se

National assessment materials• Formative materials / Diagnostic materials,

including models for self and peer assessmentA widening range of subjects

• National tests: Summative subject tests (mandatory)Extensive materials, including guidelines

An increasing number of ‘core’ subjects (Sw, Maths, Eng, Science…)

• Electronic assessment bank (formative & summative materials) A widening range of subjects

The Swedish National Agency for Education commissions different university departments in the country to take responsibility for test

development (and research)

www.gu.se

Reactions to national tests• Students usually positive to varied, authentic, “different” tasks/tests

• > 95 % of teachers positive to the national testsfunction, content, level of difficulty, guidelines, individual proficiency profiles, layout…

• [Teacher] discussions, e.g., about workload; weight/importance of test results; adaptation/accommodation; standards…

• [Political] discussions, e.g., about number of tests; aim(s) of national tests; role of national test; stability over time; reliability…

www.gu.se

Test grade: national subject tests (2008)year 9 – upper secondary education

% “Not pass”/ Pass Pass with Pass withFail distinction special

distinction

Swedish9 3% 41% 45% 11%B 11% 46% 34% 9%

Mathematics9 17% 48% 26% 10%A 23% 45% 23% 9%

English9 4% 35% 43% 18%A 6% 39% 43% 11%

www.gu.se

Final Grade & national Test Grade– end of compulsory school (year 9) – 2008

FG<TG FG=TG FG>TG

Swedish 6 % 79 % 15 %

Mathematics 2 % 74 % 24 %

English 7 % 84 % 9 %

www.gu.se

Can teachers’ ratings be trusted?• Joint study (2008) by the Swedish NAE and four university departments

responsible for test development; to be published shortly on the NAE website

• Random sampling: 100 teacher-rated grade 9 tests of Swedish, Mathematics and English (oral subtests not included, since recording is not a requirement)

• Re-rating by three independent raters; 10-point scale

• To some extent, varying results between subjects (higher correlations for Mathematics and English than for Swedish)

• Example: EnglishListening & Reading comprehension (50/50 constructed and selected response): r = > .99; Written production (“essay”): r = .86-.93; Generalizability coefficient: .85 [Analyses of oral test results in similar studies show comparable results]

www.gu.se

Collaborative approaches to national assessment

• Political level (NA) – Universities

• University level (national and international)

• Test development (researchers from different disciplines, teacher trainers, teachers, students…)

• Local test administration

• Rating by teachers

• Reporting by teachers (results and reactions)

• Data collection – Analyses – Research

• Public reporting / Dissemination

www.gu.se

A collaborative test development process (FL)

• Analyses of literature, research, curricula, examples of tests• Development of items and tasks based on common, explicit

principles and specifications• Piloting with a limited number of students > Adjustment of tasks• Large scale pre-testing in randomly selected classes in the country

(n ≈ 400 students/task); systematic collection of feedback from all participating teachers and students

• Analyses of test results and feedback (a "q+q” approach)• Selection/Sequencing of tasks; Standard setting and Benchmarking• Nation-wide test administration• Collection of data / Analyses / Research > Reporting (publicly

available on the web)

www.gu.se

Contributors and contributions • Teachers and Teacher trainers

Development groups; Task/Item construction; Mini trials – Pre-testing (administrating, observing, analysing, discussing, reporting): Selection, composition, sequencing, standard setting; Rating and benchmarking; Reporting and responding after administration of tests…

• Students of ‘all’ agesProviding information and sharing their views in interviews and (regular) questionnaires in connection with pre-testing; comments on, e.g, relevance; perceived level of difficulty; content; vocabulary; clarity of instructions; time, speed of speech; Retrospective, task related self-assessment…

• ResearchersAspects of language; Gender; Quantitative properties of item and test data; Teachers’ handling of constructed response; Progression in tasks and tests; Dimensionality; Rater introspection; Test-taker and teacher feedback…

www.gu.se

Current discussions• Number/Type of aims for national assessment system?

• Clarifying/Strengthening the role of national tests?

• Formative and Diagnostic materials – where, how and from/by whom?

• Effects and implementation of computer assisted testing and assessment?Aspects of validity? Linear – Sequential – CAT..?

• Increasing standardization - if yes, concerning what, and how?

• Clearer emphasis on aspects of reliability (equity)?

• How to avoid narrowing the curriculum - negative impact/washback?

• Emphasizing common basic principles for all types of assessment !

www.gu.se

Good [language] testing and assessmentaccording to approximately 1 400 teenagers in 10 European countries

Breadth and VariationFocus on ”usefulness” / communication

Learning potentialClarity

FairnessEnough timeChallenge

(Erickson & Gustafsson, 2005)www.ealta.eu.org/resources.htm

www.gu.se

EALTAwww.ealta.eu.org

European Association for Language Testing and Assessment

• Broad membershipTeachers, Teacher trainers, Developers of large scale testing systems

• Non-commercial• Individual membership free

• Founded with support from the EU

• Resources page; Discussion list for membersAnnual conferences (June 2009: Turku, Finland)

797 individual members (41 European countries); 81 associate members(27 countries, 5 continents); 51 institutional members; 16 expert members

www.gu.se

EALTA Guidelines for Good Practice in Language Testing and Assessment

Adopted in 2006; Currently translated into 34 languagesTranslations freely available on the web

Address the activities of the three membership categoriesvia questions

Initial emphasis on basic principles for all types of assessment:

TRANSPARENCYRespect for students/examinees, responsibility, fairness, reliability, validity,

collaboration among the parties involved

www.gu.se

Concluding remarks – Common concerns• Creating a reasonable balance between local and central initiative,

interpretation and responsibility

• Maintaining high standards of validity as well as reliability

• Aiming for positive impact on learning and teaching

• Creating a reasonable balance, and maintaining a close cooperation, between R & D – research and development / researchers and test

developers

• Further developing and elaborating methods of collaboration with wide groups of stakeholders

• Bridging the gap between formative and summative assessment – emphasizing common, basic principles