inspired 2014- using assessments for systemic reform- ver 1.1

8/10/2019 InspirED 2014- Using Assessments for Systemic Reform- Ver 1.1

1/61

Using Assessments for Systemic Reform

InspirED 2014

Jan 11, 2014

Vishnu Agnihotri

Educational Initiatives

For any queries, contact

[email protected]


2/61


1. What are we trying to reform?

2. Making the problem visible and providing specific insights

3. Experiences in using assessments for systemic reform

4. Beyond student assessments of learning outcomes

5. Some technical challenges in assessments

6. Assessments for systemic reform- Points to keep in mind


3/61

3

Education for the Poor: Very Low Learning

Levels

Class 5

Less than 50% of class 5

children could write a sentenceon the picture Correctly answered by only

40% students

Class 4

Subtract 26 from 51

Sample of about 10,000 students per class

Source: EI Student Learning Study 2008


4/61


5/61

5

Private schools: Learning happens, but by

rote

Calculate the LCM of 12, 16 and 20.

Define density

What IS LCM and where is it used?

If some ice is dropped into a glass of water,

what will it look like?

Sample of about 8,000 students per class

Source: EI SLIMS 2006


6/61

A passage on

brushingones teethand whatworks..

How didIndian statesdo?

Comparison with International Levels


7/61PISA 2009+

TN 63%

HP 53%PISA Avg 85%

Comparison with International Levels


8/61


9/61


10/61

Why do so

many studentsstruggle to

measure a

length orcompare

angles?


11/61

Teaching Vs Learning


12/61









13/61

Which of the following are examples of respiration?

1. Humans use oxygen and release carbon dioxide.

2. Plants use carbon dioxide and release oxygen.

3. Burning dry leaves uses oxygen and releases carbon dioxide.

A. only 1

B. only 2C. only 1 and 2

D. 1, 2 and 3

Virtually no

change in

understandingfrom 9.5 to 14.8

years!

Option A

Option D

Performance

in entire paper


14/61

What students at different ability levels can do?- Class 4


15/61

What students at different ability levels can do?...


16/61

Comparative performance across regions1


17/61

Comparative performance across regions2


18/61

State students typically score

3 to 8 percentage pointslower than Indian Private

Schools

However in some questions

involving familiarity in real

life, they score higher

Comparison across School Systems

Sample Question: This question tests knowledge of the living

environment. 83% of the tested students could answer this

question correctly compared to 48% of ASSET students.

Social ScienceClass 7 EVSClass 5

Sample Question: This question tests the students

understanding and interpretation of geographical maps. 40% of

the tested students could answer this question correctlycompared to 67% of ASSET students.

The shaded dark grey part on this map shows the continent of

______________.

A. Asia

B. Australia

C. Antarctica

D. South America

Here is a picture of grains of wheat. Manan buys some wheat

grains to make into bread,chapatisand biscuits.

To make these, the wheat grains should first be____________.

A. Put into the soil

B. Mixed with water

C. Cooked over fire

D. Made into powder


19/61


20/61

Detailed, Question-wise Analysis Available for all

Teachers and Researchers

http://www.educationalinitiatives.com/GujaratDiagnostic/GD_StudyGraph_Display.php
http://www.educationalinitiatives.com/GujaratDiagnostic/GD_StudyGraph_Display.phphttp://www.educationalinitiatives.com/GujaratDiagnostic/GD_StudyGraph_Display.php


21/61

Overall Summary Of

School Performance

Class Performance Table

Students Score Table

Recommendations

Question Wise Comparison

Of Your School Versus All

Schools

Skill Performance

Report & Questions

Found Difficult/Easy


22/61









23/61


24/61


25/61

Salient Features of Gunotsav

%

Largest school assessment program -

53.5 lakh students in 34,000 Government

primary schools, 715 Ashramshalas and

593 granted schools across the state

every year since 2009 on the following

holistic parameters:

Learning Outcomes for each Class

Co-curricular Activities

Community Participation

Availability and Use of Infrastructure

Self evaluation of all schools by

teachers,further 25%(9000 Schools) also

externally assessed by officers(Ministers, IAS,IPS,IFS officers, ClassI &

II officers) every year since 2009

www.gunotsav.org Education Dept., Government of Gujarat
http://www.gunotsav.org/http://www.gunotsav.org/


26/61


27/61


28/61

Transparency of the Initiative

www.gunotsav.org Education Dept., Government of Gujarat

Association of three well known organizations in the

field of education

Assessment of post Gunotsav Remedial class

programme by UNICEF

Evaluation of assessment processes during theofficer assessment-2011 by Pratham-ASER

Continuous support and analysis at all stages by

Educational Init iat ives
http://www.gunotsav.org/http://www.gunotsav.org/


29/61


30/61


31/61

Making Learning Levels Visible through aStudent Progress Tracking System


32/61


33/61

SPTS allows action and remediation,

school level assessments, progress

monitoring, talent identification etc.

Over time, the system can enabledetection of educational learning

patterns that show the effectiveness of

specific textbooks, educational

methods or policies by providing

longitudinal data for comparison.

Student Progress Tracking System (SPTS)

The SPTS allows a school to

identify its strength and

weakness areas vis-a-vis allother schools. This allows

specific resources to be

focused for those who need

them most.


34/61

Three year ComprehensiveAssessmentdriven School

Excellence Reform

All students of Classes 3 and 6 Multiple Mediums of Instruction

System Level ReportIndividual School and Ward Level

Reports Dissemination Workshops

Municipal Corporationof Greater Mumbai

MCGM


35/61


36/61


37/61









38/61

QES aims to find answers

to what constitutesQuality education

A multi-year study to expand the meaning ofquality in education to include educationaloutcomesbeyond student performance insubjects and study the attributes of qualitylearning environments.

The study relates quality to a reflection of

outcomes in larger sense:scholastic, co-scholastic and affective (specifically values andattitude)

Quality Education Study

Assessing Attitudes


39/61

Assessing Attitudes

People throw garbage out in the street, from the

bus or train, in the river, in empty buildings, in thetheatre, in parks and canals.

When do you think people could throw outgarbage in the street, from the bus or in the

canals?A. when there are no garbage binsB. when others do the sameC. never

Class 4, 8 Civic Responsibility

Source: QES

Beliefs about discipline vs Learning outcomes


40/61

Source: Quality Education Study 2011

Beliefs about discipline vs Learning outcomes

Beliefs about discipline vs Classroom Climate


41/61

Beliefs about discipline vs Classroom Climate

Source: QES

C t ki b d t h b li f


42/61

Country ranking based on teacher beliefs

Source: TALIS (OECD) + Quality Education Study 2011

How do these beliefs link with learning outcomes?


43/61

The GRIT Scale


44/61

The GRIT Scale

Only 8 items!

Consistency of interests &

Persistence through failure

Source: Angela Duckworth

Students assessing teachers


45/61

Students assessing teachers

Source: Roland Ferguson, Harvard Economist

5 items most strongly associated with student learning- classroom control &

challenge

1. Students in this class treat the teacher with respect.

2. My classmates behave the way my teacher wants them to.

3. Our class stays busy and doesnt waste time.

4. In this class, we learn a lot almost every day.

5. In this class, we learn to correct our mistakes.

Students rating of teachers!

i f i f


46/61







6. Using assessments for systemic reform- important points


47/61

Sample Selection (sample or census?)

Control Group Selection

Test Development (item types; testing formats)

Test Administration (complexity, centralisation, fairness)

Points related to transparency (instrument, item-wise data)

Points related to standardisation (procedures)

Points in analysis and reporting (IRT, explaining unexpected

findings)

Some Technical Challenges in Assessments

U i A f S i R f


48/61








E l ti & A t F k K P i t


49/61

Evaluation & Assessment Framework- Key Points

Governance: Striking the right balance between central efforts and

local initiatives and between accountability and development State vs District initiatives

Procedures: Designing the right instruments to ensure evaluation

and assessment contribute to improvement of teaching and learning Training on what to assess, building diagnostic items

Capacity: Developing competencies within the state for evaluationand for using feedback at all levels of the education system Balance external assessments with teacher-led assessments

Balance and align Summative and Formative assessments (CCE)

Use of results: Organizing evaluative information for effective use;avoiding misuse of E&A results Differentiating between use for accountability and use for development

Reports designed to meet objectives

Based on OECD Paper


50/61

When Large Scale Assessments Fail to Create Impact

50

Regarded as a stand-aloneactivity with little

connection to the Ministry

Inadequate involvement ofstakeholders in design andimplementation of an

assessment

Failure to communicatefindings to all in a position

to act

Lack of confidence in thefindings of a national

assessment

Political sensitivity tomaking findings public

Failure to devise

appropriate actionfollowing an assessment atthe level of general policies

or at the school level

Source: CSF presentation based on Using The Results Of A National

Assessment Of Educational Achievement, Greaney, Kellaghan & Murray, 2009

Integrate assessment activity into

existing structures, policy, and

decision-making processes

Involve all relevant stakeholders in

design and implementation of an

assessment

Increase the likelihood of making

findings public by holding regular

stakeholder discussions Make provision in the budget for

dissemination, plan activities, and

prepare a number of reports

tailored to user needs

Integrate national assessment

activity into policy and managerial

activities, and review findings todetermine implications and

strategies

Actions


51/61

Thanks!

For any queries, contact

[email protected]


52/61

Educational Initiatives 52

7. Higher classes / Board Exams - matter for EarlyLearning

Source: Examples are from Gujarat Board, CBSE and IGCSE Class 10 papers of 2009

From a State Board Paper

From the CBSE Paper

From the IGCSE Paper


53/61

The Plan of my Talk


54/61

Need for a New Science of Learning

Actions

Large Scale Assessments

Student Progress TrackingSystems

Student Interviews

Diagnostic Sheets

Personalised LearningSolution

Strategies

Make the Problem Visible Provide Insights / Diagnosis

Create New Tools to Solve

the Problem

Goal: Ensure Quality Learning for All Children

Government Schools Private Schools

The Plan of my Talk

School tests/ Board exams make us blind to


55/61


the consequences



56/61


the consequences

Evaporation is the process of a liquid (like water) changing into a gaseous state. Which of the

following observations can be explained by the process of evaporation?

1. Water spilled on the floor dries up after some time.2. Sweat from our skin disappears when we sit under a fan.

3. Water from the ocean changes into gas and forms clouds.

Option Option Percent Correct

A only 1 15.1%

B only 3 31.8%

C both 1 and 3 23.4%

D 1, 2 and 3 27.6%

Describe Evaporation

Vs 32% of nearly 10,000students believe onlythis is evaporation

Source: ASSET Class 6

Board Exams in some progressive countries.


57/61

The more

progressive

countries are

moving towardsgiving students

greater choice

be it in terms of

number of

subjects, levels,or even attempts

allowed at the

exams

Source: Board ExamReform Study-

Gujarat



58/61

The more

progressive

countries are

moving towardsgiving students

greater choice

be it in terms of

number of

subjects, levels,or even attempts

allowed at the

exams

Source: Board ExamReform Study-

Gujarat

Different levels of depth/difficulty for asubject. For example,

IB a Higher level paper and a Standard levelpaper,

Finland offers advanced courses in certainsubjects.

Flexible choices in the same subject.Alberta - a Pure Science Subject or a

CombinedScience subject.Hong Kong - IntegratedScienceorCombined

ScienceorIndividualScience subjectssuchas Physics, Chemistry and Biology.

More than one attempt to improve

scoresChina, Finland - 2 attempts. The best attemptis then counted towards total marks.

New Zealand unsuccessful can appear againwithin the same academic year.



59/61

Source: Board Exam Reform Study- Gujarat



60/61

Source: Board Exam Reform Study- Gujarat

The case of Finland- No 1 since 2000


61/61

The case of Finland No. 1 since 2000

inspired 2014- using assessments for systemic reform- ver 1.1

Documents