national assessment program information communication .information communication and technology...

Download NATIONAL ASSESSMENT PROGRAM INFORMATION COMMUNICATION .INFORMATION COMMUNICATION AND TECHNOLOGY LITERACY

Post on 28-Jul-2018

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • NATIONAL ASSESSMENT PROGRAM

    INFORMATION COMMUNICATION AND

    TECHNOLOGY LITERACY

    YEAR 6 AND YEAR 10

    TECHNICAL REPORT

    2005

    John Ainley

    Julian Fraillon Chris Freeman Martin Murphy

    February 2008

  • Prepared in 2008 by The Australian Council for Educational Research Ltd 19 Prospect Hill Road, Camberwell, Victoria, 3124, Australia.

    Copyright 2008 Australian Council for Educational Research

  • CONTENTS

    Chapter 1 Introduction............................................................................ 1 Project Overview.........................................................................................................................1 ICT Literacy National Sample Assessment Survey ....................................................................1 ICT Literacy Strands ...................................................................................................................2 Participants in the Assessment ....................................................................................................2 Assessment Format .....................................................................................................................2 Reporting of the Assessment Results ..........................................................................................2 Structure of the Technical Report ...............................................................................................3

    Chapter 2 Assessment Domain and Instrument Development............ 4 Developing the Assessment Domain...........................................................................................4 Application of ICT Literacy........................................................................................................4 The ICT Literacy Assessment Instrument...................................................................................4 Structure of the Instrument..........................................................................................................7 Assessment Item Types...............................................................................................................7 Items and modules.......................................................................................................................8

    The General Skills Test .......................................................................................................9 The Hybrid Assessment Modules .......................................................................................9

    Summary ...................................................................................................................................10

    Chapter 3 Sampling and Weighting ..................................................... 11 Sampling Design .......................................................................................................................11

    Sampling schools ..............................................................................................................11 Sampling Students.............................................................................................................12 Planning sample size .........................................................................................................12 Designed Sample...............................................................................................................13

    Sampling Process ......................................................................................................................14 First sampling stage...........................................................................................................14 School exclusions..............................................................................................................14 Replacement schools .........................................................................................................14 Second sampling stage ......................................................................................................15 Student exclusions.............................................................................................................16 Participation rates..............................................................................................................17

    Weighting..................................................................................................................................17 The First Stage Weight......................................................................................................18 The Second Stage Weight .................................................................................................19 Overall Sampling Weight..................................................................................................19

    Characteristics of the Achieved Sample....................................................................................20 Calculating the precision of estimates.......................................................................................21 Structural differences in State and Territory education systems ...............................................21 Summary ...................................................................................................................................22

  • Chapter 4 Field Administration and Data Management ................... 23Administration...........................................................................................................................23

    Contact with schools .........................................................................................................23 Information management ..................................................................................................25 Within-school procedures .................................................................................................25 Quality control ..................................................................................................................27 Marker training and marking procedures ..........................................................................27

    Data management......................................................................................................................28 Data-entry procedures .......................................................................................................28 Coding of the student background survey.........................................................................28

    School reports............................................................................................................................30 Summary ...................................................................................................................................31

    Chapter 5 Scaling Procedures............................................................... 32 The scaling model .....................................................................................................................32 Assessment of item fit ...............................................................................................................33 Interaction between Test implementation and Item calibration ................................................33 Equating ....................................................................................................................................37 Item calibration .........................................................................................................................37 Plausible values.........................................................................................................................42 Summary ...................................................................................................................................43

    Chapter 6 Reporting of Results ............................................................ 44 Standardisation of student scores ..............................................................................................44 Scaled Score Result by State or Territory .................................................................................44 Estimation of sampling and measurement variance ..................................................................45 Reporting of mean differences across States and Territories ....................................................46 Summary ...................................................................................................................................47

    Chapter 7 Proficiency Levels and Standards-setting .......................... 48 Proficiency levels ......................................................................................................................48

    Principles...........................................................................................................................48 Technical information .......................................................................................................48 Specifying Proficiency Levels ..........................................................................................49 Describing proficiency levels............................................................................................50 Distribution of students across proficiency levels.............................................................50

    Standards-setting .......................................................................................................................52 Methods.....................................................................................................................

Recommended

View more >