heart online large-scale assessment data management system

HEART Online Large-Scale

Assessment Data Management

System District-Wide

Multiple Assessments

HEART District-Wide Multiple Assessments

What Other District-Wide Assessments?

Import Other Student InformationImport Other Test InformationImport Other Student Achievement

Data


• Identify by name and content area other district-wide assessments planned for import into HEART.

• How many years of data exist for each?• Is the data on a data disk, a computer

file, or a paper record?• Are descriptive materials available for

the assessments?


Three New Practices to be Prepared to Use Other District-Wide

Multiple Assessments in HEART

1. Import Other Student Information2. Import Other Test Information

(Planned Assessment File)3. Import Other Student Achievement

Data(Student Information File)

1. Import Other Student Information

Current Practice• A district’s ITBS/ITED bar code file

is imported.• A district’s data is imported from

the ITBS/ITED data disk.• The bar code file does not have

information about students in grades not taking the ITBS/ITED.


• If multiple assessment data to be imported is for students already in your district’s ITBS/ITED bar code file, no change is necessary.

• Continue to annually send a copy of your district’s bar code file to Lisa Sines to be imported.


New/Best Option:

• Provide for import a K12 file instead of the ITBS/ITED bar code file.

• This is an extract from your district’s student information system that includes all students, not just those taking the ITBS/ITED. Who does this extract?


Another Option:

• Create and provide an Other students file (grades not in the bar code file)

• Information about the file layout and an Excel template are downloadable from the Iowa Testing Programs website

(see next slide)

http://www.education.uiowa.edu/itp/index.htm

A description of the bar code file

layout and an Excel

template to create a bar

code file can be

downloaded from the Iowa

Testing Program website.


Implications of a K12 File:

• An additional file may need to be created beyond the one sent to Iowa Testing Programs that generates ITBS/ITED bar code labels.

• More student information needs to be provided, and protected.

• More student information will need to be checked/cleaned in the HEART Data Cleanup section.

• Others?

2. Import Other Test Information

Test Name Content Area PRK RSO GEO RIT NCE STN STS GEI RSI PCT

lo

cut

K

hi

cut

K

lo

cut

1

hi

cut

1

lo

cut

2

hi

cut

2

lo

cut

3

hi

cut

3

lo

cut

4

hi

cut

4

lo

cut

5

hi

cut

5

lo

cut

6

hi

cut

6

lo

cut

7

hi

cut

7

lo

cut

8

hi

cut

8

lo

cut

9

hi

cut

9

lo

cut

10

hi

cut

10

lo

cut

11

hi

cut

11

lo

cut

12

hi

cut

12

Heart Online Multiple Assessments I nformation to be I mported

Enter an A in the appropriate

column to indicate what type of

score will be used to compute

achievement levels f or a test (ie, if

PRK is used to compute achievement

levels f or Test 1, type the letter A

in the corresponding cell). Only one

score type can be used per test.

PRK = Percentile Rank

RSO = Raw Score (ordinal)

GEO = Grade Equivalent (ordinal)

RIT = RI T score (NWEA tests)

NCE = National Curve Equivalent

STN = Stanine score

STS = Standard Score

GEI = Grade Equivalent (interval)

RSI = Raw Score (interval)

PCT = Percent (percent correct)

Enter the test

name as you want

it to appear on

graphs and tables.

Remember that

long test names

may not print well,

but short names

will not describe

the test clearly.

You may enter as

many as ten (10)

diff erent tests.

Reading

Math

Science

etc.

The low cut (lo) represents the

highest score that is part of the low

band (below proficient). The high cut

(hi) represents the lowest score that

is part of the high band (advanced).

Enter the cut scores f or all grades f or

which there are data in the import.




RI T = RI T score (NWEA tests)


STN = Stanine score





What district-wide multiple assessments do

you intend to import? HEART needs details

about the assessments.

Planned Assessment File


lo

cut

K

hi

cut

K

lo

cut

1

hi

cut

1

lo

cut

2

hi

cut

2

lo

cut

3

hi

cut

3

lo

cut

4

hi

cut

4

lo

cut

5

hi

cut

5

lo

cut

6

hi

cut

6

lo

cut

7

hi

cut

7

lo

cut

8

hi

cut

8

lo

cut

9

hi

cut

9

lo

cut

10

hi

cut

10

lo

cut

11

hi

cut

11

lo

cut

12

hi

cut

12


Enter the test

name as you want

it to appear on

graphs and tables.

Remember that

long test names

may not print well,

but short names

will not describe

the test clearly.

You may enter as

many as ten (10)

diff erent tests. Examples of test names currently used in HEART and

how they appear.



lo

cut

K

hi

cut

K

lo

cut

1

hi

cut

1

lo

cut

2

hi

cut

2

lo

cut

3

hi

cut

3

lo

cut

4

hi

cut

4

lo

cut

5

hi

cut

5

lo

cut

6

hi

cut

6

lo

cut

7

hi

cut

7

lo

cut

8

hi

cut

8

lo

cut

9

hi

cut

9

lo

cut

10

hi

cut

10

lo

cut

11

hi

cut

11

lo

cut

12

hi

cut

12


Enter the test

name as you want

it to appear on

graphs and tables.

Remember that

long test names

may not print well,

but short names

will not describe

the test clearly.

You may enter as

many as ten (10)

diff erent tests.

To encourage standardization, Great Prairie AEA will support common multiple assessment naming practices.

Examples of likely supported names:

ASVAB

BRI (Basic Reading Inventory)

Explore Reading (ACT)

ICAM Comprehending Literature

ITP Constructed Response Math

Plan Reading (ACT)

SCASS Weather Maps

Star Reading

TIMSS

Works Keys (ACT)


Test Name Content Area PRK RSO GEO RI T NCE STN STS GEI RSI PCT

lo

cut

K

hi

cut

K

lo

cut

1

hi

cut

1

lo

cut

2

hi

cut

2

lo

cut

3

hi

cut

3

lo

cut

4

hi

cut

4

lo

cut

5

hi

cut

5

lo

cut

6

hi

cut

6

lo

cut

7

hi

cut

7

lo

cut

8

hi

cut

8

lo

cut

9

hi

cut

9

lo

cut

10

hi

cut

10

lo

cut

11

hi

cut

11

lo

cut

12

hi

cut

12

Heart Online Multiple Assessments I nformation to be Imported

Enter the test

name as you want

it to appear on

graphs and tables.

Remember that

long test names

may not print well,

but short names

will not describe

the test clearly.

You may enter as

many as ten (10)

diff erent tests.

Reading

Math

Science

etc.

What other content areas will have multiple assessments?

Again, standardization of naming will be encouraged

(Example: Social Studies)



lo

cut

K

hi

cut

K

lo

cut

1

hi

cut

1

lo

cut

2

hi

cut

2

lo

cut

3

hi

cut

3

lo

cut

4

hi

cut

4

lo

cut

5

hi

cut

5

lo

cut

6

hi

cut

6

lo

cut

7

hi

cut

7

lo

cut

8

hi

cut

8

lo

cut

9

hi

cut

9

lo

cut

10

hi

cut

10

lo

cut

11

hi

cut

11

lo

cut

12

hi

cut

12















STN = Stanine score





Enter the test

name as you want

it to appear on

graphs and tables.

Remember that

long test names

may not print well,

but short names

will not describe

the test clearly.

You may enter as

many as ten (10)

diff erent tests.

Reading

Math

Science

etc.













STN = Stanine score






TYPES of District-wide Multiple Assessment Scores that can be Imported into HEART

(Source: Using Data to Improve Student Achievement, Deborah Wahlstrom)

PRK Percentile Rank

Percentile ranks express the percentage of scores in the norming group that are less than or equal to the raw score. Percentile scores are the most commonly reported scores and are best used to describe a student’s standing in relation to the norming group at the time of testing. National Percentile Rank is a score that shows how a student performed in relation to a group of students who were tested under the same conditions at the same time of the year during national standardization. If a student received a percentile rank of 68 on a nationally normed test, the student scored at or better than 68 percent of the individuals in the norm group.

RSO Raw Score (Ordinal)

The number of questions or items that a student answers correctly on a test. A score that has not had anything done to it – has not been converted or transformed to any other scoring system. Raw scores are converted to other scoring systems such as percentile rank, scaled scores, and percent correct scores.

GEO Grade Equivalent (Ordinal)

Grade equivalent scores are used to relate a given student’s test score to the average score of other students who took the test based on grade and month.

RIT RIT score (NWEA only)

The score indicates the current learning level for a student. The level tests also provide a percentile score that compares a student with other students at the same grade across the nation who have taken the tests. RIT scores range from approximately 150-250. (Source: NWEA Parent Guide)

NCE Normal Curve Equivalent

Normal curve equivalent scores are standard scores based on the normal curve. NCE’s have a standard deviation of approximately 21 so that the range of probable scores is 1 – 99. NCE’s are commonly used with the measurement of performance in federal programs such as Title 1.

STN Stanine A standard score system that represents groupings of percentile ranks into a nine-unit scale. A stanine is one of the steps in this nine-point scale of standard scores.

TYPES of District-wide Multiple Assessment Scores that can be Imported into HEART

(Source: Using Data to Improve Student Achievement, Deborah Wahlstrom)

STS Standard Score A scoring system designed to show student performance on an achievement test. A standard score is expressed as a deviation from a population mean. Standard scores may also be called growth scale values, developmental standard scores, and scaled scores. Each scoring system has its own units to represent the scores. Thus, a scaled score of 388 on one test may not be the same as a scaled score of 388 on another test. Standard scores have equal interval units of measurement, so you can use them to compute averages and gains. Examples of standard score systems include T-scores, Z-scores, stanines, and Normal Curve Equivalents. Standard scores can be used to compare a student’s standing on two or more different tests.

GEI Grade Equivalent (Interval)

Grade equivalent scores are used to relate a given student’s test score to the average score of other students who took the test based on grade and month.

RSI Raw Score Interval

The number of questions or items that a student answers correctly on a test. A score that has not had anything done to it – has not been converted or transformed to any other scoring system. Raw scores are converted to other scoring systems such as percentile rank, scaled scores, and percent correct scores.

PCT Percent Scores that represent the percentage of test items a student answers correctly. Percent correct scores are calculated by dividing the number of questions a student answered correctly by the number of questions he or she could have answered correctly and then multiplying by 100.

Ordinal A way of measuring that ranks (puts them in order) on some variable. The difference between the ranks need not be equal.

Interval A scale or measurement that describes variables in such a way that the distance between any two adjacent units of measure (or intervals) is the same, but in which there is no meaningful zero point.

PRK Percentile Rank 1-99 (actually >0 and <100) Ordinal scale – not equal intervals

RSO Raw Score (ordinal)

Numeric scale 0-?? Ordinal scale – not equal intervals Rubric scores, other ratings of level

GEOGrade Equivalent

(ordinal) Typical for grade equivalents on most tests Ordinal scale – not equal intervals

RITRIT score (NWEA

testing only) Interval scale special standard score scale used in

NWEA tests

NCENormal Curve

Equivalent Interval scale specialized standard score

STN Stanine Interval scale specialized standard score

STS Standard Score Interval scale general standard scores Numeric scale – accepts values greater than 0

GEIGrade Equivalent

(interval) ITBS and ITED grade equivalent scores Interval scale

RSI Raw Score Interval Raw score from tests Interval scale, equal intervals

PCT Percent

Percent correct is typical 0-100 Interval scale


lo

cut

K

hi

cut

K

lo

cut

1

hi

cut

1

lo

cut

2

hi

cut

2

lo

cut

3

hi

cut

3

lo

cut

4

hi

cut

4

lo

cut

5

hi

cut

5

lo

cut

6

hi

cut

6

lo

cut

7

hi

cut

7

lo

cut

8

hi

cut

8

lo

cut

9

hi

cut

9

lo

cut

10

hi

cut

10

lo

cut

11

hi

cut

11

lo

cut

12

hi

cut

12















STN = Stanine score





Enter the test

name as you want

it to appear on

graphs and tables.

Remember that

long test names

may not print well,

but short names

will not describe

the test clearly.

You may enter as

many as ten (10)

diff erent tests.

Reading

Math

Science

etc.













STN = Stanine score





Cut scores ARE NOT a guess, an

arbitrary decision, a dart throw, a roll of

the dice, or one person’s opinion.


2. Import Other Test InformationIowa Tests of Basic Skills

Interpretive Guidance:PROFICIENCY

For purposes of NCLB, the achievement of Iowa students is described in NPR score ranges as Low, Intermediate, and High. Students with NPRs of 1 to 40 are in the Low

level, those with NPRs from 41 to 89 are in the Intermediate level, and those with values from 90 to 99 are in the High level. Each state is required to decide

which levels of achievement are considered proficient and less-than-proficient. In Iowa, the Low level as defined

above is “Less-than-Proficient” and the Intermediate and High levels together are regarded as “Proficient”.

Define proficiency

2. Import Other Test InformationEXAMPLE: ITBS 4th Grade Reading Comprehension

Achievement Level Descriptors

Low: Seldom understands factual information or new words in context. Sometimes is able to make inferences and interpret either non-literal language or information in new contexts. Rarely can determine a selection’s main ideas or analyze its style and structure.

Intermediate: Usually understands factual information and new words in context. Often is able to make inferences and interpret either non-literal language or information in new contexts. Usually can determine a selection’s main ideas and analyze its style and structure.

High: Understands factual information and new words in context, is able to make inferences, can interpret either non-literal language or information in new contexts, and can determine a selection’s main ideas and analyze its style and structure.

Describe proficiency


EXAMPLE: ITBS 4th Grade Mathematics (Math Total)Achievement Level Descriptors

Low: Sometimes can understand math concepts. Seldom is able to solve word problems, use estimation methods, or interpret data from graphs and tables.

Intermediate: Usually can understand math concepts, solve word problems, and interpret data from graphs and tables. Sometimes is able to use estimation methods.

High: Understands math concepts, solves word problems, and interprets data from graphs and tables. Usually is able to use estimation methods.

Describe proficiency


MODULES

Reading Comprehending Literature (4, 8, and 11)Comprehending Academic TextsComprehending Functional TextsLiterary Elements & Techniques

Mathematics Problem-Solving Process & StrategiesNumber Concepts & Operations (4 and 8)MeasurementGeometryData Analysis, Statistics, & ProbabilityPatterns, Functions and Algebra (11)Solving Work-Related Math Problems

ICAM example


Points

Earned

01234567891011121314151617181920

Level 1

Level 2

Level 3

I CAM 4th Grade Mathematics: Number Concepts and Operations

Achievement Level Descriptions

Score Conversion Table

I s beginning to develop an understanding

about whole numbers, f ractions, decimals,

percents, and mixed numbers; and is

beginning to develop the ability to solve

problems using number concepts.

Understands little about whole numbers,

f ractions, decimals, percents,and mixed

numbers; is generally unable to solve


Understands whole numbers, f ractions,

decimals, percents, and mixed numbers; is

developing the ability to solve problems

using number concepts.

Form Z

Achievement Levels


Points

Earned

01234567891011121314151617181920

Level 1

Level 2

Level 3

I CAM 4th Grade Mathematics: Number Concepts and Operations

Achievement Level Descriptions

Score Conversion Table

I s beginning to develop an understanding

about whole numbers, f ractions, decimals,

percents, and mixed numbers; and is

beginning to develop the ability to solve


Understands little about whole numbers,

f ractions, decimals, percents,and mixed

numbers; is generally unable to solve


Understands whole numbers, f ractions,

decimals, percents, and mixed numbers; is

developing the ability to solve problems

using number concepts.

Form Z

Achievement Levels

The selection of cut points

is a judgmental

decision that should be made only

after content experts have reviewed the actual items on the test.

Descriptions about what students know and can do in relationship to the content

standards.

3. Import Other Student Achievement Data

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AA AB AC AD AE AF AG AH AI AJ AK AL AM AN AO APRed columns

are not

currently

used

Yellow

columns are

required. End - fi ll this column with periods down to the last row with data.

Last Name First Name

Birth

Date

Gende

r Grade

State Building

I D ––

State

Student I D Am I nd

Asian/ P.

I sl Af r. Am. Hisp White –– –– –– FAY I EP ––

Free/

Reduced –– ELL Migrant –– –– –– ––

District

Student

I D

Test

Year

Test

Month

Test

Name PRK RSO GEO RI T NCE STN STS GEI RSI PCT .

I n the Test Name column, insert exact test name as listed in

planned assessments fi le and fi ll down to the bottom of the column

of data (The import requires an exact match on test name.)

Student Demographic and

Achievement DataPaste data in the column that represents the data type

being imported.

Student Data File:

Demographic and Achievement Data Template

3. Import Other Student Achievement DataRed columns

are not

currently

used


Birth

Date Gender Grade

State Building

I D ––

State Student

I D Am I nd

Asian/P.

I sl Af r. Am. Hisp White ––

–– –– FAY I EP –– Free/ Reduced –– ELL

Migrant –– –– –– ––

District

Student I D Test Year Test Month

Test Name PRK RSO GEO RI T NCE STN STS GEI RSI PCT .

Paste data in the columns that represent the data type being imported

End - fi ll this column with

periods down to the last row

with data.

I n the Test Name column, insert exact test name as listed in planned assessments fi le and fi ll down to

HEART Multiple Assessment

Student Data File

Yellow columns are required.


Field Required Comments

Last name X Use exact spelling as the student import file

First name X Use exact spelling as the student import file

Date of birth X Use mm/dd/yyyy or mm/dd/yy

Gender X Use F or M

Grade X Use K-12 or 00, 01, – 12.

Building name X Building name (as spelled in HEART)

State student ID X Use state assigned student ID numbers here

Ethnic – American Indian

X Mark only one ethnic code per student by putting a 1 in the correct column.

Ethnic – Asian, Pacific Islander

Ethnic – African American/Black

Ethnic – Hispanic

Ethnic - White

FAY OUsed for ITP/NCLB: 0 = not FAY building or district, 1 = FAY district, not building, 2 = FAY in

both

IEP X Use: 1 = special education, blank or 0 for non-special education

Free/Reduced X Use: 1 = free/reduced, blank or 0 for non-free/reduced

ELL X Use: 1 = ELL, blank or N for non-ELL

Migrant X Use: 1 = migrant, blank or 0 for non-migrant

District student ID O Recommended until transition to State ID is completed

Year Tested* X Use: 4 digit year at time of testing (2004)

Month Tested* X Use: 1-2 digit month of testing (i.e., 9 = Sept., 12 = Dec.)

Refer to these Data Field Definitions when completing

the Student Data File


Field Required Comments

Last name X Use exact spelling as the student import file

First name X Use exact spelling as the student import file

Date of birth X Use mm/dd/yyyy or mm/dd/yy

Gender X Use F or M

Grade X Use K-12 or 00, 01, – 12.

Building name X Building name (as spelled in HEART)

State student ID X Use state assigned student ID numbers here

Ethnic – American Indian

X Mark only one ethnic code per student by putting a 1 in the correct column.

Ethnic – Asian, Pacific Islander

Ethnic – African American/Black

Ethnic – Hispanic

Ethnic - White

FAY OUsed for ITP/NCLB: 0 = not FAY building or district, 1 = FAY district, not building, 2 = FAY in

both

IEP X Use: 1 = special education, blank or 0 for non-special education

Free/Reduced X Use: 1 = free/reduced, blank or 0 for non-free/reduced

ELL X Use: 1 = ELL, blank or N for non-ELL

Migrant X Use: 1 = migrant, blank or 0 for non-migrant

District student ID O Recommended until transition to State ID is completed

Year Tested* X Use: 4 digit year at time of testing (2004)

Month Tested* X Use: 1-2 digit month of testing (i.e., 9 = Sept., 12 = Dec.)


are not

currently

used


Birth

Date Gender Grade

State Building

I D ––

State Student

I D Am I nd

Asian/ P.



Migrant –– –– –– ––

District






with data.



Student Data File


All student information must be exactly the

same as the

Student Information imported previously.

If not:

DATA CLEAN-

UP!


are not

currently

used


Birth

Date Gender Grade

State Building

I D ––

State Student

I D Am I nd

Asian/ P.



Migrant –– –– –– ––

District






with data.



Student Data File


The Test Name

must be exactly the

same as the

Planned Assessmen

t File Name

imported previously.

If not:

BAD IMPORT!


are not

currently

used


Birth

Date Gender Grade

State Building

I D ––

State Student

I D Am I nd

Asian/ P.



Migrant –– –– –– ––

District






with data.



Student Data File


All student scores

must be entered as the same

type as the Planned

Assessment imported previously.

If not:

BAD IMPORT!


Red columns

are not

currently

used

Yellow

columns are

required.


Birth

Date Gender Grade

State Building

I D ––

State

Student I D Am I nd

Asian/ P.

I sl Af r. Am. Hisp


Student Data File


lo

cut

K

hi

cut

K

lo

cut

1

hi

cut

1

lo

cut

2

hi

cut

2

lo

cut

3

hi

cut

3

lo

cut

4

hi

cut

4

lo

cut

5

hi

cut

5

lo

cut

6

hi

cut

6

lo

cut

7

hi

cut

7

lo

cut

8

hi

cut

8

lo

cut

9

hi

cut

9

lo

cut

10

hi

cut

10

lo

cut

11

hi

cut

11

lo

cut

12

hi

cut

12

Templates for your data are available from Evan, Julie, Lisa or the

GPAEA website


Red columns

are not

currently

used

Yellow

columns are

required.


Birth

Date Gender Grade

State Building

I D ––

State

Student I D Am I nd

Asian/ P.

I sl Af r. Am. Hisp


Student Data File

Good News for ICAM and NWEA Users!

•This HEART file does NOT need to be created if you can provide your original data disks or data files. They will import directly.

•A Student Information File still needs to be imported if there is achievement data for students other than those in the ITBS/ITED bar code file.

•A Planned Assessment File still needs to be imported first.


Three New Practices to be Prepared to Use Other District-Wide

Multiple Assessments in HEART

1. Import Other Student Information2. Import Other Test Information3. Import Other Student Achievement

Data

Evan McCormick, HEART Online [email protected] ext. 1298District-Wide Student Achievement Data: ITBS/ITED Generating Graphs, Analysis, UseDistrict-Wide Student Achievement Data: Multiple Measures in HEAR T (ICAM, BRI, local assessments)HEART assessment data connections: CSIP, IPDM, IDM, SINAHEART Student Behavior ComponentHEART Online Technical Assistance Manual

Julie-A. Barwick,HEART Online Technology [email protected] ext. 5250User setupClassroom setupStudent class assignment setupStudents records cleaningOver-the-phone technology technical assistance/coaching for HEART data management

Lisa Sines,HEART Online Data [email protected] ext. 5230Maintains list of local school HEART Online contacts: HEART District Coordinators, HEART ManagersReceives school data disks (ITBS & ITED) and student bar code files from LEAs annually to be imported into HEART OnlineReturns data disks & student bar code files to schools

mailto:[email protected]



heart online large-scale assessment data management system

Documents

districts bar code file

bar code file layout

districtwide assessments

k12 file

computer file

additional file

student informationimport

itbsited bar code labels