Advancing Assessment Literacy

Data Analysis II:

Examining & Interpreting Data

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 2

Predicting

All data are meaningless until we attach meaning through interpretation.

NATIONAL PERCENTILE

(NP)NATIONAL

STANINE (NS)

GRADE EQUIVALENT

(GE)

WHAT IT IS Places students on a scale of 1 to 99, where 99 is highest. An NP score of 99 means the student performs better than 99 per cent of the national sample of the population at that grade level. A national percentile of 50 means that one is average for one's grade level.

Places scores into nine divisions (1 to 9) called stanines. The higher the stanine, the better the performance

Compares a student's performance with that of students in other grades

WHY IT IS USEFUL

Shows precisely how a student is performing in comparison with others at the same grade level

Provides quick and simple information

Shows a student's progress from year to year

WHY IT CANBE

CONFUSING

It is so precise that variations are bound to occur, partly from chance

Simplicity means that a student may make tremendous efforts and actually improve, without moving up a stanine, or stop working without it showing for a while

Grade equivalent does not indicate that the student should be moved up or down to another grade

From http://www.canadiantestcentre.com/Teachers/InformationForTeacher.asp#result

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 4

Cut Scores

• On page 4 of the detailed reports you will find the cut scores detailing the percentage correct required for students to be classified at one of two levels:– Threshold of adequacy– Threshold of proficiency

• Reader response and Math challenge scores are presented on a five-level scale (1-low to 5-high).

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 5

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 6

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 7

Locating Cut Scoresin the Report

• Turn to pg. 4 in the detailed report for your grade level.

Opportunity-to-Learn Elements Performance Component

Excellent Standard

Sufficient Standard

Proficient Standard

Adequate Standard

A score out of 5 (1-low to 5-high) % correct required to reach the standard

• You will need to refer to these scores during the following prediction activity.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 8

Predicting

AFL Math OTL – R & M AFL ReadingOn the chart for your grade level, predict how many students achieved the ADEQUATE Standard on the 2007 AFL in Math for each strand:•Math Content Skills•Integrated Applications•Estimation Skills•Calculator Skills•Computation Skills

On the reverse are the Math Content Skills.

On the charts, predict what percentage of students had SUFFICIENT opportunity to Learn in the following areas:

Reading:•Student preparation and commitment to learn•Student knowledge & use of reading strategies•Home support for reading

Math:•Student preparation and commitment to learn•Student persistence•Home support for learning•Home support for learning math

On the charts, predict what percentage of students met the ADEQUATE Standard in the following strands:•Reading Comprehension•Explicit Comprehension•Implicit Comprehension•Critical Comprehension•Reader Response

On the reverse are the Reading Comprehension Skills.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 9

AFL MathPredictions of Percentage Correct

0

10

20

30

40

50

60

70

80

90

100

Num & Op Ratio & Prop Geom & Meas Data Mng't Algebra CPR Q's APS Q's

AFL Math

Percentage of Students who met the Adequate Standard set by Saskatchewan Educators

AFL MathPredictions of Percentage Correct

0

10

20

30

40

50

60

70

80

90

100

Num & Op Ratio & Prop Geom & Meas Data Mng't Algebra CPR Q's APS Q's

Shade in your prediction on the supplied prediction chart.

Wellman, B. & Lipton, L. (2004). Data driven dialogue. Mira Via, LLC.

AFL Math

Percentage of Students who met the Adequate Standard set by Saskatchewan Educators

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 11

Hypothesesand Assumptions

Based on your predictions, create a set of hypotheses for some or all of them. As you create each hypothesis, identify the underlying assumptions.

• Hypothesis – “X” will contain the highest scores.• Assumption – we created a common assessment for “X” in 2005.

• Hypothesis – students will report higher on digging in and starting reading than looking at the front and back covers to get an idea of what the book is about.

• Assumption – students expect to discover what the book is about in the first few pages.

Write each hypothesis and its accompanying assumption on the cards provided. Please write legibly.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 12

Sharing

• Gather the cards together at your table and either pass them to the next table or shuffle them to discuss as a group.

• When you receive a set of cards, discuss the hypotheses and assumptions.

• Do you see any patterns?• Are any similar or different?• Why might this be?

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 13

Comparisons

The completed bar graphs have been supplied to your table.

• What are you noticing about the data?• What surprised you?• What are the benefits of approaching data in

this manner?• What other data would you like to see to

better inform the results you’ve seen so far?

Wellman, B. & Lipton, L. (2004). Data-driven dialogue. Mira Via, LLC.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 14

Designing Interventions

• Assumptions must be unpacked, because our interventions will be based on them.

• We must strive to correctly identify the causal factors.

• Don’t fall in love with the theory until you have other data.

• Use a strength-based approach to interventions.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 15

Data Displays

• Create groups of three – appoint a recorder, materials manager, and facilitator.

• On the wall is a piece of chart paper and:– AFL – The test questions with results by question

• Package with objectives to supplement.

– CAT3 – Criterion Referenced Scores

• Observe the data silently.

Wellman, B. & Lipton, L. (2004). Data-driven dialogue. Mira Via, LLC.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 16

Data Displays

• After 5-8 minutes share your observations, questions or comments about the data.

• What is unique? Unexpected?• Don’t interpret, just look for what pops out.• Please record your comments and

questions on the chart paper provided.

Wellman, B. & Lipton, L. (2004). Data-driven dialogue. Mira Via, LLC.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 17

Gallery Tour

• Leave a representative behind to answer questions.

• Members of other groups may now circulate and look at the other groups’ findings.

• Return to your original display. Discuss what you saw on your tour.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 18

Team Action Plan

• At your data display, complete the Team Action Plan sheet.

• Write one observed strength and one opportunity for improvement in student learning on the post-it notes provided and post them on the chart for your school at the front of the room.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 19

Fishbone Tool

• At your table, analyze one strength and one area for improvement. Consider all possible causes.

• When you are done, compare and contrast the two diagrams.

Advancing Assessment Literacy Modules: Data Analysis II (February 2008) 20

Creating a Statementof Success

• From your analysis of what is working, develop a statement of success. Write it in the space provided on this worksheet.

• What elements from your statement of success might we include in the beginning stages of an action plan?

• This process will be continued and expanded upon in the next module.