data analysis - tools and processes (school level) food for thought how does your school use data to...

Data Analysis - Tools and Processes (School Level) Food for Thought How does your school use data to inform instruction and improve student achievement? Tuesday, February 21, 2012 Hawaii Department of Education Office of Curriculum, Instruction and Student Support

Webiquette 1.Before the meeting starts, close any other applications running on your computer. 2. Use the Hand icon if you wish to speak or have a question. 3. Mute your microphone to eliminate ambient noises. 4. Make sure that your Chat box is set for Everyone. 5. Use the Chat box for questions. 6. Restrict the use of text-speak, please respond using standard English to text. 7. Your collaboration is vital. Every perspective contributes to the whole picture. Muted Unmuted 2

Group Norms for Webinar Self-directed Learner Make personal connections to your position Community Contributor Honor the expertise of ALL Complex Thinker Synergize Collective thoughts Quality Producer Grow professionally Effective Communicator Seek first to understand, then to be understood Effective & Ethical User of Technology Remove all other distractions 3

Hawaiis Five RTTT Pillars 6 Improved Student Outcomes Data for School Improvement Longitudinal Data System Balanced Scorecard Data Governance Using data to inform instruction Common Core Standards Career & College Ready Diploma Curriculum Framework Common Instructional Materials Formative Assessments Interim Assessments Summative Assessments STEM Focused support on lowest-performing schools - Zones of School Innovation Flexibility Great teachers and great leaders Remove barriers to learning Performance-based evaluation system New Teacher Induction & Mentoring Incentives Leadership development Alternative pathways Systems of Support to enable schools to do their best work reprioritize and reorganize State resources; establish Human Resources Unit in Zones of School Innovation; automate 6

Essential Question How does data analysis help in school improvement efforts? 7

Desired Outcomes A common understanding of the various purposes for analyzing data An understanding of how to analyze data using a variety of tools and processes 8

Data Analysis (School Level) Agenda Reason We Analyze Data Basic Information We Use to Analyze Data Processes We Can Use to Analyze Data Finding Root Causes 9

Reason We Analyze Data Why do we need to use data? Why do we want to use data? 10

Why We Need to Use Data

Formative Assessment / Instruction Data for School Improvement (DSI) as a formative assessment tool Using DSI Reports to inform instruction Deconstructing the Standards Process K-12

Why We Want to Use Data 1. Pick a number from 1-10. 2. Multiply that number by 9. 3. Add up the digits of the answer. 4. Subtract 5 from the number. 5. Find the letter that corresponds to the number (example: 1=A, 2=B, 3=C, etc.) 6. Think of a country whose name starts with that letter and write it down. 7. Take the 2nd letter in the country's name, and think of an animal whose name starts with that letter and write it down. 8. Write down the color of that animal. 9. Take the last letter of the country's name and write down an animal whose name starts with that letter. 10. Use the last letter of that animal and write down a fruit that starts with that letter. 18

Types of School Teams Principal Counselor Curriculum Coordinator GL Reps/Dept Chairs Kindergarten First Second Teacher Science Math Language Arts Teacher Leadership Team Grade Level/ Department Classroom 19

Take a Minute How does your school use data to help students and teachers succeed? What types of data teams do you have at your school? 20

State Goals Vision of a Hawaii high school graduate is that all public school graduates will : Realize their individual goals and aspirations; Possess the attitudes, knowledge and skills necessary to contribute positively and compete in a global society; Exercise the rights and responsibilities of citizenship; and Pursue post-secondary education and/or careers without need for remediation. 21

State Goals 22 Vision of a Hawaii high school graduate There are six General Learner Outcomes (GLOs) that are the goals of standards-based learning in all content areas: Self-Directed Learner: The ability to be responsible for one's own learning Community Contributor: The understanding that it is essential for human beings to work together Complex Thinker: The ability to be involved in complex thinking and problem solving Quality Producer: The ability to recognize and produce quality performance and quality products Effective Communicator: the ability to communicate effectively Effective and Ethical User of Technology: the ability to use a variety of technology effectively and ethically.

Inverted Data Pyramid Summative High Stakes Assessment Other 23 N. Love, 2010

Data Pyramid: What kinds of data do teams and coaches use? Summative High Stakes Assessments Demographic, Process and Perceptual Data Benchmark Common Assessment Formative Common Assessment Formative Classroom Assessment 24 N. Love, 2010

Using Data at the School Level Where are we now? Requires measures and data What measures do you have available to help to determine what may be the problem? Where do we want to go? Where are we going? Determine the goal know the target What are the needs of the school or the students? How do we get there? Identify a strategy or process What will you put in place in order to achieve the outcome? 26

Too much data? Diverts attention away from the primary purpose: improving instruction. Leads to overload creating long, comprehensive plans that few read. Reveals too many things to address so too many goals and initiatives are created.

Multiple Measures Demographics Perceptions Student Learning Processes 28 V. Bernhardt

Demographic Data Clarifies who our clients are. Builds on the context of the school Helps to predict future conditions to best serve the needs of our future students. Demographic information is crucial in data analysis as it helps us understand the context within which schoolwide change is planned and takes place. (V. Bernhardt, 1998, p-25) 29

Examples of Demographic Data Number of students in the school Number of students with special needs Ethnicities of the students in the school Number of graduates Number of disadvantaged students 30

Tools Finding Data Demographic (Type any other places that you get this type of data into the Chat box) Longitudinal Data System (LDS) http://employees.hidoe.k12.hi.us

Tools Finding Data Demographic (Type any other places that you get this type of data into the Chat box) United States Census http://quickfacts.census.gov/qfd/states/15000.html

Tools Finding Data Demographic (Type any other places that you get this type of data into the Chat box) Hawaii Department of Business, Economic Development & Tourism http://hawaii.gov/dbedt/info/census/

Tools Finding Data Demographic (Type any other places that you get this type of data into the Chat box) School Documents Online http://iportal.k12.hi.us

Perceptual Data A view, judgment or appraisal formed in the mind about a particular matter. A belief stronger than impression and less strong than positive knowledge. A judgment one holds as true. In organizations, if we want to know what is possible...we need to know the perceptions of the people who make up the organization. V. Bernhardt, 1998,pg. 41 35

Examples of Perceptual Data Observations Person-to-person interviews Telephone surveys Focus groups Parent surveys 36

Tools Finding Data Perceptions (Type any other places that you get this type of data into the Chat box) School Quality Survey (SQS) http://arch.k12.hi.us

Process Data Programs can include a wide variety of offerings, from specially funded programs to academic curricular sequences to extracurricular programs. 38

Examples of Process Data (Type any other places that you get this type of data into the Chat box) Grant data Program data Comprehensive Needs Assessment (continuous improvement process) Curriculum mapping Data Teams 39

Student Learning Data Most important type of data to focus on. Annual Large-Scale Assessment Data Periodic Assessment Data Ongoing Classroom Assessment Data 40

Examples of Student Learning Data Hawaii State Assessment (HSA) Terra Nova DIBELS/DIBELS Next Reading Inventories Classroom Assessment Data 41 LDS

Tools Finding Data Student Learning (Type any other places that you get this type of data into the Chat box) Accountability Resource Center Hawaii (ARCH) http://arch.k12.hi.us

Tools Finding Data Covers Demographic, Perceptual, and Student Learning (Type any other places that you get this type of data into the Chat box) School Status & Improvement Report (SSIR) http://arch.k12.hi.us

Disaggregating Data 44

Disaggregating Data Typically, student achievement data are reported for whole populations, or as aggregate data. It is not, however, until the data are disaggregated that patterns, trends and other important information are uncovered. ** Disaggregated data simply means looking at test scores by specific subgroups of students. 45

Disaggregating Uncovers... Achievement gaps are differences in academic achievement amongst different groups of students. It is important to examine these differences in order to find ways that we can address some of the inequities. The disaggregated data and the dialogue that arises can transform beliefs and practices. 46

Trend Data Data that shows a pattern over time. Time is the variable over which one constant is being compared. The more years of data that you have, the more reliable are the trends and patterns. Statistically three years of data just barely indicates a trend. Five years provides more confidence to your inferences. N. Love, 2008 47

Several Ways to Disaggregate Data Gender Socio-Economic Status Mobility Special Education and Disability ELL English Language Learners Grade level Classroom or course 48

Take a Minute What student data/information does your school use to make decisions? 49

Problem Solving Processes 52

Root Cause Analysis - Process Evaluate Programs Improvement Planning Determine Root Causes Conduct Data Analysis Define Problems Organize Teams 53 P. Preuss, 2003

Structured Collaboration BUILD Foundation IDENTIFY Student Learning Problems VERIFY Causes GENERATE Solutions RESULTS 54 N. Love, 2010

The Problem Solving Cycle 1. Identify the Problem 2. Describe Hunches and Hypotheses 3. Identify Question and Data 4. Analyze Multiple Measures 5. Analyze Political Realities 6. Develop Action Plan Resolution 7. Implement Action Plan 8. Evaluate Implementation 9. Improve the Process and System 55 V. Bernhardt, 2011

Step 2: Analyze Data to Prioritize Needs Step 3: Establish SMART Goals Step 4: Select Specific Strategies Step 5: Determine Results Indicators Step 6: Monitor and Evaluate Results Step 1: Conduct a Treasure Hunt Decision Making for Results Doug Reeves 56

Inquiry Cycle Annenberg Institute for School Reform Establish Desired Outcomes Define the Questions Collect and Organize the Data Make meaning of the data Take Action Assess and Evaluate actions 57

What are the Causes? 60

Data helps us get to the root causes of a problem so we solve the problem and not just the symptom. V. Bernhardt 61

What is Root Cause? The deepest underlying cause, or causes, of positive or negative symptoms within any process that, if dissolved, would result in eliminating or substantial reduction of the symptom. 62 P. Preuss, 2003

Levels of Root Cause External Level Systemic Level Programmatic or Process Level Incident/Procedural Level Families Communities Supporting agencies Leadership Values/Beliefs Instructional Process Time Staff Development Students Teachers Incidents 63

When is a Cause a Root Cause? Would the problem have occurred if the cause had not been present? Will the problem reoccur as the result of the same cause if the cause is corrected or dissolved? Will correction or dissolution of the cause lead to similar events? If yes, then it is a contributing cause. If no, then it is a root cause 64

Tools for Root Cause Analysis The Five Whys (Roots Toyota Corporation) The Questioning Data Process (P. Preuss) Causes for Student Learning Problems (N. Love) System Planning Process (P. Preuss) The Diagnostic Tree Process (P. Preuss) 65

The Five Whys Why is it happening? Problem: 10 th grade is not making significant progress in reading Why? Students not progressing are not doing work Why? Students not progressing miss multiple days a quarter Why? Students miss instruction because they are not in class Why? Students are not in class because they babysit siblings Why? Elementary school and high school schedules are different 66

Questioning Data Process (P. Preuss) Step 1: Ask What do you see in this data set? Step 2: Ask What questions do you have about what you see? 67

Possible Causes for Student Learning Problems (N. Love) Did we teach it? In enough depth? Placed in the right sequence? Frequently enough? Curriculum: Did we use a variety of research-based instructional approaches? Are we sharing successful practices? Did we reteach using a different approach to individuals or groups who didnt yet get it? Instruction: Do we use ongoing formative assessment to explore student thinking and built on it in our instruction? Communicate to students how to improve? Help them self-assess? Assessment: Did we examine attitudes or practices that might contribute to achievement/relationship/teaching gaps? Equity: Did we identify students who need additional help and provide them with it? Individual Assistance: What knowledge/skills would help us improve student achievement? Teacher Development: 68

System Planning Process EXAMPLE (P. Preuss) 18 of 20 IEPs for high school students with disabilities lack post secondary goals. Where are we now? Post secondary goal statements must reflect goals after leaving high school Where are we going? Involve the student. career interest Inventory How will we get there? Making time for students to explore options after leaving high school What is holding us back? 69

Sample Diagnostic Tree (P. Preuss) HSA Math Scores below standard Math Achievement Score in Grade 3 Student Demographics CurriculumInstruction Math Achievement Score in Grade 4 System Processes Organizational Culture Math Achievement Score in Grade 5 70

How does data analysis help in school improvement efforts? My answer: It allows us to see everything that may affect student learning It allows school level leaders identify ways to support student learning It complements classroom (instructional) level data teams It allows us to predict success! 72

Question & Answer 73

Thank you for joining us! A recording of this webinar will be posted on the Standards Toolkit website. If there are any questions, please e-mail: Dewey Gottlieb, Mathematics Specialist Monica Mann, Acting Administrator Petra Schatz, Language Arts Specialist, or Derrick Tsuruda, Science Specialist 74