arthur l. petterway, phd proposal dissertation defense, dr. william allan kritsonis, committee...
DESCRIPTION
Dr. William Allan Kritsonis, PhD Dissertation Committe for Arthur L. Petterway, PhD Program in Educational Leadership, PVAMU, Member of the Texas A&M University System.TRANSCRIPT
MAY 2006 1
A Mixed-Methods Analysis of the Impact of High Stakes Testing on
English Language Learners in Major Urban High Schools in
Texas
A Dissertation Proposalby
Arthur L. Petterway
William Allan Kritsonis, PhDDissertation Committee Member
MAY 2006 2
Proposal Defense Format
I. Purpose of StudyII. Research Questions(4)III. Hypotheses(2)III. Significance of the StudyIV. Review of LiteratureV. Research Design
MAY 2006 3
Purpose of the Study
The purpose of the study is two-fold:
1. Determine whether there is a significant relationship between the ratio of ESL students taking the TAKS test, relative proportion of the school’s faculty who are certified to teach English as a first language and the Grade 10 TAKS in Reading/English Language Arts and Mathematics.
MAY 2006 4
Purpose of the Study
2. Explore what certified ESL teachers, non – certified ESL teachers who teach ELLs, campus administrators, and district ESL personnel view as the impact that high stakes standardized assessments have on ELLs, ELL curriculum and instruction.
MAY 2006 5
Research Questions(1): What is the effect of high
stakes standardized assessments and their impact on ELLs’ motivation, state of mind or behavior / view of the test?
MAY 2006 6
Research Questions(2):• What is the influence of
teacher certification status on high stakes standardized assessments on ELLs?
MAY 2006 7
Research Questions(3):
• What is the impact of high stakes standardized assessments on ELLs?
MAY 2006 8
Research Questions(4): Is there a relationship between
TAKS performance in Reading/English Language Arts and Mathematics of 10th Graders and designation of ESL or non-ESL, type of teacher (certified or non-certified to teach ESL) and % of students who do not speak English as a first language ?
MAY 2006 9
Hypotheses(1):
Ho1: There is no statistically significant relationship between the ratio of ESL students taking the TAKS test, relative proportion of the school’s faculty who are certified to teach ESL students and the % of students who do not speak English as a first language and the school’s performance in the Grade 10 TAKS test in English/Language Arts.
MAY 2006 10
Hypotheses(2):
Ho2: There is no statistically significant relationship between the of ratio of ESL students taking the TAKS test, relative proportion of the school’s faculty who are certified to teach ESL students and the % of students who do not speak English as a first language and the school’s performance in the Grade 10 TAKS test in Mathematics.
MAY 2006 11
Significance of the Study:• Expected outcome of the study
will be to provide additional data for standardized assessment writers in regards to biases and to school districts in developing assessments that truly measure learning without the nullifying effects of linguistic and cultural biases.
MAY 2006 12
Significance of the Study
•It will also help to enhance the reliability of standardized assessments as a tool to determine accountability for student performance of English language learners.
MAY 2006 13
Review of Literature
• No Child Left Behind (NCLB)• Adequate Yearly Progress
(AYP)• Limited English Proficient
(LEP) Students• High Stakes/Statewide
Testing
MAY 2006 14
Review of Literature
• No Child Left Behind (NCLB)–Historical Note–Description of the Key Factors
–Expectations for Parents–Response to NCLB
MAY 2006 15
Review of Literature
• No Child Left Behind Act of 2002 (NCLB)–Historical Note: In 2003, the Center of Educational Policy clarified why accountability was not a part of ESEA in 1965:
MAY 2006 16
Review of Literature “At that time the federal role in education was marginal, most state education agencies had very limited authority and capabilities, and local people were extremely wary that more federal aid would bring federal control”(p.iv).
MAY 2006 17
Review of Literature
• No Child Left Behind Act of 2002 (NCLB)-Description of the Key Factors (Rosenbusch, 2005)(4):
1. Accountability2. Testing3. Teacher Quality4. Scientifically-Based Research
MAY 2006 18
Review of Literature
• Accountability to Parents– Beginning in 2005, grades 3 – 8 must be tested
in Math and English– By the end of SY 2005 – 2006, teachers must
be “highly qualified”– Number of students achieving state standards
must increase each year until reaching 100% at the end of 12 years
– Schools must notify parents if their child’s school is targeted for improvement
MAY 2006 19
Review of Literature
• Expectations for Parents (cont.)– If a school targeted for improvement fails after
two years, parents may choose to transfer their child to another school or enroll in free tutoring.
Source: collegeboard.com
MAY 2006 20
Review of Literature
• Response to NCLB– Controversy (Rosenbusch, 2005)– Majority of Americans believe in local control of
schools (Rose & Gallup, 2003)– Many believe Math and English testing not
sufficient to give accurate picture of the school (Rose & Gallup, 2003)
– Could result in narrowing of the curriculum and “sorting of students” (Marshak, 2003, p.229)
– NEA says focus on punishment is an obstacle (National Education Association, n.d.)
MAY 2006 21
Review of Literature
• Adequate Yearly Progress (AYP)
–Purpose and Support to NCLB
–Changes and Updates
MAY 2006 22
Review of Literature
• Limited English Proficient (LEP) Students–Definition of English Language Learners (ELLs)
– Issues and Other Considerations of LEP
MAY 2006 23
Review of Literature
• High Stakes/Statewide Testing– Principles of Testing Programs– Accountability in Testing– Effects of High Stakes Testing
on Student Motivation– Other Considerations of
Assessment on Testing
MAY 2006 24
Review of Literature• Identification of ELLs
– Home Language Survey– Proficiency tests
(Abedi, 2004b)
– Assessment instruments may not be sufficient
– Leads to inconsistency
(Zehler, Hopstock, Fleischman & Greniuk, 1994)
MAY 2006 25
Review of Literature
• Strategies to improve LEP instruction– Improve classification methods– Improve monitoring– Improve teacher quality– Include redesignated LEP students as
part of the LEP subgroup
(Abedi, 2003)
MAY 2006 26
Review of Literature
• High Stakes Statewide Testing– States required to administer Reading and
Math tests annually in grades 3 – 8 and during one year in high school starting in 2005 – 2006 (National Center for Education Statistics, 2002)
– States must meet AYP goals (Abrams & Madaus, 2003)
– Use of testing to change pedagogical priorities has a long history (Madaus & O’Dwyer, 1999)
MAY 2006 27
Principles of Testing Programs
MAY 2006 28
Review of Literature
• Principle 1–The power of tests is a
perceptual phenomenon
–All parties must believe the results are important
(Airasian, 1988)
MAY 2006 29
Review of Literature
• Principle 2– The more a quantitative social indicator
is used for social decision making, the more likely it will be to distort and corrupt the social process it is intended to monitor(Amrein & Berliner, 2002; Haladyna, Nolen & Haas, 1991; Klein, Hamilton, McCaffrey & Stecher, 2000; Linn, 1998)
MAY 2006 30
Review of Literature• Principle 3
– If important decisions are based on test results, then teachers will teach to the test.
(Jones et al., 1999; Madaus, 1991; Madaus, 1991; McMillan, Myran & Workman, 1999; Pedulla et al., 2003; Stecher, Barron, Chun & Ross, 2000)
MAY 2006 31
Review of Literature
• Principle 4– In every setting where high-stakes
tests operate, the exam content eventually defines the curriculum
(Herman and Golan, n.d.; Hoffman, Assaf, & Paris, 2001; Jones et al., 1999; Pedula et al., 2003)
MAY 2006 32
Review of Literature
• Principle 5– Teachers pay attention to the form
of the questions of high-stakes tests and adjust their instruction accordingly.
(Taylor, Shepard, Kinner & Rosenthal, 2003; Kortz, Barron, Mitchell, & Keith, 1996a)
MAY 2006 33
Review of Literature
• Principle 6– When test results are the arbiter of
future education or life choices, society treats test results as the major goal of schooling.
(Holmes, 1911, p.128; Edwards, 2003)
MAY 2006 34
Review of Literature
• Principle 7–A high-stakes test transfers
control over the curriculum to the agency that sets or controls the exam.
(Myers, 2003)
MAY 2006 35
Review of Literature
• Accountability in Testing– Cut-off scores are arbitrary (Horn,
Ramos, Blumer & Madaus, 2000)
– Test scores are fallible (Rhoades & Madaus, 2003)
– No test can be truly comprehensive (Harlow & Jones, 2003)
MAY 2006 36
Review of Literature
• Effects of High-Stakes Testing on Student Motivation and Learning– Evidence shows that such tests actually
decrease student motivation and increase the number of students who leave school early.
(Arein & Berliner, 2003)– Attaching high stakes to tests apparently
obstructs students’ paths to becoming lifelong, self-directed learners and alienates students.
(Sheldon & Biddle, 1998)
MAY 2006 37
Review of Literature
• Effects of High-Stakes Testing on Student Motivation and Learning (cont.)– Many researchers hold high-stakes testing at
least partly to blame for climbing dropout rates.
(Rothstein, 2002; Jacob, 2001; FairTest & Massachusetts CARE, 2000)
– More teenagers are exiting school early to earn a General Educational Development (GED) credential
(Murnane, Willett, & Tyler, 2000)
MAY 2006 38
Review of Literature
• Effects of High-Stakes Testing on Student Motivation and Learning (cont.)– May increase drop-out rate when promotion to
next grade hinges on passing state exams
(Goldschmidt & Wang, 1999)– Results in a high percentage of minority and
low socio-economic background students being retained (McNeil, 2000; Haney, 2000, 2001; Klein, Hamilton, McCaffey & Stecher, 2000; Yardley, 2000; Fisher, 2000)
MAY 2006 39
Review of Literature
• Other Considerations of Assessment and Testing– Can affect the number of students, especially
black and Hispanic students, who are classified as Special Education
(Thurlow, Neilson, Tellucksingh,& Ysseldyke, 2000; Haney, 2000; D’Emilio, 2003, June; Zehler, Fleischman, Hopstock, Pendzick, & Stepherson, 2003)
MAY 2006 40
Review of Literature
• Other Considerations of Assessment and Testing (continued)– Can negatively affect teacher and student
morale (Anderson, 2004; Flores & Clark, 2003)– Can decrease student motivation (Lane &
Stone, 2002)– May increase drop-out rate for English
Language Learners ( Hood, 2003; Anderson, 2004; Barro & Kolstad, 1987; Kaufman, alt & Chapman, 2001)
MAY 2006 41
Review of Literature
• Other Considerations of Assessment and Testing (continued)– May contribute to teacher burnout (Hinde,
2003)– Vitally connected to socio-cultural, economic
and psychological issues (solano-Flores & Trumbull, 2003
– Can change the way in which teachers and students interact (Cheng, 1999)
MAY 2006 42
Review of Literature
• Other Considerations of Assessment and Testing (continued)– Requires administrators, teachers
and students to be motivated and invested in demonstrating achievement in order to be successful (Lane and Stone, 2002)
MAY 2006 43
Research Design
• Research Methodology– Descriptive Comparative Research
– Triangulation (open-ended questionnaire, focus groups, and interviews)
– Explanatory Design
– Correlational Research
MAY 2006 44
A Mixed Methods Analysis of Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas
QUANTITATIVE
QUALITATIVE
Explanatory Design
Predictors% ELLs passing
% Non-ELLspassing
% Non-Certified passing
Views/Opinions
Administrators
TeachersDistrict
Personnel
Dependent Variable
Grade 10
TAKS
Reading/ELA &Math
Impact of
Statewide
Testing
On
ELLs
Student
Performance
ESL
Curriculum
And
Instruction
Multiple Regression
MAY 2006 45
Research Design
• Correlational Research– Independent Variables:
1. The ratio of ESL students taking the TAKS test2. The relative proportion of the school’s faculty who are certified to teach ELL3. The percentage of students who do not
speak English as a first language– Dependent Variable:
10th grade Reading/English Language Arts and Mathematics TAKS scores
MAY 2006 46
Research Design• Subjects of the StudyQuantitative:
All major high schools in selected major urban school districts in Texas
MAY 2006 47
Research Design
• Subjects of the StudyQualitative:
Per SchoolTotal
1. ESL Teachers 3 302. Non-Certified ESL 3 30
Teachers1. Principals 1 102. Assistant Principals 2 203. District ESL Personnel 8
Total 98
MAY 2006 48
Research Design• Pilot Study - Qualitative
– 2 HISD schools
Three basic considerations:1. Administer pre-test under conditions
comparable to those anticipated in the final study.
2. Analyze results to assess the effectiveness of the trial questionnaire.
3. Make additions, deletions, and/or modifications to the questionnaire.
(Isaac and Michael, 1995)
MAY 2006 49
Research Design• Instrumentation
Quantitative:
The records section of TEA
Qualitative:
• Open-ended questionnaire www.apetterway.speedsurvey.com
• Focus groups
• Interviews
MAY 2006 50
Research Design
• Reliability and Validity – Qualitative
The triangulation method will involve (1) the analysis of the quantitative data, (2) collation of data from the on-line questionnaire, and (3) interviews/focus groups.
MAY 2006 51
Research Design• Data Analysis – Quantitative
Descriptive and Comparative Statistics will be used to calculate the means and standard deviations between the variables.
MAY 2006 52
Research Design
• Data Analysis – QuantitativeCorrelational Research
Two separate multiple regressions will be computed:
y=a+b1x1+b2x2+b3x3
– Reading/English Language Arts – Mathematics
MAY 2006 53
Research Design• Data Analysis – Quantitative
Correlational Research
SPSS computations:
1. A Pearson r correlation coefficient will be calculated
2. Multiple R and R squared
3. Degree of freedom, the F-value, and the level of significance
4. Regression formula for predictability
MAY 2006 54
Research Design
Data Analysis – Qualitative
The information gathered from the qualitative portion will be organized under different categories in a frequency table. Percentages will be calculated and listed in descending order.
MAY 2006 55
A Mixed Methods Analysis of Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas
QUANTITATIVE
QUALITATIVE
Explanatory Design
Predictors% ELLs passing
% Non-ELLspassing% Non-
Certified passing
Views/Opinions
Administrators
TeachersDistrict
Personnel
Dependent Variable
Grade 10
TAKS
Reading/ELA &Math
Impact of
Statewide
Testing
On
ELLs
Student
Performance
ESL
Curriculum
And
Instruction
Multiple Regression