the purpose and design of the special education ...the purpose and design of seep about the authors:...

36
Special Education Expenditure Project The Purpose and Design of the Special Education Expenditure Project Report 00 December 2003 Prepared by: Jay G. Chambers Thomas B. Parrish Jamie L. Shkolnik Roger Levine Freya E. Makris Submitted to: United States Department of Education, Office of Special Education Programs

Upload: others

Post on 05-Sep-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

Special Education Expenditure Project

The Purpose and Design of the Special Education Expenditure Project Report 00 December 2003

Prepared by: Jay G. Chambers Thomas B. Parrish Jamie L. Shkolnik Roger Levine Freya E. Makris

Submitted to: United States Department of Education,

Office of Special Education Programs

Page 2: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

About the authors:

Jay G. Chambers is the Project Director for the Special Education Expenditure Project (SEEP). Dr. Chambers is also a Senior Research Fellow and a Managing Director of the Business Development Committee on Economic Indicators and Education Finance within the Education Program at the American Institutes for Research (AIR). He served as a member of the President’s Commission on Excellence in Special Education and served on the Task Force on Finance and on Systems Administration. During 2001 and 2002, Dr. Chambers was President of the American Education Finance Association and a consulting professor at Stanford University’s School of Education. He is a nationally recognized expert in school finance and educational cost analysis. Thomas B. Parrish is the Director of the Center for Special Education Finance (CSEF), and is also Deputy Director of the Education Program at AIR. Dr. Parrish specializes in education policy, with special expertise in analyzing the costs of, and fiscal policies for, special education programs. He has appeared before numerous legislative bodies and written extensively on issues relating to special education finance and state funding formulas. Jamie L. Shkolnik is a Senior Research Scientist in the Education Program at AIR, and earned her Ph.D. in Economics from the University of California at San Diego. Her areas of expertise are education finance, cost analysis, school resource allocation, and econometrics. She was the Project Manager for Analysis for SEEP and for several state SEEP evaluations, and also served as the Deputy Project Director for the Evaluation of Class Size Reduction in California. Roger Levine has been actively involved with social science data collection and analysis for over 20 years. He established and is the Director of the American Institutes for Research’s Cognitive Survey Laboratory. He has worked on numerous task order contracts and other sponsored research projects, for the U.S. Departments of Education, Housing and Urban Development, Transportation, Labor, and the National Institutes of Health; for various state agencies, foundations, other public organizations and private organizations. Freya E. Makris is a Research Associate in the Education Program at AIR. She has contributed to numerous evaluations and reports on issues relating to class size reduction, arts education, and special education finance. She also assisted with data collection for SEEP.

American Institutes for Research, Page i

Page 3: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Acknowledgements

Primary support for this research comes from the U.S. Department of Education, Office of Special Education Programs (OSEP). The authors wish to express their appreciation for the guidance and suggestions of Louis Danielson in his capacity as Director, Research to Practice Division, Office of Special Education Programs, and Scott Brown in his capacity as Project Officer for the Special Education Expenditure Project (SEEP). The authors would like to express a special note of thanks to Jenifer Harr and Jean Wolman for their conceptual and editorial support. Their contributions have made this a better report.

The following is a comprehensive list of all the individuals who have contributed to the SEEP during the course of the past two years and their various capacities with the project. Project Design Team: Jay Chambers (Project Director), Tom Parrish (Director, Center for Special Education Finance), and Roger Levine (Task leader for Sample Design). Senior Consultants: Margaret McLaughlin, Institute for the Study of Exceptional Children and Youth, University of Maryland; Margaret Goertz, University of Pennsylvania, Philadelphia, Pennsylvania. Technical Work Group: Stephen Chaikind, Gallaudet University; Doug Gill, Office of Superintendent of Public Instruction, Washington State; Diane Gillespie, Virginia Tech, Blacksburg, Virginia; Bill Hartman, Pennsylvania State University, University Park, Pennsylvania; John Herner, Division of Special Education, Ohio Department of Education; Donald Kates, Georgetown University, Child Development Center; Brian McNulty, Adams County School District 14, Commerce City, Colorado; Jim Viola, New York State Education Department. State Directors of Special Education in the nine extended sample states: Alabama, Mabrey Whetstone, State Director, and Barry Blackwell, liaison; Delaware, Martha Brooks, State Director and Debbie Stover, liaison; Indiana, Robert Marra, State Director, and Hank Binder, liaison; Kansas, Bruce Passman, State Director, and Carol Dermyer, liaison; Missouri, Melodie Friedebach, State Director, and Bill Daly, liaison; New Jersey, Barbara Gantwerk, State Director, and Mari Molenaar, liaison; New York, Larry Gloeckler, State Director, and Inni Barone, liaison; Ohio, Ed Kapel, State Director; Rhode Island, Tom DiPaola, State Director, and Paul Sherlock, member, Rhode Island legislature. Managers of data collection and processing: James Van Campen, Rafi Youatt, Marie Dalldorf, and Kristi Andes Peterson. Data collectors and support teams include the following: Team leaders: Peg Hoppe, Michael “Chad” Rodi, Jennifer Brown, Andy Davis, Leslie Brock, Jeanette Wheeler, and Jean Wolman. Team members: Mary Leopold, Claudia Lawrence, Patrice Flach, Bette Kindman-Koffler, Brenda Stovall, Danielle Masursky, Ann Dellaira, Eden Springer, Jack Azud, Nancy Spangler, Melania Page-Gaither, Raman Hansi, Chris White, Lori Hodge, Freya Makris, Megan Rice, Amynah Dhanani, Melinda Johnson, Carmella Schaecher, Iby Heller, Hemmie Jee, and Irene Lam. Data collection support team: Emily Campbell, Ann Win, Sandra Smith and Diana Doyal. Data analysis team: Maria Perez, Gur Hoshen, Jamie Shkolnik, Amynah Dhanani, Irene Lam, Bob Morris, and John DuBois. Report production team: Phil Esra, Jenifer Harr, Jamie Shkolnik, Jean Wolman, and Michelle Bullwinkle.

American Institutes for Research, Page ii

Page 4: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

SEEP Reports This document is a part of a series of reports based on descriptive information derived from the Special Education Expenditure Project (SEEP), a national study conducted by the American Institutes for Research (AIR) for the U.S. Department of Education, Office of Special Education Programs (OSEP). SEEP is the fourth project sponsored by the U.S. Department of Education and its predecessor, the Department of Health, Education and Welfare, in the past 40 years to examine the nation’s spending on special education and related services. See Kakalik, Furry, and Carney (1981), Moore, Strang, Schwartz, and Braddock (1988), and Rossmiller, Hale, and Frohreich (1970). The SEEP reports are based on analyses of extensive data for the 1999-2000 school year. The SEEP includes 23 different surveys to collect data at the state, district, and school levels. Survey respondents included state directors of special education, district directors of special education, district directors of transportation services, school principals, special education teachers and related service providers, regular education teachers, and special education aides. Survey responses were combined with other requested documents and data sets from states, schools, and districts to create databases that represented a sample of approximately 10,000 students with disabilities, more than 5,000 special education teachers and related service providers, approximately 5,000 regular education teachers, more than 1,000 schools, and well over 300 local education agencies. The series of SEEP reports will provide descriptive information on the following issues:

• What are we spending on special education services for students with disabilities in the U.S.?

• How does special education spending vary across types of public school districts? • What are we spending on due process for students with disabilities? • What are we spending on transportation services for students with disabilities? • How does education spending vary for students by disability and what factors explain

differences in spending by disability? • What role do functional abilities play in explaining spending variations for students with

disabilities? • What are we spending on preschool programs for students with disabilities? • Who are the teachers and related service providers who serve students with disabilities? • How are special education teaching assistants used to serve students with disabilities? • What are we spending on special education services in different types of schools? • How does special education spending vary across states classified by funding formula,

student poverty, special education enrollment levels, and income levels?

American Institutes for Research, Page iii

Page 5: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Table of Contents I. Introduction ............................................................................................................................. 1 II. Research Questions ................................................................................................................. 3 III. Design of the Study ................................................................................................................. 4

Sample Design .................................................................................................................... 4 State Sample........................................................................................................................ 4 School District Sample ....................................................................................................... 4 School District Central Office Staff Sample....................................................................... 5 School Sample .................................................................................................................... 5 Teacher and Aide Samples.................................................................................................. 6 Student Samples.................................................................................................................. 6

IV. Data Collection Instruments and Procedures .......................................................................... 8 State-Level Data Collection................................................................................................ 8 District-Level Data Collection............................................................................................ 8 School-Level Data Collection............................................................................................. 9 State and National Databases............................................................................................ 11

V. Response Rates and Data Acquisition Rates ......................................................................... 12 VI. Analysis Methods .................................................................................................................. 14 The Resource Cost Model..................................................................................................14 Student Resource Cost Database .......................................................................................14 References..................................................................................................................................... 19 Exhibits

Exhibit 1. Response Rates by Questionnaire Type................................................................ 11 Exhibit 2. The Student Resource Cost Database ....................................................................15 Exhibit A-1. Response Rates by School Size (Enrollment)................................................A-1 Exhibit A-2. Response Rates by District Poverty Level ....................................................A-2 Exhibit A-3. Response Rates by Proportion Minority Students in District...........................A-2 Exhibit A-4. Response Rates by Urbanicity.....................................................................A-2 Exhibit A-5. Response Rates by Proportion of Students with IEPs .....................................A-3 Exhibit A-6. Representation Rates by School Size (Enrollment) .......................................A-3 Exhibit A-7. Representation Rates by District Poverty Level.............................................A-4 Exhibit A-8. Representation Rates by Proportion Minority Students in District ...................A-4 Exhibit A-9. Representation Rates by Urbanicity .............................................................A-4 Exhibit A-10. Representation Rates by Proportion of Students with IEPs .............................A-4 Appendices

Appendix A Response Bias Analysis ......................................................................................... A-1 Appendix B Development of Weights.........................................................................................B-1

American Institutes for Research, Page iv

Page 6: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

I. Introduction In 1973, Congress passed Section 504 of the Rehabilitation Act, affirming the principle that children with disabilities should be educated in public schools. Two years after this legislation, in 1975, Congress passed the Education for All Handicapped Children Act (EAHCA) and expanded the federal special education financial commitment into a sizable program of grants to the states. Known as Public Law 94-142, and later as the Individuals with Disabilities Education Act (IDEA), this legislation guaranteed a free, appropriate public education to each child with a disability in every state and locality across the country. IDEA articulated for the first time a national mission to improve access to education for children with disabilities. Changes in the law since 1975 have included efforts to improve how children with disabilities are identified and educated, provisions to evaluate the success of these efforts, and due process protection for children and their families. Interest in special education finance policy and the implications of IDEA have grown across the states, as well as at the federal level, in recent years. According to a survey conducted by the Center for Special Education Finance in 1999-2000, 28 of 46 reporting states have reformed the way they fund special education during the past six years and 21 states are considering future formula changes. Eleven of the reporting states have both reformed the way they fund special education in the past six years and are already considering future reforms. As 38 of 46 reporting states have either already reformed funding, are considering future reforms, or both, it is clear that the current context of special education funding is one of frequent change. In spite of increased attention, special education expenditure data have generally been lacking. Until very recently, the most current national study of special education expenditures was the one conducted by Decision Resources Corporation during the 1985-86 school year (Moore, Strang, Schwartz, and Braddock, 1988). Reflecting the need for updated, comprehensive, and accurate information regarding special education expenditures and their relationship to regular education, the reauthorization of IDEA in 1997 mandated studies to measure and evaluate the impact of IDEA and the effectiveness of state efforts to provide a free, appropriate public education to all children with disabilities (per Sections 618 and 674 of Part B). Under this authorization, with intentions of informing the 2002 reauthorization cycle, the Office of Special Education Programs (OSEP), U.S. Department of Education, funded the Special Education Expenditure Project (SEEP)—a national study conducted by the American Institutes for Research (AIR), and the first of its kind in 15 years. SEEP is the fourth project sponsored by the U.S. Department of Education and its predecessor, the Department of Health, Education, and Welfare, in the past 40 years to examine the nation’s spending on special education and related services. For information on these previous studies, see Rossmiller, Hale, and Frohreich (1970); Kakalik, Furry, and Carney (1981); and Moore et al. (1988). This document describes the purpose and design of SEEP, and is part of a series of reports and analyses derived from the study. The SEEP reports are based on analyses of

American Institutes for Research, Page 1

Page 7: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

extensive data for the 1999-2000 school year. SEEP includes 23 different survey questionnaires to collect data from states, districts, and schools.1 Questionnaire respondents included state directors of special education, district directors of special education, district directors of transportation services, district central office staff, school principals, special education teachers and related service providers, regular education teachers, and special education aides. Information about special education students was collected through questionnaires completed by special education teachers and related service providers. Questionnaire responses were combined with other requested documents and data sets from states, districts, and schools to create databases that included a sample of nearly 10,000 special education students, more than 5,300 special education teachers and related service providers, approximately 5,300 regular education teachers, over 1,050 schools, and 350 local education agencies.

1 Copies of the SEEP survey questionnaires may be obtained from the SEEP website at http://csef.air.org/about_seep_instruments.php

American Institutes for Research, Page 2

Page 8: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

II. Research Questions To address the need for updated information related to special education expenditures in the nation, SEEP was designed to investigate the following research questions:

• What are we spending on special education services for special education students in the U.S.?

• How does special education spending vary across types of public school districts?

• What are we spending on due process for special education students?

• What are we spending on transportation services for special education students?

• How does education spending vary for students by disability and what factors explain differences in spending by disability?

• What role do functional abilities play in explaining spending variations for special education students?

• What are we spending on preschool programs for special education students?

• Who are the teachers and related service providers who serve special education students?

• How are special education teaching assistants used to serve special education students?

• What are we spending on special education services in different types of schools?

• How does special education spending vary across states classified by funding formula, student poverty, and income levels?

This report is devoted to describing the purpose and design of the study. It details the design of the sample, and describes the state, district, district central office staff, school, teacher, special education aide, and special education student samples. Discussion of the data collection instruments and procedures follows. Response rates and data acquisition rates are also presented, as are descriptions of the resource cost model and the analysis methods used.

American Institutes for Research, Page 3

Page 9: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

III. Design of the Study Sample Design

Data were collected from a nationally representative sample of districts and schools in 50 states and the District of Columbia. The study used a stratified random sample of 1,769 schools in 448 school districts and 30 affiliated intermediate education units (IEUs). Twenty state-run special education schools were also randomly selected for participation. It should be noted that the original national sample included about half of the total number of schools, districts, and IEUs ultimately included in the sample. However, early in the planning stages for the study, the SEEP study team sent letters to state directors of special education offering to gather data that was representative at the state level and could be used for internal state policy analysis. As a result, nine states contracted with AIR for extended participation in the study.2 The State Departments of Education in these nine states requested that the SEEP study team survey additional districts beyond the selected sample and provide an individual state-level analysis. This expanded sample comprises about half of the overall SEEP sample, and the data collected from this supplemental sample doubled the sample size and are reported with the data collected from the base sample. All data are appropriately weighted so that samples within the nine states are representative of each of the nine states, and the remaining data are weighted to reflected the population in the remaining 41 states. For a complete discussion of how data are weighted, see Appendix B. State Sample

The state-level data collection included all 50 states and the District of Columbia. Questionnaires were send to state directors of special education, and completed state-level questionnaires were received from 41 states. The SEEP study team followed up with telephone calls and emails until data were collected from all 50 states. School District Sample

Samples of school districts were selected within each of the states, with a minimum of two districts in each state.3 Larger numbers of districts were included from larger states. The total sample consisted of 498 regular school districts, IEUs, and state-run special education schools. The sample included 448 regular school districts and 30 IEUs, selected randomly, with the probability of selection proportional to the number of students enrolled in the district. This sample was selected from a sampling frame of more than 14,000 school districts

2 These nine states include Alabama, Delaware, Indiana, Kansas, Missouri, New Jersey, New York, Ohio, and Rhode Island. 3 As Hawaii and the District of Columbia are each considered a single school district, only one school district could be selected from each of these entities.

American Institutes for Research, Page 4

Page 10: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

listed in the Common Core of Data (CCD). Of these 448 districts, 248 districts made up the extended participation sample from the nine contracted states. Districts selected for the national sample were removed from the frame when selecting this extended sample so that they could not be re-selected. In addition, 20 state-run special education schools were treated as individual school districts, so that fiscal data could be collected using the district-level questionnaires. The 20 state-run special education schools were randomly selected from a frame believed to contain all of the state-operated special education schools in the nation. School District Central Office Staff Sample

Within each school district, a sample of up to six certified central office staff was selected. Staff were considered to be central office staff if they had offices in the district central office and only worked in schools on an as-needed basis. Examples of such staff include district-level special education administrators, psychologists, social workers, counselors, nurses, and learning consultants. The district director of special education and up to two psychologists were selected to receive this questionnaire for central office staff, along with three other central office staff who were randomly selected using an alphabetized roster. A total of 2,960 central office staff members were included in the sample. School Sample

Samples of elementary, secondary, and special education schools were selected from among the sampled districts and IEUs. Schools from within the total selected sample of 448 districts and 30 IEUs were classified as being either elementary, secondary, or special education schools. A total of 1,769 elementary schools and secondary schools were chosen. This total includes 50 district-run special education schools and one school from each of the 30 IEUs. To represent the nationwide ratio of elementary schools to secondary schools, roughly two thirds of the selected schools were elementary schools. A minimum of two elementary schools, whenever possible, were selected from each district. Within the regular school districts selected, a sample of 70 schools containing “cluster programs”4 was assembled. These schools were selected to ensure that sufficient numbers of students with low-incidence, high-cost disabilities who are served in regular schools were included in the sample to generate stable estimates. Cluster program schools that did not serve students with low-incidence disabilities, or had already been selected for the sample, were excluded from the sampling frame. If more than one cluster program school remained in a selected district, one elementary and one secondary school were randomly selected.

4 Within a particular district, special education students with certain low-incidence disabilities are clustered into selected non-special education schools in order to take advantage of economies of scale in meeting their specific needs. For example, if a particular school has a cluster program for visually impaired students, visually impaired students from all over the district would attend that school.

American Institutes for Research, Page 5

Page 11: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Teacher and Aide Samples

Samples of regular education teachers, special education teachers and related service providers, and special education aides were chosen from each selected school. All of the special education teachers and related service providers at each selected school were included in the sample. In addition, six regular education teachers at each selected elementary school and nine regular education teachers at each secondary school were randomly selected using an alphabetized roster and a random number generator. Up to four special education aides were also randomly selected from each elementary school in the sample and up to six special education aides were randomly selected from each secondary school in the sample. The total numbers of teachers and aides included in the sample are listed in the Response Rates and Data Acquisition Rates section of this report. Student Samples

Two types of students were sampled: those who were served within the public schools operated by the sample districts or IEUs, and those who were served outside of their local school district in a non-public school or other public agency. The data for the group of special education students served within the public schools come primarily from questionnaires filled out by special education teachers and related service providers. All special education service providers in the selected schools were asked to fill out questionnaires for two students, and instructions were provided to ensure random sampling.5 Students with low-incidence disabilities were over sampled (i.e., a higher-than-proportionate number were included) to ensure adequate sample sizes for these less common disability categories. If the class or caseload of a teacher or related service provider included one or two students with low-incidence disabilities, these students were automatically selected for the sample. If there were more than two such students in the class or caseload, two of them were randomly selected using specific procedures provided in the survey materials. If there were no students with low-incidence disabilities, respondents used the procedures to select a random sample of two students with high-incidence disabilities. Special education students being served outside of their local school district were also included in the sample. District directors of special education randomly selected up to three students placed in nonpublic schools or other public agencies paid for by the school district, selecting from those students with low-incidence disabilities first. The total selected sample of these special education students was 1,520. To prevent the possibility of a student being selected multiple times, the research team developed sample selection procedures so that students were only selected from the most 5 The sample selection procedures were designed to ensure that the service provider most knowledgeable about the student was asked to complete the student questionnaire. The sampling instructions given to teachers are available online at http://csef.air.org/seep_instruments_student.php

American Institutes for Research, Page 6

Page 12: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

restrictive placement possible for any given student. The sample selection procedures were designed to ensure that the service provider most knowledgeable about any student was asked to complete the Special Education Student questionnaire about the student.

American Institutes for Research, Page 7

Page 13: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

IV. Data Collection Instruments and Procedures Data collection was conducted between February and July 2000. All data are for the 1999-2000 school year unless otherwise indicated. Study team members first sought state assistance in notifying sample districts and securing their participation in the study. Districts were also asked to assist in securing the participation of their selected schools. Data collection instruments included survey questionnaires as well as requests for existing documents and materials such as budgets, enrollment reports, and personnel and payroll records.6 To ease respondent burden, data were accepted in whatever format was easiest for the respondent to provide, including electronic files and pre-existing printouts or reports. State-Level Data Collection

A two-part questionnaire was sent to each state director of special education. The first component of the questionnaire requested lists of personnel and non-personnel resources allocated to the special education division within the state department of education. The second component requested information about special education funding to districts and other local educational agencies (LEAs) within the state, which included allocations of federal IDEA funds and state special education funds to each LEA, including regular school districts and any intermediate educational units funded directly by the state or federal government. District-Level Data Collection

The selected districts received a six-part questionnaire. Part I – General Information Part I consisted of a request for general background and enrollment information for the sample districts. These variables were used to classify districts into various categories with regard to size and the composition of student populations with respect to certain special needs characteristics (e.g., race-ethnicity, poverty, and English language proficiency). Part II – Special Education Program Part II of the district questionnaire gathered detailed information about the district special education program, including service arrangements provided to students, special education enrollments, expenditures at the district central office and at the school site, expenditures on preschool special education and on homebound and hospital programs, external special education placements, summer school services for special education students, procedural safeguards, and assessment, as well as federal funds in special education. 6 The survey questionnaires used in this data collection effort are available online at http://csef.air.org/about_seep_instruments.php

American Institutes for Research, Page 8

Page 14: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Part III – Fiscal Information & Request for Documents and Materials Part III of the district questionnaire collected two types of fiscal information: general information on sources and dollar amounts of funding and specific payroll information for the sample schools. Part IV - Transportation Programs in your District Part IV requested information about special versus regular education spending on transportation services, as well as information about the parameters that help local district decision makers decide how to allocate spending among various transportation service components. This questionnaire was designed to collect information about regular education transportation as well as the expenditures transportation for special education students. Central Office Special Education Staff Questionnaire A random sample of up to six certified district central office staff was selected by the district director of special education with the help of a study team data collector. Each selected central office staff member received a questionnaire designed to obtain information about how these personnel allocate their time among certain administrative and support functions such as assessment, due process, litigation, professional development, and eligibility determination. Information About a Special Education Student with an External Placement In addition, the district director of special education selected up to three externally placed special education students, (students placed in nonpublic schools or other public agencies paid for by the school district). A questionnaire regarding the types special education services provided to the student, as well as the tuition paid for these services, was completed for each. Data collected from Part II of this questionnaire were combined with items from the Central Office Staff questionnaire and the Special Education Teacher and Related Service Provider questionnaire, described below, to enter district-level expenditures into the Student Resource Cost Database. The database was used to organize spending on special education by specific types of services and resources. Breakdowns of total district-level expenditures on central office administration, homebound and hospital programs, and summer school programs were generated using this database. School-Level Data Collection

A three-part questionnaire was sent to the principal of each selected school, to be filled out by the principal or the most knowledgeable respondent.

American Institutes for Research, Page 9

Page 15: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Part I – General Information and School Programs Part I of the school-level questionnaire requested general information about the school and school programs, such as the types of services being offered to students, grade levels served, and enrollment composition. Part IIA – School-Level Special Education Programs and Services Part II-A was designed to collect information about special education programs operated within the school. Part IIB – Special Education Programs Operated by the District Office Part II-B was designed to collect information about special education programs operated by the district central office, if such programs existed at the school. Part IIC – Special Education Programs Operated by Intermediate Educational Units Part II-C was designed to collect information about special education programs operated by external agencies, if such programs existed at the school. Part II of the school-level questionnaire was analyzed with payroll and benefit information from Part III of the district-level questionnaire and state personnel data files, as well non-personnel cost information from NCES databases, to develop estimates of expenditures for school administration and support services at the school site (e.g., the principal’s office, instructional support, maintenance and operations) for both regular and special education. Part III – Request for Documents and Materials Part III of the questionnaire was a request for personnel lists and school budgets, which provided detailed breakdowns of expenditures by program, including regular education, programs for English language learners, Title I, gifted education, and special education. Where these documents were not provided, information from state personnel data files and NCES databases was used to supplement the data. Special Education Teacher and Related Service Provider Questionnaire

The study team also collected data from all special education teachers and related service providers assigned to the selected schools. The special education teacher and related service provider questionnaire collected information about a teacher’s employment status, the direct services he or she provides to students, the settings in which he or she provides those services, the disability categories of the students served, assessment, evaluation, Individual Education Program-related activities, education background and experience, and compensation. Regular Education Teacher and Special Education Aide Questionnaires

Samples of regular education teachers and special education teacher aides were surveyed as well. The regular education teacher and special education aide questionnaires also

American Institutes for Research, Page 10

Page 16: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

requested information about employment status, services provided to students, the settings in which services are provided, disability categories of students served, education background and experience, and compensation. Special Education Student Questionnaire

The Information about a Special Education Student Questionnaire was designed to gather information about a student’s background, the nature of his or her disability, the educational services provided, the contexts in which each service is provided, the professionals involved in providing each service, and a measure of student abilities. State and National Databases

Above and beyond the survey data collected directly by AIR, State-level data files on school personnel were also obtained directly from state education departments. These were used to estimate expenditures for salaries and benefits (or rates of pay) where schools and districts did not provide these data. The SEEP study team also obtained enrollment files and fiscal data from some state departments of education, along with fiscal data from NCES, to generate estimates for expenditures on school and district administration. Costs for school administration and ratios of the costs of fringe benefits to total compensation were also gathered from NCES data and used to estimate these costs when necessary.

American Institutes for Research, Page 11

Page 17: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

V. Response Rates and Data Acquisition Rates The response rates for each type of questionnaire at the state, district, and school levels are presented in Exhibit 1 below. The response rates were generally higher for the state- and school-level questionnaires than for the district-level questionnaires. Fiscal data, such as school district budgets, were acquired from some of the districts that did not return the fiscal information (Part III) questionnaire. Note also, that the response rates for the Central Office Staff and Externally Placed Student questionnaires are artificially low. All districts received six central office staff and three externally placed student questionnaires. However, some districts had fewer than six central office staff or fewer than three externally placed students, while response rates are based on the full number of questionnaires sent. Exhibit 1: Response Rates by Questionnaire Type

Questionnaire Type

Sample Size

(Questionnaires Sent)

Completed Questionnaires

Returned

Response

Rate State-Level 50 50 100% District-Level Part I – General Information 498 214 43% Part II – Special Ed. Program 498 231 46% Part III – Fiscal Information 498 202 41% Part IV – Transportation 498 203 41% Central Office Staff 2,960 888 30% Externally Placed Students 1,520 380 25% School-Level Part I – General Information 1,769 885 50% Part II – Special Ed. Program 1,769 859 49% Part III – Request for Documents 1,769 719 41% Regular Education Teacher 10,022 5,475 55% Special Education Teacher/Related Service Provider 11,567 5,445 47%

Special Education Aide 4,772 2,633 55%

Special Education Student Information Forms 23,134 10,271 44%

In schools where one or more teachers returned a special education teacher questionnaire, overall 68 percent of the teachers returned a questionnaire. When school averages are calculated, in the average school with at least one respondent to the special education teacher's questionnaire, 72 percent of the teachers returned a questionnaire. In schools where no teachers responded it is possible and likely that the questionnaires were never distributed. Districts that were unwilling to participate were encouraged, at least, to send in completed Special Education Student Information Forms, as these student questionnaires

American Institutes for Research, Page 12

Page 18: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

form the foundation of the Student Resource Cost Database, which is the centerpiece of the SEEP expenditure analysis. When student questionnaires were returned, but no school or district questionnaires were available, school and district information were estimated based on a combination of available surveys and the data collected from state education departments. The students in the database represent 1,053 schools (out of the total of 1,769 or almost 60 percent of the original school sample) and 350 districts and IEUs (out of the total of 498 or 70 percent of the original district and IEU samples combined) across the U.S. See the following section for a more detailed explanation of the procedures for organizing the database and imputation of missing values. The project team analyzed each data set in the sample for response bias, then developed and applied a system of weights to adjust for non-response as well as to enable generalization from the sample to the population of interest. Appendices A and B describe the response bias analysis and the calculation of the weights in detail.

American Institutes for Research, Page 13

Page 19: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

VI. Analysis Methods The data collected from completed questionnaires were combined with other requested documents and data sets from states, districts, and schools to create a Student Resource Cost database, and were then analyzed using the Resource Cost Model. The Resource Cost Model

To determine the patterns of expenditure on students with disabilities, SEEP uses an “ingredients” approach to data collection and analysis. This approach, referred to as the Resource Cost Model (RCM), organizes detailed information on individual resources according to the services they are designed to provide. These resources include the teachers, related service providers, or paraprofessionals providing these services; the class size or number of students receiving these services at the same time; special equipment; and supplies and materials. Services include classroom instruction, consultation of resource teachers with regular classroom teachers, pullout programs in resource rooms, integrated services provided in regular classrooms to students with special needs, and overall administration and support.7 The RCM requires detailed information on the allocation and utilization of both the personnel and non-personnel resources required to provide education services to students with disabilities. The approach organizes the data collection to address two major questions:

• What specific ingredients (i.e., resources) are used to serve students with disabilities?

• How are these ingredients organized for service delivery?

To estimate expenditures for serving individual students with disabilities, we collected detailed information from individual teachers about the allocation of their time, the students they serve, and the composition of services these students receive. This information was then used to create the SEEP Student Resource Cost Database. Student Resource Cost Database

The SEEP Student Resource Cost Database includes a record of each ingredient used to educate the approximately 10,000 students in the SEEP sample who receive special education services. These ingredients make up all of the special education services and regular education services that each special education student receives. Ingredients differ from student to student and occur in a variety of combinations. For example, one student 7The RCM approach has a substantial history of applications to special as well as to regular education expenditure analysis. Perhaps most importantly, the RCM approach was used for the previous major special education expenditure survey conducted by Decision Resources Corporation (Moore et al., 1988). For a more complete description of the development of a resource cost database, the reader is referred to Chambers and Parrish (1994).

American Institutes for Research, Page 14

Page 20: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

who receives a service (such as a third grade class) might share a regular education teacher with 18 other students and have a personal special education aide. The same service (a third grade class), when provided to another student, might be made up of a different combination of ingredients, such as an interpreter and a special education teacher shared with eight other students. Detailed knowledge of the services provided, the ingredients used to provide these services, and the cost of each ingredient, along with the cost of school and district administration and support, allow for the calculation of the total expenditures required to educate each student. The information contained in the Student Resource Cost Database comes from data collected from the district, school, teacher, and student questionnaires, as well as from state-level and NCES databases. This is illustrated below in Exhibit 2. Questionnaires and existing data sources, such as NCES databases, are shown in the right-hand column in Exhibit 2. Existing data sources are shaded in gray. Examples of the types of information taken from these data sources are in the center column of Exhibit 2, and the student-level, school-level, and district-level expenditures represented in the Student Resource Cost Database are illustrated in the left-hand column of Exhibit 2.

American Institutes for Research, Page 15

Page 21: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Exhibit 2: The Student Resource Cost Database

District-Level Admin Per Pupil (Special Ed)

District-Level Admin Per Pupil (Regular Ed)

School-Level AdminPer Pupil

Tuition, Fees, Other Services Paid by Dist

Transportation Expenditures

# Teachers, Aides, Related Service

Providers

Student Resource Ingredients/Costs Questionnaire/ Cost Database Existing Data Source

Student-Level Expenditures

School-Level Expenditures

District-Level Expenditures

Special Education Student

Special Ed Teacher/ Related Svcs Provider

Special Education Instructional Aide

General Education Teacher

District Part IV

School Parts III ! Rosters ! Master Class Schedules! Budgets

Externally Placed Student

District Parts II – III! Payroll Files

Rates of Pay Salaries/Benefits

Hours/Types of Services

NCES Fiscal Reports

State Personnel Data Systems

District Part III ! Rosters ! Budgets

NCES Benefit Ratios by State

District Part II Summer School Expenditures

American Institutes for Research, Page 16

Page 22: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

The special education student questionnaire was the primary source of information about the specific services received by each student. Data collected through this questionnaire include the number and types of services a student receives, the number of hours a student spends receiving a given service, the full-time equivalents quantity of teacher, aide, or related service provider time involved in providing the service, and the number of other students receiving the service at the same time.8 The hours of classroom services students receive were analyzed in conjunction with the estimated rates of compensation for the various categories of school personnel. Rates of pay were estimated using econometric models based on information provided in the Special Education Teacher/Related Service Provider questionnaire, the General Education Teacher questionnaire, and the Special Education Instructional Aide questionnaire. State level fiscal files available from the National Center for Education Statistics (NCES) were used to estimate benefit rates for school personnel. For a description of the national and state level databases used in this analysis, see the Data Collection Instruments and Procedures section of this report. Information from the student questionnaire was used to determine whether or not individual students received transportation or summer school services. Estimates of per-pupil transportation expenditures were derived using information collected in Part IV of the district questionnaire. Likewise, the average per pupil expenditures for summer school services in each district were calculated using information provided in Part II of the district questionnaire. For districts that did not provide transportation or summer school information, estimates of the per-pupil expenditures on these services were generated using an econometric model. These expenditures were then added to individual student records in the database. The student-level expenditures described above make up the bulk of the expenditure information represented in the database. Another element of the total expenditure for a student includes the school administration costs. Part III of the district questionnaire and Part III of the school questionnaire were combined with information derived from state personnel data systems and the NCES state level fiscal files to estimate expenditures for school administration. The per-pupil expenditures on school administration are calculated by dividing the estimate of total administration costs for the school by the total number of students in the school. A record for per-pupil school administration expenditure is included for each student in the database. Students in the Resource Cost Database have two records for per-pupil district administration expenditures: one for administration expenditures specific to the special education program, and another for overall district administration. Parts II and III of the district questionnaires, respectively, were combined with the NCES state fiscal files to calculate these expenditures, as they contained information about such district administrative costs as general administration, fiscal administration, personnel and

8 The FTE amounts of time include both the hours of direct contact time as well as the hours, where appropriate, of time for preparation, record keeping, consultation, and travel required to provide the direct contact time associated with instruction or related services.

American Institutes for Research, Page 17

Page 23: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

payroll administration, and district maintenance and operations expenses. The district level administrative expenditures specific to the special education program were estimated based on data provided in Part II of the SEEP District Questionnaire. Total expenditures on special education administration was divided by the total number of special education students in the district, and the general district administration expenditures were divided by the total number of students in the district. Information about expenditures and number of students who received homebound or hospital services came from the District Part II questionnaire. These expenditures are not included in the Student Resource Cost Database, but they are a part of the total expenditures to educate students who receive special education services. Therefore, national estimates of expenditures for students who received homebound or hospital services were estimated from district-level expenditures for such services. These records of ingredients and their costs, when taken together, provide a comprehensive picture of expenditures for providing education services to special education students and were used as a basis for many of the analyses presented in the series of SEEP reports.

American Institutes for Research, Page 18

Page 24: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

References Chambers, J.G. (2000, November). Measuring Resources In Education: A Comparison

Of Accounting And The Resource Cost Model Approach. School Business Affairs 66(11), 26-34.

Chambers, J.G. (1997). Measuring Geographic Differences in Public School Costs

(working paper). Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Chambers, J.G., & Parrish, T.B. (1994). Developing a resource cost database. In S.

Barnett & H. Walberg (Eds.), Cost Analysis for Educational Decision Making (pp. 23-44). Greenwich, CT: JAI Press.

Chambers, J.G. & Parrish, T.B. (1986). The AEFP/RCM Documentation for Version 86.

4. Prepared for users of the AEFP/RCM System Computer Simulation Programs. Mountain View, CA: Associates for Education Finance and Planning, Inc.

Chambers, J.G. & Parrish, T.B. (1984). The Development of a Program Cost Model and

a Cost-of Education Index for the State of Alaska: Final Report. Stanford, CA: Associates for Education Finance and Planning, Inc.

Finn, C.E., Jr., Rotherham, A.J., & Hokanson, C.R., Jr. (2001). Rethinking Special

Education for a New Century. Washington, DC: Thomas B. Fordham Foundation and the Progressive Policy Institute.

Kakalik, J., Furry, W., Thomas, M., & Carney, M. (1981). The Cost of Special Education.

Santa Monica, CA: Rand Corporation. Moore, M.T., Strang, E.W., Schwartz, M., & Braddock, M. (1988). Patterns in Special

Education Service Delivery and Cost (ERIC Document Reproduction Service No. ED 303 027). Washington, DC: Decision Resources Corp.

Palmaffy, T. (2001). The Evolution of the Federal Role. Washington, DC: Thomas B.

Fordham Foundation and the Progressive Policy Institute. Parrish, T., Anthony, J., Harr, J., Merickel, A., & Esra, P. (2003). State Special Education

Finance Systems, 1999-2000. Palo Alto, CA: American Institutes for Research, Center for Special Education Finance.

Rossmiller, R.A., Hale, J.A., & Frohreich, L. (1970). Educational Programs for

Exceptional Children: Resource Configurations and Costs. Madison, WI: University of Wisconsin, Department of Educational Administration.

U.S. Department of Education, Office of Special Education Programs. (2001). Twenty

Five Years of Progress. Washington, DC: Author. Retrieved 12/8/03. from http://www.ed.gov/policy/speced/leg/idea/history.pdf

American Institutes for Research, Page 19

Page 25: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

.

American Institutes for Research, Page 20

Page 26: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Appendix A Response Bias Analysis

A thorough response bias analysis was conducted on each data set in the sample to determine differences between questionnaire respondents and non-respondents and to ensure that the results reported in this study are not based upon a biased sample. Analysis of the sample revealed no evidence of response bias in the district or special education school samples. Responding and non-responding school districts and special education schools were compared across a variety of demographic characteristics, including percentage of minority students enrolled, poverty level, urbanicity, and proportion of students with Individual Education Programs (IEPs). The study team found no statistically significant differences between those who responded and those who did not with respect to these characteristics. A response bias analysis conducted on the elementary and secondary school samples, however, did reveal some evidence of response bias, particularly for elementary schools. Chi-square tests revealed statistically significant differences between respondents and non-respondents with respect to several characteristics. Below are descriptions of the response bias discovered at the elementary and secondary school levels. Following the presentation of the response bias analysis are the results of a second response bias analysis, which compares the characteristics of schools represented in the Student Resource Cost Database with schools that are not represented in the database.9 Response Rates

As illustrated below in Exhibit A-1, a comparison of elementary school response rates by school size, for which the sample of selected schools was divided into thirds by enrollment, indicates that small and medium-sized schools were more likely to respond (with 53 and 56 percent response rates, respectively) than large schools (46 percent). This was not the case at the secondary level, however, as there were no statistically significant differences in response rates by school size.

Exhibit A-1: Response Rates by School Size (Enrollment) Small Medium Large P Value for Chi-

Square Statistic Elementary Schools 53% 56% 46% 0.0332 Secondary Schools 45% 47% 49% 0.8349

Elementary schools in high poverty districts were less likely to respond. Fifty-six percent of the selected elementary schools in districts with low or medium levels of poverty participated in the study, compared to 43 percent of elementary schools in high poverty districts. Schools in the 9 A school can be represented in the Student Resource Cost Database if a student-level questionnaire was returned but school-level questionnaires were not returned from that student’s school. In these cases, school-level data could often be imputed.

American Institutes for Research, Page A-1

Page 27: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

sample of secondary schools, however, responded at similar rates regardless of poverty level. This is illustrated below in Exhibit A-2. Again, there were no statistically significant differences at the secondary level. Note that the SEEP study team divided the sample of selected schools into thirds by the number of students receiving free lunch.

Exhibit A-2: Response Rates by District Poverty Level Low Medium High P Value for Chi-

Square Statistic Elementary Schools 56% 56% 43% 0.0021 Secondary Schools 47% 47% 48% 0.9696

Elementary schools in districts with low and medium proportions of minority students responded at similar rates of 59 and 58 percent respectively, but only 25 percent of schools in districts with high proportions of minority students responded. As shown below in Exhibit A-3, a similar discrepancy was found in the secondary school sample. Fifty-three and 50 percent of the secondary schools in districts with small and medium proportions of minority students responded, though only 35 percent of the secondary schools in districts with large proportions of minority students responded. Schools were considered to have a high proportion of minority students if more than 50 percent of the students enrolled in the district are members of minority groups. Schools with fewer than 10 percent minority students were considered to have a low proportion of minority students.

Exhibit A-3: Response Rates by Proportion of Minority Students in District Small Medium Large P Value for Chi-

Square Statistic Elementary Schools 59% 58% 25% <0.0001 Secondary Schools 53% 50% 35% 0.0056

As Exhibit A-4 illustrates, a comparison of school response rates by urbanicity indicates that a lower percentage of urban schools participated in the study. The response rate for urban schools (48 percent) was statistically significantly lower than the rates at rural (59 percent) and suburban (57 percent) schools. The same statistically significant pattern was found for secondary schools.

Exhibit A-4: Response Rates by Urbanicity Rural Suburban Urban P Value for Chi-

Square Statistic Elementary Schools 59% 57% 48% 0.0090 Secondary Schools 56% 50% 39% 0.0061

Response rates by proportion of students in the district with IEPs were also compared for the elementary and secondary school samples (see Exhibit A-5). The higher the percentage of students with IEPs at the district, the more likely a school is to respond. Sixty-three percent of elementary schools in districts with a large proportion of students with IEPs participated, whereas only 40 percent of elementary schools in districts with a small proportion of students with IEPs participated. Secondary schools with a large proportion of students with IEPs appear

American Institutes for Research, Page A- 2

Page 28: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

to have responded at a slightly higher rate, though this difference was not significant at the 5 percent level. Note that both samples of schools were divided into thirds by proportions of students with IEPs.

Exhibit A-5: Response Rates by Proportion of Students with IEPs Small Medium Large P Value for Chi-

Square Statistic Elementary Schools 40% 56% 63% <0.0001 Secondary Schools 41% 49% 50% 0.1524

Representation Rates

For the response bias analysis discussed above, characteristics of schools that completed and returned questionnaires were compared to characteristics of schools that did not. The analysis presented in this series of reports, however, does include many schools from which school-level questionnaires were not completed and returned. These schools are represented in the Student Resource Cost Database because student-level questionnaires were completed and returned and school and district-level data could be imputed. A second response bias analysis was conducted in order to compare schools represented in the student database with schools not represented in the student database. For this second response bias analysis, the definition of participating schools was expanded to include schools for which student-level surveys were received, and school and district-level data were imputed. The results of this second response bias analysis are presented below. As Exhibit A-6 indicates, the elementary schools represented in the Resource Cost Database are represented at equal rates with respect to school size. Small, medium, and large elementary schools were all equally likely to participate. In the secondary school sample, however, medium and large size schools were more likely to participate.

Exhibit A-6: Representation Rates by School Size (Enrollment) Small Medium Large P Value for Chi-

Square Statistic Elementary Schools 62% 61% 60% 0.9179 Secondary Schools 49% 61% 65% 0.0309

High poverty elementary schools were less likely to participate than were elementary schools with low and medium levels of poverty. While 54 percent of elementary schools in high poverty districts participated, 70 percent of elementary schools in low poverty districts participated. A similar trend was observed in the secondary school sample, though it was not significant at the 5 percent level.

American Institutes for Research, Page A- 3

Page 29: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Exhibit A-7: Representation Rates by District Poverty Level Low Medium High P Value for Chi-

Square Statistic Elementary Schools 70% 62% 54% 0.0002 Secondary Schools 68% 62% 57% 0.1037

As shown in Exhibit A-8, schools in both the elementary and secondary school samples in districts with small and medium proportions of minority students were more likely to participate than schools in districts with large proportions of minority students. Elementary and secondary schools with large proportions of minority students participated at rates of 48 and 53 percent, respectively, while 66 to 67 percent of schools with small and medium proportions of minority students participated.

Exhibit A-8: Representation Rates by Proportion of Minority Students in District Small Medium Large P Value for Chi-

Square Statistic Elementary Schools 67% 67% 48% <0.0001 Secondary Schools 66% 66% 53% 0.0122

As Exhibit A-9 illustrates, in both the elementary and secondary school samples, rural and suburban schools were more likely than urban schools to participate. Sixty-nine percent of rural elementary schools, and 70 percent of rural secondary schools participated, compared to 53 percent of urban elementary schools, and 54 percent of urban secondary schools.

Exhibit A-9: Representation Rates by Urbanicity Rural Suburban Urban P Value for Chi-

Square Statistic Elementary Schools 69% 63% 53% 0.0004 Secondary Schools 70% 64% 54% 0.0119

Elementary and secondary schools in districts with large proportions of IEPs were more likely to participate than schools in districts with medium or small proportions of students with IEPs. Elementary schools in districts with small proportions of students with IEPs participated at a rate of 55 percent, whereas elementary schools in districts with large proportions of students with IEPs participated at a rate of 71 percent. This difference was not as pronounced at the secondary level. Sixty-eight percent of schools in districts with large proportions of students with IEPs participated, and 61 percent of schools in districts with small proportions of students with IEPs participated. In the secondary school sample, schools with medium proportions of students with IEPs participated at the lowest rate, 56 percent.

Exhibit A-10: Representation Rates by Proportion of Students with IEPs Small Medium Large P Value for Chi-

Square Statistic Elementary Schools 55% 58% 71% <0.0001 Secondary Schools 61% 56% 68% 0.0689

American Institutes for Research, Page A- 4

Page 30: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Adjusting for Non-response Bias

Whenever questionnaire response rates are less than 100 percent, there is a possibility of response bias. This bias can be ameliorated by non-response adjustment factors. Accordingly, a system of weights was developed and applied in the analysis to enable generalization from the sample to the population of interest (see Appendix B for a more detailed discussion).

American Institutes for Research, Page A- 5

Page 31: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Appendix B Development of Weights

Overview

Weights were calculated to permit generalization of findings to the populations of interest. In most cases, analytic weights were calculated using a three-step process.

• First, a sampling weight (equal to the inverse of the probability of selection) was calculated.

• Next, adjustment factors were calculated to compensate for non-response and sampling error.

• Finally, an analytic weight was created by multiplying the sampling weight by the adjustment factor.

State Sample Weights

No weights were calculated for the state-level data collection (i.e., the state directors of special education questionnaire instruments). This reflects the fact that every state director of education was surveyed, and all states provided data.

District (Regular and IEU) Sample Weights

The district questionnaires were comprised of six separate and distinctive components. A regular school district or Intermediate Educational Units (IEU) might complete and return any or all of these sections. Separate weights were developed for each of the six components of the district questionnaire, using the following procedures. See the District-Level Data Collection section of this report for a description of the six components.

Parts I-IV: Regular districts. Weights were developed for the regular district sample, reflecting each district’s probability of selection. Since regular districts were selected from a sampling frame that included all of the regular districts within a state with probabilities proportional to the square root of enrollment, the sampling weight assigned to a regular district (denoted as i) was:

(1) i

dists

ii

districtregularinenrollment

stateindistrictsregularinenrollment∑=

#

1

These weights were adjusted for non-response and also adjusted to ensure that the sum of the weights for all responding districts in a state would equal the number of regular districts in the state. The adjustment factor for regular districts was:

(2) respondingdistrictsregularselectedstateindistrictsregular

##

American Institutes for Research, Page B-1

Page 32: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

This adjustment factor was different for each of the parts of the questionnaire. Many districts returned partially complete questionnaires, and therefore separate analytic weights were calculated for each part of the questionnaire. The analytic weight for each part was calculated by multiplying the sampling weight (1) by the adjustment factor (2):

Regular district analytic weight = (1) x (2) Through the use of these adjustment factors, the sum of the regular district analytic weights for a state must equal the number of districts within this state. These procedures were employed for districts in the nine extended sample states. All of the non-extended sample states were combined into a single ‘super-state’ for purposes of calculating adjustment factors.10

Parts I-IV: Intermediate Educational Units (IEUs). The selection of IEUs was based on the random selection of sampled districts. That is, 30 regular districts were randomly selected from the regular school districts selected for the sample. The IEU that served this district was then selected for the IEU sample. Accordingly, the IEU’s probability of selection was equal to the regular district’s probability of selection11 and the inverse of the sampling weight assigned to the IEU:

(3) i

dists

ii

districtregularinenrollment

stateindistrictsregularinenrollment∑=

#

1

In order to generalize results to all IEUs in the nation, an adjustment factor was calculated:

(4) respondingIEUsselectednationinIEUs

##

The analytic weight for generalizing to all IEUs in the nation was calculated by multiplying the sampling weight (3) by the adjustment factor (4):

IEU analytic weight = (3) x (4)

As with the regular district questionnaires, separate analytic weights were created for each of the six parts.

District Central Office Staff Questionnaires. Weights for district central office staff were calculated using a two-step process. In the first step, the individual’s probability of selection from the pool of all similar individuals in the central office was determined. Since there was only one district director of special education per district, this individual was selected with certainty. Up to two psychologists were selected per district. If there were only one or two psychologists in

10 Special procedures were employed throughout for districts in New York State. New York State was divided into six sample strata corresponding to Need/Resource Capacity “NRC” classifications, as defined by the New York State Department of Education. Each NRC was treated as if it were a separate state for the purposes of calculating regular district weights. 11 Since IEUs were typically associated with more than one district, the potential for multiplicity arose. That is, the same IEU could be selected from more than one regular district. Since this did not happen, no adjustments were made for multiplicity.

American Institutes for Research, Page B-2

Page 33: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

a district, they were selected with certainty; if there were three or more, each psychologist’s probability of selection was equal to two divided by the number of psychologists in the district. Up to three other central office staff members were also randomly selected. So, if there were three or fewer other central office staff, each was selected with certainty. If there were more than three, the probability of selection of any of these staff members was three divided by the number of other staff in the district. The inverse of any individual’s probability of selection was their initial sampling weight. Thus, the sampling weight for each type of central office staff was equal to:

(5) respondingstaffofficecentralselected

districtinstaffofficecentral#

#

The individual’s weight was multiplied by the district analytic weight to produce a central office staff analytic weight:

Central office staff analytic weight = (5) x Regular district analytic weight

Externally Placed Student Weights. Weights were also developed for special education students served in non public schools or other public agencies where the tuition and fees are paid by the district. These students were identified by district staff. The non-response adjusted sampling weight for students was:

(6) respondingdistrictinstudentsedspecplacedexternally

districtinstudentsedspecplacedexternally#

#

To adjust for sampling error within the sample of selected districts, the non-response adjusted sampling weight is multiplied by the district analytic weight:

(7) = (6) x District analytic weight In order to generalize findings nationally, further adjustments were made to compensate for sampling error at the state level. To ensure that the sum of the weights for the externally placed students in a state would be equal to the number of externally placed students in the state,12 the sampling error adjustment factor was:

(8) ∑ − weightsstudentplacedexternallyadjustedresponsenon

stateinstudentsplacedexternally#

For national estimates, the analytic weight was calculated by multiplying (7) by this factor:

Externally placed student analytic weight = (7) x (8)

12 These adjustments were made for each of the extended sample states and the non-extended sample states. However, since there were no student data associated with certain states, all of the non-extended sample states were combined and treated as a single state for purposes of calculating this factor.

American Institutes for Research, Page B-3

Page 34: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

School Sample Weights

Weights for elementary schools, secondary schools, special education schools, and IEU schools were developed to permit generalizations to the populations of interest. Since a two-step sampling plan was used for the selection of elementary schools, secondary schools, and IEU schools, analytic weights were developed that took into account the school district’s probability of selection and the school’s probability of selection. This was then adjusted for non-response and sampling error. Special education schools were selected from a frame of state-operated special education schools. Accordingly, different procedures were employed for calculating their analytic weights.

Elementary and secondary school weights. The number of elementary schools selected from regular school districts was a function of the number of elementary schools in the district. If the district contained only one or two elementary schools, these schools were selected with certainty. If the district contained more, at least two elementary schools were selected. Similarly, the number of secondary schools selected from regular school districts was a function of the number of secondary schools in the district. If the district contained only one secondary school, this school was selected with certainty. The sampling weight assigned to an elementary (or secondary) school was:

(9) districtfromselectedtypeeachofschools

districtintypeeachofschools)(#

)(#

In order to deal with non-response, non-response adjusted school weights were calculated. These weights were:

(10) respondingdistrictfromselectedtypeeachofschools

districtfromselectedtypeeachofschools)(#

)(#

This weight was multiplied by the district’s weight to create an analytic weight, permitting generalization within a state.

School analytic weight (within state) = (9) x (10) x District analytic weight

In order to generalize findings nationally, further adjustments were made to compensate for sampling error, to ensure that the sum of the weights for elementary (or secondary) schools in a state would be equal to the number of elementary (or secondary) schools in the state. The sampling error adjustment factor was equal to:

(11) ∑ stateinweightsschoolanalytic

stateinschoolsregular#

This weight was multiplied by the within-state analytic weight to produce national estimates:

School analytic weight (national) = School analytic weight (within state) x (11)

American Institutes for Research, Page B-4

Page 35: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

Special education school weights. In non-extended sample states, non-response adjusted special education school weights were equal to:

(12) respondingstatessampleextendednoninschoolsspecialstatessampleextendednoninschoolsspecial

−−

##

For extended sample states, the non-response adjusted special education school weights were equal to:

(13) respondingstateinschoolsspecialstateinschoolsspecial

##

Weighting special education schools to the state level is equivalent to weighting them to the nation, as the total for all states equals the national total.

Intermediate Educational Unit (IEU) school weights. Non-response adjusted IEU school weights were equal to:

(14) respondingIEUinschoolsIEUinschools

##

These weights were then multiplied by the IEU’s weight to produce an analytic IEU school weight.

IEU school analytic weight = (14) x IEU analytic weight

Teacher and Aide Sample Weights

Weights were calculated for general education teachers, special education teachers, related service providers, and special education aides. Non-response adjusted sampling weights for each type of educator were equal to:

(15) respondingschoolattypeeachofeducators

schoolattypeeachofeducators)(#

)(#

These weights were multiplied by the school weight, to produce a respondent weight.

Teacher analytic weight (within state) = (15) x School analytic weight

In order to generalize findings nationally, further adjustments were made to compensate for sampling error, to ensure that the sum of the weights for each type of educator in a state would be equal to the number of educators of that type in the state. The sampling error adjustment factor was equal to:

(16) ∑ − stateinweightseducatoradjustedresponsenon

stateintypeeachofeducators )(#

American Institutes for Research, Page B-5

Page 36: The Purpose and Design of the Special Education ...The Purpose and Design of SEEP About the authors: Jay G. Chambers is the Project Director for the Special Education Expenditure Project

The Purpose and Design of SEEP

American Institutes for Research, Page B-6

This weight was multiplied by the within-state analytic weight to produce national estimates:

Teacher analytic weight (national) = Teacher analytic weight (within state) x (16)

Student Sample Weights

Students served in public schools Weights were calculated for special education students served in public schools. Sampling weights for each type of special education student were equal to the inverse of the student’s probability of selection:

(17) caseloadsteacherrespondingfromselectedstudentseducationspecial

teacherrespondingbyservedstudentseducationspecial'#

#

Non-response adjusted weights were then calculated. These non-response adjusted weights were equal to the number of students with each type of disability served by the teacher divided by the number of students with high (or low) incidence disabilities for which questionnaires were completed by the teacher.

(18) completedsurveyswhichforstudentseducationspecial

caseloadsteacherrespondingfromselectedstudentseducation#

'special#

These weights were multiplied by the educator’s weight to produce a non-response adjusted sampling weight.

(19) = (17) x (18) x teacher analytic weight In order to generalize results, further adjustments were made to compensate for sampling error and to ensure that the numbers of students with specific types of disabilities would be equal to the numbers of such students nationally.13 The national adjustment factor was equal to:

(20) ∑ −

iAllweightsdisabilityspecificwithstudentadjustedresponsenon

stateindisabilityspecificwithstudents)(

)(#

For national estimates, the analytic student weight was calculated by multiplying the non-response adjusted student weight by this factor:

Student analytic weight (national) = (19) x (20)

13 These adjustments were made for each of the extended sample states and the non-extended sample states. However, since there were no students with certain specific, low-incidence disabilities in many of the non-extended sample states, all of the non-extended sample states were combined and treated as a single state for purposes of calculating this factor.