6. implications for analysis: data content. 1 prerequisites recommended modules to complete before...
TRANSCRIPT
6. Implications for Analysis: Data Content
6. Implications for Analysis: Data Content
2
Prerequisites
• Recommended modules to complete before viewing this module 1. Introduction to the NLTS2 Training Modules 2. NLTS2 Study Overview 3. NLTS2 Study Design and Sampling For parent/youth data sources:
• 4. NLTS2 Data Sources: Parent and Youth Surveys For school or student data sources:
• 5. NLTS2 Data Sources: School Surveys, Student Assessments, and Transcripts
6. Implications for Analysis: Data Content
3
Overview NLTS2 sample Multiple data sources NLTS2 data collection Availability of data Data discrepancies Missing data Response rates Restricted-use NLTS2 data Closing Important information
6. Implications for Analysis: Data Content
NLTS2 restricted-use data
• NLTS2 data are restricted.• Data used in these presentations are from a
randomly selected subset of the restricted-use NLTS2 data.
• Results in these presentations cannot be replicated with the NLTS2 data licensed by NCES.
4
6. Implications for Analysis: Data Content
5
NLTS2 sample
• Sampled in two stages First stage: Local education agencies (LEAs) and state-
operated special schools. Second stage:
• Students sampled to represent those in the 12 federally defined disability categories.
• Students were 13 to 16 in grade 7 or above, and in middle or high school.
• Complex sampling has implications for weighting.
6. Implications for Analysis: Data Content
6
Multiple data sources
• Parent/youth survey A parent interview or mail survey
questionnaire completed by youth’s parent or guardian
A youth interview or mail survey questionnaire completed by youth
• School characteristics survey School-level information about the
school that the student attended• Teacher survey
A general education academic subject teacher about student’s experiences in general education academic classroom
• Student’s school program survey School staff member who knew about
student’s program, including transition and vocational education
• Student assessments Direct assessment of student’s
scholastic abilities Student interview on attitudes about
school, self-determination Alternate Assessment completed by
student’s teacher if student was unable to participate in the Direct Assessment and interview
• Secondary school transcripts
6. Implications for Analysis: Data Content
7
Data sources• Although data were collected from many sources (parents,
school administrators, teachers, youth), the unit of analysis is the individual youth. In a given wave, a single teacher may have responded to multiple
surveys for students within the teacher’s school or classroom, but each survey is in reference to a single student.
• Generalizations cannot be made about families, communities, LEAs, schools, teachers, or classrooms. Although many youth may have attended any single school, data
from the School Characteristics Survey are stored in a youth-level record not a school-level record.
6. Implications for Analysis: Data Content
8
NLTS2 data collection• Parent/guardian interviews/surveys
Collected in 5 waves• Youth interviews/surveys
Collected in 4 waves• School data
Collected in 2 waves• Student assessments
Assessed once when youth was 16 years or older• Multiwave collection resulting in one file included with
Wave 2 data
• Secondary school transcripts Collected for the school years when youth in secondary school
included with Wave 5 data
6. Implications for Analysis: Data Content
9
NLTS2 data collection: TimelineWave 1 Wave 2 Wave 3 Wave 4 Wave 5
Year 1
2000-2001
Year2
2001-2002
Year 3
2002-2003
Year4
2003-2004
Year 5
2004-2005
Year6
2005-2006
Year 7
2006-2007
Year8
2007-2008
Year 9
2008-2009
Year10
2009-2010
Parent telephone interviews or surveys
Youth telephone interviews or surveys
Direct assessment andin-person interviews
Teacher Survey
Student’s School Program Survey
School Characteristics Survey
Transcripts
6. Implications for Analysis: Data Content
10
Availability of data
• Not all sources of data are available for a given youth. Within a wave, a youth may have one or more sources of
data, but not necessarily all sources.• For example, there may be a Wave 1 Parent Interview for a youth but
no Wave 1 school-level data or vice versa. Across waves, a youth may have data for a given source in
one wave and not another.• For example, there may be a Wave 1 Parent Interview and a Wave 5
Parent/Youth Interviews but no Wave 2 through 4 Parent/Youth data.
6. Implications for Analysis: Data Content
11
Data discrepancies
• Similar items in multisource, longitudinal data do not always agree. For example, all youth in the sample were in special
education in 2000. However• Some parents indicated that their son or daughter was never in
special education.• Parents reported disabilities other than those that the district and/or
school has reported. A parent may have reported the youth dropped out in one
wave as well has having reported he or she graduated from high school in another wave.
6. Implications for Analysis: Data Content
12
Data discrepancies• A secondary school transcript may report a different high school
leaving date than reported in a parent/youth interview.• A parent and a youth may provide conflicting responses on such
things as high school leaving status, postsecondary attendance, or employment status. Example: Parent/guardian and youth responses for why youth left
secondary school may differ. Example: Parent/guardian and youth responses for whether youth
attended a 2-year/junior college may differ.
Implications for Analysis: Data Content
Data discrepancies: Examples
13These results cannot be replicated with full dataset; all outputin modules generated with a random subset of the full data.
6. Implications for Analysis: Data Content
14
Missing values• There are many reasons for data to have missing values,
which are an unavoidable part of survey work.• Data can be completely missing for a given source.
The appropriate respondent to complete the survey could not be located.• Example: School had no record of student attending that school.• Example: Parent had moved and we no longer had contact information to
reach the family .• Example: Foster parent no longer had youth in his or her care.
Respondent refused to complete a questionnaire or participate in an interview.
Youth attended a known school but school did not compile a transcript for that youth.
6. Implications for Analysis: Data Content
Missing values• Respondent may have had data for a given data source, but
data can be missing within a file for that source. Respondent did not know or refused to answer a question. The question was not applicable. The question was skipped by design. Respondent missed a page in a questionnaire. Respondent terminated the interview before it was
completed. Respondent completed a different version of an instrument
that does not have that particular item.• For example, respondent completed the mailed Family Survey in lieu
of the Parent/Guardian interview.
15
6. Implications for Analysis: Data Content
16
Missing values
• Pay attention to the number of respondents when conducting any analysis. Across waves, the number of respondents for a given data
source will vary. Within a wave, the number of respondents for each data
source will be different from file to file. Within a data collection instrument, n’s will vary item by
item.
6. Implications for Analysis: Data Content
17
Missing values
• Longitudinal and/or multisource analyses are affected by missing values. The more sources of data that are being combined, the
larger the likelihood that data will be missing for at least one of those sources.
The number of respondents is typically smaller for cross-instrument or cross-wave analysis than for analysis of a single instrument.• For example, an analysis that requires a respondent to have all 5
waves of Parent/Youth Survey data will have a smaller number of respondents than any single wave of Parent/Youth Survey data.
6. Implications for Analysis: Data Content
18
Response rates• Data collected from multiple sources and multiple points
in time generated different response rates.• Generally there were higher response rates in earlier
waves than in later waves. Lower response rate between Wave 1 Parent and Wave 2
Parent/Youth largely due to attrition.• Combining data from multiple sources can result in a
smaller n if requiring that all respondents have all sources.
6. Implications for Analysis: Data Content
19
Response ratesParent/Youth Surveys
W1 Parent Survey 82%
W2 Parent/Youth Survey 61%
W3 Parent/Youth Survey 50%
W4 Parent/Youth Survey 50%
W5 Parent/Youth Survey 48%
School Surveys
W1 (Teacher, Program, School) 36%,53%, 57%
W2 (Teacher and Program) 41%, 52%
Student Assessments
W1 Administration 63%
W2 Administration 72%
Transcripts
[multiple waves] 81%
6. Implications for Analysis: Data Content
20
Restricted-use NLTS2 data
• NLTS2 Web Seminar data NLTS2 data and statistical output are restricted and require
licensure for use. Data used in these presentations are a random subset of
the full NLTS2 database. Results cannot be replicated with fully licensed data as
demonstrations and examples are generated using a random subset of the data.
• If you plan to use NLTS2 data, you will need to obtain a license through NCES. http://nces.ed.gov/statprog/rudman/
6. Implications for Analysis: Data Content
21
Closing
• Topics discussed in this module NLTS2 sample Multiple data sources NLTS2 data collection Availability of data Data discrepancies Missing data Response rates Restricted-use NLTS2 data
6. Implications for Analysis: Data Content
22
Closing
• Next module for Parent/Youth data sources: 7. Implications for Analysis: Parent/Youth Survey Data
• Next module for school or student data sources: 8. Implications for Analysis: School Survey, Student
Assessment, and Transcript Data
6. Implications for Analysis: Data Content
23
Important information NLTS2 website contains reports, data tables, and other
project-related information http://nlts2.org/
Information about obtaining the NLTS2 database and documentation can be found on the NCES website http://nces.ed.gov/statprog/rudman/
General information about restricted data licenses can be found on the NCES website http://nces.ed.gov/statprog/instruct.asp
E-mail address: [email protected]