Results from the AVID Program in Chicago

Jenny Nagaoka, Jonah Deutsch, Melissa Roderick, and Andy Brake

January 29, 2008

Today

• Brief overview of findings on AVID in Chicago– Ninth grade course performance– Student and teacher survey responses

• How to read individual school reports

• How to use the information in your individual school reports

Are AVID students doing better than other students on 9th grade

outcomes?

A preliminary analysis of 6 AVID schools in 2004-05

Summary of findings

• AVID shows positive effects on 9th grade grades in math and English and weighted GPA.

• This cannot simply be explained by “creaming” within AVID schools or that AVID students systematically differ in other ways (i.e., coming from more supportive families)

• Some of the AVID effect seems to be linked to its practice of placing AVID students in honors courses

2006-2007 AVID Survey Overview

• AVID student survey in April 2007– 9th and 10th graders– 32 of 39 AVID Schools participated in the survey – 2,890 of 3,752 AVID students surveyed (77%

response rate)

• 2007 CCSR teacher survey (April)– 28 AVID schools– Over 1,000 teachers surveyed

Results provided to schools to highlight areas of success and points needing improvement in the AVID program across the district and within schools

AVID students are very positive about their experiences in the program

42

51

49

56

22

35

22

22

64%

86%

71%

78%

0 20 40 60 80 100

My AVID teacher made a bigdifference in my life this year.

AVID gives me the support I needto be successful in high school.

I am a better writer since I joinedAVID.

I look forward going to my AVIDclass each day.

Agree Strongly agree

However, most AVID students do not report having in-depth conversations with their AVID

teachers about postsecondary planning.

32

44

36

18

38

49

0 20 40 60 80 100

My AVID teacher has met with me one-on-one this year to discuss my college

plans

How much has your AVID teacherdiscussed how to decide which college

to attend

Talked to you about honors/AP classesto take next year

Discussed briefly Discussed in-depth

On all measures, students in schools in the “high” and “highest” categories are overwhelmingly positive about the impact that the program has had on them.

• 100% (81% in high category) report that they are a better writer.

• 83% (76% in high category) agree or strongly agree that their AVID teacher made a big difference in their life this year.

Even students who attend schools in the “lowest” category are fairly positive about the program

• 70% agreed or strongly agreed that AVID gives them the support they need to be successful in high school.

• 79% agreed or strongly agreed that their AVID teacher had earned their respect.

Teachers in AVID schools are also positive about the program.

47

61

58

52

18

23

27

26

65%

84%

85%

78%

0 20 40 60 80 100

Have implemented AVID strategies in theclassroom

The AVID program supports my goals

The AVID program teaches studentsuseful tools

AVID is well-implemented in this school

Agree Strongly agree

Organization of Report

Four main areas:– Students’ and teachers’ evaluation of the AVID program– Skills learned through AVID– Relationships built with AVID teachers and tutors– Postsecondary planning in AVID

Four groupings of results:– Results by school– Results by type of school (certified and new)– Results by type of students (gender, 7th grade ITBS math score

categories, and grade)– Results by type of students within schools

Seven survey measures in four areas:1. Students’ and teachers’ evaluation of the AVID program:

– AVID Program Satisfaction (Student) – Students’ overall satisfaction with the AVID program, including skills learned and supports received.

– AVID Program Impact (Teacher) – Teachers’ reports on the impact of the AVID program on their practice and on their students.

2. Skills learned through AVID:

– AVID Skills – Students’ evaluation of whether AVID has taught them skills that impact their effectiveness as students in all their classes.

3. Relationships built through AVID:

– AVID Teacher Support – Students’ perceptions of the amount of personal support they receive from their AVID teacher .

– AVID Tutors – Students perceptions of the quality of guidance and academic and personal support they receive from their AVID tutors.

4. Postsecondary planning and preparation in AVID:

– AVID Postsecondary Planning Discussions – Student reports of the extent their AVID teacher has discussed planning for students’ postsecondary education.

– AVID Postsecondary Academic Planning & Preparation – Student reports of how often their AVID teacher has talked about the importance of course grades, course-taking, test preparation, and college planning for future success.

Organization of Report

Summary page of all measures:– School rating category

– Summary rating across all measures

On each measure: • School-by-school means graph• Table of survey question responses by school categories• Table of school results by student type

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

28 14 6 26 9 1 25 31 20 3 7 11 27 5 15 12 16 24 21 18 8 10 22 30 2 4 13 19 29 23 32 17

32 AVID high schools

Mea

n s

tan

dar

diz

ed s

core

(Mea

n=1

, St.

Dev

.=1)

Lowest Low Average High Highest

Program Satisfaction (Student), Page 8

Program Satisfaction (Student), Page 8

Lowest0.50 or

lower

Low0.51 to 0.75

Average0.76 to 1.24

High1.25 to 1.49

Highest1.50 or

higher

Your School1.02

N of schools 4 3 15 5 5

N of students 462 708 1,101 327 295 150

How much do you agree with the following . . .

% Agree and Strongly Agree to Each Question(% Strongly Agree)

In AVID, I am learning theinformation I will need to go tocollege

71%(14%)

78%(20%)

88%(34%)

93%(49%)

97%(59%)

71%(10%)

AVID gives me the support I need tobe successful in high school

70%(12%)

78%(16%)

89%(28%)

93%(40%)

97%(46%)

67%(10%)

I have become a better student since Ienrolled in AVID

58% (10%)

60% (10%)

75% (18%)

83% (26%)

87% (36%)

60% (12%)

I look forward to going to my AVIDclass each day

57% (10%)

65% (12%)

80% (19%)

81% (27%)

96% (44%)

59% (13%)

I would recommend the AVIDprogram to my friends

50%(11%)

62%(15%)

77%(25%)

82%(36%)

92%(47%)

50%(8%)

Program Satisfaction (Teacher), page 10

Lowest0.50 or lower

Low0.51 to 0.75

Average0.76 to 1.24

High1.25 to 1.49

Highest1.50 or higher

N of schools 4 6 9 6 3

N of teachers 92 185 359 197 67

How much do you agree with the following . . .

% Agree and Strongly Agree to Each Question(% Strongly Agree)

The AVID program supports my goals in the classroom.

70%(8%)

72%(11%)

88%(24%)

95%(36%)

95%(41%)

AVID students are getting useful tools to improve their performance in class.

69%(7%)

73%(17%)

91%(29%)

95%(39%)

95%(47%)

AVID has a positive effect on its students’ academic performance

65%(8%)

74%(17%)

88%(29%)

95%(41%)

96%(48%)

I have implemented AVID strategies in my classroom.

56%(6%)

54%(11%)

67%(19%)

82%(28%)

95%(40%)

I have a clear understanding of AVID strategies and classroom activities.

49%(7%)

59%(13%)

71%(23%)

80%(35%)

90%(38%)

AVID has a positive effect on my instruction.

50%(2%)

57%(10%)

76%(20%)

84%(34%)

91%(36%)

The AVID program is well run and implemented in our school.

47%(8%)

64%(10%)

85%(30%)

92%(41%)

95%(46%)

AVID students are prepared to move into honors and AP courses.

42%(7%)

62%(10%)

70%(17%)

83%(28%)

80%(31%)

Skills, Page 12

Lowest0.50 or lower

Low0.51 to 0.75

Average0.76 to 1.24

High1.25 to 1.49

Highest1.50 or higher

N of schools 1 3 21 5 2

N of students 98 513 1,863 290 95

How much do you agree with the following . . . % Agree and Strongly Agree to Each Question(% Strongly agree)

Since I joined AVID: I am a better writer

87%(27%)

57%(11%)

71%(20%)

81%(31%)

100%(69%)

I am more organized in school 73% (9%)

69% (16%)

74% (19%)

87% (28%)

87% (32%)

I ask better questions than I did before 71%(17%)

56%(12%)

71%(19%)

86%(27%)

96%(50%)

I feel more comfortable asking for help from adults

56%(8%)

60%(13%)

71%(16%)

78% (27%)

90%(30%)

I speak up with my ideas in class more often 42%(11%)

48%(10%)

64%(17%)

79%(25%)

83%(39%)

I am more organized outside of school 41% (2%)

49% (12%)

54% (12%)

60% (15%)

67% (20%)

Postsecondary Academic Planning and Preparation, page 20

Lowest0.50 or lower

Low0.51 to 0.75

Average0.76 to 1.24

High1.25 to 1.49

Highest1.50 or higher

N of schools 3 4 15 5 5

N of students 205 816 1,226 302 248

How much do you agree with the following . . . % of students saying that they have discussed the following with their AVID teacher either A little or A lot

(% saying A lot)

This year, has your AVID teacher: Encouraged you to do well in your classes

89%(47%)

94%(58%)

96%(68%)

100%(88%)

99%(90%)

Helped you set goals for your grades 88% (32%)

91% (33%)

94% (48%)

99% (75%)

99% (79%)

Explained to you the importance of grades in preparing for college

87%(41%)

90%(53%)

96%(67%)

98%(81%)

99%(87%)

Taught you how to make long-term goals for the future

85% (42%)

86% (39%)

93% (54%)

97% (72%)

98% (75%)

Talked to you about honors or AP courses to take next year

52%(14%)

70%(27%)

86%(48%)

98%(69%)

97%(72%)

Helped you prepare for standardized tests such as the PSAT, ACT, or SAT

42%(10%)

75%(29%)

75%(30%)

83%(40%)

91%(57%)

Met with you one-on-one to discuss your college plans

27%(6%)

44%(15%)

49%(17%)

56%(24%)

73%(33%)

Talked to your family members about college options

23%(6%)

34%(1%)

36%(14%)

46%(21%)

56%(25%)

• Each student was given a value based on how they responded to the items in a measure. – Measures are standardized to have a mean of 1 and

standard deviation of 1

We grouped schools into categories based on their mean score:

– Lowest: Schools that were at least half a standard deviation below the mean (0.50 or below)

– Low: Schools that were between one quarter and one half standard deviation below the mean (0.50 to 0.75)

– Average: Schools that were between one quarter standard deviation below or above the mean (0.75 to 1.25)

– High: Schools that were between one quarter and one half standard deviation above the mean (1.25 to 1.50)

– Highest: Schools that were at least one half standard deviation above the mean (1.50 or higher)

Rating AVID Schools

Over a third of the schools had average scores that placed them in the “high” or “highest” categories

2

5

14

7

4

0

5

10

15

20

Lowest Low Average High Highest

Categories based on schools’ overall score

Num

ber

of s

choo

ls

Page 6

Comparing AVID outcomes by type of student, page 7

OverallN=120

ITBS categories Gender Grade

Your schoolSystem Average

LowN=18

MiddleN=54

HighN=23

MaleN=43

FemaleN=48

9th

N=6410th

N=67

ProgramSatisfaction

1.001.00

0.301.07

0.460.98

0.510.93

0.510.94

0.481.04

0.570.91

0.391.12

Skills 0.441.00

1.001.11

0.780.99

0.790.89

0.770.87

0.861.08

0.910.95

0.681.05

Teacher Support0.661.00

0.701.08

0.380.94

0.761.00

0.450.90

0.81.07

0.580.98

0.391.03

Tutors0.991.00

0.711.11

0.460.99

0.570.89

0.681.00

0.81.00

0.910.99

1.021.02

PostsecondaryDiscussions

0.781.00

0.861.05

0.700.98

0.800.95

0. 890.96

0.751.02

0.740.85

0.761.20

PostsecondaryPreparation

1.101.00

0.951.06

0.950.99

0.720.94

0.910.97

0.801.03

0.770.91

0.881.14

Additional copies of this report

• Go to the Consortium’s website: ccsr.uchicago.edu/AVID

• Your user name and password are in your AVID report.

• Release forms are in your packet.