Program SLO Assessment for English as a Second Language Upon completion of the credit ESL program, students will have adequate listening, speaking, reading, writing and academic skills to successfully complete transfer-level classes. PLANNING Assessment Describe the Planned Program SLO Assessment

[from the original Program-level SLO Proposal]: “Collect data from [English] 1AX classes as a sample of transfer-level courses, to determine the preparation students have had. Collect data on the retention and success [rates] in these 1AX classes. Compare student retention and success in 1AX (ESL) classes with the retention and success in 1A [non-ESL] classes.”

1. In Spring 2009, students in all sections of English 1AX will complete a survey eliciting information about their prior preparation for the course, including ESL and English classes, and classes taken through the El Camino Language Academy (ECLA). The purpose of the survey will be to identify ESL students and to track the progress of students who will go on to take IGETC critical thinking courses in Fall ’09:

English 1AX Survey

Spring, 2009

Instructor’s Name:__________________________________________ Section #:________

Your Student ID #:____________________

1. Is English your first language? Yes No

1. How long have you lived in an English-speaking country? _____years, _____months

1. 3. Which high school did you attend and where is it located?

________________________________________________________________________

1. 4. Please put a check mark (ü) next to all of the classes that you’ve completed OR that you are taking this semester at ECC:

ESL writing classes:

_____English AX

_____ESL 53B

_____ESL 53A

English writing classes:

_____English A

_____English B

ESL reading classes:

_____ESL 52C

_____ESL 52B

_____ESL 52A

ESL listening/speaking classes:

_____ESL 51C

_____ESL 51B

_____ESL 51A

English reading classes:

_____English 84 (formerly English 2R)

_____English 82 (formerly English R)

_____English 80

English grammar class:

ESL multi-skills class:

_____ESL 55

ECLA classes:

_____College Preparation—Level 3

_____College Preparation—Level 2

_____College Preparation—Level 1

_____English 4

Other ECC English classes (please list):

1. 5. Please list your other classes this semester:

Thank you for your participation.

2. Some of the survey data will be collected and entered into a spreadsheet. This spreadsheet will include information that is not currently available through the Institutional Research (IR) department (i.e. Survey Items 1-3 and ECLA courses taken before entering the ECC ESL program)

ESL Program-level SLO Survey

Excel Spreadsheet Items

Spring, 2009

Col:

Original

Survey Item:

Column Heading and Description of Item:

A

Course Section number / Student ID number

Section Number (bold); ECC Student ID Number

B

1. Is English your first language (LI)?

“Engl. is L2” =”English is my second language (L2)”

Y = Yes

C

D

2. How long have you lived in an English-speaking country?

“Years in Engl Spkg Country?” = “Number of years

living in an English-speaking country?”

“Months?” = “Number of months”

3. Which high school did you attend and where is it located?

E

F

“US HS” = “Did you attend High School in the U.S.?

Y = Yes

“HS Country” = “In which country is your

High School located?”

Refer to “ESL Program-level SLO: English 1AX

Survey: Country Codes” list

Attendance in ECC Language Academy (ECLA)

“Attend ECLA” = “Did you attend the ECC Language

G

Academy?”

Y = Yes

H

I

J

Which ECC Language Academy courses did you take?

“Did you take…”

“…Level 3?”

“…Level 2?”

“…Level 1?”

Y= Yes

3. Country codes will be developed by D. Mochidome for determining students’ first language background (based on the location of these students’ high schools):

ESL Program-level SLO

English 1AX Survey

Country Codes

Spring 2009

Code:

Country:

?

ALG

BG

BZ

CB

CHN

CHL

COL

No notation

Algeria

Bulgaria

Brazil

Cuba

China

Chile

Colombia

ESV

ETH

IDO

IND

IRN

JA

KO

ML

MX

NI

PRU

El Salvador

Ethiopia

Indonesia

India

Iran

Japan

South Korea

Malaysia

Mexico

Nigeria

Peru

SA

SNG

SRB

SRL

TH

TK

TW

VN

Saudi Arabia

Singapore

Serbia

Sri Lanka

Thailand

Turkey

Taiwan

Vietnam

4. The spreadsheet items will be tabulated separately from the information provided by IR. [See Attachment]

5. Subsequently, IR will track improvement and retention rates for Spring ‘09 English 1AX students enrolled in Fall ‘09 IGETC critical thinking courses (i. e. English 1C, Philosophy 5, and Psychology 3)

Standards or Rubric Describe the standards you will use to determine success in your assessment

• Success and retention rates for English 1AX courses

• Comparison of these success and retention rates with those of non-ESL English 1A courses

• Correlation, if any, between success in English 1AX and success in ESL 51ABC (listening/speaking) course series

• Improvement and retention rates for former English 1AX students in IGETC critical thinking courses taken the semester after completing English 1AX

Assessment Leaders Mochidome, Debra Uyemura, Evelyn Uyemura, Evelyn Cortez-Perez, Aurora Location and Semester Semester of Current Assessment: Campus of Assessment: El Camino College Semester of Next Planned Assessment: Spring 2013 Linking Course Assessments No Course SLOs have been matched to this Program SLO REPORTING Assessment Data Provide the survey or assessment data.

IR completed the first 4 of the 5 items in the Research Plan; Item 5 was not completed as IR felt it to be “rather nebulous.” It was suggested, however, that it would be possible to collect this data in the future if the department would like to have it.

Research Plan, Item 1:

With regard to their previous ESL courses (excluding ECLA courses), the average success and retention rates for the Spring 2009 English 1AX students surveyed were:

Average Success Rate for previous ESL Courses:

Average Retention Rate for previous ESL Courses:

Successful Spring 2009 English 1AX students

85.9%

92.5%

Unsuccessful Spring 2009 English 1AX students

51.9%

78.2%

Difference between Successful and Unsuccessful Students

34%

14.3%

These data indicate that:

• The majority of successful English 1AX students (85.9%) had also been successful in their previous ESL courses and, to a lesser extent, the converse was also true; unsuccessful English 1AX students (51.9%) tended to have been unsuccessful in their previous ESL courses. The difference between the success rates for successful and unsuccessful English 1AX students was 34%.

• The retention rate for successful English 1AX students in their previous ESL courses was 92% whereas the retention rate for unsuccessful English 1AX students in their previous ESL classes was 78.2%. The difference in these retention rates was 14.3%.

Item 2:

IR analyzed the Spring ‘09 success and retention rates of English 1AX classes and compared them with those of English 1A classes.

Because students must now register for classes online, and the “X” is not shown in the online version of the Schedule of Classes, some English 1AX students are native English

speakers (i.e. non-ESL students)

Also, because a clear separation was not made between the ESL students and the non-ESL students before the English 1AX Survey data was submitted to IR, the data IR provided below reflects all students, ESL and non-ESL, who were enrolled in English 1AX:

Spring 2009

Course:

No. of Students

(initially enrolled in all sections of course):

Success:

Retention:

English 1AX

161

65.8%

84.5%

English 1A

2140

58.5%

77.6%

Differences between English 1AX sections and other English 1A sections

7.3%

6.5%

The corresponding English 1AX and English 1A grade distributions were:

English 1AX Sections—Spring ’09:

Grade:

Count:

Success:

Retention:

A

43

65.8%

85.4%

B

48

C

15

D

9

DR

5

F

19

2

I

W

20

Total:

161

All Other English 1A Sections—Spring ’09:

Grade:

Count:

Success:

Retention:

A

385

58.5%

77.6%

B

476

C

391

D

125

DR

74

F

241

I

43

405

W

Total:

2140

Difference between English 1AX sections

and all other English 1A sections:

Success:

Retention:

7.3%

6.9%

• IR’s finding here was that students coming through the ESL program were considerably more successful (7.3%) than non-ESL students who attempted English 1A. In addition, the overall retention rate for English 1AX students was 6.5% higher than that of English 1A students

Item 3:

Eligibility for English 1AX requires credit in English AX and in ESL 52B (i.e. completion of the ESL program’s pre-transfer-level writing series and its intermediate-level reading course).

In an attempt to see whether completion of the 51 series (listening/speaking) courses had an impact on the success and retention of English 1AX students, the correlation between successful Fall ‘08 1AX students and their completion of 51 series courses was examined.

Of the 161 students enrolled in Spring ‘09 English 1AX sections, 110 students completed the English 1AX Survey, and of those students, 13 (11.82%) identified themselves as native-English speakers (i.e. non-ESL students). The remaining 97 students (88.18%) identified themselves as non-native English speakers (i.e. ESL students). These ESL students were educated in high schools located in 27 countries (including the U.S.). The following data were collected from the ESL students’ responses to the survey:

English-1AX Success and Retention Rates by Placement/Enrollment in ESL-51 Series

Course/Placement

Group

Count

% of Grp

Success

Retention

ESL-51A

Took

12

67%

58%

67%

Did not take

6

33%

67%

67%

Difference (% pts)

6

33

-8

0

ESL-51B

Took

27

43%

85%

89%

Did not take

36

57%

78%

89%

Difference (% pts)

-9

-14

7

0%

ESL-51C

Took

6

9%

83%

83%

Did not take

58

91%

88%

93%

Difference (% pts)

-52

-81

-5

-10

Overall

Took

45

31%

78%

89%

Did not take

100

69%

83%

95%

Difference (% pts)

-55

-38

-5

-6

Of the students who placed into ESL 51A, 66% enrolled in the course. Forty-three percent of students placed into 51B enrolled in 51B, and 9% of students who placed into 51C enrolled in 51C.

IR’s report indicated “mixed” results in terms of student success and retention:

• Students who had “…enrolled in ESL 51B were [7%] more likely to be successful” in English 1AX than “those who did not”

• However, “the outcome reverses” for those who enrolled in ESL 51A and 51C. The lower counts in these courses may be a factor here

• IR’s conclusion for this portion of the SLO assessment was, “While no clear conclusions can yet be drawn as to whether the oral series provides an added benefit to transfer-level writing, results from the 51B level are promising, given its larger sample size”

Item 4:

After the Fall ‘09 semester, data on students who took IGETC critical thinking courses (English 1C, Philosophy 5, and Psychology 3) were collected. The results for Fall ‘09 courses were:

Success and Retention in Critical Thinking Courses by English 1A Course Status

Course

Engl-1A Status

Total

Success

Retention

ENGL-1C

ESL

26

77%

81%

Not ESL

340

75%

89%

Difference (% pts)

2

-8

PHIL-5

ESL

*

100%

100%

Not ESL

11

64%

73%

Difference (% pts)

36

27

PSYC-3

ESL

*

50%

50%

Not ESL

55

58%

75%

Difference (% pts)

-8

-25

Overall

ESL

*

77%

80%

Not ESL

406

72%

86%

Difference (% pts)

4

-6

*Data suppressed due to small sample size.

Overall, the results of the IGETC course analysis were mixed.

• Twenty-six English 1AX students (roughly 26.8% of the original 97 ESL students surveyed) successfully completed English 1C, representing roughly 6.4% of the total number of all successful 1C students. These 26 students also represented 23.64% of the original 110 respondents to the Spring ‘09 survey.

• Although the number of successful ESL students was small, in comparison to the total number of successful 1C students, the ESL students’ improvement rate was 2% higher than that of their non-ESL peers. However, the retention rate of ESL students in 1C classes was 8% lower than that of their non-ESL peers.

Observable Patterns Summarize the patterns observed in the data.

• ESL students who had come through the ESL program were more successful in English 1AX than non-ESL students who attempted English 1A.

• Successful English 1AX students tended to have been successful in their previous ESL classes and to have high retention rates as well.

• There is a substantial number of non-ESL students who take English 1AX. It is possible that these students, who tend to be more likely to drop the class, may be affecting retention rates in English 1AX.

• While a correlation between students’ success in English 1A and these students’ having taken 51 series courses is inconclusive, nevertheless, students who had taken 51B were 7% more successful than those who had not.

• In addition, Jenny Simon noted that the higher students placed into 51 series courses, the higher their success rate was in English 1AX. Specifically, students who placed into ESL 51C had an average 1AX success rate of 86%, students who placed into 51B had an

average 1AX success rate of 81.5%, and students who placed into 51A had an average 1AX success rate of 62.5%.

• Regarding IGETC courses, English 1AX ESL students overwhelmingly tended to take English 1C, rather than Phil. 5 or Psych. 3. The success rate for these 1C students was slightly higher than that of their non-ESL peers. On the other hand, the retention rate for ESL students in English 1C was notably lower than that of their non-ESL peers.

Implications and Future Directions What are the implications of the data? How should the institution as a whole or related programs act on the data?

• Make eligibility for ESL 51C “recommended preparation” for English 1AX

• Make sure that English 1AX instructors are using the rubric for English 1A to score students’ essays. The same goes for English AX instructors’ using the English A rubric for essay scoring

• Schedule essay-norming sessions for all ESL writing courses (including English !AX) for instructors teaching these courses. The purpose of the norming to ensure that grading standards are consistent department-wide

• Schedule norming sessions for ESL 52B and 52C where instructors can evaluate students’ written summaries of their reading assignments in order to ascertain the extent of these students’ reading comprehension.

Changes to the SLO statement: None

Changes to the assessment:

• Meet with Irene Graff (IR) to discuss measurement and assessment options before designing and implementing the assessment

• Include data from Compton Center’s non-credit ESL program to include its role in preparing students for the ECC ESL program, and omit ECLA data; in this report, only 4 student respondents to the survey (3.6%) indicated that they had taken ECLA courses prior to taking English 1AX

Changes to the rubric:

• A number of ESL students take English 1B after completing English 1AX, so instead of examining improvement and retention rates for IGETC critical thinking courses per se, examine improvement and retention rates for English 1C and English 1B. Omit improvement and retention rates for Philosophy 5 and Psychology 3. since so few students in this assessment’s survey sample took these courses

• Take into account the educational goals of non-university-bound students by adding a survey item such as “Educational goals: Transfer to a 4-year university? Associate’s degree? Certification? Other?”

• Include success and retention rates for students who go through the ESL program and who take English 1A (in addition to the success and retention rates for students who go through the ESL program and who take English 1AX)

• Include the results of this SLO assessment in the next SLO assessment so that the results of the next assessment can build on the results of this one. In this way, factors (e.g. progress/increases in success rates, etc.) can be tracked from this point (June, 2010) on into the future

Changes to the method:

If we choose to survey students again:

• Use a scan-able survey form to expedite data collection. IR has already developed a preliminary version of this type of survey.

• Be sure to clearly separate the non-ESL students from the ESL students before sending data to IR

• Elicit more specific information related to students’ first language backgrounds (vs. Survey Item 3. “Which high school did you attend, and where is it located?”). For example, this item can be reworded as: “Country?”; “Number of years speaking English?”; “Language spoken at home?”; etc.

• Develop a more “user-friendly,” possibly numerically-based, method for coding students’ home countries, if this data is germane to the next cycle’s assessment

• Omit Survey Item 5 “Please list your other classes this semester:”

If we choose NOT to survey students:

• To expedite the data-gathering process, get enrollment, success, retention, and improvement data from IR instead of gathering these data via a survey.

• Have IR track English 1AX students over a longer period (perhaps as long as 10 years) in order to reinforce the preliminary correlation between the successful completion of 51 series courses and/or the initial placement of students into 51 series courses, and success in English 1AX.

• Request that IR track success and improvement rates for English 1AX students who have successfully completed ESL 52C prior to taking 1AX

Additional Attached Files No Additional Files have been attached