edtech 505 eval project

Evaluation Report: English Department at the New Beginning School1

EDTECH 505 - Evaluation for Education Technologists

Evaluation Report – Course Project

Done by: Kirkland Lawrence

Submitted to: Dr. J. Thompson

Boise State University


Summary

English Language is said to be one of the universal languages available to mankind and it is spoken globally. However, there are times when people who do not speak English go to countries where English is the native language and find it difficult to communicate through speaking, reading, and/or writing the language. The Haitian who speaks French and the Dominican Republicans who speak Spanish are two groups of people who find themselves in this position when they seek to work and live in the Turks and Caicos Islands (TCI). As result, the New Beginning School (English department) was established to bridge the gap that existed because of the language barriers and to help the two nationals integrate in the society. The program is funded by the government of the TCI through its Ministry of Social Development department. It is against this background that evaluator examined the English Department to determine its efficiency, effectiveness and impact in helping non-English speaking students integrate in the public school system and the society in general and to provide feedback to stakeholders for decision making. The aim of the program is to integrate/ reintegrate students into the Government’s public school system and the wider society of the TCI at large with objectives revolved around students’ abilities to write and speak English fluently, as well as to practically demonstrate how to complete job applications and construct resume’s. Data were gathered from instruments such as pretest/posttest, surveys in the form of questionnaires, obtrusive and unobtrusive observations, school records and interviews. Analysis of data was presented using graphs and narrations.

The discussion section highlighted some of the highpoints of the data presented where stakeholders can garner pertinent information to guide them during the decision making process.

Seven invaluable recommendations are made with the intention of convincing stakeholders of the importance of the program as it impacted students’ lives. It is the evaluator’s belief that these recommendations will influence stakeholders’ decisions in the affirmative to continue support the program and provide more funding when it is available. The appendix section contains artifacts used in the evaluation process. Stakeholders are expected to honor the invoice supplied for completing the evaluation exercise.

Description of the Program

The English Department of New Beginning School (NBS) in the Turks and Caicos Islands (TCI) was evaluated to determine the extent to which it is helping non-English speaking students to integrate in the government's public school system. The institution mainly caters for at-risk children who include high school dropouts, and some underprivileged children who are not eligible to be enrolled in the government's public school system because of immigration status and/or their inability to read and write the English language fluently and clearly. The overall aim


of the program is to integrate/ reintegrate students into the Government’s public school system and the society at large.

The program was started seven years ago because teenage students who got pregnant were not being allowed back into the education system. In addition, TCI was considered to be the hub for economic growth, mainly because of its offshore banking opportunities and its rapid development in tourism and the hospitality industry. It attracted people from all over the globe: both investors and workers who traveled there to earn US dollars in order to improve their standard of living. According to statistics from the Statistical Institute, there are over 128 different nationals that speak about16 different languages. Expatriates accounted for approximately 60% of the inlands’ population and worked in both the private and public sectors with the majority being Haitians who mainly do menial tasks. As result of the economic boom, English and non-English immigrants alike came to the TCI shores with their families to settle down and to “rake- in” some of the monies. However, the non-English immigrants especially the Dominicans and the Haitians encounter greater challenges to gain employment and for their children to integrate in a school system in which teachers and students speak an entirely different language; hence, the language barrier prevented them from enrolling in the school system if they are not bi-lingual.

The institution is funded by the Ministry of Social Development and it is providing learning opportunities for the less fortunate of the TCI who would otherwise find it extremely difficult to receive an education which would help them integrate into the school system and the everyday activities that take place in the country. In fact, over three hundred (300) students have passed through the institution over the past three years. However, raising adequate finances to keep the program active is one of the most serious challenges facing the institution at this time.

Program Objectives

The program goal is to integrate/ reintegrate students into the Government’s public school system and the wider society of the TCI at large. At the end of the of the evaluation process, students at the New Beginning School will be able to:

1. Integrate in the public school system

2. Speak English fluently

3. Write English clearly

4. Write cover letter, resumes, and complete job application forms

5. Express a desire/ ability to learn practical skills that will help them to be gainfully employed.

Program Components


The English Department of the New Beginning School has one instructor or teacher on staff and 20 students enrolled.

The timetable so that students meet three times each week for 2 hours on Mondays and Tuesdays and 3 hours on Thursdays.

It is the primary hub where foreigners come to learn English as a second language so that they can be integrated into the society.

The program comprises a syllabus which the teacher adheres to by teaching the fundamentals of English Language with emphasis on the letters of the alphabet, pronunciation, basic sentence construction, spelling, etc.

Evaluation Method

The evaluation program looked at the efficiency, effectiveness, and impact of the program as it relates to the program goal and the extent to which students are achieving it. The Decision-Making Model as discussed in Chapter five of the Evaluation textbook by (Boulmetis & Dutwin, 2011), was used to present my findings to the stakeholders for them to continue support the program’s existence in the future. Survey instruments were developed to be administered to 20 students during the summer session of the school term. The initial intention of the evaluator was to have students complete the survey online using Google Docs Survey - http://bit.ly/MQePwh , but because of confidentiality of information and policy of the institution, the evaluator was not given students’ email addresses in order for him to grant them online access; hence, the instruments have to be administered manually with pens/pencils and papers. However, the principal of the institution gave the evaluator his email address; hence he was able to complete the survey online at: http://bit.ly/OfsrAO. In addition, the evaluator consulted the school register for the records of past students in an attempt to interview some of them to determine the extent to which the program had impacted their lives. The evaluator’s description plan (EDP) below gives a general overview of how the evaluation questions related to the objectives and other variables involved in the evaluation process at NBS.

Evaluator Description Plan (EDP)

Evaluation Questions

Activities to Observe

Data Source

Population Sample Design

Data Collection

Responsibility Data Analysis

Audience

Are students able to be integrated in the public school system?

Track students progress

Survey All 20 subjects

Questionnaire Evaluator Qualitative description

Funding agency

http://bit.ly/OfsrAO

http://bit.ly/MQePwh


Can students speak English fluently?

Listen to students speak/read English

Obser-vation

All 20 subjects

Obtrusive and unobtrusive observations

Evaluator Analysis of variance

Funding agency

Can students write English clearly?

Test Test All 20 subjects

Pretest/Posttest


Funding agency

Can students write cover letter, Resume’, and fill out job application form?

Students write cover letter, Resume’, and fill out job application form.

Test All 20 subjects

Pretest/Posttest


Funding agency

Are students learning any practical skills that will help them to be gainfully employed?

Observe students in practical skill areas.

Survey All 20 subjects

Checklist Evaluator Qualitative description

Funding agency

Participants

Fifteen out of twenty students or 75% of participants responded to the questionnaire that was administered. This represents a good sample of the population.

Consult records such as the registers for pass students’ data that can be used to track their current locations.

The principal of the institution was also interviewed to get background information of the institution and how the programs at the school in general impacted students.

Procedures

Pretest was conducted in order to determine how efficiently students are writing Resume, completing job application forms, in addition to completing some official documents such as bank lodgment forms, work permit forms, labor clearance forms, etc.

The in-class observation had occurred obtrusively when evaluator listened to how students speak to teachers and peers. Unobtrusive observation was done when the evaluator went to the tuck-shop, canteen, and dining area to purchase and eat meals respectively with the students in an


attempt to hear them converse. At times, the evaluator deliberately initiated conversations with students to hear the fluency of their speeches and record his observations without students knowing that they are being studied or measured.

School records were consulted to gather information on students who have left the program in an attempt to track their location and how the program has impacted them.

Survey was administered to students who were present at school that day. No effort was made to contact those students who were absent from school for the day.

Survey was administered to the principal who completed it online in Google Docs survey.

Data Sources

Data were gathered from various sources as listed below:

Pretest Posttest Survey instrument for students Survey instrument for administrator – principal School record - register Obtrusive and unobtrusive observations

Results

A total of fifteen participants were surveyed out of a group twenty students: http://bit.ly/MQePw.

The results for all 15 participants were collected and analyzed. The ages of participants range from 12 to 30 years old.

When questioned about their nationality, 20% reported that they are Turks and Caicos Islanders, 14% are Dominicans, 13% are from different territories while an alarming 53% of respondents are Haitian nationals as can see in the graph below:

http://bit.ly/MQePw


20%

53%

14%13%

What is your nationality?

TCI IslanderHaitianDominicanOthers

When asked about the reason for attending school, of the possible options listed below, respondents were asked to select more than one if they find it applicable: (i) kicked out of high school = 0%, (ii) want to learn a skill = 43%, ((iii) want to learn speak English language = 21.7%, (iv) want to learn to read and write = 8.6%, (v) want to learn to do Math = 17.4%, and a surprising 8.6% who ‘do not have anything better to do.’ It is important to note that some of the selections overlap.

86.6% of respondents selected a resounding yes, when asked if they could speak English fluently, while the other 13.3% selected no. At the same 80% 0f respondents indicated that they can write English clearly while the other 20% are unable to write clearly. For the statement – I can read English Language clearly, 73.3% of respondent said yes, 13.3% said no while the other 13.3% are not sure. To follow up with that statement, the evaluator intentionally asked respondents who read the survey for them, and 80% reported that they did it for themselves while the other 20% sought help. This can be seen in the Pie Chart below:

20%80%

Someone read this survey for me.

YesNo


According to data provided 73.3% of respondents have never been employed before, while the other 26.6% have done some form of work before.

The Likert Scale was used to measure students’ positions on their abilities to be integrated in the other school system because of language barriers and the inability to pay school fees. Their responses are summed up in the graph below: 6% strongly disagree, 14% disagree, 40% don’t know and agree respectively while none strongly agree.

SD D DK A SA0%

5%

10%

15%

20%

25%

30%

35%

40%

Strongly DisagreeDiagreeDon't KnowAgreeStrongly Agree

Responses to the statement: “I will be able to get a job after I graduate from the program” are shown below:

Strongly Disagree = 20%Disagree = 0%Don’t Know = 0%Agree = 40% Strongly Agree = 40%

The following graph is used to represent the Likert Scale that measure students’ abilities to write Resume, cover letters, and fill out job application forms. 20% strongly disagree, 0% disagrees, 13.3% don’t know, while 33.3% agree and strongly agree respectively.


SD

D

Dk

A

SA

0.00% 10.00% 20.00% 30.00% 40.00%

Strongly DiagreeDisagreeDon’t KnowAgreeStrongly Agree

When participants were asked to response to whether or not they are computer literate, 20% strongly disagree, 6.6% disagree, 13.3 don’t know, 33.3% agree and 26.6% strongly disagree.

It is interesting to note that when participants were asked how the program impacted their life styles an overwhelming 93.3% responded yes while only 6.6% responded don’t think it does. The result can be seen in the following graph:

In terms of funding, 13.3% of the respondents’ school fees are paid by government, while the other 86.6% have to pay the required fees.

Participants are desirous of pursuing a practical subject that will enable them to be gainfully employed. As a result, all respondents (100%) said they would like to be enrolled in a skills area.

Finally all 100% of respondents said the program is satisfying their needs and they would like to be enrolled into a college after they were asked to comment on the statement and answer the question that follow respectively: (1) I would like to be enrolled in college after completing the program, and (2) Is the program satisfying your needs?

Discussion

Yes = 93.3%

No = 6.7%

Does the program improve your life styles in any way?

YesNo


From records reviewed, I actually contacted five past students who said that the program impacted them positively. Two of them are currently working in the hospitality industry; one of them is working as a pump attendant at a gas station, another one recently graduated from the Clement Howell High School and applied to the TCI Community College, while the final one is attending the TCI Community College where she is pursuing an associate degree in Hospitality and Tourism Management.

Although 80% of respondents selected a resounding yes, when asked if they could speak English fluently, 20% said no, but both obtrusive and unobtrusive observations revealed otherwise. The fact that 13.3% students said they could not read and another 13.3% are not sure means that remediation is needed.

Recommendations

I. Students should be encouraged to converse in English rather than their native language. This will strengthen their grip on the English Language

II. Establish a reading lab using both French-to- English and Spanish-to English software and any other appropriate strategy so that students can be engaged at all times writing, reading and conversing in English

III. The institution should employ bilingual teachers whenever possible. This will facilitate easy interpretation among teacher and learners.

IV. Use students who develop language skills quickly to help peers communicate in EnglishV. In addition to computer studies, include programs in the curriculum that can lead to a

professional career. This will provide the opportunities for students to develop marketable skills that will qualify them to become gainfully employed. For example, the institution can include courses such as hospitality and tourism, electrical technology, cosmetology, construction technology, plumbing, etc.

VI. Present a career day and mock job fair where students can get a feel of what potential employers are looking for. Give the students the opportunity to fill out official documents such as: (a) bank lodgment and withdrawal forms, (b) work permit application forms, (c) labor clearance forms, and (d) job application forms.

VII. Allow students to frequently practice how to write cover letters and resumes for advertised jobs.

Project Cost

Please click on the link to view invoice online: http://www.aynax.com/lh5l-0fn8-wbjh.view

INVOICE

Fr: Zae Evaluation Services (ZES) To: Ministry of Social Development

http://www.aynax.com/lh5l-0fn8-wbjh.view


122 Blue Hills Rd Down Town Mall Center

Provo Provo

Turks and Caicos Islands Turks and Caicos Islands

Invoice # 0000001

Invoice Date 07/23/2012

Item Description Unit Price Quantity Amount

Days Five (5) 500.00 5.00 2,500.00

NOTES: This bill is for services rendered doing the evaluation at the New Beginning School.

Subtotal = 2,500.00 >>>>>>> Balance Due $2,500.00


Appendixes

Appendix A

Survey Form: Administrator - Principal

You can view the published form here: https://docs.google.com/spreadsheet/viewform?formkey=dEIwUFBhOEI1cDdCX1dYZzhiWUdBMXc6MQ

Is the program goal consistent with the needs of the students?

yes

No

To some extent

When was the program started?

Why was this program started?

What is the name of the program?

Check the boxes that are applicable. The participants of the program are from

Turks and Caicos

Haiti

Dominica Republic

Jamaica

Guyana

Others

Who fund the program?

Does the program get adequate funding?

https://docs.google.com/spreadsheet/viewform?formkey=dEIwUFBhOEI1cDdCX1dYZzhiWUdBMXc6MQ

https://docs.google.com/spreadsheet/viewform?formkey=dEIwUFBhOEI1cDdCX1dYZzhiWUdBMXc6MQ


Yes

No

How do you recruit/ enroll students?

What type of skills are the students participating in the program are equipped with?

Literacy

Numeracy

Business Education

Electrical Installation

Costomology

Other:

What employability plan is in place for students when they graduate from the program?

What is the population of the institution over the pass few years?

2009 to 2012

What are the main challenges faced by the institution?

Appendix B

Survey Form: Students

EDTECH 505 - Evaluation Survey

See form online: https://docs.google.com/spreadsheet/viewform?formkey=dGlWQm5BNUo1emdwbTM1ZmNPdDVVVFE6MQ#gid=0

This evaluation survey is to find out some information about the program you enrolled in at the New Beginning School of Continuous Education. Your responses are greatly appreciated and will help the evaluator gain more insight about your learning experience and also about institution.

https://docs.google.com/spreadsheet/viewform?formkey=dGlWQm5BNUo1emdwbTM1ZmNPdDVVVFE6MQ#gid=0

https://docs.google.com/spreadsheet/viewform?formkey=dGlWQm5BNUo1emdwbTM1ZmNPdDVVVFE6MQ#gid=0


Which age group do you fall in?

12 - 14 years

15 - 17 years

18 - 20 years

Above 20 years

What is your nationality?

Turks and Caicos Islander

Haitian

Dominican

Other:

What is your reason for enrolling in the program at the school?

Click all that are applicable.

Kick out of high school

Want to learn a skill

Want to learn English Language

Want to learn to read and write

Want to learn to do Maths

Did not have anything better to do.

Can you speak English fluently?

Yes

No

I can write English clearly

Yes

No


Not Sure

I can read English Language clearly

Yes

No

Not sure

Someone read this survey for me.

Yes

No

Other:

Have you ever being employed before?

Yes

No

I cannot integrate in the other school systems because of my language barrier and the inability to pay school fee.

1 Strongly Disagree

2 Disagree

3 Don't Know

4 Agree

5 Strongly Agree

I will be able to get a job after I graduated from the program.

1 Strongly Disagree

2 Disagree

3 Don't Know

4 Agree


5 Strongly Agree

Have you ever being employed before?

Yes

No

I learned how to write Resume, cover letters, and fill out job application forms.

1 Strongly Disagree

2 Disagree

3 Don't Know

4 Agree

5 Strongly Agree

I am computer literate.

1 Strongly Disagree

2 Disagree

3 Don't Know

4 Agree

5 Strongly Agree

Does the program improve your lifestyle in anyway?

Yes

No

I don't think it does

The State pays my school fee to enroll in the program


Yes

No

Partially

Which skill training course(s) are you enrolled in?

Which skill training program would you like to be enrolled in if it was offered at the institution?

**Do not select from the list if you have stated it above.**

Business Education

Computing

Cosmotology

Electrical Installation

Plumbing

Woodwork

Auto mechanics

Building Construction

Hospitality

Other:

Is the skill learned will allow you to be gainfully employed?

Yes

No

Don't Know

Will you be able to satisfactorily integrate in the society after completing the program?

Yes

No


Not Sure

I would like to be enrolled in College after completing the proggram

Yes

No

Other:

Is the program satisfying your needs?

Yes

No

Unsure

Would you recommend this program to others who fall in the same position as you? Give reasons.

Appendix C

Bank Voucher


Appendix D

Job Advertisements


Appendix E


References

Boulmetis, J. & Dutwin, P. (2011). The ABCs of evaluation. Timeless Techniques for Program

and Project Managers. Third Edition: John Wiley & Sons, Inc. San Francisco: CA.

Dick, W., & Carey, L. M. (1991). Formative evaluation. In L. J. Briggs, K. L. Gustafson,& M.

H. Tillman (Eds)., Instructional design: Principles and applications (2ndEd.), pp. 227-267. Englewood Cliffs, NJ: Educational Technology Publications.

edtech 505 eval project

Documents