diagnostic test for social english programs

7/29/2019 Diagnostic Test for Social English Programs

1/17

Diagnostic Test for Social English Programs

Ramon Mislang

December 12, 2012


2/17

Ramon Mislang MATESOL Component 8 Assessment Instrument 1

Audience and Intended Purpose

Trends inusing social media platforms such as Facebook, You Tube, and Gmail, have

changed the way people communicate. As a result, language learners must now deal with a genre

of English that demands the integration of listening and writing skills as well as knowledge of

idiomatic language to communicate effectively in Social environments. I have designed a

Diagnostic Test (Appendix A) for Japanese learners of English who are preparing for study in

America. The purpose of the test is to determine the degree to which the specific instructional

objectives of Social English programs have already been met. The test results will inform

teachers which areas of English students still need support.

Definition of the Constructs

The first construct the test seeks to measure is vocabulary, which is the total number of

lexical items a learner knows. Davies (1999) states that the function of a vocabulary test is to test

a learners breadth (number of words) of vocabulary as well as the depth (how well words are

known). My test is concerned with idioms, which according to Oxford Dictionary (2011), are

groups of words established by usage as having a meaning not deducible from those of the

individual words. My test will help teachers observe learners depth of vocabulary. A description

of how idioms will be tested will be explained in the prompt and response attributes section.

The test also seeks to measure listening comprehension. Listening is a process that is

difficult to observe directly (Davies, 1999) because meaning is not simply extracted from input

but involves test-takers knowledge of the language system, their prior knowledge, and the

context of the interaction (Vandergrift & Goh, 2009). However, there are tasks that test-

developers can create to elicit behaviors that are desired from language learners engaged in the

process of listening.


3/17


Buck (2001) describes three types of observable behaviors, which include; (1) the ability

to process extended samples of realistic spoken language automatically and in real time, (2) to

understand the linguistic information that is unequivocally included in the text, and (3) to make

whatever inferences are unambiguously implicated by the content of the passage (Buck, 2001).

Listening ability will be evaluated on the basis of recalling content heard in the audio clip and

applying pragmatic knowledge to display understanding of the content recalled. Pragmatic

knowledge refers to the understanding of illocutionary force of an utterance and interpreting the

intended meaning in terms of that (p.104).

The final construct my diagnostic test seeks to measure is summary writing. According to

Brown and Day (1983), summary writing is the process of deleting trivia, avoiding redundancies,

substituting a super-ordinate action for a list of actions, selection of a topic sentence, and

inventing a topic sentence.

Rationale and Framework for Design

Following principles of communicative testing (Buck, 2001), this test is less concerned

with how much learners know about the target language and more concerned with how

effectively they can communicate in the target language. Buck (2001, p.253) states that in many

communicative situations, verbal information is accompanied by visual information and playing

a disembodied recording from an audio-player does not create a very realistic listening

situation. Therefore, audio and video stimuli are supported by contextual information such as

the name of the audio and video clip and character profile pictures.

One ofSwains (1984) principles of communicatve language development, bias for best,

refers to encouraging and eliciting the test takers best performance, which is directly related to

the way a test is designed. Bias for best features include the clarity of instructions, explicit


4/17


attention to time for completing a task, suggestions for how to proceed with a particular task,

visual aids that support mental processes for completing a task, and space for note taking.

Listening comprehension and composition sections include all of the above elements for the

purpose of encouraging the best possible performance out of test-takers.

Prompt and Response Attributes

The Vocabulary Knowledge section (Appendix B) consists of ten multiple choice discrete

point items. Each discrete point item has a sentence which contains an idiomatic expression that

is underlined and embedded in a sentence to give it context because as Nation and Chung (2009)

state, words often do not make much sense unless they are in phrases or larger units. Below each

sentence are four choices from which test-takers must choose which best represents the meaning

of the underlined idiomatic expression. The second section, listening comprehension, contains a

dictocomp or integrative skills section where test-takers listen to auditory stimuli in the form of

two audio clips and then respond to a prompt by writing a short answer

The final section is composition for which test-takers watch a short video clip and carry

out mental processes similar to those of the listening section, but must also connect paralinguistic

(body language) features with sociolinguistic variables. Sociolinguistic knowledge refers to the

understanding of the language of a particular socio-cultural setting and interpreting utterances in

terms of the context of the situation (Buck, 2001). After watching the video clip, test-takers must

compose a summary of fifty to one hundred words. Answer keys and scoring rubrics for response

attributes are located in Appendix B.


5/17


References

Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language

assessment. New York: McGraw-Hill.

Brown, A. L., & Day, J.D. (1983). Macrorules for summarizing texts: The development of

expertise.Journal of Verbal Learning and Verbal Behavior, 22, 1-14.

Buck, G. (2001). Assessing listening. Cambridge: Cambridge University Press.

Davies, A. (1999).Dictionary of language testing. Cambridge: Press Syndicate of the University

of Cambridge.

Idiom. (2010). In Oxford Dictionaries. Retrieved April, 15, 2010 from Oxford Dictionaries.

http://oxforddictionaries.com/view/entry/m_en_us1256944.

Nation, P., Chung, T. (2009). In M.H. Long & CJ. Doughty (Eds.) The handbook of language

teaching (p.543-559). West Sussex, UK: Wiley-Blackwell.

Swain, M. (1984). Large-scale communicative language testing: A case study. In S. Savignon &

M. Burns (Eds.), Initiatives in communicative language teaching (pp. 185-201). Reading,

MA: Addison-Wesley

Vandergrift, L. & C. Goh, Teaching and testing listening comprehension, in Long, M. &

Doughty, C. (eds.)Handbook of language teaching(Blackwell, 2009), pp. 395-411.
http://oxforddictionaries.com/view/entry/m_en_us1256944http://oxforddictionaries.com/view/entry/m_en_us1256944http://oxforddictionaries.com/view/entry/m_en_us1256944


6/17


Appendix A: Diagnostic Test

Section A: Vocabulary Knowledge (1 x 10 points)

Choose the best definition that you think represents the meaning of the words in bold type. You

have 10 minutes to complete this section.

Example:

English is too hard. I give up!

a. Quit a habit b. Please help me

c. Offer to you d. Stop trying

1. You got the job! Good for you!a. You are smart! b. Healthy!

c. Congratulations! d. Rich!

2. Tom said he will be right back.a. turn around b. return soon

c. never be wrong d. stand behind you

3.

Hes rich, but moneyisn't everything.a. everywhere b. a lot

c. important d. useful

4. An increase of 25 cents per gallon of gas is nothing to sneeze at!a. quite significant! b. hard to understand!

c. not really important! d. something that makes you sneeze!

5. Paul is down on himself just because he got one bad grade.a. lying on the floor b. criticizing

c. disappointed with d. agreeing with


7/17


6. The test will be next month, so make a mental noteofthat!a. remember b. write down

c. believe d. forget

7. John will break upwith Mary today.a. divide into pieces b. stop working

c. end the relationship d. start laughing

8. You got to be kidding me!a. teasing me! b. joking with me!

c. serious with me! d. making fun of me!

9. I never know where a pimple will pop upnext.a. burst upwards b. fly outwards

c. suddenly appear d. go away

10.Things rarely work out in such a simple way.a. get exercised b. get resolved

c. get created d. get calculated

This is the end of Section A. If you have finished early, please remain in your seats

and wait for further instructions.


8/17


Section B: Listening Comprehension (2 x 5 Points)

This section has two audio clips. Before listening to each audio clip, take 30 seconds and read

information about each audio clip:

Name of audio clip 1:Elaines Birthday

Information about audio clip 1: The two characters in audio clip 1 are Jerry and Elaine. The

audio clip starts with Jerry and Elaine talking at Jerrys house.

Elaine Jerry Setting

Audio Clip 1

- Before audio clip 1 starts, take a few seconds to preview the question below.- You will soon listen to an audio recording that is 1 minuteand 30 seconds long.- After the audio recording stops, you will have 5 minutes to write your answer to the

question below.

*You may use this space to take notes while you are listening.

Question 1: Please describe what is happening in the audio clip.

__________________________________________________________________

__________________________________________________________________

__________________________________________________________________

__________________________________________________________________


9/17


Name of audio clip 2:Elaines Birthday

Information about the audio clip 2: The three characters in the audio clips are Jerry, Elaine,

and Kramer. The audio clip starts with Kramer visiting Jerrys house.

Elaine Jerry Kramer Setting

Audio Clip 2

- Before audio clip 2 starts, take a few seconds to preview the question below.- You will soon listen to an audio recording that is 1 minuteand 30 seconds long.- After the audio recording stops, you will have 5 minutes to write your answer to thequestion below.

*You may use this space to take notes while you are listening.


__________________________________________________________________

__________________________________________________________________

__________________________________________________________________

__________________________________________________________________

This is the end of Section B. If you have finished early, please remain in your seats and wait

for further instructions.


10/17


Section C: Composition (10 points)

A short video clip (5min) will be shown. After watching the video clip, you will have 20 minutesto write a summary about the video clip you just watched. Your summary should be between a

50 and 100 words. But before watching the video clip, take 30 seconds and look at the name of

the video clip as well as the characters below:

Name of Video clip:Elaine goes to the beauty salon

Characters:

Elaine George Mr. Costanza(Georges Father)

Kramer Jerry

While you are watching the video clip, you may use the space below to take notes.

*Notes:


11/17


______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

This is the end of the test. If you have finished early, please remain in your seats and wait



12/17


Appendix B: Test Answer Key

Section A: Vocabulary Knowledge (1 x 10 points)

Choose the best definition that you think represents the meaning of the words in bold type. You

have 10 minutes to complete this section.

Example:

English is too hard. I give up!

a. Quit a habit b. Please help me

c. Offer to you d. Stop trying

1. You got the job! Good for you!a. You are smart! b. Healthy!

c. Congratulations! d. Rich!

2. Tom said he will be right back.a. turn around b. return soon

c. never be wrong d. stand behind you

3.

Hes rich, but moneyisn't everything.a. everywhere b. a lot

c. important d. useful

4. An increase of 25 cents per gallon of gas is nothing to sneeze at!a. quite significant! b. hard to understand!

c. not really important! d. something that makes you sneeze!

5. Paul is down on himself just because he got one bad grade.a. lying on the floor b. really criticizing

c. disappointed with d. Agreeing with


13/17


6. The test will be next month, so make a mental noteofthat!a. remember b. write down

c. believe d. forget

7. John will break upwith Mary today.a. divide into pieces b. stop working

c. end the relationship d. start laughing

8. You have got to be kidding me!a. teasing me. b. joking with me.

c. serious with me. d. making fun of me.

9. I never know where a pimple will pop upnext.a. burst upwards b. fly outwards

c. suddenly appear d. go away

10.Things rarely work out in such a simple way.a. get exercised b. get resolved

c. get created d. get calculated

This is the end of Section A. If you have finished early, please remain in your seats

and wait for further instructions.


14/17


Section B: Listening Comprehension (2 x 5 Points)

This section has two audio clips. Before listening to each audio clip, take 30 seconds and look at

the name of the audio clip, information about each audio clip, and the characters that will be in

the audio clip:

Name of audio clip 1: Elaines Birthday

Information about audio clip 1: The two characters in audio clip 1 are Jerry and Elaine. The

audio clip starts with Jerry and Elaine talking at Jerrys house.

Elaine Jerry Setting

Audio Clip 1

- Before audio clip 1 starts, take a few seconds to preview the prompt below.- You will soon listen to an audio recording that is 1 minuteand 30 seconds long.- After the audio recording stops, you will have 5 minutes to write your answer to the

prompt below.

Scoring Rubric

Score General Description

0 Missing Content/completely Misunderstood1 Complete and correct information

Rater 1 and Rater 2 scores will be averaged to calculate final score.


(1)Jerry surprises Elaine with a birthday present.

(1)But Elaine is unhappy with the birthday present because it is cash.

(1)Because they are friends, it appears that she expects more than cash.

(1)Elaine seems to have expected a more thoughtful present that reflects their friendship.

(1)So she is surprised that Jerrys gift is less than expected.


15/17


Name of audio clip 2: Elaines Birthday

Information about the audio clip 2: The three characters in the audio clips are Jerry, Elaine,

and Kramer. The audio clip starts with Kramer visiting Jerrys house.

Elaine Jerry Kramer Setting

Audio Clip 2

- Before audio clip 2 starts, take a few seconds to preview the question below.- You will soon listen to an audio recording that is 1 minuteand 30 seconds long.- After the audio recording stops, you will have 5 minutes to write your answer to thequestion below.

Scoring Rubric


0 Missing Content/completely Misunderstood

1 Complete and correct information



(1)Elaine is surprised that Kramer gave a birthday present to her.

(1) The present turned out to be a bench, which is actually something that she wanted.

(1)In addition, Kramer gives Elaine a birthday card with a poem written in it.

(1) Elaine seems touched by the poem.

(1) So she is happy with both Kramers present and his birthday card.

This is the end of Section B. If you have finished early, please remain in your seats and wait



16/17


Section C: Composition (10 points)

A short video clip (5min) will be shown. After watching the video clip, you will have 20 minutesto write a summary about the video clip you just watched. Your summary should be between 50

and 100 words. But before watching the video clip, take 30 seconds and look at the name of the

video clip as well as the characters below:

Name of Video clip:Elaine goes to the beauty salon

Characters:

Elaine George Mr. Costanza

(Georges Father)

Kramer Jerry

While you are watching the video clip, you may use the space below to take notes.

*Notes:


17/17


Scoring Rubric


0 Missing Content/completely Misunderstood

1 Partial Information/mostly Correct

2 Complete and correct information


(2)Elaine is late for her appointment at the nail salon, which is run by Korean

ladies.(2)Because she is late, she has to wait for the next available salon

employee. (2)While she is waiting for someone to work on her nails, she hears

the Korean women speaking Korean and thinks they are speaking about her. (2)

After she suspects that they are calling her a dog, Elaine gets Georges dad

who speaks Korean and brings him to the salon. (2)Georges father becomes

angry and confronts the Korean ladies because he knows they are saying bad

things about him. (97 Words)

This is the end of the test. If you have finished early, please remain in your seats and wait


diagnostic test for social english programs

Documents