Indonesian Oral Proficiency Guidelines

Ellen Rafferty, Juliana Wijaya, Erlin Barnard

COTSEAL/SEASSI 20th Annual ConferenceJuly 16 – 17, 2010

University of Wisconsin – Madison

Southeast Asian Languages Oral Exams:

Assessing What?Achievement (exit) or Proficiency?

Assessing Who?

For What Purposes?

Placement and Achievement Tests:university-based (designed and produced locally)

program-based (in the US: SEASSI, FLAS)in SEA: Advanced SEA Language Study Abroad (COTIM,

VASI, AFAP, ASK, AST)

National Standard Tests: Measuring proficiency by ACTFL or ILR scales

Problematizing Standards:ACTFL and ILR guidelines

ACTFL and ILR scales: Linear progressionsvs Speakers’ non linear progressions

(e.g. heritage language speakers) Eurocentric script disjunction tasks: simple descriptive language may be

easier/harder in one language than the other context and content registers, speech acts

Proficiency-referenced Scales

ILR

0 (No Proficiency) 0+ (Memorized Proficiency) 1 (Elementary Proficiency) 1+ (Elementary Proficiency, Plus) 2 (Limited Working Proficiency) 2+ (Limited Working Proficiency,

Plus) 3 (General Professional

Proficiency) 3+ (General Professional

Proficiency, Plus) 4 (Advanced Professional

Proficiency) 4+ (Advanced Professional

Proficiency, Plus) 5 (Functionally Native Proficiency)

ACTFL

Novice: Low, Mid, High

Intermediate: Low, Mid, High

Advanced: Low, Mid, High

Superior

SEASSIPE: Southeast Asian Summer Studies Institute Proficiency Examinations

Developed by Brown, Ramos, Cook, Lockhart (UH-Manoa) 1988-99

User’s Manual published in 1991 Reports on the design, administration, revision

and validation of SEASSI Proficiency Examinations Goal: to develop overall language proficiency

examinations in: Indonesian, Khmer, Tagalog, Thai and Vietnamese

Objective: to assess the grammatical and communicative abilities of students studying these languages in order to gage their overall language proficiency

Paradigm: ACTFL guidelines: novice-advanced plus (speaking)

SEASSIPE: Oral Exam Format

Students’ responses to the interviewers’ questions are measured by:

1) Meaning: Interviewer’s general meaning was understood and interviewer followed the student’s answer.

2) Fluency: The student answered quickly and with few pauses and hesitations

3) Accuracy: Very few grammatical errors

Thai Oral Proficiency Exam: Chulalongkorn University – UH Manoa

Speaking test formats:CommunicativeReportInterview

What guidelines?Rubrics are similar to ACTFL

Purpose?Exit, proficiency, placement, certification

Indonesian Oral Proficiency Guidelines (COTIM):

developing national standards for Indonesian oral proficiency levels

defining linguistic features associated with levels based on a set of sample interviews collected from seven different Indonesian language programs across the US (bottom up approach)

Test format integrates 3 communicative modes: Interpersonal, Interpretive, Presentational communicative skills

ACTFL scales adapted: novice low - superior

Test Format: (Rubric 1): Novice High – Intermediate Mid

to obtain language samples from students and thus to demonstrate proficiency levels from the novice-high to intermediate-mid levels.

The oral interview consists of two parts: 1) a student presentation based on an illustration (The

presentation can be a concrete description of the contents of the illustration or an interpretation of what is going on with evaluative comments creating a story), followed up by questions about the illustration to determine the student’s ability to express him/herself. 2) a conversation lead by the interviewer (to push to a higher level: the interviewer will request the student to describe and narrate in paragraph level format discussing high frequency concrete topics such as school, work, education, current events, social issues, comparisons).

(Rubric 2): Intermediate High and Above

to obtain language samples from students and thus to demonstrate proficiency levels from the intermediate-high through superior levels.

The oral interview consists of two parts: 1) a student presentation based on a reading. (The presentation should begin with a description/summary of the contents of the article and end with supported opinion about the topic), followed up by questions about the content of the article checking comprehension and pushing the student to achieve the highest level of proficiency obtainable.

2) a conversation lead by the interviewer about a variety of topics (e.g. freedom of expression, human trafficking, etc.) to elicit a language sample that evidences a wide range of vocabulary and different registers. The objective is to push the student to his/her highest skill level.

Indonesian Oral Proficiency Descriptors:

Global tasks/Functions Context Content/Topics Accuracy Text type

Novice Mid:

speak in discrete words and phrases. produce mostly memorized utterances. The context is very predictable and limited to self and

immediate surroundings, dealing with common discrete elements of daily life.

respond to direct questions with limited words, and they need frequent prompting from the interlocutor.

They are highly reactive. Long pauses are frequent, with frequent hesitation, lack

of vocabulary and inaccuracy (pronunciation, grammar, structure, diction, use of question words in answers).

They are understood with difficulty by sympathetic interlocutors familiar with non-native speakers.

Novice High:

can convey information related to self and immediate environment (study and kinship, places, preferences) but producing mostly formulaic and memorized phrases.

can respond to simple and direct questions tend to repeat and rephrase and clarify the interlocutor’s

utterances in their response. use simple sentences although they often still use

incomplete sentences and in phrases (resort to 1st language when tasks are incomplete)

inaccurate pronunciation influenced by his/her First Language, but can still be understood by sympathetic interlocutors accustomed to non-native speakers

Errors are frequent, for example word order, use of negation, omissions of prepositions, affixation, and deletion of verbs

are understood with difficulty by sympathetic interlocutors familiar with non-native speakers.

Intermediate Low:

communicate some basic ideas, although with difficulty; using simple sentences with many long pauses, searching for words.

demonstrate limited ability to create with the language to convey basic, limited personal message.

responses are typically reactive, indicating inability to initiate conversations. Topics include self, family, friends, places, likes and dislikes.

pronunciation is influenced by 1st language (English), understood only by sympathetic interlocutor.

patterned errors are evident, such as word order preposition, deletion of ‘ada’ in negative, prepositional phrase.

Intermediate Mid:

demonstrate ability to create with the language both combining and recombining learned materials.

speak in strings of sentences. can talk about concrete topics, such as self, family,

occupation, travel, school, and leisure activities can communicate uncomplicated idea:

personal experience, future plans, simple comparisons utilize time and sequence marker, use simple

connectors, more frequent use of affixation common patterned errors include passive voice,

frequent inaccurate word choice. when hesitate, revert to their first language.

Intermediate High:

demonstrate emerging ability to describe and narrate. show emerging paragraph, but mostly operate in

strings of sentences. incorporate some communicative strategies such as

asking for clarification, repair (self correct to more formal expressions/forms).

can make more complete comparisons. utilize more variety of connectors. occasional lapses into informal forms & pronunciation,

while formal forms are called for. venture to use more complex patterns, such as more

varied use of affixes and passive voice.

Advanced:

can narrate different time frames (past, present and future).

can describe with greater details and a wide range of word choices.

full conversational partner, functioning in informal and formal settings, but mostly formal settings.

concrete topics of personal and public interest e.g. education, economy, environment, social issues and current events.

can use complex sentences and morphology involving accurate of passive and active sentences, subordinate clauses, nominalization

can self correct and be understood without difficulty by speakers unaccustomed to non-native speakers

connected discourse with adequate cohesive devices

Advanced High:

can state and fully support opinion and explain in details in concrete topics.

emerging some ability to synthesize. emerging ability to conceptualize and discuss abstract

topics. strategies to persuade and convince the interlocutors. emerging ability to demonstrate the socio-linguistics

skills to achieve your communicative tasks, such as to persuade, to apologize, to criticize.

functioning in informal and formal settings, but mostly formal settings with higher presentational skills.

increasing control of verbal affixes and nominal affixes use more extensive vocabulary increasing control of verbal affixes and nominal affixes more extensive vocabulary

Skills of Advanced and Superior Speakers

Advanced level Ability to describe and narrate in paragraph discourse. Some ability to appropriately shift from informal to

formal register. Ability to discuss concrete and factual topics (social and

professional topics) Ability to negotiate a culturally appropriate manner out

of an uncomfortable situation and in so doing demonstrate socio-cultural knowledge.

Superior Level Ability to tailor language to fit the audience, persuade,

negotiate, advocate for a position and interpret. Ability to discuss a wide range of concrete and abstract

topics in extended discourse. Ability to elaborate complex concepts and choose

appropriate words Ability to handle most formal and informal settings.

Future:

More data collection Dissemination Standardization for different purposes Insights for Indonesian language

curriculum and material development: Communicative based Content based Task based Standards or proficiency based ***

THANK YOU

TERIMA KASIH