The Computerized ACTFL-based Speech Tool (CAST)

Dr. Mary Ann Lyman-Hager and Ms. Kirsten Barber

San Diego State UniversityMerlot Conference, August 2004

Who are our sponsors?

U.S. Department of Education (International Research and Studies) Grant and National Foreign Language Resource Center

National Science Foundation (Ed Center) U.S. Department of Defense (National

Security Education Program) Samsung Corporation (English as a Foreign

Language)

Who are our partners?

• ACTFL American Council on the Teaching of Foreign Languages)

• CAL Center for Applied Linguistics

• BYU Brigham Young University

• DLI Defense Language Institute

• SDSU San Diego State University

What are the purposes of our talk?

Provide context of the CAST test/tool, including how it will help assess student proficiency in speaking a foreign language and in giving diagnostic feedback

Provide examples of the work to date and how the technologies employed are optimal for testing large numbers of students and for performing linguistic research

Why do we care about proficiency in foreign languages?

Our nation is grossly unprepared to face challenges of an increasingly global world…marketplace, science and math education, politics and terrorism

Less commonly taught languages and heritage languages are particularly vulnerable to neglect, given the over-emphasis on English language proficiency

What is oral proficiency?

Ability to comprehend and orally respond to predictable and unpredictable situations with language that is culturally appropriate, with increasing accuracy and fluency…

Novice, Intermediate, Advanced, Superior, Distinguished, Educated Native

WHY is the TEST ACTFL BASED?

• Unifying professional organization for foreign language teaching in this country (K-18 STANDARDS)

• Creating guidelines for proficiency assessment in all four skills of language (reading, writing, speaking, listening), plus culture

• Providing professional development (K-18+), including training of oral proficiency testers and trainers

How will two specific teams, two separate grants, produce the test?

ACTFL and CAL: Establish a framework for the test

SDSU, BYU, and DLI: Develop the technical specifications for the test and deploy the test

What does the ACTFL oral proficiency test look like?

Two people sit in a room, with a tape recorder

One asks questions, the other answers

Occasionally roles are reversed

Some task cards are introduced at some point in the interview (easier, harder)

The tester makes the candidate feel like this is a normal conversation (warm-up, wind down)

Goal: to elicit a ratable sample of speech

What should the CAST oral proficiency test look like?One person sits in front of a computer with a microphone,

equipped with video and audio capabilities

The computer generates a series of questions, based on on-line background data (interests, background, etc.)

The items are related and sequenced based on an algorithm, with harder/easier tasks introduced at the discretion of the person taking the test and the computer.

The computer makes the candidate feel like this is a “normal conversation” (warm up, wind down)

Goal: to elicit a ratable sample of speech from the testee

What is the CAST test?

Online low stakes, high volume diagnostic assessment instrument that predicts ACTFL Levels from Novice to Distinguished (Levels 0+ to 4+ on the Interagency Roundtable Language Scale used in the government)

Initially, for Spanish and Arabic, later for English, Persian, Chinese, Iraqi dialect, Syrian dialect, Egyptian dialect

What questions are the first team (the Framework Writers) working with?

1.      What are the characteristics that make an oral proficiency assessment task elicit a clear speech sample ratable using the criteria of the ACTFL Guidelines?

2.      What combination of tasks makes for an efficient oral assessment, at each of the four levels of the ACTFL Guidelines?

3.      What adjustments must be made to the general specifications to accommodate specific languages, including less commonly taught languages? 

What questions are the Second Team (the Technical Team) dealing with?

4.      What hardware and software considerations must inform the design of such a test?

5.      What Web design and data encoding standards must inform the framework?  

Answers to these questions will inform the test framework and specifications produced and disseminated under this project.

The Ultimate CAST RESEARCH QUESTION

Has the examinee reached a specific proficiency level of the level-specific test or not?

What does it sound, feel, and look like at various levels of language proficiency?

How can the databank of tests and responses better inform how we teach and learn languages in this country?