fact sheet-course design, competency or proficiency, amep research centre

July 2007 Teaching issues 9 Adult Migrant English Programme Research Centre 1

Context

The Adult Migrant English Programme (AMEP)curriculum, along with English Language Teaching(ELT) curricula generally, has undergone considerablechanges over the past several decades. Although theAMEP has its origins in the early fifties it was notuntil the early eighties that AMEP descriptions ofoutcomes were formalised across Australia. Sincethen, two different approaches to assessment andcurriculum design have been used nationally acrossthe AMEP proficiency based and competency based.

In the last decade, there have been substantialchanges to the delivery environment. No longer areAMEP providers serving only AMEP learners.Currently, many providers form classes of studentsfunded from both the AMEP and a number of otherfederal and state sources. Additionally, manyproviders deliver different curriculum approacheswithin the same organisation depending on thefunding source. These changed circumstances haveled to provider concerns about how to best servetheir various clients within the same organisation oreven class.

An additional complicating factor is that proficiencyis used as the language measure for placement withinthe AMEP (using the International Second LanguageProficiency Ratings ISLPR), while competency isused as a measure of learner achievement in theprogramme (using the Certificates in Spoken andWritten English CSWE).

This fact sheet addresses the basis of curriculumdesign and the tension between proficiency-basedand competency-based programmes.

The nature of curriculum and syllabus

While the two terms, curriculum and syllabus, havebeen used interchangeably in the ELT field, manywriters differentiate between them. In the AMEP

context, it is useful to make such a distinction asAMEP instruction is based on the CSWE which is acurriculum framework, not a syllabus. Syllabusdesign is left to providers at the local level whereparticular contexts and learners can be accom-modated. The term course design is used in the titleto reflect this.

Curricula

A variety of curricula have been used in Englishlanguage teaching, both in the AMEP and inter-nationally. The basis of curriculum design can belinguistic-based or subject-based.

Linguistic-basedA number of different approaches fall withinlinguistic-based curricula:

Grammar-based approaches begin with sentencestructures, such as tense, and vocabulary.

Lexical approaches use lexical units as thebuilding blocks for syllabus design. Such unitsinclude chunks and collocations. Choices forinclusion are based on extensive linguisticcorpora such as COBUILD.

Functional/notional approaches begin with thefunctions (such as apologising) that learners needto perform to be communicatively competent. Infunctional/notional approaches, the linguisticfeatures taught are dictated by the function.

Genre- or text-based approaches begin with texttypes (such as short recounts) that learners willneed for the contexts in which they will use thelanguage. While a number of teaching metho-dologies could be used in a text-based approach,this approach is most commonly used withexplicit instruction of the linguistic features of the text and the staging that makes the textcoherent. Texts can be oral or written, monologuesor dialogues.

teaching issues 9Fact sheet Course design:Competency or proficiencyJuly 2007

AMEP Research Centre Fact sheets are developed by the AMEP Research Centre to provide AMEP teachers withinformation on issues of professional concern. They provide a summary of the issue and provide annotatedreferences that can be used to broaden knowledge and extend understanding. These references can be obtainedthrough the AMEP Resource Centre at [email protected]

The AMEP Fact sheets are funded by the Department of Immigration and Citizenship (DIAC) through the AMEPSpecial Projects Research Programme and are informed by the Australian-based research that is funded by thisprogramme. The AMEP Fact sheets can be accessed through the Professional Connections website:http://www.nceltr.mq.edu.au/pdamep/factsheets.html

2 Adult Migrant English Programme Research Centre July 2007 Teaching issues 9

http://www.nceltr.mq.edu.au/pdamep

Subject-basedSimilarly, a number of different approaches usesubject matter as the basis for constructing acurriculum. This approach is usually called content-based instruction (CBI) and refers to the integrationof content and language learning. The choice oflanguage is dictated by the content. Content can beacademic for example, the subject matter requiredby another curriculum in schools or universities. Itcan also be vocational that is, the content learnersneed for specific occupations. In adult ESL, thecontent is often survival content, such as going tothe doctor or getting a job.

Another approach within CBI is theme-based, wherethe syllabus is designed around themes of interest tothe learners such as the environment but notnecessarily serving any other specific outcome.

Although most curricular frameworks have tradition-ally been associated with particular methodologies,they do not necessarily prescribe methodologies. CBI,for instance, can follow communicative metho-dologies or linguistic ones. Similarly, task-basedlearning can incorporate explicit teaching of gram-matical structures as well as communicative tasks.Methodologies are the language-learning activities,tasks, subject matter and learning experiencesselected by the teacher to facilitate learning. Theyare usually based on the teachers theoreticalassumptions about language, language learning andteacher and learner roles.

AMEP curriculum frameworkA curriculum framework provides the theoreticalunderpinnings and the overarching framework forinstruction, including learning objectives and, often,assessment and learning arrangement. A syllabus,on the other hand, provides content and strategiesfor achieving outcomes. It is a coherent course ofstudy for specific learners with the scope andsequence determined by the curriculum framework.

We can therefore consider the current AMEP curricu-lum framework, the CSWE, to be contentless andmethodologically neutral, where the content, metho-dology and learning activities are developed locally,either by providers or individual teachers. While theCSWE does not specify a particular methodology, itdoes require the explicit teaching of the structure andfeatures of texts through modelling, deconstruction,and scaffolded construction prior to independentconstruction by the learner. This teaching/learningcycle underpins the CSWE framework.

A proficiency perspective: ISLPR

As a response to the Galbally Report (1978), theAustralian Second Language Proficiency Ratings

(ASLPR) were introduced to provide a commonmechanism for describing outcomes of the AMEPprogramme. These ratings were based on languageproficiency as a measure of student use of language,whether through formal instruction or informallearning. Proficiency scales set out to establish whatlearners should be able to do with language acrossvarious staged developmental levels. Most norm-referenced assessments, such as the InternationalEnglish Language Testing Scheme (IELTS) or the Testof English as a Foreign Language (TOEFL), adoptproficiency scales where the language targets can beacquired through a variety of ways. In 1997 theASPLR became the International Second LanguageProficiency Ratings (ISLPR) because of its increasinguse outside Australia.

Most proficiency-based assessments rate learnersseparately in the four macro skills of listening,speaking, reading and writing. Systems such asISLPR and IELTS use criterion-referenced assessment that is, learners are assessed against specificcriteria, rather than rated against each other. Thecriteria are simple statements of performance, suchas able to satisfy own simple everyday transactionalneeds and limited social needs (Reading 1+). Thesesimple statements are then extensively expanded todefine the contexts, the types of texts or interactions,the types of grammatical features that should bepresent, and the level of understanding of genre andregister the learner needs to perform to achieve theparticular rating. The ratings also provide examplesof language behaviour to guide assessors as well as comments, such as understanding depends,however, on there being significant controls over theinput (Reading 1+).

ISLPR is a twelve-level scale going from 0 to 5, where5 is proficiency equivalent to that of a native speakerof the same socio-cultural variety.

A competency perspective: CSWE

In the 1980s, the focus in the AMEP was on a learner-centred curriculum, variously implemented from astrong version to a light version. The strongversion involved negotiating with learners all aspectsof course content and methodology, while the lightversion involved teachers conducting a needsanalysis with learners and developing content,materials and methodology based on this analysis.In 1986, the Federal Government commissioned areview of the AMEP which found that, althoughteachers supported the principles behind such acurriculum, in practice they found it placedtremendous demands on their time and expertise.The review also found that learners and teachersconsidered that such a curriculum did not providecontinuity and progression across levels. A key



recommendation, therefore, was to develop curricu-lum guidelines with clear learner pathways in otherwords, to reduce the reliance on teachers as curricu-lum developers and provide a less negotiated syllabus.

These issues were addressed at state level, wheregroups of teachers, working with curriculum coordi-nators, developed curricula for local use. Eventually,NSW Adult Migrant English Service (AMES) developedan integrated curriculum document which theytrialled with their own teachers and learners andthen had accredited within the Australian vocationaleducation system. The Federal Government eventuallyadopted this curriculum framework for the entireAMEP. This framework is the Certificates in Spokenand Written English (CSWE) the current frameworkfor the AMEP nationally against which learners andproviders are currently measured.

The CSWE curriculum framework adopts acompetency-based and text-based approach.Competency approaches to curriculum design arethe basis for the vocational education and trainingsector in Australia the sector in which the CSWE isaccredited. This approach requires learners todemonstrate learning achievements by performingreal-world tasks, usually in simulated situations. To be assessed as competent, learners need to beable to perform all aspects of the task. In thevocational sector in which this approach is situated,it is reasonable to expect that an electrician canperform all stages in rewiring a house or a mechanicperform all stages in wheel alignment. Such anapproach becomes more problematic, however,when applied to communication where not all stagesin an interaction or text type are necessarilycompulsory.

The second theoretical foundation of the CSWE, atext-based approach, is based on systemicfunctional grammar a social theory of language inwhich language description takes the appropriateunit of analysis as whole text in its socioculturalcontext, as opposed to a sentence or utterance.

The current version of the CSWE melds this text-based approach with learning outcomes, rather thancompetencies. Outcomes-based education has along history in general education, where it has hadvery specific approaches to defining and assessingoutcomes (see, for example, Spady 1993). Within theCSWE, learning outcomes are defined as what anindividual can do with language. However, it can beargued that the learning outcomes described in theCSWE still take a competency-based approach inthat the assessment criteria are described as wholetexts with specific, required and optional elements.To achieve a particular learning outcome, learnersneed to meet all the required criteria within thespecified conditions of assessment (such as lengthof text).

A competency perspective: The Canadian LanguageBenchmarks (CLB)

The CLB is also competency-based (that is, itidentifies what learners are able to do), task-based(having learners perform tasks to demonstrateproficiency), and text-based (using texts rather thandiscrete linguistic items as measures of proficiency).CLB, however, differs from the CSWE in that it is nota curriculum framework, but a scale of Englishproficiency. It differs in the way teachers assesswhether learners have met the competency.

Genres are described generally (for example, WritingBenchmark 7: Convey a personal message in a formalshort letter or note, or through email, expressing orresponding to appreciation, complaint, disappoint-ment, satisfaction, dissatisfaction and hope).Instead of required elements, CLB uses PerformanceIndicators (for example Addresses the purpose ofthe task; Expresses main ideas and supports themwith details). These Performance Indicators do notindicate every element that might be in a particulargenre or text, but rather convey descriptions of whatone would expect a particular text to do.

Each benchmark includes competency in fourdistinct areas:

social interaction

reproducing information

business/service messages

presenting information and ideas.

The CLB, therefore, differs significantly from CSWEassessment since, while based on outcomes orcompetencies, measures them in terms of languageproficiency.

Issues

Student placementLearners in the AMEP are placed in specific classesbased on a number of factors. Learners new to theAMEP are assessed using the ISLPR, a proficiency-based assessment. They are also interviewed andtheir previous educational experiences ascertained.They are then assigned to courses at one of the fourCSWE levels based on their scoring on the ISLPR.Courses are:

A Course in Preliminary Spoken and WrittenEnglish students with no previous exposure toEnglish or with very limited skills

CSWE I ISLPR 0

CSWE II ISLPR 1

CSWE III ISLPR 1+



Additionally, because learners come to the AMEPwith diverse previous experiences, within eachcertificate level, learners can be assigned to one ofthree bands:

Band A: slow pace for those with limited formaleducation or low literacy in their first language

Band B: average pace for those with some learningstrategies and education (at secondary level) intheir home country

Band C: fast pace for those with high levels oflearning strategies and some post-secondaryeducation (see Annotated Bibliography belowfor NSW AMES Certificates in Spoken and WrittenEnglish for complete details of levels and bands).

Students who have completed a CSWE level mayprogress to the next level. Those who havecompleted only some of the outcomes for thatcertificate will remain in a course at that level. Thus,in any one AMEP class at a specific Certificate level,there will be learners with the appropriate ISLPR,those who have completed the previous levelCertificate and those who have completed some ofthe learning outcomes for the current Certificatecourse in which they are enrolled. This recognitionof different types of prior learning means they haveacquired different linguistic skills and in differentcontexts. Teachers, therefore, need to have a firmunderstanding of the language achievements of all of their learners.

Our examination of the degree of mapping of theISLPR and the CSWE learning outcomes indicatesthat learners placed through any of the threeprocesses will have gaps. Learners who havecompleted the compulsory learning outcomes in the CSWE may not have reached the equivalentISLPR proficiency level. This is to be expected, giventhat proficiency-based assessments differ fromcompetency-based ones.

A national curriculumAs a result of the Campbell report (1986), the nationalCSWE curriculum was introduced. Not only did thisaddress the issue of learner pathways, it alsoprovided the funder the Department of Immigrationand Citizenship with an accountability measure.Through the national curriculum framework learningoutcomes, DIAC can measure learner progress.However, not only instruction but many variablesaffect learner progress. As well as learners previousschooling experience and literacy, other factors suchas trauma or the exigencies of the settlement processcan affect learning. Additionally, such accountabilityis premised on teachers assessments being validand reliable.

Assessing learner competenceA criticism of the learner-centred, classroom andteacher assessment of proficiency was that suchassessments were not objective. It was claimed thatcompetency-based assessment was more objectivesince the elements of the text were clearly delin-eated for teachers. Since it is an all or nothingassessment, there should be no subjectivity.However, all forms of criterion-referenced assess-ment require professional judgment, whether theyare proficiency- or competency-based. Both theISLPR and the CSWE use training of raters to ensurenorming of assessors. NSW AMES, the developers ofthe CSWE, conduct national moderation, and theNational Working Group in conjunction with theAMEP Research Centre moderates and trials assess-ment tasks (nationally available on the web) alongwith their assessment criteria. However, teachersexercise their professional judgment in situ as towhether a learner has met the criteria fully. Inproficiency scales, such judgments can lead tomovements on the scale. Since competencies are not scaled, teachers have a bi-polar choice only.Some outcomes-based assessments, such as thoseadvocated by professional organisations (for example,Teachers of English to Speakers of other Languagesin their standards documents), provide levels ofperformance of the learning outcome often a three-point scale of: approaches standard, meetsstandards, or exceeds standard.

A number of studies conducted by the AMEPResearch Centre (Brindley 2001) have shown thatCSWE assessments have low reliability whenadministered under certain conditions. Thesestudies suggest that a) assessments of a single taskby a single rater are highly unreliable and b) experienced CSWE assessors do not consistentlyagree on the classification of texts as achieved or not achieved.

Multiple learner outcomesIn many adult English language classrooms inAustralia, AMEP-funded learners are in the sameclass as learners funded through other sources whether by the Commonwealth of Australia (forexample, Language, Literacy and NumeracyProgramme) or by the state. These other fundingagencies require different measures of studentlearning. Thus, while teachers can use the CSWE asa framework for instruction, they need to conductdifferent assessments for different learners. In somestates, institutions map CSWE learning outcomesonto their local or state-mandated curriculum. As we have seen in our mapping of the ISLPR and the CSWE Learning Outcomes, such mappings arerarely completely aligned (see NCELTR websitereferenced in the Annotated Bibliography below forthis mapping).



Classroom implications

For teachers to both teach and assess their learnersin the context described above requires a high levelof professionalism which can only be achievedthrough ongoing professional development. Suchprofessional development needs to include agrounding in assessment both proficiency-basedand competency-based as well as the managementof assessment.

If we expect providers, and therefore teachers, to beaccountable for student learning, we also need tobetter understand the validity of the CSWE. To date,there have been no validation studies of the CSWE sothis question remains unanswered. There is noevidence available concerning the construct validityof the CSWE as a curriculum framework. It thereforeremains unclear whether or not the CSWE can besaid to reflect current understandings of languageability and language use. In terms of content validity,although the CSWE was designed to reflect thelanguage learning needs of the population for whomit is intended, no studies have been conducted todetermine if this is in fact the case.

Annotated bibliography

Bottomley, Y., Dalton, J., & Corbel, C. (1994). From proficiencyto competencies. Sydney: NCELTR.

This volume reports on a research project funded by theDepartment of Immigration and Multicultural Affairsthat documents the introduction of the competency-based CSWE curriculum framework in the Adult MigrantEducation Service (AMES) in Victoria. It describes howAMES teachers worked through curriculum change asthey moved from the proficiency-oriented ASLPR to thecompetency-based CSWE.

Brindley, G. (Ed.). (2001). Studies in immigrant Englishlanguage assessment. (Vol.1). Sydney: NCELTR.

Brindley, G. & Burrows, C. (Eds.). (2001). Studies in immigrantEnglish language assessment. (Vol. 2). Sydney: NCELTR.

These two volumes include several studies conductedby the National Centre for English Language teachingand Research (NCELTR) on assessment within the CSWEcurriculum framework.

Campbell, W. J. (1986). Towards active voice. Report of theCommittee of Review of the Adult Migrant EducationProgram. Canberra: Australian Government PublishingService.

This review was the impetus for a move from anegotiated syllabus to a more systematic curriculumthat provided learners (and teachers) with pathwaysacross language levels. It also led ultimately to theadoption of a national curriculum framework used tomeasure learners and providers.

Galbally, F. (1978). Migrant services and programmes: Reportof post-arrival programmes and services for migrants.Canberra: Australian Government Publishing Service.

This review of the AMEP led to a wide range of post-arrival services for adult immigrants, including unlimitedentitlements to free educational and settlementservices. Provision of English tuition was consideredbasic to successful settlement. The assurance of stablefunding led to the professionalisation of the teachingforce in the AMEP.

Murray, D. E. (2005). AMEP Fact sheet Teaching issues 5.Vocational training and the AMEP. Sydney: AMEPResearch Centre.

This fact sheet describes how vocational content can beincorporated into the AMEP CSWE curriculum. Itdiscusses the issues around trying to develop suchcontent-based courses.

National Centre for English Language Teaching and Research(NCELTR) website:http://www.nceltr.mq.edu.au/pdamep/

The NCELTR website, AMEP Professional Connectionspage, contains links to a literature review of curriculumdesign and mapping of CSWE outcomes to ISLPRproficiency levels.

New South Wales Adult Migrant English Service (2003).Certificates in Spoken and Written English (4th ed.).Sydney: NSW AMES.

The curriculum framework is available through NSWAMES. As well as providing the learning outcomes andassessment criteria, the document includes a detaileddescription of the theoretical bases of the curriculum.

Nunan, D. (1987). The teacher as curriculum developer.Adelaide: National Curriculum Research Centre.

This book describes the role and function of teachers ina learner-centred curriculum. It demonstrates howteachers need to develop considerable expertise towork in such a system and how they also need to beprovided with sufficient time and resources to developcurricula at the classroom level.

Spady, W. (1993). Outcome-based education. Belconnen:Australian Curriculum Studies Association.

This document provides details of how to developcurricula that are outcomes based.

Teachers of English to Speakers of other Languages website:http://www.tesol.org/assoc/

The TESOL website hosts information about the variousstandards projects that TESOL has sponsored over thepast decade, both learner standards and teacherstandards. These standards all take a performance-based or outcomes-based approach. Included in allthese standards are indicators of student learning.



Williams, A. (2004). Fact sheet Teaching issues 3: Enhancinglanguage teaching with content. Sydney: AMEP ResearchCentre.

This fact sheet describes the nature of content-basedinstruction (CBI), possible topics useful for AMEPclients, and factors that teachers need to take intoaccount in applying CBI.

Willis, D. (1990). The lexical syllabus: A new approach tolanguage teaching. London: Collins COBUILD.

This volume describes in detail the lexical approach tosyllabus design. The lexical basis for the syllabus is theCOBUILD database of language corpora.

Yates, L. (2005). AMEP Fact sheet Teaching issues 4:Generic skills. Sydney: AMEP Research Centre

This fact sheet describes the nature of generic skills andwhat teachers in the AMEP can do to help learnersacquire the generic skills essential for their workplaceparticipation. Many of these generic skills are part ofVocational Education and Training (VET) sector trainingpackages.

Compiled byDenise E MurrayExecutive DirectorAMEP Research CentreMacquarie University

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

/Description > /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ > /FormElements false /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles false /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /DocumentCMYK /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /UseDocumentProfile /UseDocumentBleed false >> ]>> setdistillerparams> setpagedevice

fact sheet-course design, competency or proficiency, amep research centre

Documents