performance assessment in vocational training

Upload: hotsaxon

Post on 13-Jan-2016

16 views

Category:

Documents


0 download

DESCRIPTION

Performance Assessment in Vocational Training

TRANSCRIPT

  • PERFORMANCE ASSESSMENT IN VOCATIONAL EDUCATION AND TRAINING Geoff Hayton and Zita M Wagner ~Mh;;Jt1lt@dr'l$*~f'~ ;ax;.v..S31H@~::mt l4f!;ti;$n{~ i$ ... W @ikMtiWiQiM#W !41i-$4

    This paper describes performance assessment and some of its important attributes in the vocational education and training setting. It is asserted that the widespread introduction of competency-based education and assessment in recent years has reinforced the use of performance assessment. Some examples of performance assessment in vocational education and training are examined to illustrate some of the important attributes of performance assessment. The paper then discusses some of the key issues concerning performance assessment, including validity, reliability, costs and consequences of assessment. It is argued that performance assessment is a very useful tool in vocational education and training, and may not need to satisfy all the reliability criteria of paper and pencil tests.

    Introduction

    In recent years the use of performance assessment has increased steadily in vocational education and training and other sectors in education systems across the world (Broadfoot 1995). Increasingly educators are called upon to promote the learning of skills, knowledge and attitudes that cannot be tested by the traditional paper and pencil assessment techniques. Performance assessments are expected to deliver a wide range of benefits to the learner such as higher motivation, deeper and more meaningful learning, better connection between assessment and learning and more valid assessment.

    There has been much debate in the United Kingdom and the United States concerning the use of performance assessment for elementary school students and particularly the use of performance assessments for accountability purposes to make comparisons across schools and regions. The proponents of performance assessment (Wiggins 1989; Frederickson & Collins 1989; Linn & Baker 1996) have mounted a sustained attack on the traditional measurement model of assessment and especially on standardised norm-referenced testing using multiple choice items. The opponents of performance assessment have questioned its validity (Messick 1994) and outlined its psychometric problems (Shavelson et al. 1993).

    Performance assessment In vocational education and training 69

  • Performance assessments have a long history of use in the vocational education and training sector and in vocational courses in higher education. This paper will seek to define the characteristics of performance assessment in the vocational education and training setting, with reference to some examples, and examine some of the key issues concerning performance assessment.

    What is performance assessment?

    The term performance assessment covers a wide range of approaches to assessment. It is easier to say what it is not than to define what it is and indeed much of the debate in north America focusses on performance assessment as an alternative to standardised testing using multiple choice items. Hence the use of the terms 'alternative' and 'performance' assessment interchangeably. Other terms used for performance assessment include 'performance based', 'authentic' and 'direct' assessment.

    Performance assessment is a technique or method of assessment which seeks to assess a wide range of outcomes such as skills, attitudes and knowledge in a multi-dimensional way (Green 1995). While there is no agreed specific defining characteristic, a key characteristic of performance assessment is the requirement that students be assessed on performance of a practical activity which is the criterion activity (e.g. performance of a job task at the workplace), or which is a simulation of the criterion activity. An extended definition has been provided by Linn (1993, p.9). He states that the term performance assessment:

    70

    ... refers to assessment tasks that require students to perform an activity (e.g. laboratory experiment in science) or construct a response. Extended periods of time, ranging from several minutes to several weeks, may be needed to perform a task. Often the tasks are simulations of, or representations of, criterion activities valued in their own right. Evaluations of performance depend heavily on professional judgement.

    (Linn 1993, p.9)

    Australian and New Zealand Journal of Vocational Education Research

  • In vocational education and training the set activity is usually the criterion of competent performance on the job, or a task which is to be performed off the job, the task being a close representation of the criterion activity. Examples of the latter include constructing a wall frame, writing out an airline ticket, and detecting faults in a computer.

    The distinction needs to be made between performance assessment and competency-based assessment. Competency-based assessment is the. assessment of evidence to determine a person's current abilities against a given set of competency standards (Rutherford 1995). Competency-based assessment then is a system in which a number of assessment techniques can be used, of which performance assessment is just one technique (Hager et al. 1995). However, performance assessment is likely to be used in a competency-based system because both the system and the technique have a focus on criterion activities or outcomes.

    Performance assessment is an assessment technique which can be used in a wide range of contexts such as in a workplace competency scheme, in a vocational education and training, school or university curriculum or indeed to select the world champion in formula one motor racing or the winner of the Archibald portrait painting prize.

    Attributes of performance assessment

    We have identified six attributes of performance assessment in vocational education and training which are seen as important in the literature on performance assessment. These include:

    .:. the assessment activity reflects the criterion activity or realistic workplace activity

    .:. assessment is multi-dimensional, encompassing more than knowledge

    .:. assessment can be of a product or a process or both

    .:. assessment spans a continuum from simple to complex activities

    .:. assessment is open ended

    .:. scoring requires human judgement

    Performance assessment in vocational education and training 71

  • As already indicated in the definition of performance assessment, the first two attributes may be regarded as essential. Each of the six attributes now will be briefly described.

    Realistic workplace activity

    The criterion for a vocational education and training program is usually competent performance on the job, as a whole or in some part of the job. So performance assessment usually should be based on actual job performance at the workplace, as a whole, or in particular aspects or tasks that comprise the job. The performance assessment can occur in the workplace or off-the-job and can be a simulation or a 'real' activity (Linn 1992).

    Assessment is multi-dimensional

    Performance assessments are practical assessment methods which assess a wider range of outcomes than knowledge alone.

    Performance assessments require students to demonstrate skills by actually performing. They involve doing rather than just knowing about . ..

    (Linn & Gronlund 1995, p.241)

    Most 'real' workplace activities will call for the application of knowledge, skills and attitudes rather than just abstract knowledge alone. For example, a child care worker interviewing a parent about a child's progress will use the knowledge of child development, the skills of interviewing and the attitudes and values of concern, information giving and following a job through to the end. In this way performance assessment seeks to capture much more than a paper and pencil test for knowledge:

    Product or process

    Performance assessments can involve the assessment of a product (e.g. a business plan, a soup or a stained microscope slide) or the observation of

    72 Australian and New Zealand Journal of Vocational Education Research

  • a process (e.g. a presentation, the use of a fire extinguisher, operating a fork lift) (Messick 1994, p.14; Fitzpatrick & Morrison 1971; Linn & Gronlund 1995). In some settings it will be more appropriate to assess the produCt rather than the process. For example, in the domains of . painting and creative writing there may be very many acceptable processes to achieve a product, so that it is more feasible to assess the product rather than the process. In other domains such as auto mechanics both process and product warrant sc~tiny. Another example of assessment involving both process and product is observation of th~ taking of a clinical history from a patient, where both the process (the' interaction) and the product (the written history) can be assessed.

    In other domains such as acting and dancing the process and the product ' are essentially the same thing. A product from a performance assessment may well be just a piece of paper with writing.on it-for example, a business plan-but it, and the processes leading up to its completion, need to reflect realistic workplace activity.

    Simple to complex

    Within the area of performance-based assessment there is'a continuum of complexity ranging from the performance of a small, simple skill sample (turning on an oven) through to a complex multi-dimensional activity (planning, preparing and serving a three-course meal for a function). Linn and Gronlund (1995) use the terms restricted performance to refer to assessment of specific skills and extended performance to refer to the integration of knowledge, skills and attitudes in the asse,ssment of more complex learning outcomes.

    Openendedness

    Another attribute of performance assessment is its openendedness (Baker et al. 1993). The assessments are not fixed-choice and a stu~ent may respond to a task in a number of ways, some of which may be unexpected by the assessor. This is more likely to occur as the complexity of the set task increases.

    Performance assessment in vocational education and training 73

  • Human judgement Another feature of performance assessments is that they are usually rated by human scorers using predetermined criteria (Green 1995). Some degree of assessor expertise is therefore required. The assessors can score the performance holistically or globally, based on an overall impression, or analytically, using a list of criteria (Athanasou 1997). The development of rating sheets or assessment guidelines and the training of assessors to do the scoring have become aspects of assessment that are of great importance.

    Examples of performance assessment

    Performance assessment is not new. Mehrens (1992) cites the biblical example of the Gilead guards who challenged the fugitives from Ephraim:

    Are you a member of the tribe of Ephraim? they asked. If the man replied that he was not, then they demanded, 'Say Shibboleth'. But if he could not pronounce the 'sh' and said Sibboleth instead of Shibboleth he was dragged away and killed. As a result 42000 people of Ephraim died there at that time.

    (Judges 12: 5-6).

    Performance assessments have also been used in the military for at least 50 years and in industry for 70 years (Bond 1995). Performance assessments have a long history in being employed in examinations for professional certification such as medicine (van der Vleuten &. Swanson 1990) law and architecture. Projects, portfolios, extended problems, presentations and others have long been used in school, vocational and higher education.

    Example from vocational education: Tourism Certificate course

    In the Tourism Certificate course in New South Wales, students are studying to be travel agents. In the subject International Airfares and

    74 Australian and New Zealand Journal of Vocational Education Research

  • Ticketing 1 the assessment task requires students to write out an airline ticket to suit the requirements of a specific customer. This is a simulation of a specific workplace activity common to all travel agents around the world. The assessment activity is 'open book' and requires students to use the various air tariffs, airline information and other industry manuals, maps and guides. This is an important aspect of the assessment activity since a travel agent at the workplace would normally complete the activity having all these guides at hand.

    The assessor bases the assessment on the accuracy and completeness of the written-out .ticket using a marking guide in the form of a correctly filled out ticket and the assessor also has instructions on how to mark incorrect or missing sections. This is a performance assessment based mainly on the product although assessors can follow the students' progress through the task as various sections of the ticket are filled in and can therefore assess the process too. This assessment is a complex performance assessmenf and displays in a holistic way the achievement of four main learning outcomes in International Airfares and Ticketing 1 listed below:

    .:. interpret airline guides and manuals

    .:. construct airfares and air itineraries

    .:. provide accurate quotations and advice on air fares

    .:. accurately issue documents .

    This performance assessment fulfils the first two essential attributes namely reflecting a realistic' workplace activity and it is multi-dimensional. It is also a complex activity of a combined product/process and assessor judgement is required to' score the activity. The only attribute it does not appear to have from the list of six is openendedness. Assessors on the subject International Airfares and Ticketing 1 inform the authors that there is only one way to fill in the ticket correctly.

    Example from the workplace: Printing machining

    An example in a workplace training program of performance assessment is provided by the Printing Machining Trade Recognition Scheme in NeW South Wales. People seeking recognition as a trade skilled worker

    Performance assessment in vocational education and training 75

  • in printing machining must seek assessment and certification from the relevant government agency-the New South Wales Department of Training and Education Coordination (DTEC). The normal path to certification is a three-year training and employment apprenticeship involving a three-year part-time course in printing machining at a technical and further education (TAFE) college and formal and informal on-the-job training.

    People not in this path (e.g. people who were trained overseas) may seek trade skilled recognition through DTEC. This recognition scheme is based on an assessment schedule which contains units of competency and associated elements of competency drawn from the national competency standards for the printing machining occupation. The assessment is conducted by an industry training adviser from DTEC. For each element of competency on the schedule the adviser seeks evidence of competency. Depending on the individual case, for each element the evidence may be completion of a college course, observation by the person's workplace supervisor or workplace trainer, observation by the adviser, or other documentary evidence. The particular type of evidence used to assess each element of competency is noted on the assessment schedule.

    The observations by the supervisor, workplace trainer or DTEC advisor are clearly assessments which are performance based, and in the case of printing machining are mostly process rather than product assessments. The activities assessed are nearly always actual workplace tasks which are multi-dimensional, requiring the application of knowledge, skills and attitudes. Typical assessment activities involve the setting up of a particular type of printing machine in the workplace to produce printed product according to the written requirements of a customer. This requires several steps which may be performed in various ways. The task is both complex and open-ended, and requires holistic judgement by the assessor observing the process. In some cases the performance assessment is supplemented by oral questioning to assess the person's underpinning knowledge.

    76 Australian and New Zealand Journal of Vocational Education Research

  • Issues concerning performance assessment

    The literature identifies several issues of quality and usefulness associated with performance assessment. Most of these issues also apply to other types of assessment. The key issues concerning performance assessment include:

    .:. validity

    .:. consequences of assessment

    .:. use of resources

    .:. reliability

    .:. holistic assessment

    Each of these are discussed below.

    Validi1y

    Some argue that in comparison with traditional assessment approaches, performance assessment provides more valid information. Performance assessments appear to have good face validity (Mehrens 1992) in being acceptable to industry and the community. This face validity assists in the acceptance of performance assessment to the various stakeholder groups but it is not sufficient on its own and cannot take the place of overall test validity. Overall test validity may be defined as the 'degree to which a certain inference from a test is appropriate and meaningful' (Athanasou 1997, p.160).

    Two related aspects of validity are authenticity and directness. It is claimed that compared to' other approaches to assessment, performance assessment has the central advantages of authenticity and directness (Frederickson & Collins 1989). Authenticity in assessment means that all or nearly all of the criterion construct (usually a vocational education and training goal expressed as a job performance at the workplace) is captured by the assessment task. Directness in assessment means that none or few skills outside the criterion construct are captured by the assessment task. Performance assessment is often thought to be more reflective of workplace requirements because of its authenticity and

    Performance assessment in vocational education and training 77

  • directness, by assessing in a way as close as possible to a real world situation (Linn & Burton 1994).

    There may be problems with validity related to limited sampling and lack of generalisability from the sample to the wider domain (Mehrens 1992). Because performance assessments take more time, assessors can present only one or two performance assessments to students so the sampling is less dense. There is therefore limited sampling from a domain and the difficulty then of generalising from the performance to the whole domain. Linn (1993) suggests that the number of tasks should be increased to improve coverage of the domain. This may be overcome partly by choosing a task which is very typical of the subject area being assessed (e.g. completing a typical billing transaction at a hotel desk). The other option is to choose a complex or advanced task where the assessor can infer from successful completion of the performance that the student can perform a range of less complex activities. Task choice needs to be carefully considered as it is potentially a major source of measurement error (Guion 1995).

    Messick (1994) suggests that performance assessment needs to meet more validity criteria, particularly those related to construct validity, than traditional assessment. Messick specifies that the assessment should not be too narrow or too broad and should be based on the construct that the performance represents. Linn and Baker (1996) also propose more extensive validity criteria for performance assessment including consequences (how does assessment affect learning), efficiency and economy (is it too costly to implement), generalisability (how generalisable is the performance), and fairness (do all students, including minority groups, have the opportunities to learn). In addition, Frederickson and Collins (1989) propose transparency (to what extent are the criteria for judgement clear to the learners).

    Consequences of assessment

    Authenticity and directness in performance assessment lead to positive consequences for teaching and learning (Linn 1993; Torrance 1995; Wiggins 1989; Frederickson & Collins 1989). For vocational education

    78 Australian and New Zealand Journal of Vocational Education Research

  • and training, assessment which is authentic to the performance required at the workplace is expected to provide the student with a meaningful educational experience that facilitates learning and skill development (Messick 1994). Students are more likely to be motivated to learn and have their learning reinforced when they perform assessments which are grounded in realistic workplace activities (Linn 1992). Performance assessments cover a richer array of student knowledge and skill than is possible with simple skill tests, and provide realistic contexts for the production of student work by having the tasks and processes parallel those at the workplace (Arter & Spande11992). Performance assessments are more closely linked to curriculum frameworks, particularly those that are competency-based.

    Additionally, performance assessments are claimed to be less stigmatising, more adaptable to individual student needs, more useful for employers, parents and others and more reflective of the quality of student understanding (Meisels et a1. 1995). They are also claimed to be less biased and more fair to disadvantaged groups because they do not test abilities, such as speed and recall of decontextualised information which are not connected to the experiences of students (Bond 1995).

    However, the interpretation of performance test scores is more demanding than traditional modes of assessment and teachers who make decisions about student progress must understand what is being assessed and also the procedures used to rate students and the standards required (Brennan & Johnson 1995).

    Use of resources

    There may be problems with either keeping the questions secure or developing new questions for further use (Mehrens 1992). If exactly the same performance is required semester after semester students can memorise the response to higher order questions just as they can to more basic questions. This may present no problems for some performances (e.g. baking a croissant) but any performance tasks that involve a larger meta-cognitive component cannot be so readily reused: This may be at odds with notions of transparency of assessment. In these cases there

    Performance assessment In vocational education and training 79

  • may be higher developmental costs than anticipated and there may also be difficulties making comparisons of cohorts of students/trainees across years if this is required.

    Time required and therefore the cost of performance assessment is high. There are also consumables and equipment costs. Assessors may need extra training to develop the skills of observation and recording for performance assessments. Performance assessments generally yield less information per hour than traditional assessment such as short answer supply or multiple choice tests (Green 1995).

    Wolf (1995) cautions that in the United Kingdom there has been an enormous increase in the volume of assessment (and the amount of class time devoted to it). This may be due in part to the atomistic approach adopted to assessment and the fragmentation of the curriculum. Also noted by Wolf is the observation that when the volume of assessment becomes large, formative assessment disappears.

    Reliability

    Problems with reliability centre on sampling issues, subjectivity of the assessors and to what extent we can generalise from the performance to the larger domain. Because performance assessments take more time than traditional assessments and use more assessor time, there are generally fewer tasks. This lesser quaJ.ltity of information collected from performance assessment usually will lower reliability (due to the sampling problem discussed above). For acceptable levels of score reliability there should be more than one task, and preferably several tasks. Brennan and Johnson (1995) found that the number of tasks has a strong effect on reliability. However, having several tasks in the performance assessment may be costly in time and resources if each task is time consuming. This may be justified if the assessment is critical, such as for high cost/high risk situations such as the licensing of a doctor (Linn 1993). A second justification for the use of multiple tasks is that the task performance itself is a beneficial part of instruction. When the assessment tasks are valued learning activities in their own right, the result is a better integration of instruction and assessment. According to

    80 Australian and New Zealand Journal of Vocational Educatfon Research

  • Gonczi (1996) problems with reliability may be partly overcome by focussing the assessment on constructs with multiple assessments to serve as vehicles for assessing the construct.

    Inter-assessor reliability for complex multi-dimensional assessment is low (Bond 1995). Since greater assessor expertise is required in this situation there is the need for more training of the assessors to develop the skills of observation and recording required for performance assessment. Having more than one assessor improves reliability and generalisability. High levels of reliability across assessors can also be achieved by using well defined scoring. sheets and monitoring of assessors (Shavelson et al. 1992).

    Problems with reliability mean that it may be difficult to compare performances of students or workplace trainees across workplaces, colleges or regions. The extent to which this may be needed or desired is unknown but resources are unlikely to be available to improve reliability so that these types of comparisons can be made. On the other hand if assessments are used as part of ongoing learning and teaching a much lower level of reliability will suffice. .

    Holistic assessment

    Performance assessments are able to assess complex thinking in a more holistic way relating learning outcomes in the cognitive, psychomotor and affective domains. An important feature of performance assessment is its trait of testing for complex competencies rather than their component skills which have been decomposed from the complex competency. According to Resnick and Resnick (1991) testing for the decomposed skills fails to recognise that:

    ... complicated skills and competencies owe their complexity not just to the number of components they engage but also to interactions among the components and heuristics for calling upon them.

    (Resnick & Resnick 1991, p.42)

    Performance assessment in vocational education and training 81

  • While this point is not in dispute in the literature, the implications for teaching and assessment practice are in dispute. Messick (1994) suggests that in many settings it would be most effective to teach and assess both the complex skill and its component skills. He asks: 'Might not assessment of component skills help one to understand the nature of the complex skill and the sources of its complexity, providing a functional basis for improving methods of teaching?' (p.20). However, focussing on component skills will not be sound if it means that effective teaching and practice of the complex skill are foregone. In the process of decomposing the complex skill into its components, something could be left out. The implication is that performance assessment of the complex skill as a functioning whole guarantees that nothing important will be left out (Messick 1994, p.20).

    Conclusion

    In this paper we have described performance assessment and some of its important attributes in the vocational education and training setting. Although performance assessments have a long history of use in the vocational education and training sector and in vocational courses in higher education, the widespread introduction of competency-based education and assessment in recent years has reinforced the use of performance assessment. Some examples of performance assessment in vocational education and training were examined. We then discussed some of the key issues concerning performance assessment.

    Performance assessment is very useful in most vocational education and training settings, having the key characteristics of authenticity and directness. Programs that may be flexibly delivered off the job and/or on the job may rely heavily on performance assessment because of its easy application in both settings and ease of use by most teachers and trainers. This does not mean that paper and pencil or oral tests of knowledge should not be used to supplement performance tests.

    The assessment tasks chosen need to be carefully selected to ensure satisfactory validity, and other strategies (such as increasing the number of tasks assessed) may be required to ensure adequate reliability.

    82 Australian and New Zealand Journal of Vocational Education Research

  • However, performance tests in vocational education and training are usually designed for different purposes compared with knowledge tests in non-vocational courses, so reliability and other measurement criteria are not necessarily applicable. .

    While much has been written on the theory and practice of performance assessment, very little empirical research on these issues has been identified so far, and no Australian research in the field of vocational education and training has been identified. Baker et al. (1993) report the relatively low incidence of empirical data on performance assessment, estimating that less than 5 per cent of the liter~ture cited empirical data. Most of these empirical studies are investigations of reliability of scoring procedures. Before full-scale quantitative research of these issues is feasible, conceptual and methodological issues need clarification. It is suggested that appropriate case study research clarify these issues to provide a basis for later full-scale quantitative research, and to provide tentative answers to key issues, particularly on the consequences of assessment. Since performance assessments promise benefits for teaching and learning it is important to collect evidence of these positive consequences (Messick 1994). Positive and negative consequences of assessment, whether intended or unintended, need to be investigated.

    Rsfsrsnces Athanasou, J 1997, Introduction to educational testing, Social Science Press, Wentworth

    FallS,NSW. Arter, J & Spandel, V 1992, 'Using portfolios of student work in instruction and

    assessment', Educational Measurement: Issues and Practice, vol. 11 , no.3, pp.36-44. Baker, E, O'Neil, H & Linn, R 1993, 'Policy and performance prospects for

    performance-based assessment', American Psychologist, vol.48, no.12, pp.1210-1218. Bond, L 1995, 'Unintended consequences of performance assessment: Issues of bias

    and fairness', Educational Measurement: Issues and Practice, vo1.14, no.4, pp.21-24. Brennan, R & Johnson, E 1995, 'Generalisability of performance assessments',

    Educational Measurement: Issues and Practice, vo1.14, no.4, pp.9-12 & 27. Broadfoot, P 1995, 'Performance assessment in perspective: International trends and

    current English experience', in Evaluating authentic assessment, ed. H Torrance, Open University Press, Buckingham.

    Fitzpatrick, R & Morrison, E 1971, 'Performance and product evaluation', in Educational Measurement, 2nd edition, ed. R Thorndike, American Council on Education, Washington, DC.

    Performance assessment In vocational education and training 83

  • Frederickson, J & Collins, A 1989, 'A systems approach to educational testing', Educational Researcher, voLl8, no.9, pp.27-32.

    Gonczi, A 1996, Reconceptualising competency-based education and training: With particular reference to education for occupations in Australia, University of Technology, Sydney, Sydney.

    Green, B 1995, 'Comparability of scores from performance assessments', Educational Measurement: Issues and Practice, vol.14 no.4, pp.13-15, 24.

    Guion, R 1995, 'Commentary on values and standards in performance assessment', Educational Measurement: Issues and Practice, voLl4, no.4, pp.25-27.

    Hager, P, Athanasou, J & Gonczi, A 1994, Assessment technical manual, Australian Government Publishing Service, Canberra.

    Linn, R L 1993, 'Educational assessment: Expanded expectations and challenges', Educational Evaluation and Policy Analysis, vol.15, no.I, pp.1-16.

    ----& Burton, E 1994, 'Performance-based assessment: Implications of task specificity', Educational Measurement: Issues and Practice, vol.l3, no. 1, pp.5-8, 15.

    ----& Gronlund, N 1995, Measurement and assessment in teaching, 7th edition, Prentice Hall, Englewood Cliffs, New Jersey.

    ---- & Baker, E 1996, 'Can performance-based student assessments be psychometrically sound?', in Performance-based student assessment: Challenges and possibilities, eds J Baron & D Wolf, University of Chicago Press, Chicago.

    Mehrens, W 1992, 'Using performance assessment for accountability purposes', Educational Measurement: Issues and Practice, vol.lI, no.l, pp.3-9, 20.

    Meisels, S, Dorfman, A & Steele, D 1995, 'Equity and excellence in group-administered and performance-based assessments', in Equity and excellence in educational testing and assessment, eds M Nettles & A Nettles, Kluwer, Boston.

    Messick, S 1989, 'Validity', in Educational measurement, 3rd edition, ed. R Linn, Macmillan, New York.

    ---- 1994, 'The interplay of evidence and consequences in the validation of performance assessments', Educational Researcher, vol.23, no.2, pp.13-23.

    Resnick, L & Resnick, D 1991, 'Assessing the thinking curriculum: New tools for educational reform', in Changing assessments: Alternative views of aptitude, achievement and instruction, eds B Gifford & M O'Connor, Kluwer, Boston.

    Rutherford P 1995, Competency based assessment: A guide to implementation, Pitman Publishing, Melbourne.

    Shavelson, R, Baxter, G & Gao, X 1993:Sampling variability of performance assessments', Journal of Educational Measurement, vol.30, no.3, pp.215-232.

    ---- Baxter, G & Pine, J 1992 :Performance assessments: Political rhetoric and measurement reality', Educational Researcher, vol.21 no.4, pp.22-27.

    Thomson, P 1992, 'Assessing competence: An overview of current problems and trends in Australia', What future for technical and vocational education and training?, conference proceedings, NCVER, Melbourne, December, vo1.2, pp.293-303.

    Torrance, H 1995, 'Teacher involvement in new approaches to assessment', in Evaluating authentic assessment, ed. H Torrance, Open University Press, Buckingham.

    84 Australian and New Zealand Journal of Vocatlonal Educatlon Research

  • van der Vleuten, C & Swanson, D 1990, 'Assessment of clinical skills with standardised patients: The state of the art', Teaching and Learning in Medicine, vo1.2, no.2, pp.58-76.

    Wiggins, G 1989, 'Teaching to the (authentic) test', Educational Leadership, April, pp.41-47.

    Wolf, A 1995, 'Authentic assessments in a competitive sector: Institutional prerequisites and cautionary tales', in Evaluating authentic assessment, 00. H Torrance, Open University Press, Buckingham.

    Performance assessment in vocational education and training 85