e-assessment guidelines for the vet sector: case study 1 · e-assessment guidelines for the vet...

E-assessment guidelines for the VET sector: Case study 1

Skills Institute - Mobile phones for assessing vehicle painting apprentices

6 June 2011

flexiblelearning.net.au

www.nqc.tvetaustralia.com.au


Australian Flexible Learning Framework and National Quality Council i

Acknowledgement

This report was produced for the Australian Flexible Learning Framework and National Quality Council by Rob Stowell (Learning Australia Pty Ltd) and Reece Lamshed (Binary Blue).

This case study is on interviews with:

Colin Ogden, The Skills Institute, Devonport, Tasmania

Janet Feams, Learning Consultant (Education Technology), Skills Institute, Launceston

Peter Higgs, Manager Learning Technology & Copyright Management, Tasmanian Polytechnic

Thanks to Colin, Janet and Peter for their support in developing this case study.

The views expressed herein do not necessarily represent the views of the Commonwealth of Australia.

© Commonwealth of Australia 2011. Licenced under AEShareNet Free For Education licence.

This work is copyright and licensed under the AEShareNet Free For Education Licence (AEShareNet–FfE Licence). The onus rests with you to ensure compliance with the AEShareNet-FfE Licence and the following is merely a

summary of the scope of the Licence.

You may use and copy any material covered by an AEShareNet-FfE licence, for educational purposes only, and only within or for the services of your organisation.

Conditions for the licence can be found at http://www.aesharenet.com.au/FfE2/. Queries regarding the standard AEShareNet-FfE Licence conditions should be directed to the AEShareNet website at http://www.aesharenet.com.au/help/support/.

In addition to the standard AEShareNet-Ffe Licence conditions, the following special condition applies: The licence territory is limited to Australia and New Zealand.

Requests and inquiries concerning other reproduction and rights should be directed in the first instance to the

Director, VET Technology Policy and Projects, Department of Education, Employment and Workplace Relations, GPO Box 9880, Canberra, ACT, 2601.

http://www.aesharenet.com.au/help/support/


Australian Flexible Learning Framework and National Quality Council i

Table of contents

Summary................................................................................................................................ 1

Case Study Detail .................................................................................................................. 3

Case Study assessment documents ...................................................................................11

Guidelines .............................................................................................................................17

For more information: ..........................................................................................................21


Australian Flexible Learning Framework and National Quality Council 1

Summary

Institute

Skills Institute (Tasmania Launceston)

Qualification

AUR05 Automotive Retail, Service and Repair

Cohort

Trades apprentices – on-the-job training.

E-assessment technologies

Compass – Institute database that stores all Assessment tools and generates files as PDF or for iPhone

iPhone / iQTI Player – software application loaded on iPhone that records assessment and evidence, which is compiled and sent via email to office computer.

E-File – Institute network drive where student results and evidence are stored.

Student Management System – Institute-wide system where all student records are kept that generate the student transcripts and certificates.

Description

The mobile phone (iPhone /iQTI Player) is used to conduct on-the-job competency assessment with apprentices throughout Tasmania. The device is linked to a database (Compass) that contains the competency elements and validated assessment questions, which can be accessed through an iPhone anywhere in the field.

The mobile phone is used on-the-job to assess the apprentice’s practical demonstration of skills guided by a set of questions that are verbally communicated. The iPhone can also record and store photos, videos and audio recordings that are used as assessment evidence. The assessment, result and evidence is bundled and sent immediately via email to the assessor. The employer and apprentice can also be sent the results of the assessment via email. The judgment is generally made by the assessor on-the-job, but can be moderated if required by another colleague at the assessor’s office.

The individual assessment results and evidence are stored in the Institute networked shared drive.

The final results are stored in the Skills Institute SMS (Student Management System), which generates transcripts and certificates.

Case study highlights

This case study is an effective example of the use of mobile devices (iPhones) in recording assessor observations of practical workplace tasks, using a skills checklist, photographs, videos, interviews and

Skills Institute - Mobile Phones for assessing Vehicle Repair Apprentices


employer/supervisor testimonials.

The evidence is packaged by the iPhone software (iQTI) and emailed to the assessor, keeping it digitally intact for storing and recording purposes.

It is estimated that the use of the mobile device has improved the efficiency of the assessment process by at least 50%.

The case study reveals how apprentices are assessed, privacy and confidentiality is assured and data is transferred across all the e-assessment technologies used by the institute.



Case study detail

It’s a cool autumn morning in Devonport, Tasmania and Colin Ogden has set up three visits today to conduct assessments. Colin has been conducting assessments for vehicle painting apprentices for the past 5 years. He has about forty apprentices to assess, all of whom are undertaking vehicle painting1. They are all in different stages of their qualification, are located throughout southern Tasmania and work in panel beating shops in towns such as Hobart, Kingston and Huonville.2 Colin works for Skills Institute. 3

He checks the mobile phone to make sure it’s charged. It’s an iPhone 4, on which the iQTI software has been installed.4 Colin has been using the iPhone to conduct on-the-job assessments for the past 12 months.

The iQTI software enables competency observation for the units of competency he is assessing and a set of questions that he will ask the apprentices when they demonstrate their assessment tasks to him.5 He has pre-loaded the names of the apprentices and their contact details into the ‘Address Book’ function of the iPhone.6

The assessment he will conduct today constitutes about 20% of the total for this unit of competency. Not all assessment is conducted on the job. Apprentices come to the Devonport College to do their ‘theory’ and some practical work; this is a ‘block release’ program, generally consisting of five or six weeks per year. The knowledge assessment constitutes about 40% of the total and is generally assessed using a multiple choice question test located on the Institute Learning Management System, Blackboard. The practical work for these units makes up the other 60%, which is all assessed and recorded using the iPhone, both on and off-the-job.7

Travel is a big part of Colin’s work-life. It takes him about nine working days to complete a full round of workplace visits, and this he does three times a year. Each workplace assessment might take between 30 minutes to three hours to conduct, and he can assess a number of units in the same visit.

Sometimes it’s difficult for Colin to arrange his visits to coincide with an appropriate assessment task. It depends on the actual work they might be doing at the time, so he has to be flexible and work around what the apprentices are doing that day, rather than insisting that a particular assessment task should be done. Mostly, with the employer’s cooperation,

1 The qualification is taken from AUR05 Automotive Retail, Service and Repair that includes 28 units of

competency. 2 See case study documents page 12, figure 1. 3 www.skillsinstitute.com.au 4 See case study documents page 12, figure 2. 5 See iPhone screens for assessment checklists and questions, case study documents, page 13, figures

4, and 5. 6 Colin builds his address books gradually over the process of his workplace visits. This uses an LDAP

protocol that meets an IETF standard. See case study documents page 12, figure 3. 7 The iPhone is also used as an assessment tool in workshop assessments when the apprentices do

their block release at the Institute campus.

http://www.skillsinstitute.com.au/



the assessment tasks and work tasks can be made to align. He sent a letter to his apprentices two weeks ago advising of his impending visit and confirmed this again yesterday in an SMS message, which he finds is the most reliable method of contact. The employer was also informed.

The apprentices know what they will be performing today and what criteria they will be assessed on. This information is provided in a hardcopy folder called ‘competency briefs’ that was given to them at the beginning of course. In it all the assessment tasks are outlined so they know what is expected of them.8 Also, on the phone yesterday with the apprentice, Colin confirmed they understood the assessment task and were ready to demonstrate their capability to do it.

The first apprentice he has arranged to visit, works in a small panel beating shop in Hobart. This employs ten staff, and Fred 9 is the only apprentice.

Today, the assessment task is to demonstrate how he can prepare and paint plastic components.

Colin takes out his iPhone and opens the application (iQTI Player), searches for the appropriate competency unit, which reveals the observation questions for this particular assessment task.10 He assigns the apprentice’s name. This will ensure a data file is created on the iPhone and all input information will be attributed to the right apprentice.

Colin has built a strong relationship with Fred over the past two years and understands that though Fred is not a great talker, he is adept at what he does. He remembers his first assessment when Fred appeared quite intimidated by the process, and very nervous. On that occasion, he showed Fred the questions he was going to ask on his iPhone, who then appeared more comfortable. He was then able to show Colin how he did the task. It’s about the trust and relationship built, between assessor and apprentice.

Fred demonstrates his ability to perform the assigned task. Colin asks a series of questions as he is doing it.11 If Fred doesn’t quite understand what the question is, Colin re-phrases it differently, but still sticks to the assessment ‘script’.12 He’s aware that he has to be careful not to say things that might guide the apprentice to do the activity correctly. He’s familiar with those apprentices over the years that have tried to get an assessor to ‘show’ them how to do a task by asking implied questions.13

Colin now steps back from the demonstration and takes several photos, using the iPhone, from different angles, while Fred is still spraying. For the next procedure, he uses the iPhone to capture Fred on video. He has already

8 Refer to case study documents, page 17, figure 15. 9 Not their real name. 10 See case study documents, page 13, figure 4. See also sample of full competency checklist, case

study documents, page 16, figure 14. 11 See photo of assessor taking notes while apprentice demonstrates, case study documents, pages 13

and 14, figures 6 and 7. 12 For assessment consistency reasons, a ‘script’ is followed, but the individual’s personal aptitudes (eg

literacy) are taken into account in the assessment conversation. 13 Hereby ensuring that the assessment is valid.



signed a release form with the employer of the panel beating shop, permitting him to take video and photos inside the workshop. Most importantly, he is to avoid taking shots of car number plates.14

Colin is not using his own iPhone. The Skills Institute has purchased it, and in Colin’s team, there’s about seven staff sharing four iPhones, thus avoiding complications that may occur if staff were required to purchase their own iPhones, Colin has not had an issue in obtaining a phone when he needs to conduct his assessments.

It is Janet Feam’s job15 to load all the assessment material, assessment tasks, observation questions and competency checklists, on to a database, which is then accessed by the iPhone in the field. The Skills Institute IT Department loads the software (iQTI Player) onto the iPhones. This usually takes them about five minutes. Assessors download the assessment tools, which according to Colin, is very easy to do, and for the whole set of twenty-eight competencies, takes about twenty minutes.

Colin is now adept at using the mobile technology and did not find it difficult to learn. Janet provides all the training. It takes less than thirty minutes to train a new assessor to use it.

When Skills Institute first started using them, Janet organised formal group training sessions, but now provides training on demand. As part of a 2010 e-learning Innovations Project, Janet developed a detailed ‘User Guide’ on how to load and use the iQTI software on the phone. It’s a step-by-step process. They have also developed a video and a PowerPoint presentation showing how to use the system.16

In the days before the mobile technology was introduced, the validated assessment material was loaded into a database, the assessors selected the unit that they wanted to use for an assessment, and the database generated the accompanying materials as a PDF.17 This document was downloaded by the assessor and printed out. The observation checklist was attached to a clipboard and taken out to the job. The assessment checklist was paper-based. Any checks and comments were hand written. Where photos for evidence were included, they were taken by a standard digital camera. On completion of the assessment, all the evidence was viewed, judged and compiled later into the student record’s sheet. It meant a great deal of paper in folders, double handling data and the opportunity for mismatched material.

The assessment database connected to the iPhone is called Compass18. TAFE Tasmania19 developed it in 1997 to make the assessment process easier for staff. Data in Compass is stored in .csv format.20

14 Privacy and confidentiality in data capture is an important consideration. 15 Learning Consultant, Skills Institute, Launceston. 16 http://iqti.wordpress.com 17

The Skills Institute refers to this package as an ‘assessment tool’. 18 Compass stands for Competency Assessment. 19 TAFE Tasmania split into two divisions: Skills Institute and the Tasmanian Polytechnic. 20 Database file format that complies with interoperability standards.

http://iqti.wordpress.com/



The iQTI Player was developed by the Tasmanian Polytechnic in 2007, and later as an iPhone application using development funds from a Flexible Learning Advisory Group (FLAG) grant. A QTI generator adjacent to the Compass database produces a .qti file that holds the Assessment Tool.21 This can be used to generate either a PDF or be read by the iQTI Player, which is loaded on the iPhone.

Janet is one of two specialist staff (‘Compass Champions’) who has back-end access to Compass. The new or modified assessments are given to them to upload to Compass; the assessor team does not have back-end access, because having too many people accessing the system may compromise data integrity. Janet has not found the system difficult to learn or use.22

Any changes made to materials on the assessment database are automatically accessed by the iPhone the next time an assessor opens a unit of competency that contains the modifications.23 All the assessment materials for the twenty-eight units of competency are accessible on the iPhone, and the assessor uses the iPhone search function to locate the code of the unit they are assessing against. If an apprentice is absent on assessment day, or a particular job can’t be demonstrated, the assessor can easily switch to another assessment task, or even locate another apprentice in the workshop to conduct a different assessment.

The iQTMI Player is password protected and only accessible by registered RTOS and approved assessors with recognised passwords, thus protecting the system from unauthorised use.24

Meanwhile, Colin is completing his assessment with his first apprentice. He finishes the session by using a recording application on the iPhone to record an interview with Fred about various aspects of his task. This ensures the student understood what he was doing.25

The final iPhone iQTI screen asks Colin whether he assesses the apprentice ‘competent’ or not for this particular assessment task. Based on what he has observed of Fred today, he selects ‘yes’ from the screen.26 All the assessment evidence has been stored in the one file on the iPhone.27

The iQTI software also allows Colin to attach comments to his decision on whether the student is competent or not.28

When the interview over, and assessment completed, Colin drives to his next appointment.

21 This file format is interoperable, meeting IMS QTI 2.1 standards. http://en.wikipedia.org/wiki/QTI and

www.imsglobal.org 22 See case study documents, page 15, figures 10 and 11. 23 Ensures consistency of assessment. 24 Ensures security of assessment data. 25 In this one assessment task, different forms of evidence have been gathered – practical observation,

photographic and video evidence, verbal communication (recorded). 26 Case study documents, page 14, figure 9. 27 Authenticity of the evidence is not an issue is this case, as it is the qualified assessor who collects the

evidence and is onsite to prove the candidate is authentic. 28 See case study documents, page 16, figure 13.

http://en.wikipedia.org/wiki/QTI

http://www.imsglobal.org/



Colin is still in a 3G area so selects the ‘email’ button on the iPhone to send all the data back to his network computer. If he were not in a 3G area, the iPhone would store all the data, and then immediately on reaching mobile coverage, automatically transmit the data. He is able to re-charge the phone as he drives.

The ‘email’ iPhone button has three options selecting to whom it can be sent: to the employer, to the student and/or to the assessor. Colin selects all three. He wants both the employer and apprentice (in this case, Fred) to know what his decision was, and why.29

Colin always keeps the data on the iPhone he collects on site until he returns to his office and confirms that it has been transmitted correctly, to check that there is no corruption or loss of files.30 Since the assessor team has been using this mobile technology, they have not had an instance of this happening.

Peter Higgs31 ascribes this robust data chain to the rigorous development process that the Learning Technology team had to go through to develop the iQTI software. Because it is designed as an Apple application, it goes through an Apple developer process and testing regime.32 The institute team had to first become registered Apple developers. Apple provides a developer’s kit that is used to create the software application, and this has a strict evaluation processes. Apple gives approval for the software to be stored on Apple’s iTunes store where it can be downloaded, only when the standards have been met.

With the next apprentice, Colin won’t be using the iPhone to conduct the assessment. There are several employers who for religious reasons won’t allow this electronic technology into the workshop, and the Skills Institute accommodates their request. Some apprentices (very few) prefer paper-based assessments, and their preference is supported. The paper-based version is printed from Compass download so it’s always the same version as is on the iPhone.33 However, it will mean Colin will spend more time processing the data than if he was using his iPhone.

For the third apprentice he visits that day, Colin starts out the session giving him a checklist loaded on the iPhone, which the apprentice uses to self assess34. The apprentice felt confident that he could undertake the tasks indicated. However Colin decides after observing one particular task that he

29 This provides immediate feedback to the employer and student about the assessment made. Of

course, because the assessment is made on-the-job, feedback to the student is provided this way as well. 30 Loss of data or corruptibility in transfer is possibly an issue, so the practice is to hold the data until

assured that the data has been transmitted without fault. It is the policy of the Institute that this procedure is adopted. 31 Manager Learning Technology & Copyright Management at Tasmanian Polytechnic.

32 http://developer.apple.com/library/ios/#documentation/Xcode/Conceptual/iphone_development

/000-Introduction/introduction.html 33 Thus ensuring consistency of assessment across different formats. 34 This self-assessment confirms with the student that they know what will be assessed and designed to

involve and engage them in the assessment process.



was not yet quite competent. Nevertheless, all assessment evidence is retained on the iPhone.35 He informs the apprentice of his decision, and why he has assessed him as not yet competent for this activity, and also notes this on the iPhone. He suggests another time in a couple of weeks when he can revisit and assess the apprentice on that activity again. Using his iPhone, he emails his decision to the apprentice and his employer.

Colin gathers two other pieces of assessment evidence relating to this apprentice. The employer took some photos of him doing some tasks over the past few weeks, and although Colin won’t use these as direct assessment evidence, he will look at them and see how the apprentice is progressing on the tasks.36 So he loads these onto a USB storage device. In regards to another activity, the employer has provided a testimonial about the apprentice, confirming the apprentice’s ability to do the task competently. Colin photographs this using the iPhone, and this will be stored until the formal assessment for this particular unit of competency is completed.

Having finished assessing for the day, Colin logs in to his notebook (he can do this from home or work), opens the Institute email program - Outlook, and in a dedicated folder (called IQ)37, he finds three new zip files, named and identified with the apprentice’s names.38 The data inside the zip folder created by the iPhone software is in an .xml format; photos are in jpeg; video is in .mp4 and the audio interview is in .mp3.39

The iPhone has 16-gigabyte memory capacity, which can easily store twenty or so completed assessments. Videos are large files, so these have to be short in duration, no more than a minute, and with photos, Colin restricts this to about three small photos per assessment. It’s the video and photograph files that are large, not the audio or text. However, the restriction is on the cost of data transfer rather than the speed.

Colin spends an hour or so going through the assessment evidence, checking against the competency checklist again to make sure that his judgement on site was correct. In the case of the first apprentice, there was one activity he observed that was ‘line-ball’, so he selects the video and plays it to a colleague in the office, who confirms his decision to assess him competent.40

This confirmed, Colin now enters all this information into a student-tracking system e-File located on the Institute network and drags the files from the email to the designated e-File.41 This is the formal Institute-wide system for

35 The evidence collected and the partial assessment made is stored on the iPhone (‘In Progress’) and

can be accessed at any time at a later stage. See screen interface, case study documents, page 13, Figure 4. 36 Formative assessment. 37 See case study documents, page 14, figure 8. 38 All the data sent from the mobile phone is accessible anywhere on a password-protected computer. 39 These file formats are interoperable across all platforms. iPhone also supports H.264, .m4v,

.mov. 40 The video component allows a moderated assessment; previously, the assessor could only verbally

explain what the student had accomplished. The video shows the evidence. It is also valid evidence for auditing purposes and stored with other evidence digitally. 41 See case study documents, page 16, figure 12.



Colin’s team that holds the complete record of each student’s training and assessment. The e-File contains a set of sub-folders for each unit of competency: theory test, practical task/observation, testimonial, record sheet and photos/video. So not only are the results recorded here, but all the files that relate to each assessment task (collected in this instance by the iPhone – the iQTI file) are also attached42. This system provides a complete audit trail.

Having entered all the data in the apprentice’s e-File, Colin now deletes the file from the iPhone. Finally, he enters the results for the student into the institute Student Management System (SMS) QLS.

The overall assessment process has not changed with the introduction of the iPhone technology. Rather, the process has been digitised, and as a result, streamlined.

The Institute has been delivering vehicle painting since 1978.43 The assessment process was instituted with the development of Compass. The assessments are formally validated and moderated every year by the assessor team, although observation checklists and assessment task questions are updated more regularly.44

An Industry Reference Group (IRG) representing the main car panel beating/painting employers meets generally about three times per year. The Skills Institute team has demonstrated how the technology system works with the mobile device, and this has been formally validated by the IRG.45

To explain how the iPhone technology has dramatically improved the workflow, institute staff tell of a practical test they conducted by having one staff member conduct an assessment using the old paper-based format and clipboard, and another person using the iPhone. The two assessors set out together to conduct a similar number of assessments on two different apprentices. The first assessor was dropped off at the workplace with a clipboard, camera and computer, whilst the second iPhone assessor sped off to meet an apprentice about thirty minutes away from the first.

An half hour or so later, the iPhone assessor had completed their assessment including photos, driven back to the first assessor’s site where they were just about to complete the first assessment.

It was when they returned to the office that the real time difference was revealed.

The iPhone assessor had their folder with all the assessment evidence already on their computer, it had been emailed. They then confirmed the evidence, made their final judgement, and entered the results into the student management system (SMS) and linked all the evidence files. The other assessor had to transfer all their hand-written notes, upload all the

42 This is not an automated procedure: the assessor manually drags the iQTI files into the correct folder

in the student’s e-File. 43 Long before training packages, and when the Skills Institute was part of TAFE Tasmania. 44 Validation and moderation of assessment is a procedure that is formally established across the Skills

Institute. 45 Part of the iPhone /iQTI development process was to trial the technology in an actual workplace, to

get feedback from both apprentices and employers.



photographs, make the final judgement, then enter the results in the student records system, and file the hardcopy evidence. This took a significant amount of time, the iphone assessor took much less time.

The Skills Institute assessor team estimates it takes at least fifty percent less time to conduct assessments on the mobile technology than it did using the paper-based method.

The Skills Institute development staff are aware that they have adopted an iPhone as the primary tool, and that this has restrictions on further development. According to Peter Higgs, they opted for the iPhone because the device was the most popular smart phone at the time; it provided a robust application service through iTunes and the development process was highly developed and systematised for a developer. They started initially experimenting with a Microsoft system, and then reverted to Apple. They now intend to extend the system to include the Android (Google technology), and in the near future, to iPads. They are particularly excited about the iPad, as it provides more screen space to conduct the assessment checklist tick-offs. The first generation iPad did not have a camera, so this meant the evidence collecting was limited. However, with the second generation iPad having both front and back cameras, this is more attractive.

Peter is confident about the transfer of data across the various platforms. Certainly they have considered this at the back end of the technology. The Compass database is ISO compliant, using standard .csv files, so there will be no problem in loading these files to any mobile system. At the other end, the e-File is sufficiently robust to handle the array of digital files, although there is talk (funds permitting) to replace this with a more current version (eg a Learning Management System).

Data storage is increasingly becoming an issue for the Institute as the evidence to be stored is digital rather than paper-based46.

To date, the Skills Institute has not conducted a cost analysis on the e-assessment process that they now use. The iQTI Player cost about $150,000 to create, develop and test the application and make it work fully in the iPhone framework. There are continual upgrade costs to this iQTI Player, and those associated with the development of the software for other platforms. There are two dedicated staff managing the use of mobile phones in the Institute. The other significant ongoing cost is the 3G service charges. All these costs would need to be weighed up against the paper-based assessment model. The cost effectiveness is already being demonstrated in part by the estimated 50% time saving for staff conducting the assessments.

46 For ATQF compliance, evidence needs to be stored for seven years, and linked to the student’s

results for a transparent audit trail.



Case Study assessment documents

Figure 1: Map showing southern Tasmania.

Figure 2: iPhone with iQTI application icon. Figure 3: iPhone Address Book showing names of apprentices (edited).



Figure 4: The options that are provided in the iQTI iPhone application.

Figure 5: The validated assessments menu for a particular unit of competency.

Figure 6: Using iPhone in a spray booth observing apprentice spraying.



Figure 7: Capturing evidence using iPhone camera.

Figure 8: Folder ‘qti’ in Outlook Express where the iQTI files are located when they are emailed in the field from the iPhone.

Figure 9: iQTI application showing the final screen where task is checked off as competent.



Figure 10: Screen shot of the Compass database showing interface and Units of competency list.

Figure 11: Screen shot of Compass showing where assessor comments are provided for each assessment task.



Figure 12: Folder structure on e-File. Figure 13: iQTI Comments

Figure 14. Sample of an observation checklist



Figure 15: Sample of Vehicle Painting competency brief



4. Guidelines

Group Theme Guideline Case study approach

Infrastructure provision

1.1 Web based functionality

E-assessment providers must meet minimum web based functionality requirements to support e-assessment

Those interviewed in the case study didn’t indicate any issues meeting these standards.

1.2 Desktop functionality

E-assessment providers must meet minimum desktop functionality requirements to support e-assessment.

Those interviewed in the case study didn’t indicate any issues meeting these standards.

Technical guidelines

2.1 Accessibility E-assessment providers must confirm that e-assessment resources and materials are accessible to people with disabilities.

Accessibility compliance is not an issue as the technologies are used by the practitioner. Paper-based alternatives are provided.

2.2 Portability

E-assessment materials must be transportable between different repositories and learning management systems.

All evidence is stored in formats and in systems that make content easily transferable.

2.3.Desk top content formats

E-assessment materials must meet the standard for desk top content formats.

All file formats used in the evidence gathering tool are in common and transferable formats.

2.4 Mobile content formats

E-assessment materials must meet the standards for mobile content formats.

Meets mobile content standards – QTI.

2.5 Metadata To support Not relevant to system



discovery and re-use of e- assessment resources and materials across the VET system, assessment content should be described using the Vetdata standard.

purposes.

E-assessment development and maintenance

3.1 Benchmarks

E-assessment resources and materials must meet the requirements of the relevant Training Package or accredited course.

The practitioner team work together to ensure the requirements are met.

3.2 Assessment principles

E-assessment resources and materials must provide for valid, reliable, fair and flexible assessment


3.3 Personalisation

E-assessment resources and materials must provide for personalisation of assessment.

E-assessment is highly personalised with individualised feedback to each learner at many points in the process.

3.4 Validation

E-assessment resources and materials must be systematically validated.

The practitioner team work together with industry to ensure the requirements are met.

3.5 Workplace and regulatory requirements

E-assessment resources and materials must address workplace and regulatory requirements


3.6 Candidate authentication and security

E-assessment resources and materials must provide for candidate authentication and the security of both the assessment

Candidate authentication and security is provided by the assessor attendance at the assessment activity.



process and assessment data.

3.7 Maintenance E-assessment resources and materials must be maintained.


E-assessment practices

4.1 Collaboration E-assessment must be developed in consultation with industry and other stakeholders.

Formal industry committee to validate technologies and e-assessment processes.

4.2 Evidence collection

E-assessment involves collecting quality evidence for use in assessment decision making.


4.3 Feedback E-assessment feedback must identify candidate strengths, areas for improvement and ways in which performance may be improved.

Practitioner feedback to each learner is provided at many points in the process.

4.4 Assessment judgements

E-assessment involves assessors in evaluating evidence and making assessment judgments.


4.5 Recording and reporting e-assessment outcomes

E-assessment outcomes must be accurately recorded, reported and stored.


4.6 Complaints and appeals

E-assessment processes must provide for complaints and appeals from candidates.

This is addressed in Institute policy and practice.

E-assessment context

5.1 E-assessment support services

E-assessment providers must have appropriate support services for assessors and candidates.

Institute provides support structure with assessor training and manuals showing how to use assessment tools.

5.2 E-assessment E-assessment Institute provides the



deployment strategy

providers should have an e-assessment deployment strategy.

iPhones for the assessors.



For more information: Benchmarking and Research business activity

Phone: (08) 8463 6509

Email: [email protected]

Website: flexiblelearning.net.au/research

Australian Flexible Learning Framework

Phone: (07) 3307 4700

Email: [email protected]

Website: flexiblelearning.net.au

mailto:[email protected]

http://www.flexiblelearning.net.au/

e-assessment guidelines for the vet sector: case study 1 · e-assessment guidelines for the vet...

Documents