[ieee 2009 international conference on engineering education (iceed) - kuala lumpur, malaysia...

6
2009 Inteational Conference on Engineering Education (lCEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia Outcome Based Education Performance Evaluation on Electrical Engineering Laboratory Module Husna Zainol Abidin, Norlaila Omar, Hadzli Hashim, Mohd Fuad Abdul Latip, Muhammad Murtadha Othman, Syazilawati Mohamed, Nani Fazlina Nai, Zuhaila Mat Yasin Faculty of Electrical Engineering, Universiti Teknologi MARA, Malaysia. [email protected] hadzli[email protected] Abstract - Implementing Outcome Based Education (OBE) in evaluating program outcomes is a standard practice nowadays at the Faculty of Electrical Engineering (FEE), Universiti Teknologi MARA (UiTM). Previously, laboratory assessments method was merely based only on lab report submitted by each group of students and the procedure is inconsistence as each module consists of different area of electrical engineering fields and involves many lecturers from various disciplines in the FEE. Furthermore, there were no specific guidelines for grading the report and lecturers would rely on their experiences, resulting large variance of judgments in giving the marks. To overcome such problem, an OBE assessment tool known as Laboratory Sensor Performance Evaluation Course Tool (LAB-SPECT) has been designed recently for evaluating laboratory modules. This tool has taken into account additional student's attributes that represents teamwork skill, practical skill and ethical in the lab. With rubrics table provided, it has successfully facilitated lecturers to evaluate students fairly in terms of their Group Related Skills (GRS) besides than the laboratory report. Each of the assessment section in this process is addressing the respective the course outcome (CO) and program outcome (PO). The output plots produced by this tool would be used as indicators for Continual Quality Improvement (CQI) recommendations. Kwor-assessment; outcome based education; elerical engineering; laborato moles I. INTRODUCTION Outcome Based Education (OBE) is an education system that emphasis on outcomes measurement rather than inputs of cuiculum covered. Outcomes may include a range of knowledge, skills and attitudes. In order to obtain the desired outcomes, teaching components and activities should be well orgized, planned and continuously improved [1,2]. OBE concept has been applied in many countries ranging om prima schools to universities. In Malaysia, the Engineering Accreditation Council (EAC) of Malaysia has directed that all engineering programs must adopt and implement OBE concept beginning 2007 [3]. In fact, it has 978-1-4244-4844-9/09/$25.00 ©2009 IEEE 153 become a major requirement for y degree proam to be accredited by EAC. Faculty of Elecical Engineering (FEE) at the Universiti Teknologi MARA (UiTM) has inoduced OBE knowledge ongst its staff since 2005. Beginning 2007, all deee courses have OBE elements printed in each of its syllabus. Another words, eve course has their course outcomes (CO) being mapped with the FEE tgeted progr outcomes (PO) [4]. This mapping is known as CO-PO matrix. There e eleven Proam Outcomes (PO) decided by the FEE as outlined in Table I. T«LE I. PROGRAM OUTCOMES FOR FACULTY OF ELECTRICAL ENGINEERING, UNIVERSITI TEKNOLOGI Program AUributes Outcome POI Ability to acquire and apply science and en�neering ndamentals. P02 Abili to express ideas effectively, in litten and oral form. POl Acquiring in-depth technical knowledge in one or more specializations. P04 Ability to identi, formulate and solve en�neering problems. P05 Ability to utilize systems approach to design and evaluate operational petformance. P06 Abili to use the techniques, skills, and mem engineering tools necessa for en�neering practices. P07 Ability to recognize and appreciate importance of ethical standards in professional work. POS Ability to acquire lifelong leaing. P09 Abili to apply managerial or entrepreneurship skills. POlO Ability to work as both an individual and in a team on electrical engineering or multi·disciplina projects POl l Knowledge of contempora issues and appreciation of diversi in the world and intellectual areas. Students enrolled in this pro, are expected to acquire these outcomes at the end of their four year of studies through various courses offered in the Bachelor of

Upload: zuhaila

Post on 27-Feb-2017

216 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: [IEEE 2009 International Conference on Engineering Education (ICEED) - Kuala Lumpur, Malaysia (2009.12.7-2009.12.8)] 2009 International Conference on Engineering Education (ICEED)

2009 International Conference on Engineering Education (lCEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

Outcome Based Education Performance Evaluation on Electrical Engineering Laboratory Module

Husna Zainol Abidin, Norlaila Omar, Hadzli Hashim, Mohd Fuad Abdul Latip, Muhammad Murtadha Othman, Syazilawati Mohamed, Nani Fazlina Nairn, Zuhaila Mat Yasin

Faculty of Electrical Engineering,

Universiti Teknologi MARA, Malaysia. [email protected]

hadzli [email protected]

Abstract - Implementing Outcome Based Education (OBE) in

evaluating program outcomes is a standard practice nowadays at the Faculty of Electrical Engineering (FEE), Universiti Teknologi MARA (UiTM). Previously, laboratory assessments method was merely based only on lab report submitted by each group of students and the procedure is inconsistence as each module consists of different area of electrical engineering fields and involves many lecturers from various disciplines in the FEE. Furthermore, there were no specific guidelines for

grading the report and lecturers would rely on their experiences, resulting large variance of judgments in giving the marks. To overcome such problem, an OBE assessment tool

known as Laboratory Sensor Performance Evaluation Course

Tool (LAB-SPECT) has been designed recently for evaluating laboratory modules. This tool has taken into account additional student's attributes that represents teamwork skill, practical skill and ethical in the lab. With rubrics table

provided, it has successfully facilitated lecturers to evaluate

students fairly in terms of their Group Related Skills (GRS) besides than the laboratory report. Each of the assessment section in this process is addressing the respective the course outcome (CO) and program outcome (PO). The output plots

produced by this tool would be used as indicators for Continual Quality Improvement (CQI) recommendations.

Keywords-assessment; outcome based education; electrical engineering; laboratory modules

I. INTRODUCTION

Outcome Based Education (OBE) is an education system that emphasis on outcomes measurement rather than inputs of curriculum covered. Outcomes may include a range of knowledge, skills and attitudes. In order to obtain the desired outcomes, teaching components and activities should be well organized, planned and continuously improved [1,2]. OBE concept has been applied in many countries ranging from primary schools to universities. In Malaysia, the Engineering Accreditation Council (EAC) of Malaysia has directed that all engineering programs must adopt and implement OBE concept beginning 2007 [3]. In fact, it has

978-1-4244-4844-9/09/$25.00 ©2009 IEEE 153

become a major requirement for any degree program to be accredited by EAC.

Faculty of Electrical Engineering (FEE) at the Universiti Teknologi MARA (UiTM) has introduced OBE knowledge amongst its staff since 2005. Beginning 2007, all degree courses have OBE elements printed in each of its syllabus. Another words, every course has their course outcomes (CO) being mapped with the FEE targeted program outcomes (PO) [4]. This mapping is known as CO-PO matrix. There are eleven Program Outcomes (PO) decided by the FEE as outlined in Table I.

TABLE I. PROGRAM OUTCOMES FOR FACULTY OF ELECTRICAL ENGINEERING, UNIVERSITI TEKNOLOGI MARA

Program AUributes Outcome

POI Ability to acquire and apply science and en�neering fundamentals.

P02 Ability to express ideas effectively, in ",litten and oral form.

POl Acquiring in-depth technical knowledge in one or more specializations.

P04 Ability to identify, formulate and solve en�neering problems.

P05 Ability to utilize systems approach to design and evaluate operational petformance.

P06 Ability to use the techniques, skills, and modem engineering tools necessary for

en�neering practices.

P07 Ability to recognize and appreciate importance of ethical standards in professional

work.

POS Ability to acquire lifelong learning.

P09 Ability to apply managerial or entrepreneurship skills.

POlO Ability to work as both an individual and in a team on electrical engineering or

multi·disciplinary projects

POl l Knowledge of contemporary issues and appreciation of diversi�' in the world and

intellectual areas.

Students enrolled in this program, are expected to acquire these outcomes at the end of their four year of studies through various courses offered in the Bachelor of

Page 2: [IEEE 2009 International Conference on Engineering Education (ICEED) - Kuala Lumpur, Malaysia (2009.12.7-2009.12.8)] 2009 International Conference on Engineering Education (ICEED)

2009 International Conference on Engineering Education (lCEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

Engineering (Hons) Electrical Engineering program or EE220. At the FEE, the course assessment activities are divided into four components i.e. examinable courses, non­examinable course, laboratory courses and fmal year project courses. Each of these components has its own customized measurement tools for OBE performance as described in Table II below [5,6].

No.

I

2

3

4

TABLE II. FACULTy OF ELECTRlCAL ENGINEERING OBE MEASUREMENT TOOLS

Name Code Formative Summative Activity Activity

Summative Dynamic Assessment SAMOBEC

Test, Assignmen� Final Exam

Model with OBE Compliance Quiz

Outcome Based Perfonmance for OPNEC

Test, Assignmen� Presentation,

Non·Exam Courses Quiz, Project Demonstration

Laboratory Sensor Perfonmance LAB·SPECT

Group Related None

Evaluation Course Tool Skill, Report

Project Sensor Perfonmance Progress Report, Project Proposal,

PRO·SPECT Presentation, Evaluation Course Tool Seminar

Technical Paper,

Before 2007, laboratory assessments method in the FEE was merely based only on lab report submitted by each group of students. The lab coordinator would then compile all the graded reports and transformed them into grades. This procedure is inconsistence as each module consists of different area of electrical engineering fields and involves many lecturers from various disciplines in the FEE. Furthermore, there were no specific guidelines for grading the report and lecturers would rely on their experiences, resulting large variance between the seniors and juniors judgments in giving the marks. To overcome such problem, an OBE assessment tool has been designed recently for evaluating laboratory modules. This measurement tool, known as Laboratory Sensor Performance Evaluation Course Tool (LAB-SPECT) shown as in Figure I, is an Excel based designed software that has taken into account additional students' attributes that represents leadership and team-skill in the lab.

It has two category sections; Group Related Skills (GRS) and laboratory report. In evaluating students fairly, lecturers are being provided by rubrics marking scheme for each topic in the sections above. The system tool fmally will produce performance plots that show students' POs achievement scores. In addition, lecturers can also analyze other plots such as COs and POs density, as well as students' population density in achieving the scores. All these plots can be used by the lecturer to prepare a Continual Quality Improvement (CQI) report for the laboratory module.

154

LAB·SPEeT �

Designed By Assoc. Pror. Or. Hadzli Hashim Developed By Mohd Fuad Bin Abdul Latip

httpJlwww.fke.uitrn.edu.my

All RICht ReHtYed c..llt fKE Urt'M MiilaysJalOO9

Figure l. LAB-SPECT Introductory Page

This paper describes the process of how students are being assessed when taking an electrical engineering laboratory module. EEE36 I , an Electrical Engineering Laboratory lab course in semester 2 is selected as a model case to be presented in this paper. Experiments involved in this module will be elaborated in Section II, while Section III dwells with the assessment process and the utilization of LAB-SPECT system. The output results and recommendations for improvement will be presented in Section IV and Section V respectively. Finally the whole paper will be concluded in Section VI.

II. ELECTRIC AL ENGINEERING LABOR ATORY MODULES

Engineering Electrical Laboratory courses are offered to enhance the understanding of the electrical engineering knowledge learnt in the classes through practical work done in the labs. There are five lab courses known as Electrical Engineering Laboratory I until Electrical Engineering Laboratory 6 that are being offered for students in semesters 1, 2,3,5, 6 and 7 respectively. Assessment methods involve two components; Group Related Skills (GRS) and laboratory report. The COs basically focus on having attributes in GRS; such as team work, practical skills and fmally safety and ethics in the lab. While, the lab report focuses on writing skill, formulation of solving problem and designing elements. The COs and POs addressed by all laboratory modules are tabulated in Table III. It can be observed that, there is a direct or one-to-one mapping between each CO and its respective PO.

As mentioned in previous section, EEE361, a semester 2 lab course is selected as a model case in this study. Therefore, it's COs and POs would be referred to Lab 1 and Lab 2 specifications. In this course, students are expected to complete nine experimental modules within nine weeks in a group of at least two students. These experiments comprise of knowledge area in power, electronics and computer. As this laboratory module is considered as lower level module,

Page 3: [IEEE 2009 International Conference on Engineering Education (ICEED) - Kuala Lumpur, Malaysia (2009.12.7-2009.12.8)] 2009 International Conference on Engineering Education (ICEED)

2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

the students are expected to acquire the basic laboratory works knowledge. The experiments that are carried out throughout the semester are design to enhance their understanding on the fundamental of electrical engineering courses.

Lab

(Sem)

",N' or. E = � .... '" ...... c c = = - -or. • = E .... �

... Vi' or. E = � .... '" ...... c c = = "'''' or. • = E .... �

",r:c � � .... '" ...... c c = � on", or. • � E .... �

TABLE III. COs AND POs SPECIFICA nONS FOR ELECTRICAL ENGINEERING LABORATORY IN FEE UITM

Component CO CO Attributes Addressing PO

COl Teamwork: Adhere 10 instructions and attend the

POW laboratory on time.

GRS CO2 Skills: Use effectively the technology, tools and

P06 instruments.

C03 Ethics and Safety: Dress appropriately in the

P07 laboratory

Written: Present and label clearly the figures and C04 P02

graphs in the report.

Report C05 Identify the engineering problems: Identify and

P04 explain the electrical engineering problems

C06 Apply fundamental knowledge: Relate and apply

POI basic knowledge to the experimental work.

COl Teamwork: Work effectively with team members. POW

GRS CO2 Skills: Utilize techniques and skills in the laboratory

P06 experiment.

Ethics and Safety: Adhere to the instructions of C03 P07

laboratory safety.

Written: Express points clearly and effectively C04 P02 through presentation and/or report.

Formulate to solve the engineering problems: Report C05 Formulate and analyze to solve electrical P04

en2ineerin2 problems.

Design and evaluate operational performance: Use

C06 the fundamental principles to design the system or P05

task based on the 2iven specification.

COl Teamwork: Give commitments to the group in

POW achieving the objectives of laboratory.

Skills: Use logical skills in troubleshooting the GRS CO2 problems encountered while performing the P06

laboratory procedure .

Ethics and Safety: Demonstrate positive attitude C03 P07

towards the group.

Report C04 Written: Discuss effectively the results and conclude

P02 the main points in the report.

III. ASSESSMENT PROCESS AND LAB-SPECT

In all the laboratory courses, students are evaluated based on the components specified in the GRS as well as their laboratory reports. Despite of the many experiments with their GRS and report assessments, laboratory coordinator will select only three samples representing first lab session, mid semester and the last lab session, for the purpose of OBE measurements. All these raw marks plus information on the appropriate CO and PO, will be used as the input when using the LAB-SPECT. The first step of using the tool is to identify the CO-PO mapping of GRS and laboratory reports with respect to Table II. The outcome will be printed in Table IV and Table V where shown are the CO-PO

155

mapping of the respective GRS and report component in EEE361. Only the strongest PO is identified for each CO. These mapping enables the distribution of the CO addressed by each evaluation components and eventually, the relationship between COs and POs that are being addressed and can then be observed.

TABLE IV. CO-PO MAPPING FOR EEE361 (GRS COMPONENT)

GROUP RELA 11:0 SKILL (GRS) Eksperimenta:z> EXP • EX" EXP.

.. ""= .. ., . ., .. ., . ., • .00 . ., '.00 . ., • .00 CourseOL/tcon'r.;Ii:IQ C01IGRS1 CQ2K;RS2 C03IGRS3 C01IGRS1 C02IGRS2 COllCRS3 C01IGRS1 C02JGRS2 C03IGRS3

PO.

. PO, Eo

.� �- PO, o· 6� PO.

u g PO • . " � . POll , , , ,. .-u> P07 , , , " �

i� PCB

�;; !! P09

. " P010 , , , � P011

TABLE V. CO-PO MAPPING FOR EEE361 (REpORT COMPONENT)

LAB REPORT Ekiiperimenl==> EXP1 EX .. EXP9

Markii=> .. ., .. ., '.00 '.00 '.00 '.00 '.00 '.00 '.00 CourseOulcomH="> C04 co. coo co. coo cos co. co. coo

PO.

. PO, :Eo

�� PO, o •

6 � PO.

ug PO. ." �� POS 0-u� P07 oZ .0 PO S a PO.

2� P010 � P011

Once the CO-PO mapping is done, plots that show the CO density as well as PO density is depicted in Figure 2. As can be seen from Figure 2(a), this laboratory is concerning on six COs with equivalent marks for each GRS and lab report component. Thus, each CO would carry an even weight of 16.67%. This indicates that students are expected to familiar with all the six outcomes that are considered to be important aspects should be practiced when working in the lab. The GRS assessments address the first three COs. COl, and C02 are focusing on the teamwork and skills respectively while C03 is related to the ethics and safety in the lab. The last three COs of the course are addressed by the lab report. C04, COS and C06 are related to written skill, identifying the engineering problems and applying the fundamental knowledge respectively. All of these COs are mapped to six POs, respectively. The first three COs that are addressed by the GRS are linked to PO 1, P02 and P04 while the last three COs (which are expected to be achieved through the reports) address P06, P07 and P09. Since each CO addressing one PO, thus all the six POs have a weight of 100%, as depicted in Figure 2(b).

Page 4: [IEEE 2009 International Conference on Engineering Education (ICEED) - Kuala Lumpur, Malaysia (2009.12.7-2009.12.8)] 2009 International Conference on Engineering Education (ICEED)

2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

IV. PER FORMANCE ANALYSIS

Once the CO-PO mapping is done, students' marks for the evaluation components are ready to be entered and processed for any output measurements. However, before that lecturers need to evaluate their students' performance for each of the GRS component as well as when grading the reports. Guidelines for giving the marks can be referred to the rubrics table shown in Table VI.

16.67

LABORATORY ANALYSIS PERFORMANCE

ELECTRICAL ENGINEERING LAB 2 (EEE361 ): APRIL 2009

Plot: CO Density

16.67 16.67

C02/GRS2 C03/GRS3

16.67

Course Outcome (CO)

(a) CO Density

16.67

LABORATORY ANALYSIS PERFORMANCE

ElECTRICAL ENGINEERING LAB 2 (EEE361 ): APRIl2009

Plot: PO Density DC01IGRSI _C02JGRS2 _C03IGRS3 _COot IICOS

16.67

,-- ,- ------n'---- .- -,------ �----,

- - - - - - - - - - -

Program OUtcome (PO)

(b) PO Density

Figure 2. Distributions of COs and POs for EEE361

These rubrics mapping of students' performance exercise will actually allow lecturers to justify any marks given to their students. They just have to select the right cell for marks after observing the students' activity in the lab or reading through their reports. These marks will be filled in a customized evaluation form provided by the lab coordinator beginning of the semester. The lecturers will need to return these forms to the coordinator, usually after the mid-semester break. In this case, coordinator can start early in managing raw marks keying exercise into the appropriate data entry sheet in the LAB-SPECT.

Since LAB-SPECT produces output measurement of POs and indirectly, the COs achievement by the students, therefore the processing engine in the tool is designed specifically for these purposes. The algorithm is being formulated to map and compute all input marks with its respective PO. The fmal measurement score is computed by normalizing the actual student's earned point with the total maximum possible point. Equation 1 describes the

156

calculation of a PO score respectively. During the process of calculating a specific PO score, all other non-relevant POs and COs are disabled by labeling them as logic '0'. Alternatively, only the interested PO is labeled as logic '1'. In that case, only the respective student's mark for this PO is considered in the calculation process.

TABLE VI. RUBRICS MARK FOR EEE36 I

LAB COMPONENT CO-OETAILS

TealThYO�:

GRS S��

Ell'i::sand Salet(

Wntten:

2

kErt�tte lJIlREPORT erg�1\l IXOlierns:

IWf IiMinrenial

�.

SMC, T

SMc,

c. SM '

%po'tudent '=1,2,3.. ..

RMc, T

RMc,

c RM '

where SM

RM PO C

RUBRICS TABLE

1 (Very weak)

attell1tte�b15 rri1JIes�e

carrdusedtte IW(gf!odsall1

i"61nrnen 1 al aD

wiIIIi�sale�sOO€s all1�dlel

rxirouil!d�aD

l1'eM'lSeoitte �batteqll!soonlo re answered dllirg tte�b5�elevart

Rep:xt1L5l� 1es IreMrrI

lllllerslardngol Iliscooocei/S

ulI!r¥l\ltte9b

POc, , POc, ,

c POj

2 3 4 (Weak) jModerate) (Strong)

attefdttelab10 attell1tte�b5 attefdtte�bontirre

min�es�le nillies�e usedtte usedtte usedtte

led'oci:igf!oo�all1 IW(gf!ods all1 led'oci:igf!oo�and

i"6lrurrertafier ilslnrnenl afi� i"6lrurrertafier ex�in all1derro� ex�inandil!lIO� ex�in� i"6 1n.dor.(lIOrelOOn i"61n.dor.(crelire i"6b1.dorloretinre

21irrres) orll) ooi{)

�ess aooadrdflo �lIn.1 salayslm IIiIIIlijadiei

tte�b�essooil!

IlIIp"operlorrnal Derool'61rales Mm eITIXS a mlellOO

ard'orroo�r erroffi omol'6

aom�ol'6

l1'e1ll�ol tte l1'e1ll�oltte

Ttelll�oltte �batteqll!Soonlo �borttelp!sOOnlo

real'6 wereddllil\l rearmredilri"g 9bortteqll!slKxi10

ttelab5�rtia� tte9b5KErtfied, reansweredrlirir\l

K1entifed,aIl15 1xJ5s�led�a tte�b5eIT�I.5.

slaled�a sornewhat slXl'elllmitde ar

urdearroomer. roomer.

Rep:lrtlt6tra:es Rep:xt IL5IraIes an lia<ruale

Rep:lrtill6lralesa accrJale

�niled�rs1all1rg �ffi�lI1rgol

ofbascoon:ei/S �ffi�nfllJol

bascoorc¢ �r¥l\ltte�b.

IIOSIllisclXlOCei/S UllJertiiigtte9Q ullJertiiigtte�b.

totalcomponen'( C. c. ) L SM' x PO

; ,

5 (Very Strong)

attell1tte�brelNe lire

canusedtte IW(gf!odsall1 i"6IrurrertOOIT� wiIInI!/IKI!iom

U"ib1.da.

dressilCCOrdrg�1o tte�bdressooil!

and neal

Preserrted�adli� � lie,mgaiXt IrdilEsan

allXolliatelille,and 5 acuale, oompiele, all1�operIy�re�

l1'eM'lSeoitte �batteqll!Soonlo reanswereddlJil\l tte�b5c1early

KErtfiedands� 1ed.

Rep:lrtlL5tr�es an iIl:U�leand IIixollJh

underslardngol Iliscooocei/S

UlI!iMllltte 9b

,

POc, x 100= j=1

total'I0nen'( RM

Cj xPO�j )

xlOO (I ) ,

pd ' )=1 ;

c. PO' , student's marks

total marks program outcome (logic' l' or '0') assessment component

Page 5: [IEEE 2009 International Conference on Engineering Education (ICEED) - Kuala Lumpur, Malaysia (2009.12.7-2009.12.8)] 2009 International Conference on Engineering Education (ICEED)

2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

Table VII shows an example of thirteen EEE361 students together with their earned marks that are entered in each of the respective GRS components. Raw marks from the evaluation form submitted by the lecturers are compiled by the coordinator and later handed to a special unit formed by the FEE for the purpose of doing the data entry. At the faculty, this unit is named as Data Management Unit (DMU), and it will manage all raw marks data entry using the LAB­SPECT. Once the students' marks are entered, measurement of their POs score will be automatically produced. Simultaneously, an average score for the respective POs will be computed which represent the overall performance of the students taking this lab. All of these scores are displayed in terms of plot and thus, conclusion can be derived from them. An example of this plot is shown in Figure 3.

No.

1 2 3 4 5 6 7 8 9 10 11 12 13

TABLE VII. GRS SPECIFICATION TABLE FOR RAw MARKs DATA ENTRY

GROUP RELATED SKILL IGRS) Ell,! EJlP4 EIIP1

ot.mt COtIGRSt COZGRSl CO!IGRS3 COtIGRSt C02IGRSl COlIGRSl COtIGRSt COZGRSl CO!IGRS3

2tre298478 3 3 3 5 4 5 4 4 4 :lII8298464 3 3 3 4 3 3 5 3 4 :lmII!I58 4 4 4 4 3 3 4 3 3 :!II82!I823Ii 4 4 4 4 3 3 4 3 3 m298452 4 4 4 4 3 3 4 3 3 _48 4 3 3 5 3 4 4 3 4 m298252 4 4 4 4 3 3 4 3 3 21m291!i24 4 4 4 4 3 3 4 3 4 2OO829Il374 4 4 4 4 3 3 4 3 4 2008298454 4 4 4 4 3 3 4 3 4 2Ire298362 4 4 4 4 3 3 4 3 4 - 4 4 4 4 3 3 4 3 3 - 4 4 4 4 3 3 4 3 3

As illustrated in Figure 3, it can be seen that all POs can be considered as strongly being achieved where the score are more than 65%. PO 10 which is related to the teamwork has the highest score, indicating students within a group managed to work as a team even though they were paired or grouped randomly by the coordinator. P04 which is focusing on solving the engineering problems has the lowest PO score that is around 67%. This is due to some of the experiments that were carried out with some of the students did not even yet learn about the theory part during the lecture class. This is due to the reason that some of the course theory is thought simultaneously with the assigned experimental module at the same semester. For such case, an instruction by the FEE management is given to the lecturers involved. A briefmg must be conducted prior to the experimental work to accommodate the students' lacking in the theoretical part of the experiment. However, this was not enough to help these students to really understand the purpose of the experiment. In addition, students sometimes do not know how to apply the knowledge that they have learnt in the class in solving the problems given in the lab.

157

The next phase involves transforming the PO scores into a qualitative ranking level by using the guide line table given by the FEE. Table VIII refers also as the Key Performance Index (KPI) to be achieved where a score of more than 65% will be defmed as very strong. Actually, this discretion table is originated from the applied conventional grade point average system where anything less than 50% is considered to be non-performing or below grade 'C', and a score more or equal than 65% reflects a grade 'B' [7]. Sustaining a strong ranking or 'B' for every course will eventually result in the students to at least graduate with an upper second class honors. Besides that, performance of the students in EEE361 can always be easily monitored, tracked or compared regularly between one cohort to another.

LABORATORY ANALYSIS PERFORMANCE ELECTRICAL ENGINEERING LAB 2 (EEE361): APRIl2009

AverageScorefor PO

100,--------------------------------,

74.59 71.41

70 67.85

30

10 0.00

80.00 73.19

0.00

Program Outcome (PO)

0.00 0.00 0.00

Figure 3. Distributions of PO Average Score for EEE361

TABLE VIII. KEy PERFORMANCE INDEX RANKING

PO Score (%) Rank Description Color Code Level 0-49 I Weak Red

50-64 2 Moderate Yellow

65-100 3 Strong Green

Figure 4 represents the PO average score in terms of ranking level of achievement. As shown in the previous figure, the six POs addressed by this course had achieved score more than 65% which is very strong (level 3). Therefore, this batch of students has shown strong attributes in the identified POs. LAB-SPECT can also provide avenue for the lecturer to dissect his/her students' density performance so that future improvement can be made during delivery in the lab. Such indicator is described in Figure 5, where it can be observed that majority of the students have strongly achieved the addressed POs. Infact, more than 90% of students have strongly achieved P02, P07 and PO 1 0 which are related to the soft skills of the students. About 1 % of students are found to be weak in P02 which is related to written comprehension of language used in expressing ideas.

Page 6: [IEEE 2009 International Conference on Engineering Education (ICEED) - Kuala Lumpur, Malaysia (2009.12.7-2009.12.8)] 2009 International Conference on Engineering Education (ICEED)

2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

LABORATORY ANALYSIS PERFORMANCE ELEC TRICAL ENGINEERING LAB 2 (EEE361): APRIl2009

Average Ranking for PO

Program Outcome (PO)

o I

Figure 4. Distributions of PO Average Ranking for EEE361

70

.60

{so � 40

10

_Levell·weak

IILevel2-Moderlilie

aLeve!3.Slrong

LABORATORY ANALYSIS PERFORMANCE ELECTRICAL ENGINEERING LAB 2 (EEE361): APRil 2009

Ranking Level: Students' Population Density

r r r P01 POl P04 P05 P06 P07 PCO P09 0.0 0.00 0.00 0.00 0.00 0., 0., 0.00

11.11 0.00 22.22 0.00 22.22 4 ... 0.00 0.00 ..... ".44 0.00 77.78 0.00 77.78 95.56 0.00 0.00

I

POlO P011 0.00 0.00 4 ... 0.00

95.56 0.00

Figure 5. EE361 Students' Density for Different Ranking Level

V. CONTINUOUS QU ALITY IMPROVEMENT

As this course is considered as a lower level laboratory course, students are expected to familiar with the important elements to be practiced when doing the lab work. From all the plots observed, at this stage it can be considered that all POs that are addressed by this course had achieved the target. However, there is still room for improvement especially to increase the students' performance for P02 and the P04 score. The recommendation for improvement includes the followings:

1. Increase the understanding of the concept of electrical engineering knowledge so that students can apply the knowledge in solving the problems given in the lab.

2. Encourage and motivate students to express their ideas through discussion during the lab session.

VI. CONCLUSION

The process of evaluating the outcomes of electrical engineering laboratory modules in FEE, UiTM used to be very complicated as the laboratory modules normally consists of experiments from different electrical engineering fields. Thus, a user friendly and reliable supporting tool known as LAB-SPECT has been developed to facilitate the

158

lecturers to do the evaluations. The system offers a systematic ways in evaluating lab performance that consists of GRS and laboratory report. The system will use raw marks gained by students during the lab assessment activities and compute their measurement score of the respective POs. The system outputs are in the form of various plots that can provide indicators to the lecturers for recommending further improvement.

This paper has described the step by step algorithm used by the LAB-SPECT using Electrical Engineering Laboratory 2 (EEE361) as a model case. Students' raw marks from the assessments activities during the December 2008-April 2009 semester session were used as inputs for the system. Outputs plots of average score and ranking of achieved POs as well as the students' density for the three different ranking levels are shown. These plots can be used and analyzed thoroughly by the respective lecturer and later make recommendations to be implemented for Continuous Quality Improvement (CQI) exercise.

ACKNOWLEDGMENT

The authors would like to thank all members of the Infrastructure Unit of OBE Task Force and also to all laboratory coordinators at the FEE, UiTM, for their continuous support and commitment in practicing and realizing the usage of LAB-SPECT to be feasible.

RE FERENCES

[I ] W. G. Spady, "Organizing for Results: The basis of Authentic Restructing and Refonn," Educational Leadership, Vol. 46, No. 2, 1998, pp. 4-8.

[2] W. G. Spady, KJ. Marshall, "Beyond Traditional Outcome-Based Education," Educational Leadership, Vol. 49, No. 2, Oct 1991, pp. 67-72.

[3] Ministry of Education Malaysia. (2006). The Future of Engineering Education in Malaysia. United Mind.

[4] Faculty of Electrical Engineering (2007), Program Self Assessment Report, Electrical Engineering Programme, Universiti Teknologi MARA, Malaysia.

[5] M. M. Kamal, H. Hashim and W. Mansor, "Summative Assessment Of Outcome-Based Education In Electrical Engineering", in Proc. of the 41h Int. Conf. on University Learning and Teaching (INCUL T 2008), Shah A1am Selangor, 20-21 Oct. 2008.

[6] W. Mansor, H. Hashim, S. A. Che Abdullah, M.U. Kamaluddin, M. F. Abdul Latip, A.I. Mohd Yassin, T. K. Abdul Rahman, Z. Zakaria, M.

Md Kamal, "Preliminary Results on the Implementation of Outcome­Based Education on the Non-Examinable Computer Engineering Modules," 381h ASEEIIEEE Frontiers in Education Conference, 2008, pp. S4B-20-S4B25.

[7] Academic Affair Division. (2009). Academic Regulation, Diploma and Degree Program, Universiti Teknologi MARA.