[ieee 2009 international conference on engineering education (iceed) - kuala lumpur, malaysia...

2009 International Conference on Engineering Education (lCEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

Modem measurement paradigm in Engineering Education: Easier to read and better analysis using

Rasch-based approach

Hamzah A Ghulman,

College of Engineering and Islamic Architecture Umm al-Qura' University, Makkah al-Mukarramah,

KINGDOM OF SAUDI ARABIA e-mail: [email protected]

Abstract- Present students' evaluation measurement generally

practiced in Institutions of Higher Learning (IHL) is largely dependent on students' performance in carrying out tasks such as a series of tests or quizzes, final examination and submission

of assignments. However, the over use of Cumulative Grade Point Average (CGPA) which computation is purely the mean of raw scores, lacks precision and linearity hence validity required to meet the fundamental criteria of measurement

warrants a review. In mL, the theory and practice of classical test theory, the traditional approach of students' evaluation must be re-assessed. This paper provides an overview of an alternative "modern" measurement as practiced using item

response theory with focus on Rasch measurement model. An overview of Rasch measurement model and its key concepts are

presented. This assessment model proved to be a better students' assessment method and can be used to validate the

CLO of each course. A case study in the College of Engineering, Umm al-Qura', Makkah on students (N=75) was conducted to measure their command of knowledge (Kn) and

understanding (uN) for the subject 804431 -ManUfacturing

Process III; as categorised according to Bloom's Taxonomy disclosed that there is a need to take immediate action where their "Person measure level is disturbingly low; "person = -0.29Iogit.

Jlitem measurement also indicates that the undergraduates

encounter significant difficulties in grasping some fundamental engineering principles in extrusion. Only 30.67% (N=23) of the students measured found to be above the item measure �.OO logit. Whilst 6.67% (Nitem=5) of the students were found to be below the Minimumitem measure:5; -1.03Iogit. The study shows that SPELA as a model of measurement can provide better estimate of students' ability more accurately based on the CLO as compared to the traditional CGPA method of assessment using raw score.

Keywords- Outcome Based Education, Learning Outcomes,

measurement, Quality, engineering education, Rasch Model.

I. INTRODUCTION

The College of Engineering and Islamic Architecture, Umm al-Qura' University, Makkah, KSA (CEIA) has taken the challenge to improve their teaching and learning system of their undergraduate programs by attempting to obtain certification of ISO 9001:2008 - Quality Management

978-1-4244-4844-9/09/$25.00 ©2009 IEEE

Mohd Saidfudin Mas'odi, Pro-Cert. in Quality Management,

SPACE, University Teknologi Malaysia, 81300 Skudai, MALAYSIA

E-mail: [email protected]

System for the scope of service provision in teaching and learning in engineering education. They are also doing their best endeavour to obtain the accreditation of the American Accreditation Board of Engineering and Technology 2000 (ABET); an authority in quality engineering education whose approval is highly sought for by many Institutions of Higher Learning worldwide.

It is CEIA top management commitment to meet ABET program accreditation requirements where among others promote outcome based education (OBE) learning process. OBE calls for the evaluation of the course learning outcomes (CLO) as specified in each Course Outline. Performance Measurement has been largely dependent on students' performance in carrying out tasks such as a series of tests or quizzes, fmal examination and submission of assignments. Evaluation on the performance outputs; encompassing both categories, technical knowledge and generic skills gives an indication on the achievement of the subject's expected CLO. However, the current Cumulative Grade Point Average (CGPA) is only a mean average of raw scores which lacks precision and linearity hence validity required to meet the fundamental criteria of measurement.

II. FUNDAMENTALS OF MEASUREMENT

Measurement is of utmost importance in our everyday life. Mainly measurement can be used for regulate trade, monitoring; and calibration. Academicians have great need for the development of valid measures, e.g., of the quantity and quality of education services and the outcomes of these services; be it teaching and learning as well as the conduct of researches. Many researchers are frustrated when existing instruments are not well tailored to the task since they cannot expect sensitive, accurate, or valid fmdings from the use of such instruments. In CEIA, the theory and practice of classical test theory, the traditional approach of assessment and evaluation effectiveness is reviewed. It then provides an overview of "modem" measurement as practiced using item

2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

response theory with a focus on Rasch measurement model[l].

This paper describes a computational model which has been used to measure subject CLO in an undergraduate engineering program in University Teknologi Malaysia (UTM) and University Teknologi MARA (UiTM) which is further validated and confirmed in CEIA [2]. An overview of the measurement model and its key concepts are presented. This assessment model is developed based on Rasch Measurement Model which can be used to improve the students' assessment method and validate the CLO of each course. Results obtained were evaluated against the CLO map; developed based on Bloom's Taxonomy, for consistency [3]. The information generated from this measurement are of meaningful use to guide us determined the appropriate improvement of the teaching method or style employed as well as in determining of the quality of questions prepared. Questions were assessed on their discrimination index; b and subsequently scrutinised on its level of difficulty index; a before it can be considered as a bankable item in the Institution of Higher Learning (IHL) question bank. Thus, the construct validity of a particular question encompassing the topics learned, its learning outcomes and level of study is therefore resolved simultaneously.

The data is then transformed into logit, primarily to obtain uni-dimensioanality on a linear interval scale with better precision to measure the ability of students in respect of their learning difficulty encountered. It can be readily shown mathematically that a series of numbers irrespective of based used is not equally spaced but distant apart exponentially as the number gets bigger whilst a log series maintain its equal separation; thus equal interval. This equal separation we termed it logit as unit of measurement of ability akin to meter to measure length or kilogram to weight.

This provides a sound platform of measurement equivalent to natural science which matches the following SI Unit criteria; there must be an instrument of measurement with a defmed unit. It is quantifiable by mean of linearization with reasonable accuracy. The measurement shall be replicable and consistent and; is predictive to overcome missing data. [4]

III. MEASUREMENT METHODOLOGY

Responses from the students' exam results were analysed using rating scale in which the students were rated according to their achievement by topical area of study. Practically, this is only counting the responses of correct and wrong answers from the students' responses who sat for the exam that gives a raw score for each topic tested. However, rating is only an order of preference; an ordinal scale which is continuum in nature, and do not have equal intervals which contradicts the nature of numbers for statistical analysis. It does not meet the fundamentals of sufficient statistics for evaluation. Data set would normally be put on a scatter plot to establish the best regression. However,

2

prediction from ordinal responses on the ability attributes is almost impossible due to absence of intervals in the scale. The normal solution is to apply the regression approach where a line which fits the points as best as possible; which is then use it to make the required predictions by interpolation or extrapolation as necessary as shown in Figure I.

y = 130 + 131m Equ.(l) In obtaining the best fit line, there exist differences between the actual point; Yi. and the best line, the predicted point; )ri. The difference is referred to as error; e.

Equ .. (2)

Figure I. Best fit line: Linear Regression Model

By accepting the fact that there is always error involved in the prediction model, the deterministic model of equation (1) can be transformed into probabilistic model by including the prediction error into the equation;

Equ. (3)

Rasch moves the concept of reliability from establishing "best fit line" of the data into producing reliable repeatable measurement instrument. Rasch focuses on constructing the measurement instrument rather than fitting the data to suit the measurement model. By focusing on the reproducibility of the latent trait instead of forcing the expected generation of the same raw score, i.e. the common expectation on repeatability of results being a reliable test, the concept of reliability takes its rightful place in supporting validity rather than being in contentions. Hence; measuring competency in an appropriate way is vital to ensure valid quality information can be generated for meaningful use; by absorbing the error and representing a more accurate prediction based on a probabilistic model.

In Rasch philosophy, the data have to comply with the principles, or in other words the data have to fit the model. In Rasch point of view, there is no need to describe the data. What is required is to test whether the data allow for measurement on a linear interval scale specifically in a cumulative response process i.e. a positive response to an item stochastically implies a positive response to all items being easy or otherwise [5]. Rasch Measurement Model is expressed as the ratio of an event being successful as;


P(9) =

where;

Equ.(4) correlation of the person, �n and item, OJ can now be established as in equation (6).

e = base of natural logarithm or Euler's number; 2.7183

�n = person's ability

Summary statistics of Person and Items measures were next captured. It is then used to complete the PIDM indicating both the Person and Item maximum and minimum to give an indication of the person and item spread hence Standard Deviation (SD). The respective summary measurement is shown in Figure.3 -Persons Measure and Figure 4 for Item Measure.

OJ = item or task difficulty

IV.

It reveals a fair person spread of 2.35logit with

� f3:, . � !?- :II t.e=", ( O� . ) <l::.:K c::e- 1.1.e=t> I �..)j..� Qu, ...- ", t:i. OD.'" >

-+

TI � Q'§ ___ J..5

:s::9� I � �

� !Fa? I �

� F33 51

I toe =- __ ;o;x= 3. _ 092 ::'��:SULK FOR!! ING ( kr. )

GE. (�}

�=O � !F3.6 F24 I

RA I O ( � ) T I ON ( �} -}

o !F3.4 . 3. 5 !F3.? �3.8 !F20 ·23. !F29 55(�} E. (�}

- :1

� F2S F26

� F30

F_9 F34 F36 F4_ F4S

�e·;a.:n=- 0 _ 2 9

F<l2

F3 3.

FSO

If"S ?

If"S6 If"sa 1f"<; 9 �6:'2

F43

F3S

FSS

F32

If"S B

1f"6 0 If" 6:1 If"s a 15 1f"6<; I �� 1f"6 B 51

I

� IT

T I I

� I

�

IT'E.RIA(�} S - If"OR CE. (�}

!Pe':r� D%l. ��=- � _ B 0 � -+

<L�O • 4 a -<Pooor Ab :i. L :i.. t:.y > I <Ea. "' y � '" t. :i. O<r.> �O. S2

5p :rot::· -.d. : Poer",O<r.= O . S S - (- • BO ) I 1:A=n= :1 . <; :z -. (- 3. • 0 3 )

2 _ 35 � = 2 . 09 5 �

Fig.2: Person-Item Distribution Map: Students location

DATA ANALYSIS AND DISCUSSIONS

The test was administered on 3rd year students from the Department of Mechanical Engineering, Umm al_ Qura', Makkah for the course code 804431 -Manufacturing Process III . The result from the test were tabulated and run in WinSteps, a Rasch Analysis software; to obtain the logit values. Figure 2 shows the Person-Item Distribution Map (PIDM) where the person; i.e. the Students and the item; the learning topics are plotted on the same logit scale. By virtue of the same scale; then the basic rule of additivity, the

SeparatIOn, U= 1.�4 and taIT Reliability of Cronbach-a = 0.80. The major fmding is the Person Mean, f1.Person= -

O.29logit where the Students were found to be below the expected performance. Only 30.67%(N=23) of the students measured were found to have acquired the expected Learning Outcomes as compared to the Raw Score obtained; where 77.33% (N=58) students passed.

Further scrutiny is done by topic and Bloom's Taxonomy learning curve denoted as [Qn-Topic (Bloom's)]. Blooms define learning into six domain from the simplest to complex; knowledge, understanding, application, analysis,

3


evaluation and synthesis. This analysis clearly identify that there are four(4) groups of students profile from the poor to excellent as

demarcated in the person, /30 column. Whilst the Questions is basically of two(2) types; easy and very difficult, though an easy gap denoted by ( � exist between QI-Manufacturing (Knowledge) and Q3-Forging (Understanding).

Generally, the students fmd half of the topics studied are difficult to grasp; with Q8-Segmented Die (Analysis) and Q 14-Bulk Forming (Knowledge), being most difficult where most were not able to solve it satisfactorily. It was also noted there is a huge gap between Q9- (Application) and Q8-(Analysis) and, next Q8 and QI4-(Knowledge) denoted by () indicating the extent of difficulty the students encountered in attempting the question. Hence, students F5, F3, F2 and F8 are exceptional students in this cohort.

Before proceeding further on the person item map analysis, it is prudent to check whether the exam paper; as the instrument of measurement is measuring what it is supposedly to measure hence construct validity.

The Item Summary gives a good summary with Separation, G=3.45 and a very high Reliability=0.92. It has good item spread of2.45logit with SDi=0.52 but requires review on the both end of very difficult and very easy items which shows a hollow area need to be patched up. Nevertheless, students F70, F71, F72, F74 and F75 is defmitely in trouble as they have serious difficulty understanding this programme where they are located well below all items.

Analysis of the Point Measure Correlation shown in Figure.5 indicate that Q I O-Extrusion Ratio (Understanding) need review. It does not meet the discrimination criteria of a quality question. Controls applied was to checked the item as acceptable when the Point Measure=x; 0.4< x <0.8. QIOPoint Measure=O.3< 0.4; therefore need further check.

TABLE 26.1 MANUFACTURING III Final 08 Jun 5 1:152008

INPUT: 75 Persons 18 Items MEASURED: 75 Persons 18 Items 5 CAT ------------------- -+---------------------- -[]-:------------------------------------ Person REAL SEP : 1 84 REL. O. 7� --

.----�� --;�;� -;�� -

:--;�;; --��-

.-:--��;

I RAW MODEL INFIT OUTFIT

+----------------------------------------------------------- -----+

I I ENTRY RAW MODEL I INFIT I I TMEA I I I

I SCORE COU ME � ERROR MNSQ ZSTD MNSQ ZSTD I �� __ =�� __ ��:: __ ��=�� __ =_��_��=�--:=::��':':=�--:==:'� �� I ===�_: 1_______________________ _________ ______________________________________ 1 79 75 1.42 .3511.96 1.111.56 0.81 .0811 Q14 I

I 2 87 75 .95 .1811. 26 . 6 ..... ":'.-.....;;.�--':.II.If.....l..ll1.....J.,

I MEAN 46.0 18. 9 0.19 1 01 0 1 1 02 0 1 5 146 75 21 0811.25 1.5 2.47 2.81 .301 Q10 I

I . . . . i�: ;� _:g� .0811.24 1.8

SUMMARY OF 75 Persons MEASURED

I S.D. 14.0

I MAX. 75.0

I MIN. 19.0

0.0

18.0

18.0

0.45 0.08 0.36 1.0 1.18 0.8 13 213 75 -.18 :g�: :�� =�:!: :�� -:�: .3811 �; I 7 163 75 .10 .0811.07 .51 .92 -.11 .4611 Q17

0.55 0.69 2.47 4.6 8.72 5.2 11 205 75 -.14 .071 .95 -.41 .91 -.31 .5511 Q18

I 129 75 .34 .091 .75 -1.41 .50 -1.21 .5511 Q9

-1.80 0.15 0.43 -2.3 0.17 -0.9 18 331 75 -1.03 .1111.20 .81 .73 -.51 .5911 Q3

I 17 288 75 - .63 .081 .82 -1.01 .83 -.51 .6111 Q1

100_00 __ 00__ 00 __ 00 15 239 75 -.33 .071 .86 -1.11 .76 -1.01 .6311 Q12 I

I RMSE O. ,"l9 __ .0Ii1i "'O_I�O _8B!;iP,",;V_T .. [email protected] ,_PPoC ............ -iPOi!IiI>ioIsI�J8JLI TY O. 7i : --��---�;��---;;��----��-----��;----��;----��-----��-----: I

I S.D. 64.3 .0 .54 .071 .29 1.41 .43 .91 II I +-----------------------------------------------------------------------+

+----------------------------------------------------------+ Figure 5 - Point Measure Correlation: Item validity

CRONBACH ALPHA(KR20) Person RAWSCORE RELIABILITY � 0.80

Figure 3 -Summary Statistics: Person Measure

This analysis clearly identify that there are four(4) groups of students profile from the poor to excellent as demarcated in

the person, �o column. Whilst the Questions is basically of two(2) types; easy and very difficult, though an easy gap denoted by (+---+) exist between Q I-Manufacturing (Knowledge) and Q3-Forging (Understanding).

Next is to verify the suspect by looking at the Outfit Mean Square (MNSQ)= y-value to be in the range of 0.5 < y<1.5. The fmal check would be on the Outfit z-standard (ZSTD)= z-value if it is within the range of; -2< z<2. Q I 0-MNSQ=2.47> 1.5 and ZSTD>2.2; thus it confirms an item misfit.

It is considered as an misfit only when all the three(3) controls cannot be met. This is a more detailed controlled as compared to the traditional Classical Test Theory (CTT) where it only applies simple discrimination index to make an item bankable or not. SUMMARY OF 18 Items MEASURED

+----------------------,.---"f-------------------------------+ I RAW MODEL INFIT OUTFIT I I SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD I 1---------------------- ------------------------------1 I MEAN 191. 6 75.0 0.00 .10 1. 08 0.1 1. 02 0.1 I I S.D. 64.3 0.0 0.54 .07 .29 1.4 .43 0.9 I I MAX . 331. 0 75.0 1. 42 .35 1. 96 2.0 2.47 2.8 I I MIN. 79.0 75.0 .07 .66 -3.2 .50 -1. 2 I : -��no-·-;;I SD�0.52 SEPARATION 3.4) Item RELIABILITY 1':2 +-

Item RAW SCORE-TO-MEASURE CORRELATION � -0.95 1350 DATA POINT

APPROXIMATE LOG-LIKELIHOOD CHI-SQUARE: 2258.21

Figure 4 - Summary Statistics: Item Measure

4

+------ --------I CATEGORY OBSERVEI I INFIT OUTFIT I STRUCTURE CATEGORY I I LABEL SCaR COUNT \ I MNSQ MNSQ ICALIBRATN MEASURE I 1---------- ------- ------- +--------- --------+

I 1 1 741 5! I 1.00 1 . 01 NONE -.84) 11

I 2 2 61 ! I 1. 39 .67 2.01 -.33 12

I 3 3 40 I -0.84 .53 .21 -.01 13

I 4 4 74 ! I 0.91 .46 -.63 .32 14

II 5 5 434 3 I 1 . 03 1. 24 - 1 . 60 .85) 15 +-------- -------- -------+

Figure 6 -Summary of Category Structure


The structure calibration; 's' is assessed to confirm the rating classification used is applicable where s-value being the difference between each structure;

e.g; S3-2= 2.01-0.21 =1.80; > 1.4, OK. S5-4= 1.60-0.63 =1.37; < 1.4, Not OK

The result shall be in the range where s; 1.4< s<5. It is noted that the difference for each category are irregular where the difference between category 2, 3, 4 and 5 are all less than 1.4. Therefore, the classification A,5>90; B, 4>80; C,3>70; D,2>60 and Fail,1 <60 is not reflective of this cohort person separation. In Rasch, this is termed as collapsing.

In summary there are only two groups of students; between who knows and knows not. It reveals that the responses pattern is conspicuously dichotomous of I and 5 only. The rest of the other ratings were practically submerged. This call for Rasch Analysis by dichotomous approach. If the SD is found to be larger, then the dichotomous method will be used or vice-versa.

++----------------------------R 1. 0 +

I 11111111

A I 11111 . 8 + III

I 111 I 11 I 11

.6 + 11 I 11

555 55

55 55

555

I 55551

555555 I 5555 +

I 1 55 I . 4 + _ . _ . _ . _ . - r ' - ' - ' - ' - ' - ' - ' - ' �' - ' - '

55 11 I 55 11

55 11 .2 + 555 11 +

I 555 1111 I I 5555 44 4444444444444····44444444 I 1 2222* ** ** ** ** ** ** ** ** ** ... ** **.. ** ** ** ** ** ** .. 3333* 1111111" 44 44 I .0 +***********4 22222·***********+ ++---------------------------- -----------------------------++

-1 0 1 Person (MINUS) Item MEASURE

Figure 7 -Category Probability Curve

Next the students learning ability for each CLO identified can be derived from the Person Measure Table as

follows, e.g. JlPerson: P(6) = fJv - Oi

= -0.29- 0

P(6) = ePv-Oi •

l+ePv-oi

P(6) = e-O·29 •

l+e-O.29

= 0.42803 Thus, Students Learning Ability

(f rom Equ.6 )

= 0.4280

Generally the students fared poorly below the expected performance achieving a poor mean; Pperson of only 42.80% which is way below the 60% threshold limit Level of Learning Competence set by the Academic Council, Umm al_ Qura' University . It is very interesting to note that all (knowledge) questions; viz; Q18, Q7, Q2, Q17, Q4 and QI4 are located towards higher order of difficulty where students

5

fmd hard to resolve. Instead students find otherwise in attempting question on (Understanding) viz; Q3, QIl and Q 18 were found to be easier task kept at bay below the person mean; Pperson= -0.29.

TABLE 17.1 MANUFACTURING III Final 0 8 Ju1 6 11: 0 6 2 0 0 8 INPUT: 7 5 Persons 1 8 Items MEASURED: 7

-5 Persons 1 8 Items 5 CATS

Person: REAL SEP . : 1 .84 REL. : 0.77 . . . Item: REAL SEP . : 3 .45 REL. : 0.92

+-----------------------------------------------------------------------+

I ENTRY RAW MODEL I INFIT I OUTFIT I PTMEA I I I

I NUMBER SCORE COUNT MEASURE S. E. I MNSQ Z STO I MN SQ Z STO I CORR. I I erson I --------- -------- ------------------+----------+----------+-----++------1

5 75 18 .55 .2111.63 1.211.20 .51 .4311 F05 I

73 18 .47 .1911.65 1.411.46 .81 .4111 F03 I 72 18 .44 .191 .79 -.41 .54 -.51 .6811 F02 I

33 61 18 .12 .1611.28 1.11 .98 .21 .511 F33 13 59 18 · 07 .1511.12 .611.16 .51 .511 F13

58 18 · 05 .151 .92 -.311.01 .21 .551 F09 29 57 18 .02 .1512.47 4.618.72 5.21 -.161 F29 15 56 18 .00 .151 .74 -1. 21 .61 -.51 .631 Fl5 III 55 19 · 62 . IS i . 55 2. j i . 45 . g i . II i fI4 32 44 18 - .28 .161 .68 -1. 31 .52 -.51 .641 F32 69 25 18 - .96 .251 .54 -.71 .22 -.41 .561 F69 65 24 18 -1. 03 .271 .65 -.41 .59 .11 .431 F65

74 22 18 -1.20 .321 .75 .01 .17 -.51 .471 F74 72 21 18 -1.32 .371 .80 .11 .83 .31 .121 F72 70 20 18 -1.49 .461 .50 .01 .74 .31 .071 no 75 19 18 -1.80 .691 .43 .01 .19 -.41 .2911 F75

1-----------------------------------+-------- --+----------+-----++------1 I MEAN 46 .0 18.0 -.29 .1911.01 .111.02 .11 I I I

I S.D. 14.0 .0 .45 .081 .36 1.011.18 .81 I I I +-----------------------------------------------------------------------+

Figure 8 -Person Measure Table

This peculiar observation is also noted in Malaysian experience [6]. It can be deduced that engineering students dUes.J1ot go for Ieading and remember fundamental concepts . Their preference is towards mechanical questions involving applied mathematics that belongs more to application and analysis in Bloom's Taxonomy learning domain. In fact it can be seen that (Application) questions; viz. Ql5-Strain and Q5-Force are located very much towards the easy task end.

Figure 8 also exposed that student F29 who appeared to have passed the exam is found to be a misfit where he failed the 3-criteria. Scrutiny of the person key performance discloses the student response pattern does not meet Rasch Model expected outcome. Based on the pattern of his given answers, in Figure 9, Rasch Model expected him to give a response of 4 and 5 for Q 1- Manufacturing Process (Knowledge) and QII- Force (Understanding) respectively instead of I. Rasch has this special predictive feature embedded in the model to make it very reliable.

This call for reasoned argument on their level of perception seen thus far on the organisation. Study of the scalogram in Figure. I 0 clearly shows student F29 despite being ranked 22 has given several unexpected responses.

In Rasch, we are interested to fmd out why such behavioural change occurs. Two important things is happening here; first is whether the learning process is at stake or secondly the teaching process need to be reviewed whereby reasoned arguments is given emphasis rather than pure technical and mechanical . Such skewed perception requires correction in our effort to produce not only engineers but also quality human capitals who are equally "ingenious".


TABLE 7.2 MANUFACTURING III Final OB 200B

Jul 6 11: 06

INPUT:75 Persons 18 Items MEASURED:75 Persons 18 Items CATS KEY: . 1 . =QBSERVED I

UNEXPECTED l=EXPECTED, (1) �OBSERVED, BUT VERY

NUMBER - NAME ------------ MEASURE - INFIT (MNSQ) OUTFIT -S.E. 29 F29 .02 2.5 A B.7 .15 -2 -1 1---------+---------+--------+----------+--------- I NUM I tern

(5) Q14 .5. Q9

.5. Q6 .l. Q10

.5. Q4 .5. Q17 .5. Q7

.4. 9 Q2 .l. 10 Q13

.5. 11 Q1B .l. 12 Q16 .l. 13 Q5

.l. 14 Q15 .5. 15 Q12

(1) 16 Qll (1) 17 Q1

1---------+---------+--------+----------+--------- I NUM I tern -2 -1 0 2

Figure 9 -Person Key Performance

, AD AN AM AI. Ai: AJ AI AH AG AA 2 V X V VUiSRQPONtHKJJHGFE 211514 12122214111 ,

RATIIGSCALE:kU bU b71 hll 5)" 117 & 8 3 [I 8 & Il & 6 Sf n 4 211' 3 4(] II

D C LO,

B A

�rlll l1lil1li QlIII Q IQ IQ" 111111 I1Jl II1II5M012n ., • 1 , 54 l 2 1 "' -"'i' JUj�� iI� 5

5

5

5

5

5

5 5 , , 5 "1' , , I"'''''''' 620 nOt21321 ""'J< .... "" """'" • , 5 , , , .. �5 , , , , �oo , , , 00 nOli 5 3 5 ,...."'J.<ll' .... "",." • , " 5 5 , , , , , , I'T' r .. , .. • o • &2030112328 �,.�lb.t!) ,,, ... · , , 5 � , , 5 , 5 5 , 5 .S 8 0 0 8 0 II 0 0&01115&334 5 ��lb.tiJ'l>JI¥. "' .... � , , � , , , , , 5 , , , ."os"., 0 00 n I 7 G 3287 .J,.j�iJ�iJf� 12m1 " , f , 5 , , , , , 5 , , � 08 0' 03" 5S0 8011232211 "'J)..,., .... "' ..... ,,,"" •

� . I-A'I A A A 1'1 , C , � ' A ft 1 ft ", " "" " "'Ii.(.�,, ",(I. ,�

Figure 10 - Pattern Response Analysis -Guttman Scalogram

V. CONCLUSION

This model developed based on Rasch Model and Bloom's Taxonomy learning domain provides a sound platform of measurement equivalent to natural science which matches the SI Unit measurement criteria; where it behaves as an instrument of measurement with a defmed unit and therefore replicable. It is also quantifiable since it's linear. Rasch has made it very useful with its predictive feature to overcome missing data. Modem measurement paradigm: Easier to read and better analysis using Rasch-based approach

The measurement conducted reveals the true degree of learning abilities of the engineering undergraduates. Previously, lack of such measurement in Malaysia as well as in CEIA has made the necessary corrective actions in the form of skills development, education and competency training difficult to formulate. This major problem faced by

, , , , , , ,

+ , ,

-I • • " " -

6

Engineering Education Administrators in an IHL to design the necessary curriculum to mitigate the going concern is therefore resolved. A Computer Aided Test software is currently being rigorously tested for validation before used [7].

[I]

[2]

REFERENCES

B. D. Wright and M. M. C. Mok, "An overview of the family of rasch measurement models," in Introduction to Rasch Measurement: Theory, Models, and Applications, J. Everett V.Smith and R. M.Smith, Eds., 2004, p. 979

Saidfudin, M, Azlinah M , Azrilah AA, Nor Habibah, A & Sohaimi Z, "Appraisal of Course Learning Outcomes using Rasch measurement: A case study in Information Technology Education", International Journal of Systems International Journal of Systems Applications, Engineering & Development; Issue 4, vol.l, University Press, UK. pp.l 64-1 72, July 2007

[3] AChapman, "Bloom's Taxonomy - Learning Domains." vol. 2007: Businessballs.com, 2006

[4] Saidfudin, M, Azlinah M , Azrilah AA, NorHabibah, A; Hamza A Ghulman & Sohaimi Z, "Application of Rasch Model in validating the construct of measurement instrument", in International Journal of Education and Information Technologies, Issue 2, Volume 2,. pp. 105-112; May 2008

[5]

[6]

[7]

T.G. Bond and C. M. Fox, Applying the Rasch Model: fundamental measurement in the Human SCiences, 2nd ed. Mahwah, New Jersey: Lawrence Erlbaum, 2007

Saidfudin, M, Hamza A Ghulman, Razimah A & Rozeha, A, " Application of Rasch-based ESPEGS Model in Measuring Generic Skills of Engineering Students: A New Paradigm", in WSEAS Transactions on Advances in Engineering Education, Issue 8 Vo1.5, WSEAS Press. pp. 591-602, August 2008.

Saidfudin, M., Azrilah, A.A., Azlinah, M.,Nor Habibah, A., Zakaria, S., and H.A. Ghulman, "Development of Raschbased Descriptive Scale in profiling Information Professionals' Competency", in IEEE XPLORE, IEEE IT Simposium (lTSim KL),2008, pp.329-333, August 2008. Transl. J. Magn. Japan, vol. 2, pp. 740-741, August 1987 [Digests 9th Annual Conf. Magnetics Japan, p. 301, 1982].

[ieee 2009 international conference on engineering education (iceed) - kuala lumpur, malaysia...

Documents