for review only - prince of songkla universityrdo.psu.ac.th/sjstweb/ar-press/60-sep/18.pdf ·...
TRANSCRIPT
For Review Only
Elective Course Recommendation Model for Higher
Education Program
Journal: Songklanakarin Journal of Science and Technology
Manuscript ID SJST-2016-0262.R1
Manuscript Type: Original Article
Date Submitted by the Author: 15-Mar-2017
Complete List of Authors: Praserttitipong, Dussadee; Chiang Mai University, Computer Science Srisujjalertwaja, Wijak; Chiang Mai University, Computer Science
Keyword: Collaborative Filtering Technique, Courses Recommendation System, Educational Data Mining, Knowledge Discovery
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
For Review Only
Original Article
Elective Course Recommendation Model for Higher Education Program
Dussadee Praserttitipong and Wijak Srisujjalertwaja*
Department of Computer Science, Faculty of Science, Chiang Mai University,
Chiang Mai, THAILAND
Email address: [email protected], [email protected]*
Abstract
Educational data mining (EDM) is an application that applies data mining technique to
resolve educational managing problems. This paper proposes a general process of EDM for elective
courses recommendation based on student grades. Furthermore, the proper and inadequate reasons
for applying the conventional measurements to the course recommendation domain are studied and
fulfilled with the new proposed quality measurements. Several techniques were studied to establish
the model for estimating the grades of the elective courses. The experiments suggested that SVD-
based technique based on course information depicted the best results. The overall results indicate
that the proposed model is able to provide the personalized recommendation for individual students
based on the students’ abilities.
Keywords: Collaborative Filtering Technique, Educational Data Mining, Courses Recommendation
System, Knowledge Discovery.
Page 4 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
1. Introduction
The general objectives of EDM can be either improving the learning process and guiding
students’ learning or achieving a deeper understanding about educational situation (Romeron and
Ventura, 1992). One of the major problems is how to benefit the collected data for facilitating the
useful information to advise the students for planning their own enrollment courses which suitable
for their own potentials (Ray and Sharma, 2011).
Recommender system aims to provide its users with the relevance information. For modeling a
recommender system, the user personalized information is evaluated and the model for estimating
the rating scores for the items, which have not seen by its users, is developed. A successful type of
recommender system is based on collaborative filtering (CF) technique. The personalized guidance
is evaluated from the opinions of other users collected in the historical database. The similarities
between the elements are evaluated, in order to find the most suitable collaborative elements.
In the literatures, the best performance group of algorithm for CF technique are algorithms
based on the Singular Value Decomposition (SVD) technique (Cacheda, 2011; Kautkar, 2014).
Thus, SVD based algorithm was studied, its parameters were assigned their description for denoting
the terms with in the elective courses recommendation context. The general process of EDM for
elective courses recommendation based on student grades were proposed. The conventional
measurements for recommendation results quality were studied. Their inadequacies for using in
context of the elective courses recommendation were explained, besides, the new proposed quality
measurements were presented.
The remaining of this paper is organized as follows. Section 2 explains about related works
and Section 3 presents the proposed ideas. Section 4 exhibits the experimental results along with
discussions. Section 5 concludes the paper with some final thoughts.
Page 5 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
2. Related Works
Lee and Cho (2011) proposed an intelligent course recommendation system based on content
based CF technique. The system evaluated the relationship between each courses via the field of
knowledge their relevant. The recommendation process started from assessing a student’s weak
subjects and analyzing individual abilities to be able to advise individual students which fields of
study would be suitable and which courses they should take. This approach was suitable for
implementing in a small domain, where all fields of study were well specified.
Ray and Sharma (2011) presented a CF based approach for recommending elective courses.
The main idea was to provide students with the accuracy estimation of their grades. This
information about students’ performance was helpful when they decided to select elective courses.
The estimation grades of elective courses were provided to users based on the nearest neighbors of
both user-based and item-based CFs. Even though, CF recommender system based on nearest
neighbors techniques were simple, the accuracy of their results could be improved with other higher
performance CF techniques.
3. Proposed Model
This research aims to introduce the general process of EDM for elective courses
recommendation based on students’ grades. Furthermore, the conventional measurements for
recommendation results quality are studied. The usability and inadequacy of these measurements for
applying with the course recommendation domain are described. Besides, the inadequacies are
fulfilled with the new proposed quality measurements for course recommender system.
As depicted in Figure 1, the process of data mining typically consists of three main steps as:
data preparation, data modeling, and results evaluation. Data preprocessing is the process of
converting raw data collected from education systems into useful information. Subsequently, data
Page 6 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
modeling, the typical data mining techniques, such as classification, clustering, and association
rules, are applied to that information in order to find the model which represents an educational
situation issue. The interpretation is also taken into account to explain that meaning of that derived
information. Finally, the results acquired from data modeling process are evaluated along with the
results evaluation process.
3.1 Data Preparation
The input of this process is the historical data collected in the enrollment database. The
attributes of each record composes with students’ grades, students’ id, course id, course type and the
grades which they have been scored. The details of this process are as following.
1) Data extraction: the historical data collected in the enrollment database is query based on
the characteristics of the students. The studied records, which are assigned with grades which
contain grade point values (such as A, B+, B, or F), are retrieved. The values collected in
GRADEVALUE are assigned with grade point values which are numerical data.
2) Data cleansing: the data need to be cleansed. Because some students enroll in the same
courses in different semesters, these records must be grouped into one record. The grade point
values of re-enrollment courses are recalculated as the average values.
3) Data reconstruction: the values in STUDENTID and COURSEID are reconstructed to be
sequential number ordering from 1 to m and 1 to n, respectively. The value of m is the number of
students and n is the number of courses collected from historical database.
• In case of STUDENTID, the values in this field are selected with DISTINCT option
via SQL. Thus, the duplication values are removed. Then, STUDENTID is changed
to be STUDENTNO, where the values are reassigned with the sequential number.
Page 7 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
• In case of COURSEID, the value in COURSETYPE can be either ‘Compulsory
Course’ or ‘Elective Course’. Next, the data in this field is selected with DISTINCT
option and sorted ascending via SQL. The types of courses are specified in
COURSETYPE.
4) Data transformation: the historical studied records are reconstructed to the matrix
(hereafter referred to as matrix r, which is an � × � matrix). The index of each element in a matrix r
in term of (row, column) is corresponding to STUDENTNO and COURSENO, respectively.
3.2 Data Modeling
Several models can be applied for evaluating an estimated grade of a student s after attended
in courses c. The baseline estimation for an estimated grade of student s after attended in courses c
based on SVD-based technique, hereafter denoted as �̂�,�, is given by �̂�,� = �̅ + � + �� +∑ (��,� × ��,�)���� (1)
where �̅ is an average grade of all elements that have been assigned values in matrix r. �̅ is implemented in both �̅� (an average grade of a student s) and �̅�(an average grade of a course c). �, ��, ��,�, and ��,� denote model parameters, which are collected in � × 1, � × 1, m× �, and � × � matrices; respectively. m, n and k are number of students, number of courses and number of factors
for factoring matrix r. The optimal value of k for CF is 400 (Praserttitipong and Sophatsathit, 2012).
� and �� represent the observed deviations of student s from the average and the bias of observed
deviations of course c from the average, respectively. ��,� and ��,� are the extent of interest of student s on factor k-th and the extent to which the course possesses over factor k-th.
Page 8 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
The values of these parameters are initiated with the small randomized numbers. Then, these
values are recomputed via an iteration learning process with an objective for performing the
minimization of regularized square error between the students’ grades of the training elements and
the predicted values acquired from the baseline estimation equation. This iteration process performs
by utilizing a stochastic gradient descent optimization. The algorithm for SVD-based model learning
process described as following (Paterek, 2007).
o Assign the values to matrices b, d, p, and q with the small randomized numbers
o DO
1) Compute the value of the estimated grades of all studied records that have be
collected in the historical database by applying an Equation (1) as
�̂�,� = �̅ + � + �� +∑ (��,� × ��,�)����
2) Calculate the error of the estimated grade as ��,� = ��,� − �̂�,� 3) Compute the model parameters as
• � = � + �(��,� − � ∙ �) • �� = �� + �(��,� − � ∙ ��) • ��,� = ��,� + �(��,� ∙ ��,� − � ∙ ��,�) • ��,� = ��,� + �(��,� ∙ ��,� − � ∙ ��,�)
4) Calculate the value of a mean absolute error
o LOOP UNTIL meanabsoluteerror ≤ 0.5
The constants � and � are the stochastic gradient descent method constants. The values of � and � are set to 0.005 and 0.02, respectively (Paterek, 2007). This learning process repeats for
Page 9 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
setting the model parameters, which are collected in matrices b, d, p, and q, until the terminal
conditions are reached. The appropriate variable for establishing as the terminal condition is a mean
absolute error (MAE). The process is completed when the MAE is less than or equal to 0.50
(Praserttitipong and Sophatsathit, 2014), in which the process will not lead to overfitting.
The estimated grades evaluated based on classification concept to make their more
informative. The student grades are classified into 3 classes as Good recommended class (the
estimated grade is greater or equal to 3.0), Fair recommended class (the estimated grade is greater
or equal to 2.0), and Bad recommended class (the estimated grade is lower than 2.0).
3.3 Results Evaluation
The conventional measurements for recommendation results quality are studied. The usability
and inadequacy of these measurements for applying with the course recommendation domain are
described.
3.3.1 Estimation Accuracy Evaluation
The quality measurement for estimation results evaluated by the MAE that measures the
difference between the estimation results and the actual grade, which is defined as follows
./0 =∑ ∑ 123,452̂3,4167489:7389
;7×<7 (2)
where �be an � × � matrix collecting the historical grades of students. The variables ��,�and �̂�,�be an actual grade and an estimated grade of student s according to studying course c. Besides,
�= be a number of students that have been graded according to a course c and �= be a number of
courses that have been studied by a user c. Generally, lower MAE value reflects higher accuracy of
grades estimation.
Page 10 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
3.3.2 Conventional Classification Evaluation
The quality measurements for classification results assessed by means of basic information
retrieval metrics, i.e., accuracy, precision, recall, and F1-measure. These metrics are calculated from
the number of items that are either relevant or irrelevant and either contained in the recommendation
set of a user or not. These numbers clearly arranged in a contingency table that is called the
confusion matrix. The general form of confusion matrix for the course recommendation system
presented as Table 1.
The classification results categorized into 2 groups: correctly classified results and incorrectly
classified results. The results in group of correct classification results are named as TB, TF, and TG,
which are defined as the number of correctly classified results labeled as class Bad, class Fair and
class Good, respectively. The others are defined for the results in group of incorrect classification
classes. The evaluation measurements for course classification results described as following
(Cacheda et al., 2011).
1) Accuracy measures the correctness of the classification technique. The overall accuracy
measured as
/>>?�@>A = BCDBEDBFBGHIJIKL<M;NH2JOIH�IPIH;� (3)
2) Precision measures the exactness of the classification technique. The precision for
classification results is defined as
Q��>RSRT� = BGH<M;NH2JO�J22H�ILU�LK��POPHVPIH;�P<IGP��LK��BGH<M;NH2JOPIH;��LK��POPHVP<IGP��LK��
(4)
The low precision score means there is a small number of correct classified items compared with
other incorrect items classified in that class. The perfect precision score must be 1.0.
Page 11 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
3) Recall measures the completeness of the classification technique. A recall for classification
results is defined as
W�>@XX = YZ[\]^_[`abca``[cdefceghhibi[jid[^hi\dZihceghhYZ[\]^_[`abgcd]ge[e[^[\dhceghhibicgdia\`[h]edh (5)
The low recall score points out there is a small number of correct classified items compared with the
total number of items that actually belong to that class. The perfect recall score are also 1.0.
4) F1-measure is the combination of precision and recall. The quality in term of both precision
and recall are combined into a single score, which is calculated as the standard harmonic mean of
precision and recall. F1-measure for classification results is defined as
k� = l9
mno43pq6D 9no4rss
= l×t2H��PJ<×2H�KLLt2H�P�PJ<D2H�KLL (6)
However, some interested points in courses recommendation domain are not be identified by
the values of precision and recall. The inadequacies of precision values are depicted in Figure 2,
besides; the inadequacies of recall values are shown in Figure 3.
• The low precision value for class Bad indicates that there are a small number of correct
classified items in class Bad compared with other incorrect items classified in class Bad,
as shown in Figure 2(a). Because of this error, the students are informed with the
incomplete set of elective courses. This leads to a lost opportunity of enrollment with
some elective courses that probably make that students increase their accumulated grade
point average (GPA). This means there are some extra elective courses are classified in
class Bad, even though, they are fair or good courses. This situation is named as lost
opportunity recommendation case.
• The low precision value for class Good means there are some extra courses are classified
in class Good, even though, their actual classes are class Fair or class Bad, as illustrated
Page 12 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
in Figure 2(b). These classification results are serious cases. Because the students are
recommended with the wrong elective courses, these can cause many other crucial
problems, such as the problem of the students’ retirement causing from their GPA. This
situation is named as critical wrong recommendation case.
• The low precision value for class Fair means there are some extra courses are classified in
class Fair, as shown in Figure 2(c). This can be either a lost opportunity
recommendation case or critical wrong recommendation case.
• The low score is achieved from the recall measurement for class Bad implies that there
are many critical wrong recommendation results appear, as illustrated in Figure 3(a).
• The low recall score for class Good classification indicates that there are a small number
of correct classified items in class Good, as illustrated in Figure 3(b). This implies that
there are many lost opportunity recommendation situations exit.
• The low recall value for class Fair means there are some extra courses are classified in
class Fair, as shown in Figure 3(c). This can be either a lost opportunity
recommendation case or critical wrong recommendation case.
This can be seen that the values of both precision and recall values of different classes can infer
different meaning in course recommender system domain. For example, the values of both FBG and
FBF indicate there are some bad elective courses are classified in class Good and class Fair,
respectively. However, the most seriously situation in this case caused from the misclassified bad
courses into good courses. Thus, the significantly of incorrectly classified results must be defined in
different weights.
It can be concluded that the new measures for the course recommendation system domain are
called for. The confusion matrix for the course recommendation system is proposed as Table 2. The
Page 13 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
group of incorrectly classified results are designated into two subcategories; i.e., Wrong and LostOp,
in order to make the more distinguish for the degree of error values.
1) Lost metric: it uses for evaluating the degree of a lost opportunity recommendation
problem. The elective courses, which are classified in these classes, normally not be recommended
for the students. Thus, the students are informed with the incomplete set of elective courses. This
leads to a lost opportunity of enrollment with some elective courses that probably make that students
increment their accumulated GPA.
The LostOp variable is defined as the number of incorrectly classified results in case of the
misclassified items which classified into the class that lower than the actual abilities of the students.
The numerical values at the end of each category’s name; i.e., 1, 2, and 3, indicate the significance
of opportunity that the students lose. The variables LostOp3, LostOp2, and LostOp1 are sorted
according to the dramatic lost opportunity scores. The measurement for estimating the degree of a
lost opportunity recommendation problem is given as
uTSv = (w9×xJ�Iyt9)D(wz×xJ�Iytz)D(w{×xJ�Iyt{)BGHIJIKL<M;NH2JOIH�IPIH;� (7)
where the variables |�, |l, and |}, are the weighted of their impacts. The variables |�, |l, and |}
are assigned as 1, 2, and 3, respectively. The higher score of a lost opportunity (Lost) indicates the
higher numbers of students are informed with the incomplete set of elective courses.
2) Critical metric: it uses for evaluating the degree of a critical wrong recommendation
problem. These classification results are serious cases. Because the students are recommended with
the wrong elective courses, these may lead to many other crucial problems, such as the problem of
the students retirement causing from their GPA.
The Wrong variable is defined as the number of incorrectly classified results in case of the
misclassified items which classified into the class that higher than the actual abilities of the students.
Page 14 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
The degree of highest critical situation is denoted as variable Wrong3; beside, the less crucial critical
situation is denoted as Wrong1. The measurement for estimating the degree of a critical wrong
recommendation problem is given as
~�RvR>@X = (�9�2J<�9)D(�z�2J<�z)D(�{�2J<�{)BGHIJIKL<M;NH2JOIH�IPIH;� (8)
where the variables ��, �l, variables �} are the weight of their impacts. The variables ��, �l,
variables �} are assigned as 2, 3, and 4, respectively. The values of ��, �l, and �} are more than
the values specified for |�, |l, and |}. The higher score of a critical wrong recommendation
(Critical) indicates a higher number of students are recommended with the wrong elective courses
which are not suitable with their abilities. These can lead to many other crucial problems.
3) Balanced accuracy metric: it uses for evaluating the overall accuracy of courses
recommender system based on the standard harmonic mean between an accuracy and an
ErrorClassified, where ErrorClassified is defined as the balance of the Critical and Lost. Because
the values of an accuracy and an ErrorClassified are not exactly equal to Accuracy=1-
ErrorClassified, these values need to be normalized. The ErrorClassified value evaluated based on
the concept of the standard harmonic mean (Cacheda et al., 2011) as
0��T�~X@SSR�R�� = K�H2K�H(w9,wz,w{)DK�H2K�H(�9,�z,�{)r�onr�o(�9,�z,�{)
�q3� Dr�onr�o(�9,�z,�{)�np�p4rs
= lD}z
�q3�D {�np�p4rs
= �×(xJ�I×�2PIP�KL)(l×�2PIP�KL)D(}×xJ�I) (9)
Thus, the balanced accuracy based on the concept of the standard harmonic mean (Cacheda et
al., 2011) is proposed as
�@X@�>��/>>?�@>A = l9
r44�nr4�D 9(9��rsr64o��nnqn)
= l×���M2K�U×(�5CKLK<�HV�22J2)���M2K�UD(�5CKLK<�HV�22J2) (10)
Page 15 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
The higher value returned from �@X@�>��/>>?�@>A measurement indicates that higher accuracy
achieved with a tolerable error classification in elective courses recommendation results.
4. Experimental Results and Evaluation
The data in this experiment were collected from enrollment records of 438 students who
studied in Department of Computer Science, Faculty of Science, Chiang Mai University during
academic year 2006-2010. The grades which they have been scored in 74 courses collected in a
historical enrollment database, which were 21 compulsory courses and 53 elective courses. This
database collected 11,158 studied records. The studied records which have been enrolled in different
academic years were treated to be equal priority.
These studied records were divided into the training sets and the testing sets via split testing
technique. Since the studied records represented data of two different categories (i.e., compulsory
courses and elective courses), the stratified sampling was performed to select the elements for the
training sets and the testing sets. The records in the testing set were randomly sampled from the
studied records that the students have enrolled in the elective courses; besides, the rest were used as
training set. Thus, the records in the training set were the historical enrollment information of both
compulsory courses and elective courses. The testing elements were randomly selected and others
were training set. According to the small amount of experimental data, the size of each testing set
was approximately 10% of studied records a collected in the historical database for reporting the
highest powerful of each techniques. Five training and testing sets based on split testing technique
were carried out along with the several testing approaches.
4.1 Testing Approaches
The grade estimation of a students received from enrollment in an elective course c (�̂�,�) has
Page 16 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
been evaluated according to several approaches. There were 12 experiments were conducted. The
details of the testing approaches were as following.
1) An average grade of each student: the grade estimation evaluated from an average grade
earned by students, which could be modeled as
�̂�,� = ��� (11)
2) An average grade of each course: the grade estimation evaluated from an average grade
given in course c, which could be specified as
�̂�,� = ��� (12)
3) Combination of (1) & (2) course: the grade estimation calculated from an average grade
earned by students acquired from an Equation (11) and an average grade given in course c
acquired from an Equation (12) , which could be evaluated as
�̂�,� = @���@��(���, ���) (13)
4) User-based CF: This approach applied a user-based CF with Pearson correlation technique
(Cacheda et al., 2011). The detail of this technique could be described as
o First, the similarities between a current student s and other students were calculated. A
similarity value between student s and student t based on Pearson correlation technique
could be evaluated as
SR��,I = ∑ �23,452̅3�(2�,452̅�)6489�∑ �23,452̅3�z6489 �∑ (2�,452̅�)z6489
(14)
o Then, a subset of k-students who were the most similar to students are selected. In this
experiment, k was set to 7. The grade estimation based on user-based CF could be
calculated as
Page 17 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
�̂�,� = �̅� + ∑ (2�,452̅�)��89� (15)
5) Item-based CF: This approach applied an item-based CF with Pearson-correlation technique
(Cacheda et al., 2011). The detail of this technique could be described as
o First, the similarities between a course c and other course s were calculated. A similar
between a course c and a course d based on Pearson correlation technique evaluated as
SR��,V = ∑ �23,452̅4�(23,�52̅�):389�∑ �23,452̅��z:389 �∑ (23,�52̅�)z:389
(16)
o Then, a subset of k-courses which were the most similar to a course c are selected. In this
experiment, k was set to 7. The grade estimation based on item-based CF could be
evaluated as
�̂�,� = ∑ 23,���89� (17)
6) Combination of (4) & (5): the grade estimation calculated from the grade estimation based
on user-based CF technique acquired from an Equation (15) and the grade estimation based
on item-based CF technique acquired from an Equation (17) , which could be evaluated as
�̂�,� = @���@��(�̂�,���T�@�0�?@vRT�15, �̂�,���T�@�0�?@vRT�17) (18)
7) Weighted user-based CF:
This approach also applied a user-based CF with Pearson correlation technique as an
Equation (14). The weights of the similarity values were taken into account. The grade
estimation could be calculated as
�̂�,� = �̅� + ∑ �P;3,�×(2�,452̅�)��89∑ 1�P;3,�1��89
(19)
8) Weighted item-based CF: This approach also applied an item-based CF with Pearson
correlation technique as an Equation (16). The grade estimation could be evaluated as
Page 18 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
�̂�,� = ∑ �P;4,�×23,���89∑ 1�P;4,�1��89
(20)
9) Combination of (7) & (8): the grade estimation could be calculated as an average of the
results from an Equation (19) and an Equation (20).
�̂�,� = @���@����̂�,���T�@�0�?@vRT�18, �̂�,���T�@�0�?@vRT�20� (21)
10) SVD based on student info: the grade estimation calculated from SVD technique based on an
average grade earned by student s, which could be modeled as
�̂�,� = �̅� + � + �� +∑ (��,� × ��,�)���� (22)
11) SVD based on course info: the grade estimation calculated from SVD technique based on an
average grade given in course c, which could be modeled as
�̂�,� = �̅� + � + �� +∑ (��,� × ��,�)���� (23)
12) Combination of (10) & (11): the grade estimation calculated from SVD technique based on
an average grade earned by student s acquired from an Equation (22) and an average grade
given in course c acquired from an Equation (23), which could be evaluated as
�̂�,� = @���@��(�̂�,���T�@�0�?@vRT�22, �̂�,���T�@�0�?@vRT�23) (24)
4.2 Experimental Results and Discussion
4.2.1 Estimation Accuracy
The estimation accuracy evaluated by comparing the numerical estimated grades against the
actual grades via MAE as described in an Equation (1). Figure 4 shows that the estimation accuracy
results achieved from several CF techniques.
As shown in Figure 4, the MAE values returned from the SVD technique based on the
combination of SVD based on student and course information was 0.5543, which was the lowest
MAE value. This inferred that the estimation accuracy of the SVD technique based on student and
Page 19 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
course information was highest.
4.2.2 Classification Evaluation
The estimated grades derived from CF techniques were further assessed based on
classification concept to make their more informative. The student grades of both training and the
testing data are classified into 3 classes as Good, Fair, and Bad recommended class along this
simple rules-based.
/*Classification rules-based for actual grades */
IF rh,c ≥ 3.00 then actual class = Good
ELSEIF rh,c ≥ 2.00 then actual class = Fair
ELSE THEN actual class = Bad
/*Classification algorithm for recommended classes */
IF r£h,c ≥ 3.00 then recommended class = Good
ELSEIF r£h,c ≥ 2.00 then recommended class = Fair
ELSE THEN recommended class = Bad
The evaluation measurements for recommended courses classification results compared with
their actual class were evaluated in this section. The results of classification according to the
conventional measurements are shown in Figure 5. The results in terms of Lost, Critical, and
Balanced accuracy are proposed in Figure 6.
In Figure 5, the classification accuracy from SVD-based techniques returned the high values,
where an average of accuracy values was above 60%. An Accuracy achieved from SVD technique
based on course information approach was the highest scores, which was 0.6314 or approximately
63%. For illustrating an advantage of our proposed SVD-based algorithm, the results were compared
to the results presented in Anuradha and Velmurugan (2015). In that work, several classification
Page 20 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
algorithms based on content-based filtering technique were applied on students’ information. The
contents of the information were about attendance, class test, seminar, lab work and assignment
marks; etc. The overall accuracy result also revealed about 60%, even though; they acquired much
information to perform their work than SVD-based algorithm. Additionally, the degree of Precision
and F1-Measure shown that the SVD technique based on course information approach was highest.
Even though the degree of Recall of user-based CF depicted the highest score, it is a bit higher than
the result returned from the SVD technique based on course information approach.
Moreover, the degree of Lost, Critical, and Balanced accuracy were taken into account for
making the more precise in evaluation the course recommendation classification results. As depicted
in Figure 6, the degree of lost opportunity recommendation returned from the SVD technique based
on course information approach was lowest. The Lost value was 0.1768. This inferred that there
was less lost opportunity recommendation situation returned from this approach compared with
others. The degree of critical wrong recommendation returned from the SVD technique based on
course information approach was also lowest value. The Critical value was 0.2830. This signified
that there was less critical wrong recommendation result appeared from the course recommendation
classification results based on the SVD technique based on course information approach compared
with other approaches. Furthermore, the Balanced accuracy values, which were computed from the
standard harmonic mean between an Accuracy and an ErrorClassified of different approaches of CF
recommender techniques, were observed. The results from Figure 6 illustrated that the SVD
technique based on course information approach was the highest value. This figured out that the
harmony scores over the accuracy score and the inaccuracy score derived from the SVD technique
based on course information approach was the best score.
This can be concluded that the SVD technique based on course information is the most proper
Page 21 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
approach for implementing in the elective courses recommendation system. It reports the acceptable
scores in almost all aspects of consideration. This is because the data matrix, especially in part of
elective course, tends to be sparse in which the SVD technique has higher ability to extract the
hidden information in case of low density of the data matrix than others techniques. The hidden
information are excerpted and the baseline estimation for that data set are modelled with coarsely
representing the overall patterns of data collected in the data matrix.
5. Conclusion
This paper proposed the general process of EDM for elective courses recommendation based
on student grades. These grades are collected in the historical database via routine work.
Furthermore, the new quality measurements for course recommender system are proposed in order
to fulfill an inadequate of the precision and recall. The results from the experiment show that SVD-
based technique based on course information presented the best recommendation results in many
aspects compared with others. However, the SVD technique has the limitation which depends on the
density of the data matrix. The accuracy of this technique will drop with relative to the higher
density of data matrix. Because, the high density data matrix contains variety data patterns, this
causes the difficulty to model the generic baseline estimation model which represents all assortment
in data patterns. Thus, some effort for seeking the solutions to fulfill this issues is called for. Further
investigation on combining some contents about courses and students to this proposed model is a
challenging research endeavor remained to be explored.
Acknowledgement
This research has been supported by Faculty of Science Research Fund, Chiang Mai University.
Page 22 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
References
Anuradha, C. and Velmurugan T. 2015. A comparative analysis on the evaluation of classification
algorithms in the prediction of students performance. Indian Journal Of Science and
Technology. 8(15), 1-12.
Cacheda, F., Carneiro, V., Fernández D., and Formoso, V. 2011. Comparison of collaborative
filtering algorithms: limitations of current techniques and proposals for scalable, high-
performance recommender systems. ACM Transactions on the Web. 5(1), 2:1-33.
Kautkar, R. A. 2014. A comprehensive survey on data mining. International Journal of Research in
Engineering and Technology. 3(8), 185-191.
Lee, Y. and Cho, J. 2011. An intelligent course recommendation system. Smart Computing Review.
1(1), 69-83.
Paterek, A. 2007. Improving regularized singular value decomposition for collaborative filtering.
Proceedings of the KDD Cup Workshop at SIGKDD'07, 13th ACM International Conference
on Knowledge Discovery and Data Mining, 39-42.
Praserttitipong, D. and Sophatsathit, P. 2012. A distributed recommender agent model based on
user’s perspective SVD technique. International Journal of Digital Content Technology and its
Applications. 6(10), 108 -117.
Praserttitipong, D. and Sophatsathit, P. 2014. An agent model for information filtering using
revolutionary RSVD technique. Chiang Mai Journal of Science. 41(5/2), 1429 - 1438.
Ray, S. and Sharma, A. 2011. A collaborative filtering based approach for recommending elective
courses. Proceedings of the 5th International Conference on Information Intelligence, Systems,
Technology and Management, March 2011, 141, 330-339.
Page 23 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
Romeron, C. and Ventura, S. 1992. Educational data mining: a review of the state of the art. IEEE
Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews. 40(6),
601-618.
Page 24 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
Figure file
Figure 1 Elective Courses Recommendation Model.
Figure 2 Impact errors of precision values.
Page 25 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
Figure 3 Impact errors of recall values.
Figure 4 Comparison of MAE values according to different approaches of
CF recommender techniques.
Figure 5 Comparison of classification results from different approaches of
CF recommender techniques based on conventional measurements.
Page 26 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
Figure 6 Comparison of classification results from different approaches of
CF recommender techniques based on proposed measurement.
Page 27 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
For Review Only
Table file
Recommended Classes
Bad Fair Good Recall
Actual
Classes
Bad TB FBF FBG
Fair FFB TF FFG
Good FGB FGF TG
Precision
Table 1 A general form confusion matrix for the recommendation system.
Recommended Classes
Bad Fair Good
Actual Bad TB Wrong2 Wrong3
Classes Fair LostOp2 TF Wrong1
Good LostOp3 LostOp1 TG
Table 2 A confusion matrix for the course recommendation system.
Page 28 of 28
For Proof Read only
Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960