10 intelligent tutoring-spc
DESCRIPTION
TRANSCRIPT
![Page 1: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/1.jpg)
Intelligent Tutoring(Student-Problem Table)
Timothy K. Shih
![Page 2: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/2.jpg)
Outline
• Purpose of ITS (Intelligent Tutoring System) and ATS (Adaptive Testing System)
• Evaluation Methods• Student-Problem Chart• Item Respond Theory
![Page 3: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/3.jpg)
Purpose• ITS (Intelligent Tutoring System)
– Students learns in different pace– Can not provide individualized lecture– Intelligent Tutoring: students learn in different
topology– Tutorial Generation: remedial lectures after tests
• ATS (Adaptive Testing System)– Traditional test methods are not flexible– Item Response Theory: a relatively modern test
method– Different students use different number of questions
and different questions– Instance hints to students during test
![Page 4: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/4.jpg)
Evaluation Methods
• Quality of Distance Education (timely course contents, presentation quality, etc.)
• Type of Evaluations– Student to instructor: teaching evaluation– Student/instructor to content: content evaluation– Instructor to student: performance evaluation– Administrator to program: DL program evaluation
![Page 5: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/5.jpg)
InstructionObjective
ReadinessEvaluation
InstructionActivity
InstructionEvaluation
Feedback
• Is the objective satisfied?• Considering the background of students, is the
course material too difficult?• Which part of instruction was not explained clearly
by the instructor?• Which student is not working hard?
The General Model of Instruction
![Page 6: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/6.jpg)
• Placement evaluation: The evaluation assesses how a new student or a new instructor is adapted to the new environment (to test both prerequisite skill and course objective).
• Formative evaluation: During the process of instruction, informal quizzes are given to students. The outcome of evaluation are feedback to both the instructor and the students.
• Diagnostic evaluation: The identification of persistent or recurring learning difficulties that are left unresolved by the standard correction perspective of formative evaluation.
• Summative evaluation: Grades are given to the students to certify the students’ global level of knowledge on the topic taught.
• Self-evaluation: A mix of formative and diagnostic evaluations, which is taken by the students to evaluate whether to face summative evaluation.
• Self-regulation
Evaluation Methods
![Page 7: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/7.jpg)
• Norm-referenced evaluation: The credit of an individual is compared to the average credit of a group to justify the performance of the individual.
• Criterion-referenced evaluation: A standard terminal behavior is established. The credit of an individual is compared to the standard.
Evaluation Strategies
![Page 8: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/8.jpg)
Mastery Learning
Formative Assessment
Repeated Training
Course Unit A
FormativeAssessment A
FormativeAssessment B
Course Unit B
Pass ?
Pass ?
Training Preparation
Yes
Yes
No
No
![Page 9: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/9.jpg)
Student-Problem Chart
• The Student-Problem Chart Analysis Theory (S-P Table) [Sato 1975]
• Caution Indices for Students and Items [Sato 1975]
• The Student-Navigation Chart (S-N Table)
• Caution Indices for Students and Course Units
• Cross References of S-P Table and S-N Table
![Page 10: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/10.jpg)
An Example of Student-Problem Chart
S
P
2 3 7 4 9 1 6 5 10 8 grades percentage Student
caution index
Determine
classification
7 1 1 1 1 1 1 1 1 1 1 10 100% 0.00 A
5 1 1 1 1 1 1 1 1 1 0 9 90% 0.00 A
9 1 1 1 1 1 0 1 1 0 1 8 80% 0.57 A’
4 1 1 1 1 1 1 1 0 0 0 7 70% 0.00 B
10 1 1 1 0 1 0 1 0 1 0 6 60% 0.50 B’
2 1 1 0 1 1 0 1 0 0 0 6 60% 0.50 B’
14 1 1 1 0 1 0 1 0 0 0 5 50% 0.00 B
1 1 1 1 0 1 0 0 0 1 0 5 50% 0.40 B
13 1 1 0 1 0 0 1 0 0 1 5 50% 0.70 B’
6 1 0 0 1 0 1 0 1 1 0 5 50% 1.00 B’
15 1 1 1 1 0 1 0 0 0 0 4 40% 0.40 C
11 1 0 0 0 0 1 0 1 0 1 4 40% 1.20 C’
3 1 0 1 0 1 0 0 0 0 0 3 30% 0.33 C
8 0 0 1 0 0 1 0 0 0 0 2 20% 0.71 C’
12 0 1 0 0 0 0 0 0 0 0 1 10% 0.25 C
numbers 12 11 10 9 8 8 7 6 5 4 80
Correct
percentage
80 73 67 60 53 53 47 40 33 27
Problem
Caution index
0.10 0.35 0.55 0.15 0.15 0.75 0.00 0.29 0.38 0.55
Determine
classification
A A A’ A A A’ B B B B’
![Page 11: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/11.jpg)
Problem Caution Index Diagnosis
Problem Caution Index
Percentage of students who answer questions correctly
A A’
B B’
0.50 1.000
50%
100%
![Page 12: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/12.jpg)
Student Caution Index Diagnosis
Student Caution Index
Percentage of student score
A A’
B B’
0.50 1.000
50%
100%
C C’
75%
![Page 13: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/13.jpg)
Student-Navigation: The Original Table
網頁課程
學習者 1 2 3 4 5 6 7 8 9 10
精熟
學習分數 學習
總分
1 1 2 1 2 2 2 2 2 2 3 8 19
2 2 2 3 3 2 3 3 3 3 3 10 27
3 2 1 2 2 1 1 3 1 1 3 5 17
4 1 1 3 1 1 1 1 3 3 2 4 17
5 1 3 1 3 1 1 1 2 2 3 5 18
6 1 2 1 2 1 2 1 2 1 3 5 16
7 1 1 2 2 2 2 1 2 3 2 7 18
8 2 2 2 3 2 3 1 2 2 2 9 21
9 2 2 3 3 3 3 3 2 3 2 10 26
10 1 1 3 3 2 1 3 2 3 2 7 21
11 1 2 3 3 2 3 2 2 2 3 9 23
12 1 3 1 2 1 2 2 2 1 3 6 18
13 1 1 1 1 2 2 2 2 2 2 6 16
14 1 1 1 1 1 1 1 1 3 2 2 13
15 1 1 1 1 1 1 1 1 1 1 0 10
16 1 1 2 1 2 2 1 1 3 1 4 15
17 1 3 2 2 2 1 1 2 3 2 7 19
18 1 1 2 3 2 2 3 2 2 3 8 21
19 1 1 1 1 1 1 1 3 1 1 1 12
20 1 1 3 1 1 1 3 1 1 2 3 15
精熟
瀏覽分數 4 9 12 13 11 11 10 15 14 17 18.1
瀏覽總分 24 32 38 40 32 35 36 38 42 45 36.2
![Page 14: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/14.jpg)
Sorted by Mastery Learning Score網頁課程
學習者 1 2 3 4 5 6 7 8 9 10
精熟
學習分數 學習
總分
2 2 2 3 3 2 3 3 3 3 3 10 27
9 2 2 3 3 3 3 3 2 3 2 10 26
11 1 2 3 3 2 3 2 2 2 3 9 23
8 2 2 2 3 2 3 1 2 2 2 9 21
18 1 1 2 3 2 2 3 2 2 3 8 21
1 1 2 1 2 2 2 2 2 2 3 8 19
10 1 1 3 3 2 1 3 2 3 2 7 21
17 1 3 2 2 2 1 1 2 3 2 7 19
7 1 1 2 2 2 2 1 2 3 2 7 18
12 1 3 1 2 1 2 2 2 1 3 6 18
13 1 1 1 1 2 2 2 2 2 2 6 16
5 1 3 1 3 1 1 1 2 2 3 5 18
3 2 1 2 2 1 1 3 1 1 3 5 17
6 1 2 1 2 1 2 1 2 1 3 5 16
4 1 1 3 1 1 1 1 3 3 2 4 17
16 1 1 2 1 2 2 1 1 3 1 4 15
20 1 1 3 1 1 1 3 1 1 2 3 15
14 1 1 1 1 1 1 1 1 3 2 2 13
19 1 1 1 1 1 1 1 3 1 1 1 12
15 1 1 1 1 1 1 1 1 1 1 0 10
精熟
瀏覽分數 4 9 12 13 11 1 10 15 14 17 18.1
瀏覽總分 24 32 38 40 32 35 36 38 41 45 36.2
![Page 15: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/15.jpg)
Sorted by Mastery Navigation Score網頁課程
學習者 10 8 9 4 3 6 5 7 2 1
精熟
學習分數 學習
總分
2 3 3 3 3 3 3 2 3 2 2 10 27
9 2 2 3 3 3 3 3 3 2 2 10 26
11 3 2 2 3 3 3 2 2 2 1 9 23
8 2 2 2 3 2 3 2 1 2 2 9 21
18 3 2 2 3 2 2 2 3 1 1 8 21
1 3 2 2 2 1 2 2 2 2 1 8 19
10 2 2 3 3 3 1 2 3 1 1 7 21
17 2 2 3 2 2 1 2 1 3 1 7 19
7 2 2 3 2 2 2 2 1 1 1 7 18
12 3 2 1 2 1 2 1 2 3 1 6 18
13 2 2 2 1 1 2 2 2 1 1 6 16
5 3 2 2 3 1 1 1 1 3 1 5 18
3 3 1 1 2 2 1 1 3 1 2 5 17
6 3 2 1 2 1 2 1 1 2 1 5 16
4 2 3 3 1 3 1 1 1 1 1 4 17
16 1 1 3 1 2 2 2 1 1 1 4 15
20 2 1 1 1 3 1 1 3 1 1 3 15
14 2 1 3 1 1 1 1 1 1 1 2 13
19 1 3 1 1 1 1 1 1 1 1 1 12
15 1 1 1 1 1 1 1 1 1 1 0 10
精熟
瀏覽分數 17 15 14 13 12 11 11 10 9 4 18.1
瀏覽總分 44 37 40 39 37 34 31 35 31 23 36.2
![Page 16: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/16.jpg)
Student-Navigation Table網頁課程
學習者 10 8 9 4 3 6 5 7 2 1
精熟
學習分數 學習
總分
2 3 3 3 3 3 3 2 3 2 2 10 27
9 2 2 3 3 3 3 3 3 2 2 10 26
11 3 2 2 3 3 3 2 2 2 1 9 23
8 2 2 2 3 2 3 2 1 2 2 9 21
18 3 2 2 3 2 2 2 3 1 1 8 21
1 3 2 2 2 1 2 2 2 2 1 8 19
10 2 2 3 3 3 1 2 3 1 1 7 21
17 2 2 3 2 2 1 2 1 3 1 7 19
7 2 2 3 2 2 2 2 1 1 1 7 18
12 3 2 1 2 1 2 1 2 3 1 6 18
13 2 2 2 1 1 2 2 2 1 1 6 16
5 3 2 2 3 1 1 1 1 3 1 5 18
3 3 1 1 2 2 1 1 3 1 2 5 17
6 3 2 1 2 1 2 1 1 2 1 5 16
4 2 3 3 1 3 1 1 1 1 1 4 17
16 1 1 2 1 2 2 2 1 1 1 4 15
20 2 1 1 1 3 1 1 3 1 1 3 15
14 2 1 3 1 1 1 1 1 1 1 2 13
19 1 3 1 1 1 1 1 1 1 1 1 12
15 1 1 1 1 1 1 1 1 1 1 0 10
精熟
瀏覽分數 17 15 14 13 12 11 11 10 9 4 18.1
瀏覽總分 45 38 42 40 38 35 32 36 32 24 36.1
S Curve
N Curve
![Page 17: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/17.jpg)
Student-Navigation Table with Indices網頁課程
學習者 10 8 9 4 3 6 5 7 2 1
精熟
學習分數
學習
總分
平均
學習次數
學習者
注意係數
判定
類別
2 3 3 3 3 3 3 2 3 2 2 10 27 2.7 0 A
9 2 2 3 3 3 3 3 3 2 2 10 26 2.6 0.73 A’
11 3 2 2 3 3 3 2 2 2 1 9 23 2.3 0.22 B
8 2 2 2 3 2 3 2 1 2 2 9 21 2.1 0.88 B’
18 3 2 2 3 2 2 2 3 1 1 8 21 2.1 0.22 B
1 3 2 2 2 1 2 2 2 2 1 8 19 1.9 0.24 C
10 2 2 3 3 3 1 2 3 1 1 7 21 2.1 0.22 B
17 2 2 3 2 2 1 2 1 3 1 7 19 1.9 0.56 C’
7 2 2 3 2 2 2 2 1 1 1 7 18 1.8 0.03 C
12 3 2 1 2 1 2 1 2 3 1 6 18 1.8 0.64 C’
13 2 2 2 1 1 2 2 2 1 1 6 16 1.6 0.20 C
5 3 2 2 3 1 1 1 1 3 1 5 18 1.8 0.32 C
3 3 1 1 2 2 1 1 3 1 2 5 17 1.7 0.56 C’
6 3 2 1 2 1 2 1 1 2 1 5 16 1.6 0.25 C
4 2 3 3 1 3 1 1 1 1 1 4 17 1.7 0.22 C
16 1 1 2 1 2 2 2 1 1 1 4 15 1.5 0.58 C’
20 2 1 1 1 3 1 1 3 1 1 3 15 1.5 0.45 C
14 2 1 3 1 1 1 1 1 1 1 2 13 1.3 0.36 C
19 1 3 1 1 1 1 1 1 1 1 1 12 1.2 0.46 C
15 1 1 1 1 1 1 1 1 1 1 0 10 1 0 C
精熟瀏覽分數 17 15 14 13 12 11 11 10 9 4 18.1
瀏覽總分 45 38 42 40 38 35 32 36 32 24 36.1
平均瀏覽次數 2.25 1.9 2.1 2 1.9 1.75 1.6 1.8 1.6 1.2
網頁注意係數 0.39 0.53 0.52 0.02 0.26 0.11 0.34 0.17 0.54 0.25
判定類別 A B’ A’ A B B B B B’ B
![Page 18: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/18.jpg)
Course Node Caution Index Diagnosis
B
0
A A’
B’
Course Node Caution Index
0.50 1.00
2
3Average number of navigation to course nodes
1
![Page 19: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/19.jpg)
Student Navigation Caution Index Diagnosis
0
A A’
B B’
Student Navigation Caution Index
0.50 1.00
2
2.5Average of student navigation
1
C C’
![Page 20: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/20.jpg)
Cross Reference of S-P and S-N Tables
S-N S-P Observation
A A Excellent students, with stable learning progress
A A’ Good students, but careless from time to time
A B Good students, but do not work hard enough
A B’ Good students, but careless and do not work hard enough
A C Good students, but due to some reasons, do not perform well. Need special attention
A C’
![Page 21: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/21.jpg)
• How do we develop Web courses with an easy to access style? – Course Patterns
• How do we evaluate student learning performance? – Navigation, Tests
• How can we help them from the evaluation? – SPC Table
• How can computer help them? – Tutorials
Timothy K. Shih, Nigel Lin, Jianhua Ma, and Runhe Huang (U. of Aizu, Japan)
A Course Development and Student Assessment System
![Page 22: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/22.jpg)
• Objectives – Mastery Learning– Formative Evaluation– Norm-referenced Evaluation
• Methods and Systems– Course Development with Patterns– Instruction and Assessment Tools– Individualized Course Content Generation
Research Motivation
![Page 23: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/23.jpg)
• Students lost in a complicated navigation graph
• A text book has its fixed structure• Course Pattern = Structure Pattern +
Content Pattern• Only one or two course patterns are used in
a virtual university
Course Development with Patterns
![Page 24: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/24.jpg)
• Pattern definition language• Limited number of sections in a course unit• Small section size• Fixed basic structure for Structure Pattern
(i.e., tree, linked list, set)• Fixed basic structure for Content Pattern
(i.e., text, picture, Pop-up Quiz, Question)
Course Pattern Construction
![Page 25: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/25.jpg)
Reusable Course Unit
Reusable Session
Course Unit
Session
A Graphical User Interface is used to construct structure patterns.
Example: Tree and Linked Lists
Web Course Structure Pattern
![Page 26: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/26.jpg)
Example: Tree, Linked Lists, and Sets
Web Course Structure Pattern (Cont.)
![Page 27: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/27.jpg)
Formal Definition of Course PatternsStructurePattern := RCU StructurePRCU := Null | CourseUnitName @CourseUnitName := ASCIIStructureP := Session | TreePattern | ListPattern | SetPattern TreePattern := Session #< TreeP >TreeP := Null | StructurePattern TreePListPattern := #[ ListP ]ListP := Null | StructurePattern ListPSetPattern := #{ SetP }SetP := Null | StructurePattern SetP
Session := SessionName SessionContent Pop-upQuizzes SessionQuestionsSessionName := ASCIISessionContent := WebDocumentPop-upQuizzes := Null | Pop-upQuiz Pop-upQuizzesPop-upQuiz := YesNoQuestion Constraints | MultipleChoiceQuestion Constraints | FillInBlankQuestion ConstraintsSessionQuestions := Null | SessionQuestion SessionQuestionsSessionQuestion := YesNoQuestion | MultipleChoiceQuestion | FillInBlankQuestion
Constraints := Trigger | not Constraints | Constraints and Constraints | Constraints or Constraints | Constraints xor ConstraintsTrigger := TimeTrigger | ObjectNumberMetTriger | SpecificObjectMetTrigerTimeTrigger := TimeVisited >= IntegerObjectNumberMetTriger := NoOfObjectsVisited >= IntegerSpecificObjectMetTriger := CurrentMObject = MObjectYesNoQuestion := WebDocumentMultipleChoiceQuestion := WebDocumentFillInBlankQuestion := WebDocument
Note: Null is a null object @ represents the structure pattern is a reusable course unit # is an integer which represents the number of internal objectsSpecial Variables: TimeVisited CurrentMObject NoOfObjectsVisitedDefinitions omitted are: Integer is an integer ASCII is an ASCII string MObject is a multimedia object WebDocument is an HTML document
![Page 28: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/28.jpg)
Database Schema Generation
• The Structure Pattern Table: representation of tree, list and set
• The Content Pattern Table: representation of object types and object coordinates
• The Quiz Table: representation of test content and constraints
• The Question Table: representation of assignment content
![Page 29: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/29.jpg)
• Purpose: Easy to use and trace objects• Components
– Fixed Patterns– Text, Picture, Audio, and Video for Course
Materials– Pop-Up Quizzes (real-time interaction)– Assignments (as homework and references of
exams)• A Graphical User Interface is used to
construct content patterns
Web Course Content Pattern
![Page 30: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/30.jpg)
• The Web Course Pattern Tools: Pattern tools are used to help the construction of Web course patterns.
• The Web Navigation Patrol: The patrol is a mobile agent program which run on client sites. The agent collects navigation messages of Web interaction, which are used in the analysis model of student behavior.
• The Performance Analysis Model: This analysis model is based on some instruction theory from the educational literature, such as the SP-table mechanism with the computation of student caution indices.
• The Progressive Learning State Machine Constructor: The constructor generates intelligent tutorials from the course material provided by the instructor.
Instruction and Assessment Tools
![Page 31: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/31.jpg)
Pattern DDL Program
Pattern GUI
Pattern Specification
Web Course Design Editor
Web CoursePattern
Database
Curriculum Development Committee
Web CourseContentDatabase
Web Browser with Web Navigation Patrol
WebNavigationDatabase
Progressive LearningState Machine
PerformanceAnalysis Model
Pattern Formal Syntax
Students Instructors
Pattern Design
MessageSpecification File
DDL Statements
Course Pattern
Course Content
Course Content
Course Pattern
Web Navigation Records
PerformanceRecords
NavigationRecords
BNF Grammar
IntelligentTutorial
InteractionCourse Pattern
Course Design
Analysis
An Instruction Design & Delivery System
![Page 32: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/32.jpg)
• The Student-Problem Chart Analysis Theory (S-P Table) [Sato]
• Caution Indices for Students and Items [Sato]• The Student-Navigation Chart (S-N Table) [Shih]• Caution Indices for Students and Course Units
[Shih]• Cross References of S-P Table and S-N Table
[Shih]• The Student-Problem-Course Table (SPC Table)
[Shih]
Models of Learning Assessment
![Page 33: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/33.jpg)
S
P
2 3 7 4 9 1 6 5 10 8 grades percentage Student
caution index
Determine
classification
7 1 1 1 1 1 1 1 1 1 1 10 100% 0.00 A
5 1 1 1 1 1 1 1 1 1 0 9 90% 0.00 A
9 1 1 1 1 1 0 1 1 0 1 8 80% 0.57 A’
4 1 1 1 1 1 1 1 0 0 0 7 70% 0.00 B
10 1 1 1 0 1 0 1 0 1 0 6 60% 0.50 B’
2 1 1 0 1 1 0 1 0 0 0 6 60% 0.50 B’
14 1 1 1 0 1 0 1 0 0 0 5 50% 0.00 B
1 1 1 1 0 1 0 0 0 1 0 5 50% 0.40 B
13 1 1 0 1 0 0 1 0 0 1 5 50% 0.70 B’
6 1 0 0 1 0 1 0 1 1 0 5 50% 1.00 B’
15 1 1 1 1 0 1 0 0 0 0 4 40% 0.40 C
11 1 0 0 0 0 1 0 1 0 1 4 40% 1.20 C’
3 1 0 1 0 1 0 0 0 0 0 3 30% 0.33 C
8 0 0 1 0 0 1 0 0 0 0 2 20% 0.71 C’
12 0 1 0 0 0 0 0 0 0 0 1 10% 0.25 C
numbers 12 11 10 9 8 8 7 6 5 4 80
Correct
percentage
80 73 67 60 53 53 47 40 33 27
Problem
Caution index
0.10 0.35 0.55 0.15 0.15 0.75 0.00 0.29 0.38 0.55
Determine
classification
A A A’ A A A’ B B B B’
The Student-Problem Chart
![Page 34: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/34.jpg)
The Student-Problem-Course (SPC) Table
View A: Score View B: Navigation View C: Test
Each view has two caution indices
The Revised Model – the SPC Table
![Page 35: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/35.jpg)
The CSa index (caution index of student w.r.t. problem): indicates the outcome of exam. Students of a high index need special attention. On-line tutorial can be generated for these students.
The CPa index (caution index of problem w.r.t. student): indicates the quality of problems. Problems of a high index value can be re-designed.
The CSb index (caution index of student w.r.t. course unit navigation): indicates the degree of student navigation. High value means the student either works too hard (too much navigation), or too lazy (low navigation). On-line tests can be generated for those lazy students.
The CCb index (caution index of course unit w.r.t. student navigation): indicates which course unit is less visited and less effective. Course content can be revised or access paths can be re-constructed.
The CPc index (caution index of problem w.r.t. course unit): each problem is selected for an exam from a unique course unit.
The CCc index (caution index of course unit w.r.t. problem): indicate which course unit does not have a problem chosen in an exam. Problems of this course unit can be selected in the next exam of a higher priority.
The Six Caution Indices
![Page 36: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/36.jpg)
The Student-Course Table
Course Unit Number
Stu
den
t Nu
mb
er
1 2 3 4 5 6 7 8 910 11 12 13 14 TCN CSb
1 NF1,1 NF1,2 NF1,j NF1,n TCN1 CSb1
2 NF2,1 NF2,2 NF2,n TCN2 CSb2
3 NF3,1 NF3,n TCN3 CSb3
4
5
6 NFi,j NFi,n TCNi CSbi
7
8
9
10 NFm,1 NFm,n TCNm CSbm
TSN TSN1 TSN2 TSNj TSNn
CCb CCb1 CCb2 CCbj CCbn
Caution Indices Computation
![Page 37: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/37.jpg)
Average of Course Navigation = (Σi = 1 m TCN i ) / m =
Average of Student Navigation = (Σj = 1 n TSN j ) / n =
where TCN is the Total Course Navigation
and TSN is the Total Student Navigation
TCN i = Σj = 1 n NF i,j
TSN j = Σi = 1 m NF i,j
CSb i = 1 – (Σj = 1 n (NF i, j) (TSN j) )– (TCN i) / (Σj = 1
TCN i TSN j - (TCN i) )
CCb j = 1 – (Σi = 1 m (NF i, j) (TCN i) ) – (TSN j) / (Σi = 1
TSN j TCN i - (TSN j) )
Caution Indices Computation
![Page 38: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/38.jpg)
• The Assessment Criteria of Mastery Learning Navigation– The frequency of the mouse movement on
course objects– The duration of course navigation– The activation on op-up quizzes
• Navigation Factor: to evaluate how hard a student is working on the Web course material
Learning Effect Function
![Page 39: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/39.jpg)
If the degree of frequency or duration is less than the minimal threshold, the elementary message does not count.
If the degree of frequency or duration is greater the maximal threshold, it is counted as the maximal value.
Effective navigation frequency =
Σ Web object Effective Frequency / Σ Web object Actual Frequency
Effective navigation duration =
Effective Duration / Actual Duration
Navigation Frequency and Duration
![Page 40: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/40.jpg)
• Fixed Time Activation (assigned by the instructor to enforce the student)
• Criterion-based Activation (no time limitation, activated on buttons to decide pass/no pass, Mastery Learning)
• Popup Quizzes are brought up by the system at the navigation
Effectiveness of quiz interaction =
Number of quizzes answered correctly / Total number of quizzes
Activation on Pop-up Quizzes
![Page 41: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/41.jpg)
Navigation Factor = Effective navigation frequency * W1+
Effective navigation duration * W2 +
Effectiveness of quiz interaction * W3
where W1 + W2 + W3 = 1.0, and W1 > 0.0, W2 > 0.0, W3 > 0.0
Thus, 0.0 <= Navigation Factor <= 1.0
Mastery Learning Navigation Factor
![Page 42: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/42.jpg)
• Web document structure (physical links)
• Web knowledge structure (logical links in a traversal)
• Knowledge Structure can represent the dependency relations among course units, from the perspective of an instructor
Web Document and Knowledge Structures
![Page 43: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/43.jpg)
• Based on Test Results• Based on Student Navigation Factors• Based on Test Problem Used• Based on Web Knowledge Structure, or
Dependency Relations among course units
• The generated lectures are state machines which include new physical links
• Automatic lecture generation can be proceeded dynamically
Automatic Lecture Generation
![Page 44: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/44.jpg)
• Tutorial on Missing Course Content
• Based on Test Score• Based on Navigation Factor
Let alpha_1 be the max threshold of caution index CSa
Let beta_1 be the min threshold of navigation factor Fb
TutorialSetInitiation
Input: SPC table
Output: TutorialInitSet for each student
Algorithm:
For each student S with CSa > alpha_1
For each factor Fa in View A of S, where Fa = 0
Find the factor Fc in View C w.r.t. Fa, where Fc = 'Y'
Find the Factor Fb in View B w.r.t. Fc
If Fb < beta_1 then
Find course C w.r.t. Fb
Assert course C into the TutorialInitSet for student S
Automatic Tutorial Generation
![Page 45: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/45.jpg)
• On-Line Test for Lazy Student• Based on Student Caution Index• Based on Navigation Factor
Let alpha_2 be the max threshold of caution index CSb
TestSetInitiation
Input: SPC Table
Output: TestInitSet for each student
Algorithm:
For each student S with CSb > alpha_2
For each factor Fb in View B, where Fb < beta_1
Find course C w.r.t. Fb
Select problems of course C
Assert problems to the TestInitSet for student S
Automatic Test Generation
![Page 46: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/46.jpg)
• Find Inappropriate Test Problem• Based on Test Problem
Let alpha_3 be the max threshold of caution index CPa
ProblemRefinement
Input: SPC Table
Output: problem update
Algorithm:
For each problem P with CPa > alpha_3
Find the factor Fc in View C w.r.t. problem P, Fc = 'Y'
Find course C w.r.t. Fc
Refine problem P in course C
Automatic Problem Refinement Indication
![Page 47: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/47.jpg)
• Find Misleading Course Content
• Based on Course Content
Let alpha_4 be the max threshold of caution index CCb
Let alpha_5 be the min threshold of caution index CCc
CourseRefinement
Input: SPC Table
Output: course update
Algorithm:
For each course C with CCb > alpha_4
Refine content of course C and its associated Web structure
Find the corresponding caution index CCc of course C
If CCc < alpha_5
suggest to use problems in course C for the test
Automatic Course Refinement Indication
![Page 48: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/48.jpg)
Implementation Technology
• Windows NT Server, IIS (Internet Information Server)
• ASP (DHTML, VB Script, ActiveX, CSS Component)
• MS SQL Server
![Page 49: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/49.jpg)
• Student Management Tool– Manages
General Information of Students
– Shows Study Records of Students
System Implementation
![Page 50: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/50.jpg)
• Course Structure and Tutorial Management Tool– Show Course
Structure– Manage Tutorial
Designed by Instructors
System Implementation
![Page 51: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/51.jpg)
• Course Unit Design Tool– Manage General
Information of Course Units
– Allow Teacher to Design Course Unit with 5 Types of Course Unit Objects
• Text• Picture• Button• Sound• Video
System Implementation
![Page 52: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/52.jpg)
• Problem Management Tool– Manage Problem
Database– 2 Types of Problems
• Assignments• Pop-up Quizzes
– Allow Teachers to Design 3 Types of Problems
• Multiple Choice• True or False• Fill-in-Blank
System Implementation
![Page 53: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/53.jpg)
• Exam Tool– Allow Teacher to
Hold On-Line Exam in 3 Steps• Set up Exam
Time• Select Course
Units as an Exam Scope
• Select Problems in the Exam Scope
System Implementation
![Page 54: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/54.jpg)
• Assessment Tool– Create SPC Table
Based on Exam– 3 Views of SPC
Table• View A: Score• View B: Navigation• View C: Exam
System Implementation
![Page 55: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/55.jpg)
• On-Line Learning Tool– Allow Students to
Study Course Unit via Web Browser
– 3 Agents Running under the System• Course Unit
Object Navigation Tracking Agent
• Course Unit Study Record Agent
• Pop-up Quiz Agent
System Implementation
![Page 56: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/56.jpg)
• On-Line Exam Tool– Allow
Student to Attend On-Line Exam
– Also Works on PDA
System Implementation
![Page 57: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/57.jpg)
• On-Line Tutorial Tool– Instructor’s Tutorial
• Allow student to view pre-designed tutorials
– Student’s Tutorial• Each student has
his own tutorial generated according to SPC table
System Implementation
![Page 58: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/58.jpg)
DemoSystem Demo
![Page 59: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/59.jpg)
Summary of ITS
• SP Table was designed before Web technology
• Navigation factors can be tracked• Pop-up Quizzes can be used• Learning Topology can be pre-defined
• Automatically generate individualized tutorial (and Web site)
![Page 60: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/60.jpg)
Other Intelligent Technologies
• Automatic FAQ reply• Automatic grading of chat room
participation• Analysis of collaborative learning
behavior• Semantic Web for content searching
![Page 61: 10 intelligent tutoring-spc](https://reader033.vdocuments.site/reader033/viewer/2022061112/54570d50af7959b6358b7aa4/html5/thumbnails/61.jpg)
Summary
• Automatic tutorial generation• Toward automatic test generation
• Semantics• Educational Professionals