Basic Concepts of ITS

Cs5034

Material preparado por: Dr. Jorge Adolfo Ramírez Uresti

Basic Issues

2

Knowledge communication = instructional interaction

ITS/teacher and student

Object of communication = knowledge or expertise in a

domain

Revisión 200811

Traditional ITS Architecture

3

Expert module

Student model

Pedagogicalmodule

User interface

Student

Revisión 200811

Basic issues ...

4

Domain knowledge

Object of communication

First aspect that have been represented in systems

CAI

Prestored in presentation blocks (frames)

Designed by expert teacher

Displayed to the student under fixed conditions

Revisión 200811

Basic issues ...

5

Domain knowledge ...

ITS

Contained in the Expert Module

Representation of knowledge

Description of concepts and skills

Model of the domain -> dynamic expertise

Specific view of domain (from expert)

Revisión 200811

Basic issues ...

6

Domain knowledge ...

Functions of expert module

1. Source for the knowledge

Generation of explanations and responses

Generation of tasks and questions

Revisión 200811

Basic issues ...

7

Domain Knowledge ... Functions of expert module ...

2. Standard for evaluating student’s performance

Solution to problems

Generate solutions to problems (same context as student)

Generation of sensible solution paths (intermediate steps)

Generation of multiple solution paths

Comparison of respective answers

Assess student’s overall progress

Establish a measure to compare knowledge

Revisión 200811

Basic Issues ...

8

Domain Knowledge ... Aspects of communicability

Pedagogical decisions require domain related knowledge not always to be taught

Prerequisite relations, measures of difficulty

Rational for explanations

Internal structure

Open to inspection

Able to support explanations of its actions and conclusions

Transparency = “Black-box” to “glass-box”

Psychological plausibility = similarity to human experts.

Ability to modify representation to match student’s view

Revisión 200811

Basic Issues ...

9

Student model Understanding of the recipient of communication

(representation of)

Include: Aspects of student’s behaviour

Acquired knowledge

Difficult for humans -> very difficult for computers Knowing what student thinks

Communication channel is very restricted

Revisión 200811

Basic Issues ...

10

Student model: information

Infer unobservable aspects of student’s behaviour

Produce an interpretation of his actions

Reconstruct knowledge for these actions

Used for adaptability -> pedagogical decisions

Guide student’s problem solving

Organize learning experiences

Revisión 200811

Basic Issues ...

11

Student model: information ...

Formed out of representations of target expertise

Evaluation of mastery for each element of knowledge

State evaluated against the expert module

Able to provide explanatory information of student’s

suboptimal behaviour

Incorrect knowledge must be represented

Errors should be traced back for remediation

Revisión 200811

Basic Issues ...

12

Student model: representation

Use primitives of a language for the domain

Fine granularity

Spans correct and incorrect knowledge

Misconceptions and correct version can be modelled

Does not require all errors to be anticipated

Problem: search may not be tractable

Revisión 200811

Basic Issues ...

13

Student model: representation ...

Errors and misconceptions

Gather a great deal of information of users working in a domain (catalogs)

Searches for previously observed behaviour patterns

Problem: needs many subjects or models (based on users)

Advantage: knowledge can be derived empirically or from expert teachers

Revisión 200811

Basic Issues ...

14

Student model: representation …

Need of a special language for non-domain related

knowledge

“student behaves as a 4-year”

SM should be executable or runnable

Allows to run simulations

of a given student

In a particular context

Allow to check hypotheses

Revisión 200811

Basic Issues ...

15

Student model: diagnostic process

Process of forming and updating SM by analysing data made available to the system

Mostly numerical values -> statistical calculations

Detailed diagnosis require search

Search for student’s goal

Model of his knowledge (automated theory formation)

Complex internal processes

Multiple misconceptions can lead to correct behaviour

Diagnosis must assign credit (correct knowledge) and blame (incorrect knowledge)

Revisión 200811

Basic Issues ...

16

Student model: diagnostic process ...

Should deal with noise

Modelling language (simplification of human reasoning and decision making)

Students are never perfectly consistent

Learning is noisy (changes with time)

Information submitted varies:

General student’s background

Inferential = system guesses

Interactive = student tries to explain

Must be in real-time!

Revisión 200811

Basic Issues ...

17

Pedagogical knowledge

Skill of communication

Usually hardcoded in tutorial interaction

Ideally should be expressed in general principles

Declarative

Interpreted into actual decisions

May adapt and improve strategies over time

Should be domain independent

Revisión 200811

18 Revisión 200811

Basic Issues ...

19

Pedagogical knowledge ...

Didactic decisions made by references to SM and domain

Global level = sequence of instructional episodes

Local level = intervention, interruption of student, scaffolding, remediation, etc.

Control of interaction

May destroy student’s motivation to interact

Monitor = monitors student’s action, never gives control

Mixed-initiative = exchange of questions and answers

Coached = student is in full control, system only modifies the environment.

Essential elements of teaching are not clear

Revisión 200811

Basic Issues ...

20

Interface

Form of communication

Operates in coordination with diagnostic and didactic modules

Must translate between knowledge in these modules and student’s input/output

Gives the final form to a presentation

Will make a difference in its understanding

Should be easy to use and attractive

Student must have an idea of how his input will affect the system

Students may have an unrealistic expectation of ITS

Revisión 200811

Foundational Issues for AI-ED

21

The Plausibility Problem

Teaching tactics and strategies

Work well for expert human teachers

Do they work for machine teachers?

Attempt to response

LCS – provide fellow learner.

Revisión 200811

Foundational Issues ...

22

The Plausibility Problem ... Denial of Help by the System

Student surprised that the system behaved in that way.

Machine should not frustrate human learner wishes.

Issue: should the machine behave as a human?

Refusal of Help by the Users Learners have expectations of the system

Learners may choose not to receive help

Better to continue with a more challenging activity

Thought machine cannot give feedback

Issue: What can intelligent systems actually achieve?

Revisión 200811

Foundational Issues ...

23

AI not present in AI-ED AI-ED distancing from AI

Finding AI techniques to make computers better teachers - forgotten

Teaching with a computer – current trend

Reasons: Hype (disillusionment) – system won´t work in classroom

More focus on pedagogical theory (collaborative, meta-learning)

People working in another area (saying it is AI)

Revisión 200811

Foundational Issues ...

24

AI not present in AI-ED.. Machine learning

Reason and make predictions

Student model – derive equations

Pedagogical model – learning pedagogical rules

Bayesian Networks Directed cyclic graph where

each vertex represents probability of certain piece of information being true

Edges indicate relationships between pieces of evidence

Determining student goals

Determining feedback

Curriculum sequencing

Revisión 200811

Cost-Benefits for ITS

25

Funding is under constant review

Not always time to consider all info

Data unavailable or incomplete

Costs and benefits are not clearly understood

Type of advantages Technological

Financial

Differentiation – perceived by customer

Value

Motivation

Introduction of new technologies

Revisión 200811

Cost-Benefits Analysis

26

Financial basis for selecting among a set of actions or decisions.

Hayes model (1981) For each of the alternative actions or options

Identify all the important sources of costs and benefits

Estimate the values of the costs and benefits

Estimate the probabilities of obtaining the costs and benefits

Compare the expected values of the costs and benefits

Choose the action for which the expected value of the benefits minus the expected value of the costs is the greatest.

Revisión 200811

Cost-Benefits Analysis ...

27

Not always easy to:

Identify important sources of cost and benefits

Usually when ITS is beginning in an organization

Estimate probabilities

No data available

Human estimates often fail

Compare values of different types (life vs. job)

Revisión 200811

Cost categories

28

Costs

Development and Delivery – prepare lectures vs. deliver

lecture

Overhead

Labour – salaries, bonuses, vacation, benefits

Expense – travel, office supplies, facilities, rent

Capital – items over USD$1,000 (computers, printers, etc.)

Revisión 200811

Cost categories ...

29

Training Development Costs Workstation (Capital)

How long will they last?

System development (Overhead) Personnel (5 or 6 people)

Knowledge acquisition

Task analysis

Knowledge-base development

Instructional design

Domain expertise

User-interface design

System testing

Technical documentation

Revisión 200811

Cost categories ...

30

Training Delivery Costs

Delivering to student population

Training Overhead Costs

Off-site

Tuition, room and board, transportation

False idea that ITS training is quicker thus less expensive

Employee in training is expensive -> no productivity

Capital costs

Similar to costs with human lecturer

Usually HW costs - maintenance

Revisión 200811

Benefits categories

31

Three categories of financial benefits:

Efficiency (CAI claim)

Reduction in costs while output remains the same.

Reduce overall training costs

Maintain student performance levels

Cost effectiveness (ITS claim)

Costs are held level, output is increased

Productivity

Costs are decreased, output is increased

Revisión 200811

Benefits categories ...

32

Expected performance improvements

ITS typically integrated to existing training

Augment student skills and knowledge

Student-performance gains

Reduction of performance errors by 50%

Higher student skill level after training (two standard deviations)

Reduction in time to reach performance criterion by 50%

Revisión 200811

Benefits categories ...

33

Expected savings Reduce travel costs – on-site training

Reduce time of training – productivity cost

Instructor time and associated costs reduced

Reduced time using specialised training facilities

Distribution (update) of courseware is easier

Revisión 200811

Benefits categories ...

34

Differentiation advantages Value added or perceived advantages

Motivation to learn – if they like ITS

Feeling of control – internal locus of control

Make mistakes and review material without embarrassment

Practice when they have time

Training is consistent and performance assessment is the same for all students

Managers have workers on-site

Students are near home

Revisión 200811