Simulation-based tutoring systems

CPI 494/598, April 16Kurt VanLehn

Classification of step-based tutors

User interface• Forms e.g., CTAT

• Canvas e.g., Andes

• Text e.g., Criterion

• Dialogue e.g., AutoTutor

• Simulation e.g., ???

Tutoring on• Process• During e.g., Cog Tutors

• After e.g., ???• Both e.g., Andes

• Product e.g., Criterion

• Both e.g., AutotTutor

Prototypes (ill-defined classes) of simulation-based tutoring systems

• Equipment operation & maintenance– E.g., nuclear reactor operation

• Vehicle driving– E.g., flight simulator

• Tactical decision making– E.g., defend ship from possible aerial attack

• Negotiation, leadership, other soft skills– E.g., convince an NGO’s clinic to move

• Lab/field experimentation– E.g., discover cause of an epidemic

What’s different between simulation-based interface vs. others?

• Time matters in the sim• Heavy reliance on delayed feedback• Game-based expectations of sim quality• Game-based technology e.g., game engines• Just making the sim work – huge!• Game-like expense e.g., $1M each!• Most research done outside universities– Mostly military

Steamer – Very early example(Hollan, Hutchins, Weitzman 1984)

• Steam plant occupies 1/3 of space on Navy ships; miles of pipe;

• Engineer of the watch supervises 16 to 25 people who run it

• Years of experience required

Operator’s screen:conceptual, not physical, fidelity of controls

One actual operator’s consolephysical & conceptual fidelity

First, build the simulation• Underlying mathematical model – huge!– Built before Steamer

• Objects for each gauge & control authored– Visual presentation– Hooked up to underlying math model

• Augment math model: models of faults/failures• Math may not be perfect– Especially in abnormal operating conditions or failures

• Validate the sim– check that standard operating procedures work– Compare real to sim behaviors; many conditions

Sim is not an ITS but…

• Can be used for training anyway– Drill to memorize standard op procedures– Exercise in handling standard breakdowns

• Instructor(s) supervises trainees– Decides on exercise– Sets sim into initial state– Watches trainees– Gives immediate feedback occasionally– Gives delayed feedback: After Action Review (AAR)

Problem: After training, still can’t handle non-routine situations

• Need a “mental model” of the whole steam plan– Helps to diagnose what is causing trouble– Helps to hypothesize & test (mentally) repairs

• This sim is a black box; need a “glass box”– Shows multiple levels of detail– Should it to be active; can manipulate & watch– Animation; real time?

Top level view of the whole steam plant

One of about 100 subsystem diagrams

Controlling sim via the schematics

• Can click on schematics to e.g., open a valve• Can do “abstract” actions that are hard or

impossible to do in real world– Replace boiler– Raise chill-water temp to 100 degrees

Problem: Can’t understand time courses

• Neg feedback loops in steam plant control e.g., fuel flow to boiler

• Need to see first and second derivatives as the sim is running– Add new “gauges” to the sim– Some were later added to the real steam plant

• Need to see graphs

New gauges at bottom; graph at top

Was Steamer ever an ITS?• Only for safety issues– Wired to give student immediate negative feedback if

certain safety conditions violated– E.g., starting a pump without first opening the input

and output values• Doesn’t know what student is attempting to do– Can’t offer immediate or delayed feedback– Can’t offer hints– Even if it knew goals, can’t connect goals to actions– Has no representation of knowledge-to-be-learned

• Except mental model of the plant itself, which is huge

RIDES(Munro Johnson Pizzini Surmon Towne Wogulis, 1997)

• 15 years later• Like Steamer, has authoring system for

widgets• Has underlying language for implementing

Sim• Integrates widgets & sim

Has panels with widgetssome are controls, some are gauges

Has better tools for authoring instruction

Instructional design procedure

• Specify objectives; specify scenes• Develop sim– Graphical objects– Behavior rules for each– Debug sim

• Develop lessons– Author procedures using the sim– Add texts– Debug lessons

Object palette: Note how mouse controls rotation

Authoring a procedure

• Put sim in an initial state; name the procedure• Do it• End it

Put sim in an initial state

Half done with the procedure

Three modes for instructionProvided automatically

• Demo mode– Tutor executes procedure– Student controls pace

• Practice model– Student tries to execute– Gets immediate feedback before sim reacts– If authored hints exists, gets them

• Test mode– No feedback & no hints– Sim does whatever

Practice mode

Curriculum editor

Is Rides an ITS Authoring tool?

• Procedural training = CTAT = ITS• Drill on identifying parts = tutor, but not so

intelligent?