a tutoring system for novices learning object-oriented design glenn d. blank computer science &...
TRANSCRIPT
A Tutoring System for Novices Learning
Object-Oriented Design
Glenn D. BlankComputer Science & Engineering Department
Lehigh University, Bethlehem, PA, USA
With support from the National Science Foundation (Grants No. EIA-0087977 and 0624553)
and the Pennsylvania Infrastructure Technology Alliance
Outline A “design-first” approach to the first CS course
Elaboration of “objects-first” approach Multimedia support Eclipse and LehighUML plug-in support
Intelligent Tutoring System (ITS) support Expert evaluator analyzes each step of UML
design Student Model evaluates understanding of
concepts Pedagogical advisor provides feedback
Evaluation results Widening the pipeline of women and minorities
from middle school to college
Who’s on first?From syntax to objects to design
Ongoing debate about how to introduce object-oriented concepts to novices
Syntax-driven approach (traditional): Typically starts with “Hello, World” then arithmetic
Objects-first approach Starts by manipulating classes, objects and methods But objects-first textbooks still tend to emphasize coding
and syntax rather than “object think” Design-first approach:
First learn how to solve a problem Novices need to see the big picture:
Software development is more than just coding It’s understanding a customer’s problem
and designing a solution
So, what is “design-first”? Students learn object-oriented concepts
early and apply them from the start Given a problem description, students learn how to design a UML class
diagram within first two weeks of a CS1 course
Using modern software development tools Eclipse with DrJava & LehighUML
LehighUML plug-in
DrJava plugin
Eclipse IDE
Four easy lessons
1. “Big picture of software development” Compare building a house to constructing software Discussion and small group work with handout (an hour)
2. Objects and classes Multimedia lesson (preferably in lab, 90 minutes)
3. From problem description to class diagram Step by step procedure based on OO design principles
4. Design a class, with ITS help Lab exercise, takes about 90 minutes
Objects and classes: multimedia lesson Implemented in Flash with graphics, animation & sound Core concepts of classes, methods and attributes Eclipse IDE
set up for novices
LehighUML and DrJava
Manipulateconstructorsand methodsfor shapes
Create amethod thatdraws ascene
From problem description to class diagram (in eight steps)1. Read the problem description at least twice. Ask questions about anything you don’t understand or needs more explanation. (The students receive a problem description and ask questions.)
2. Put an ‘S’ next to each subject, a ‘V’ next to each verb and an ‘O’ next to each object.
The ATM verifies whether the customer's card number and his her PIN (a number) are correct. S V O Oher PIN (a number) are correct. If it is, then the customer can check the account balance, O S V O deposit cash, and withdraw cash. Checking the balance simply displays the account balance. V O V O S O V ODepositing asks the customer to enter the amount, then updates the account balance. S V O O V O Withdraw cash asks the customer for the amount to withdraw; if the account has enough cash, S O V O O V S V Othe account balance is updated. The ATM prints the customer’s account balance on a receipt. O V S V O O
Analyze subjects and objects
The ATM verifies whether the customer's card number and his her PIN (a number) are correct. S V O Oher PIN (a number) are correct. If it is, then the customer can check the account balance, O S V O deposit cash, and withdraw cash. Checking the balance simply displays the account balance. V O V O S O V ODepositing asks the customer to enter the amount, then updates the account balance. S V O O V O Withdraw cash asks the customer for the amount to withdraw; if the account has enough cash, S O V O O V S V Othe account balance is updated. The ATM prints the customer’s account balance on a receipt. O V S V O O
3. Analyze each subject and object as follows: Does it represent a person performing an action? Then is an actor, ‘R’. Is it also a verb (such as ‘deposit’)? Then it may be a method, ‘M’. Does a subject take a simple value, such as ‘color’ (string) or ‘money’ (number)?
Then it is probably an attribute, ‘A’. Which remaining unmarked noun occurs most frequently? Make it ‘C’ for class. In Eclipse, draw a class for your first class.
R
R
M
M
A A
A
A
A
A A
A
A A
A
A
A A
A M
C
C
Analyze methods and attributes
4. If a verb has a non-actor object it’s a method, M5. Distinguish attributes, A, and parameters, P: If it doesn’t distinguish instances of a class or store information
about objects, it’s probably a parameter, not an attribute Add attributes to a class in the UML class diagram Add methods in the class diagram While adding methods, include the parameters See the return type of your methods:
“get” methods take the type of the attribute returned “set” methods take “void” return type compute methods take the type of the computed value Methods that answer true/false questions take boolean type
Any more classes?
6. If any remaining classes has no attributes, it’s probably an attribute of another class
7. Otherwise, add another class to the diagram and go back to step 4 (analyze its methods and attributes)
8. Draw association links between the classes
Check your design to make sure it makes sense!
Eclipse for novices Eclipse
Open source, so it’s free! Widely used in academia and industry Students learn one IDE; learn features as needed Good error messages and syntax support But it’s complicated for novices to set up and learn! Multimedia explains how to set it up and get started using it
DrJava plug-in (www.drjava.org) Interprets Java expressions from compiled code Supports test-driven programming
LehighUML plug-in Students enter class diagram Generates code stubs for class automatically Instead of syntax, students focus on design
Lab exercise: Design a movie ticket vending machine
The movie ticket machine displays the movie title, displays the show time (e.g. “2006-11-16 7:00pm”), and displays the price of a ticket (e.g. 8.50). A customer enters money (e.g. 20.50) into the machine. The machine displays the customer’s balance (e.g. 20.50). The customer enters the number of tickets (e.g. 1) into the machine. The machine prints the tickets, and returns the customer’s balance (e.g. 12.00). Print tickets also tracks the number of available seats in the theater so it can tell the customer when the available seats are sold out.
DesignFirst-ITS architecture
Expert Evaluator – Sally Moritz
Interprets each step of a student’s solution Matches student step with pieces of expert solutions Applies concepts in Curriculum Information Network Produces info packets for rest of ITS
Allows variations from expert solutions Matches synonyms and abbreviations An attribute in expert solution can also be defined as a class
(e.g. class “Time” with TicketMachine’s attribute showTime) “Unknown” elements (can’t be matched to anything in
problem description or solution) are the most problematic Pedagogical advisor encourages student to use terms related
to the problem description
Feedback when student enters an unknown class
Expert Solution Generator
Teacher’s problem description Suggested solution
Expert Solution Generator
Generates multiple possible “expert” solutions Analyzes natural language problem description Algorithm similar to what we now teach novices Uses MontyLingua NL parser to tag parts of speech
Student Model – Fang Wei
Challenges: OO design requires understanding
of concepts (not just procedural rules) E.g., class, method, attribute, data type, etc. Concepts depend on prerequisites E.g., attribute vs. parameter attribute, parameter
Student works in an open-ended environment LehighUML plug-in provides some scaffolding Students learn how to apply concepts in design
Limited time for updating the Student Model EE+SM+PA must respond in real student time Bayesian networks can take exponential time
Authors System Context Diagnose Concept
Pre-requisites
Real Time
Murray (1998)
Desktop Associate skills √VanLehn et al.(2001)
Solve physics problems
rules, not concepts √
Millan et al.(2002)
CAT for math √Butz et al. (2004)
C++ programming √ No evaluation
Millan et al.(2005)
CAT for math √ Post-process
Wei&Blank (2006)
OO Design (UML) √ √ √
Student Model in Wei & Blank 2006 (ITS2006)compared with other Bayesian Network Student Models
actor
actor_object
object
object_class
class
class_attribute
attribute
attribute_constructor
constructor
doubleint
numeric datatype
datatype
string
datatype_variable
variable
variable_parameter
parameter
variable_returntype
returntype
pass in only
class_method
method
method_constructor
class_constructor
object_constructor
method_parameter
variable_attribute
object_attribute
object_method
double_int
int_string
double_string
method_returntype
datatype_returntype
attribute_method
attribute_parameter
actor_method
A is prerequisite of B A B
Curriculum Information Network of concepts
Two kinds of concepts (KU)
Unique concept, such as attribute or parameter Relationship concept, such as attribute_parameter Relationships emerge because of student
confusions between concepts E.g., student defines movieTitle as a parameter
when he has already defined movieTitle as an attribute
Two kinds of concepts (KU)
Unique concept, such as attribute or parameter Relationship concept, such as attribute_parameter Relationships emerge because of student
confusions between concepts E.g., student defines movieTitle as a parameter
when he has already defined movieTitle as an attribute
Prerequisite relationships
Prerequisite is relationship between concepts or knowledge units:
The concepts a learner needs to understand before understanding a concept
E.g., one needs to under int and double in order to understand numeric_datatype
Relationship concepts are prerequisites of unique concepts and vice versa
E.g., class_constructor -> constructor Understanding constructor doesn’t imply
understanding of class, just how to define a constructor for a class
……
au
ku
d-prereq(ku)1 d-prereq(ku)2d-prereq(ku)N
Atomic Bayesian Network (ABN)
A directed graph composed of one edge(ku,au)(concept or knowledge unit impliesaction unit or step in student solution)
multiple edges(direct-prerequisite(ku),ku)
Noisy-andgeneralizeslogical-and Students must understand all direct
prerequisites of the concept ku in order to understand ku
Since unique concepts (attribute, parameter) and relationship concepts(how to distinguish between attribute and parameter) are not transitive,ABN only needs to consider direct-prerequisites
Three-level architecture
• CM recognizes a student’s cognitive strategies (e.g., hacking vs. analogy)
• HM tracks history of student’s conceptual knowledge
•PDM simulates student’s conceptual knowledge
Evaluation using 240 simulated students:
• Problem Domain Layer is 94% accurate
• where accuracy means that given probabilities of a simulated student’s understanding of relevant concepts, SM correctly predicts student’s solution step
• Adding Historical Model increases accuracy to 98.5%
Pedagogical Advisor with Learning StylesShahida Parvez
What is an individual learning style?The way a learner perceives and interacts with an environment.Some people like information in the form of pictures / images,
while others like it in written form. Some like to work alone while others like to work in groups. Some prefer concept and theory while others like facts
and concrete experience.
Visual Learner
Auditory Active Learner Learner
Pedagogical Advisor architecture
Sample Pedagogical Feedback for Visual Learner
Evaluation 49 CS1 students completed four lesson introduction
to object-oriented design After multimedia, students took a multiple choice test on
object-oriented concepts Out of 31 questions, mean of 5.5 errors Confirms previous pre-test/post-test results demonstrating
effectiveness of multimedia (Blank et all 2003, 2005) All students completed TicketMachine design
Average graded score of 96.9/100 (1 point per error) Out of 31 questions, mean of 2.3 errors Using paired-samples t test, the T value is 10.2111, p<.001 Lab exercise with ITS help had a significant effect
Future studies will do a finer grain analysis with control group
Evaluation: Expert Evaluator
Overall accuracy 75% Most errors are false negatives
Student solution step marked as error, but should have been accepted as correct
Even in such a small problem, students come up with reasonable variations Example: Method “seatsAvailable” returns true if there
are enough seats to fulfill customer’s request, false if not Possible solution: Add feedback loop to
Expert Evaluator to “learn” from experience of novel correct solutions
Evaluation: Pedagogical AdvisorQuestionnaire asked students about feedback:
Of 32 returned, 23 received visual feedback (text+images), the rest received just text
based on ILS questionnaire 27 paid attention to advice in window 23 said they understood the advice 20 said it helped them figure out their errors 16 of 23 liked images in visual feedback 17 of 23 understood information in images 9 of 32 found advice annoying
Observations from student comments: Students want advice on how to fix the problem (beyond hints) They want to know correctness of their solution
(we plan an evaluation button) They want occasional positive feedback Some suggested that PA be made optional
Widening the pipeline
The pipeline for women and minorities entering computer science (CS) and information technology (IT) is shrinking, at a time when the projected demand for IT professionals is growing Camp, 1997, 1999, Cohoon, 2002
“We must take direct action to attract and retain more women to computing at all points in the pipeline (i.e., K-12, undergraduate, graduate, faculty and industry).” Camp 1999
Lehigh Valley STEM An NSF-sponsored GK-12 project Faculty and Graduate Fellows lead outreach teams Work with K-12 teachers to develop new curricula
Martian landscape with robot “rovers” created in the basement of an Allentown middle school, now a NASA Explorer school
Flash introduced in a Bethlehem middle school Design-first curriculum in an Allentown high school
NSF program manager has indicated likely recommendation for a continuing GK-12 project New project will also feature Outreach Graduate
supplements for RAs to implement curricula developed by Fellow teams
START-IT Build on success of LV STEM and S.T.A.R. Academies
Promote academic achievement for at-risk middle & high school students in the Lehigh Valley
98% of S.T.A.R. students go to college (now in 18th year) Saturdays (once a month) and summer (three weeks) Corporate partners, regional colleges & schools, and parents
Focus on Information Technology (IT) Connect & widen pipeline laid that STEM laid in schools Self-directed multimedia, intelligent tutoring, and one-on-one
mentoring by college undergraduate and graduate students Three teams led by IT faculty and Graduate Fellows
Rising 7th and 8th graders: science, math and IT ‘missions’ using remotely controlled robots in a simulated Martian landscape
Rising 9th and 10th graders: create web-based juke box or iPod™ as an interactive, animated web site using Flash™
Rising 11th and 12th graders: learn Java “design first” and get AP Java college credit