Problem-Based Learning for Foundation Computer ScienceCourses

Mike Barg, Alan Fekete, Tony Greening, Owen Hollands, Judy Kay, and JeffreyH. Kingston

Basser Department of Computer ScienceThe University of Sydney

{ mbarg, fekete, tony, owen, judy, jeff }@cs.usyd.edu.au

Kathryn CrawfordSMITE Research UnitFaculty of Education

The University of Sydneycrawfork@edfac.usyd.edu.au

Abstract

The foundationcoursesin ComputerScienceposeparticularchallengesfor teacherandlearneralike. This paperdescribessomeof thesechallengesandhow we havedesignedProblem-BasedLearning(PBL)coursesto addressthem.

Wediscusstheparticularproblemswewerekeentoovercome:thepuretechnicalfocusof many courses;theproblemsof individual learningandtheneedto establishfoundationsin a rangeof theareaswhich areimportantfor computersciencegradu-ates.Wethenoutlineourcoursedesign,showinghow wehavecreatedProblem-BasedLearningcourses.

Thepaperreportsourevaluationsof theapproach.Thishastwoparts:assessmentof a trial, with a three-yearlongitudinalfollow up of thestudents;reportsof studentlearningimprovementsafter we hadbecomeexperiencedwith full implementationof PBL.

Weconcludewith asummaryof ourexperienceoverthreeyearsof PBL teachinganddiscusssomeof thepragmaticissuesof introducingtheradicalchangein teaching,maintainingstaff support and continuing refinementof our PBL teaching. Wealsodiscusssomeof our approachesto the commonlyacknowledgedchallengesofPBL teaching.

Keywords:Problem-BasedLearning,largefirst yearclasses,life-long learning

1. Intr oduction

The foundationcoursesin ComputerScienceposeparticularchallengesfor the teacher:they developbasicskillsandattitudeswhichareimportantfor effectivelearningin latercourses;they areoften largecourseswith correspondinglylargemanagementandadministrative loads;teachingstaff oftenfind themdemanding,andfor somestaff, they areseenasonerous.

Now taking the learner’s perspective, considerthe critical role of foundationcourses.

- 2 -

They givea largecohortof studentstheir first realtasteof thediscipline. Negativeexperiencesmaydiscouragestudentsfrom further study. This is a very seriousproblemif thosenegativeexperiencesarenot indicativeof thedisciplineasawhole.

To makethisconcrete,considertheroleof groupandindividualwork in typicalfoundationcourses.The coursesare typically basedupon individual work. By contrast,much of theprogrammingworkforce operatesin teams,wherethe ability to work co-operatively and tocommunicatewell areimportant. Studentswith aptitudeanda preferencefor teamwork maygive up on ComputerScienceif their first yearexperiencesconvince themthat the disciplineis individualistic.

Consideranotherexample. A large proportionof ComputerSciencegraduateswill findemploymentwhichinvolvesbroadproblemsolvingskills,ratherthanpurelytechnicallycentredactivity. If thefoundationcourseshave a narrow technicalfocus,this maydeterstudentswithpreferencefor broadproblemsolvingthatissomuchin demandamonggraduates.Thishasbeenidentifiedasaparticularproblemfor women[8].

In addition,the majority of studentsin the first yearcomputingcoursesarealsoin theirfirst yearat University. Thesestudentsareparticularlyfragile. Researchon theexperiencesofAustralianfirst yearuniversitystudentsindicatesthatmany of thesestudentssuffer lonelinessanddoubtabouttheirchoiceof degree.

1.1. Moti vation for movefr om conventional teaching

Factorssuchastheseweighedon us in 1995aswe began the redesignof our first yearcourses.At thatpoint,welikemany others,werestill teachingPascal.For severalyears,wehadbeenacutelyawareof Pascal’sshortcomingsbut foundthechoiceof next languageverydifficult.Evenso,at thispoint,wefelt thatamovewasurgentandwechosetomovetoanobject-orientedlanguage.In fact, we moved to a clean,elegant object-oriented,introductoryprogramminglanguage,Blue[10] [12] [11].

At thesametime,wesaw majorproblemswith theoverall teachingapproachandstructureof our foundationcourses.Our foundationcourseshada conventionalformat,with six contacthoursperweek,of threelectures,atutorialandtwo-hourworkshop.Wewereconcernedthatthisconventionalcoursefailedto developskillsneededin latercourses.In particular,wewantedtheveryfirst coursesto placeemphasison threebroadclassesof skills:

1. genericskills of independentandreflective learning,problemsolving,critical thinking aswell aswrittenandspokencommunication.

2. At thesametime,wewantedto achieve increasedlearningof thetechnicalaspectsof pro-grammingandintroductorycomputerscienceaspectslikecorrectnessandtimecomplexityanalysis.

3. In themiddlegroundbetweenthepurelytechnicalandthegeneric,we wantedto providefoundationsin software engineeringareasof requirementsanalysis,design,planningandcoordinationof softwaredevelopmentandtesting,aswell asa user-centredview ofinteractive-softwaredevelopment.

Theconventionalcoursesmadeit very difficult to nurturethis rangeof skills. They generally

- 3 -

achievedthepurelytechnicalaspectsquitewell: thecourseshadbeenrefinedover many yearsandwereapolishedform of conventionalfirst yearcourses.

Thisrangeof skills is representedin mostfull degreeprogrammes.However, it is typicallynotpartof thefoundationunits. Aswehavediscussed,thismeansamissedopportunitytoshowtheimportanceof thisrangeof aspectsfrom theverybeginning. It alsomakesit moredifficultto helpstudentsestablishgoodhabitsfrom theveryfirst courses.

Consider, for example,the teachingof gooddocumentation.In a typical course,wherestudentsonly write quitesmall,soloprogramsovera shortperiod,rarelymorethanfour weeks,it is difficult to helpthemappreciatethe importanceof documentation.Certainly, onecansetguidelinesandexplainthereasonsfor them.Onecanthentieassessmenttogooddocumentation.But one is then left with largely extrinsic motivation for documentation.It is preferableifstudentscanactuallyexperiencethe benefitsof gooddocumentationandsuffer the effectsofpoordocumentation.Having thishappenat first yearmakesit easierfor studentsto form goodhabitswhichwill carrythemright throughtheirstudiesandprofessionallife. Moreover, it avoidsstudentsformingbadhabitswhichneedto beunlearnt.

Consideranotherexample.In ourconventionalcourse,wemadeextensiveuseof automatedgrading.Thismadeourwork moremanageable.It alsoconstitutedamodelof thoroughtesting:wehopedourstudentswouldlearnfromthis. However,weobservedthatit hadadifferenteffect.By seniorcourses,it meantthat studentsexpectedprogrammingtasksto have extremelytightspecificationsandthey reliedonustosetthestandardswhichdefinedthesuccessof theirwork. Itwassurprisinglydifficult to teachseniorstudentsto settheirown testingagendaandstandards.

A deeperproblemwith theconventionalcoursewasdueto thediversityin our largestudentbody. We took someaccountof this by teachingthetop studentsin anAdvancedform of thecourse.Evenso,thisleft the500to 900studentstakingthecoursein asemesterforcedto spendtheirthreelecturehoursperweekin oneof twolevelsof class.Someof thestudentshavestudiedcomputingat schoolor elsewhere.Somehavebeenprogrammingsincethey werequiteyoung.Mosthavenoprogrammingexperienceatall. Thereisalsoconsiderablediversityonmany otherdimensions.Our studentscomefrom severaldegreeprogrammes:Arts,Economics,Education,Engineering,Science,combineddegreessuchasScience/Law aswell astheComputerSciencedegree.Many studywith usfor just a semesteror two while otherswill continueto completeamajor, Honoursdegreeandpostgraduatestudy. In a monolithiccourse,it is extremelydifficultto caterto thisrange.

Certainly, therecanbesomechoicein practicaltasks.But lectureshave to compromise,aimingat themythical‘average’student.If this is well done,wecanhopeto pacethepresenta-tion to abouthalf theclass.Arounda quarterwill find thingstoo fastandanotherquarterwillbebored.

Thereareotherproblemswith thelectureformat for thetypeof learningthat is normallyrequiredin foundationcomputersciencecourses.The main learningwill invariably happenwhen the studentsactually write programsand get them working. Even aspectslike timecomplexity analysisneedto belearntby doingtheanalysis.Yearafteryear, we have seenthisproblemreflectedin studentsurveyswhereworkshopsandtutorialsareconsistentlyratedaheadof lectures.As teachers,wefelt frustratedatattemptingtheimpossibletaskof creatinglecturessuitedto therangeof students.

Anotherproblemwith theconventionalcoursefollowedfrom thestructureof thepractical

- 4 -

work. Likemany suchcourses,thereweresmallweeklyexercisesaswell astwo largerassign-mentsper semester. The free form commentsin studentsurveys madeit clearthat many stu-dentssaw theweeklyexercisesasfair but thelargerassignmentsastotally unfair. Thismaybedueto thestructureof othercourses,like mathematics,wherethelearningachievedin weeklytasksis adequatefor thecoursegoals.Wewereveryconcernedthatourstudentsbeableto inte-gratethevariouselementslearntin eachweekof thecourseandusethemto solveprogrammingproblems.

Themove to OOPprovidedadditionalimpetusfor re-examiningtheway that we taught.An importantdimensionof theOOparadigmcomesfrom thewriting of largeprogramswhichinvolve several programmers.If eachstudentdoesassignmentsalone,it is moredifficult forthemtoappreciatethisaspect.Certainly,onecanprovidelargeprogramsasastartingframeworksothattheindividualwork canhavemoreof theflavourof typicalOOprojects.But this is lesssatisfactorythangroupdevelopmentof asystem.

1.2. The choiceof Problem-BasedLearning (PBL)

Aswerefinedourunderstandingof theproblemswith theconventionalteachingframework,we identifiedtheelementsof a significantlyimprovedstructure.Theteachingapproachwhichseemedto offer mostpromisewasProblem-BasedLearning. Its literatureindicatesthat it doesdevelopjust therangeof learningwe wanted.See,for example,[1]. PBL is mostcommoninprofessionaldegrees,especiallythosein medicalandparamedicalareas.Seminalwork in movingPBL to moretechnicaldisciplineshasbeenin theareaof ChemicalEngineering[17, 18, 19].

Beforewe describewhat this is, we needto explain what it is not. We frequentlyfindpeoplewhoclaim to usePBL but they appearto usethis termin a differentway from thelargeandwell-establishedPBL community. For example,mostcomputingcoursesinvolve setting‘problems’whichstudentsarerequiredto complete.Wewill referto theseasexercisesbecausethey are small and well-defined. We usedthem extensively in our old conventionalcourse:therewereweeklyexercises,eachfocussedon particulardetailedaspectsof thecourse,usuallyonethat hadbeenon centrestagein the recentlectures;therewerelarger assignmentswhichintegratedmany aspectsof thecoursebut werestill quitetightly defined,to thepoint wherewecouldassesstheircorrectnessin ourautomaticgradingsystem.

PBL involvesmuchbroaderproblemswhich involvea largersetof problemsolvingskills.Critically, PBL placesproblemsolvingandmetacognitiveskills at theheartof thecurriculum.Classtime is devotedto suchgenericproblemsolvingskills asdefininga learningplan,brain-stormingto get startedon a problem,reflection,articulationof problemsandsolutions,self-assessment,practicein active listeningandothercommunicationskills. Theseaspectsarealsoassessedandcontributeto thegradeawarded.

Problem-basedlearning(PBL) is learningby solvinga large,real-world problem.Lecturesarereplacedby additionaltutorial andlaboratorytime. In our case,staff areonly presentforonehourin thethree-hourlaboratoryclass.Thisis necessaryto keepcostssimilar to thosefor aconventionalcourse.

PBL hasbeendefinedasfollows[4]:

The principal idea behind problem-basedlearning is that the starting point forlearningshouldbea problem,a queryor a puzzlethat thelearnerwishesto solve…

- 5 -

Problem-basedcoursesusestimulusmaterialtoengagestudentsin consideringaprob-lemwhich,asfar aspossible, is presentedin thesamecontext asthey wouldfind it in‘r eal life;’ thisoftenmeansthat it crossestraditionaldisciplinaryboundaries.Infor-mationonhowto tackletheproblemis not given,althoughresourcesareavailabletoassistthestudentsto clarify what the‘problem’consistsof andhowthey mightdealwith it. Studentswork cooperativelyin a groupor teamwith accessto a tutor whoisoftennot anexpert in thefield of theparticular problempresented,but someonewhocanfacilitatethelearningprocess.

Thisnotestheroleof authenticproblems,self-directedlearning,cooperativegroupwork andtheroleof theteacherasfacilitator. Anotherdescription,from Biggs[3]

Learners are assignedto small problem-solvinggroupsand begin interacting withteachers,peers and clients; they build up a knowledge basedon relevant materialand learn where to go to seekout more. Studentsmeetwith a tutor and discussthecasein relationto theknowledge they haveobtained.Theknowledge is applied,thecasetreated.Subsequently, there is a review processto ensure that learners developself-managementandself-monitoringskills.

emphasisesthevery importantrole of PBL in developinglife-learninglearningskills, throughvariouselementsof PBL. Critically, PBL coursesactively teachgenericproblemsolvingskills.They aremorethanhelpfulskillswhichwemighthopestudentsbringtotheircomputersciences.Weallocateteachingtimetoexplicit instructionin variousgenericskillsandweassessthem. Atthesametime,wesituatethelearningwithin thecontext of ComputerScience.

Examplesof theproblemswe offer includesimulatinga roadnetwork,maintaininginfor-mationaboutOlympiceventsandathletes,andansweringarbitrarilycomplex databasequeries.Theproblemsareopen-ended,andgroupsareencouragedto researchtheir subjectanddeveloptheirown specificationsandsolutions.A differentsetof problemsissetfor theAdvancedclass:all requireresearchinto techniquesdescribedin theComputerScienceliterature.For example,hereis oneof theproblemswehaveofferedin thefirst semestercourse:

Weareplanningto opentheBasserSoftwareMart. Thisproblemrequiresassistancein planningthecheck-outs.Somepeoplethink thatthebestthingtodoishaveasinglequeuefor customersandthepersonat theheadof thequeuegoesto thefirst availablecheckout.Otherpeoplethink it is betterto haveanExpressqueuefor two check-outsfor customerswho have lessthan6 itemsandanotherqueuefor the remainingfourcheck-outsfor all customers.

Yoursimulationwill allow theplannersto explorearangeof scenarios.For example,customersarrive at different rates,with a burst of thema little after openingtime,anotherburst during lunch time andyet anothernearclosingtime.Also, somedaysare busier than others. It would also be good to explore other possibilities,likecash-onlyqueues.

In your demonstrationof the final project, you will show the simulation of twodifferentwaysof managingthequeuesandshow how well eachdoesin termsof thingsyou considerimportant,like averagewait time for service,maximumwait time orthelike.

- 6 -

Suchproblemsprovideadriving forcefor developingmetacognitiveskills:studentsmanageandmonitortheirown learning,andreflectonhow todothismoreeffectively. Thefirst attackontheprobleminvolvesstudentsdeterminingthefollowing:

• Problemstatement- currentunderstandingof overallgoal(s)

• Currentsubgoals

• How youwill know youhavesucceeded

• Whatyoualreadyknow

• Stepsto take,by when

• Useof 3-hrclasstime

• Useof 6-hrprivatestudytime

Finally, problem-basedlearninginvolvesgrouplearning.Thisservesaspartof thedevelopmentof skills in communicationandco-operative work. It alsomeansthatstudentsneedto discusstheir knowledgeandapproachesandto justify decisions:thisexternalisationof understandingandknowledgeis anaid to improvedlearning,especiallyin thatcaseof metacognitiveskills.

Whenwe begandesignof our course,PBL wasappealing.However, we werenot abletotake its pureform wherethestudentsspendtheirwholedegreeprogramlearningin this format.We neededto adaptit to our two foundationcoursesin an environmentwheremoststudentsspendonly a quarterof their their time in our PBL course,the other threequartersbeinginvariousothercourses.

In summary, PBL is characterisedby:

• open-ended,authentic,substantialproblemswhichdrivethelearning;

• explicit teachingandassessmentof genericandmetacognitiveskills;

• collaborativelearningin groups.

2. Overview of foundation courses

This sectiongivestheflavour of thetwo coursesby describingtheway thatactivitiesarespreadover thesemester, theissuesassociatedwith groupwork, assessmentandstaff develop-ment. We describethefirst semesterin rathermoredetailsoasto communicatetheflavour ofourapproach.

2.1. Semester1: Intr oduction to Programming

Theapproachof thecourseis to view programmingasbuilding modelswhich areimple-mentedasclasses,eachinstanceof whichcorrespondsto areal-world entity.Thesystemwill re-peatedlyanalyseaneventthatcorrespondsto areal-world change,andit will evolvein responseto thisevent.

- 7 -

Thecourseobjectivesareformally setasfollows. By theendof thesemester, eachstudentwill beableto:

• define a ‘simple’ classinterface and implement it, making effective use of the Blueclasslibrary;

• usecodequality andtestingstrategies,includingusinggoodstyle,writing goodpre-andpost-conditionsandclassinvariants,runningandparticipatingin codereviews;

• reasonaboutandexplain the designof a systematic,economicalandpurposefultestingstrategy andevaluatetheextentandsuccessof asetof tests;

• readandevaluatea classimplementationin termsof modularity, codeindependence,classinterfaces,classrelationships,cohesion,coupling,overloading;

• utilisethefollowing genericskills:

• plan learningby formulatingthe problemsto be solved,establishingthe thingsthatneedtobelearnt,andwhatisalreadyknown,definingstrategiesfor learningnew thingsneeded,andmonitoringprogress;

• self-assesslearningby developingstrategiesfor testingnew programmingnotionsandmakinguseof suppliedself-assessmenttools;

• usereferencemanualsandotherprintedmaterialto find informationaboutBlue;

• usethelibrary andtheInternetto find resourcesrelevantto aproblem;

• demonstratetheability to write anEnglishreportaboutthedesignof a ‘simple’ classandits testing,includingthepurposefor eachtestandthebasisfor selectingthatasameaningful;

• giveawell structuredoralpresentationaboutthedesignandtestingof thesystemtheyhaveconstructed;

• work co-operatively, usingprogramming-by-contractandcommunicatingwith othergroupmembersto ensurethey know whateachis expectedto contributeto thegroupeffort andto assessthatcontribution.

Thissetof ‘Postconditions’is in theResourceBook [Kay1999]andclassactivitiesreferto it sothatstudentsareconsciousof whatwehopethey will learnby theendof thesemester.

Therearethreemainperiodsin thefirst semester;

• weeks1-4,thestartupProblem1;

• weeks5-11,mainwork onProblem2;

• weeks12-13,reflectiveperiod,reportwriting anddemonstrationsfor Problem2.

During thefirst four weeksof thefirst semester, enrolmentis toovolatile to form stablegroupsfor problemsolving. Instead,wepresentthefirst problemasadry runfor whatis to come.The

- 8 -

work in this periodis assessedentirely individually. However, studentswork in groups,eachgroupmemberdoingdifferentpartsof theproblem. A passon Problem1 is requiredbeforeastudentis allowedto join aProblem2 group.

Thisperiodalsoinvolvesseveralgroupactivities in eachweek’s tutorial. Theseinvolve acombinationof genericproblemsolvingskills aswell astechnicalskills. For example,anearlyactivity involvespracticein active listeningwith a partnerwho is doingproblemsolving. Weusegenericmaterialstaken from Woods[19] but setthemin thecontext of working out whatcodefragmentsdo. Wecarefullychoosecodefragments:moststudentswill needtowork outtheanswerswith theaid of theprintedresources.Theactivity endswith reflectionaboutproblemsolvingstyleandactive listeningskills, usingWoods’questionnaires.Studentsareencouragedto changegroupsfor eachactivity.

When studentsstart Problem2 in week 5, they know much of their classwell enoughto form groupswhich arereasonablycompatible.We encouragestudentsto form groupsofpeoplewith diversebackgrounds,strengthsandinterests.At thesametime,weoffer a rangeofproblemsandthegrouphasto agreeon theproblemthey will select.

Wehavealreadygivenanexampleof aProblem2 taskwith theBasserSoftwareMart. Allthetasksrequiresimulationof a complex system.All canbedesignedsothat thereis a smallcoreof essentialcodein themainsimulationdriver classes.Oncethis is written andworking,variousotherpartsof thesimulationcanbegin asverysimplestubclassesin a workingsystem.Thesecanbeupgradedasgroupmembersimplementmoresophisticatedversions.Thetutors’taskis to helpguidestudentsto a designwhich is safefor thegroupbecausetheessentialcoreis implementedearlyandnoindividualstudentcanpreventthegroupfrom producingaworkingsystem.(Thistaskis noteasy:wehavebeenevolving strategiesfor assistingtutorsin thisrole.)

Thelong durationof Problem2 would requireconsiderablestudentdisciplineif studentswereto beexpectedto work steadily, againstweeklydeadlinesin othersubjects.We providestructureto theproblemwith deadlinesfor thefollowing stages:

• groupsubmissionof structuralandfunctionalprototypesandsetof acceptancetests;

• individual submissionof a significant piece of code which contributes to the groupsystemwherea passon this stageis requiredfor the studentto earnthe groupmarksoftheirgroup;

• a final individualsubmissionof codewhich givesa largepartof themarksfor a student’spracticalassessment;

• groupsubmissionof thecodefor thesystem.

In parallelwith these,eachstudentdoesadditionalsmallexercisesof their own choice. Eachweekthey setaplanfor theseandin thefollowing week,assesstheirachievements.

In theory, astudentin thePBL coursecoulddoaverysimilarsequenceof tasksto thoseofa conventionalcourse.Eachweekshouldhave themdoinganexcercisein someaspectof thecourse.In parallel,they areworkingonthelargertask. ThedifferencebetweenthePBL studentandtheircounterpartin ourconventionalcourseisthattheformerdecideswhattaskstodo. Theymaywell getsomeassistanceandguidancefrom thetutor in selectinga taskfrom ourresourcesor they mayinventtheirown tasks.Studentsmakecommentslike

- 9 -

I knew I could not write this loop for myproblemuntil I had donea simpleonetoprint thenumbers1to 10. SoI did that andthenI couldseehowto start theloopfortheproblem.

Thefinal reflective partof thesemesteris wherestudentswrite reportson theproblemsolvedby thegroup,presentthedemonstration,andwriteareflectivereportontheir learning,with theirweeklyplansassupportingevidence.Such‘reflective’experiencesareconsideredimportanttodeeplearningandareastandardpartof thePBL approach.

2.2. Semester2: Intr oduction to Computer Science

Thesecondsemesterhastwo tasks,eachrunninghalf thesemester. A commonelementin oneof theseis anoutwardfocuson people(“users”)while theotheris technocentric.As insemester1,weoffer choicesin theproblemsfor eachtask,with careto offer tasksorientedto arangeof interests,includingbusiness,life sciences,andengineering.

Theoutward-lookingtaskhasaninformationsystemsorientation.Thismakesit feasibletocreatetherangeof problemswewantto offer. All involve inheritanceasa fundamentalaid formodelingtheentitiesmanaged.Problemsarealsodesignedtoensurethatstudentshavepracticein thelanguageandtechnicalissuesof managingfiles.

Thisproblemtypemakesit naturalto incorporateethicalissuessuchaswhomight accessor modify data. It alsoprovidesa perfectcontext for issuesof scalability: sostudentsneedtoassessthespeedof thesystemastheamountof datagrows.

Examplesof theproblemchoicesare: managingabiodiversitysurvey, managinginforma-tion for an entertainmentadvisor, managingactivities for Olympic participants,managingthedatafor aschooltimetable,andmanagingproductinventoryfor acomputervendor. In forminggroups,studentsareencouragedto find otherskeento work on thesameproblem.

The secondproblem in intended to develop students’technical programmingskills,especiallyrecursion.Hereweuseparsingasthecommonthemein thesetof problemsoffered.Eachrequiresprocessingof someinputwhoseformathasbeendefinedrecursively. Thismakesa recursiveimplementationbothnaturalandhighpayoff. At thesametime,thetaskintroducestheuseof classeswhichdonot corresponddirectly to physicalentities. It alsoprovidesa goodcontext for useof theCompositeobject-orienteddesignpattern.

Groups were re-arrangedfor this task. Choicesof problemsinclude: developing aspreadsheetincluding complex formulaedefiningcell values;a query interface(modeledonSQL)to tabular data;a prettyprinterfor a subsetof theBlue language[10] usedin thesubject,aninterpreterfor asimpleimperativelanguage,or acompilerfor thatlanguage.

2.3. Learning resources

We have developeda rangeof resourcesto supportthe learning. Oneimportanttype ofsupportisexamplesfor aproblemof thesamecharacter. Thishelpsstudentsseethepossibilitiesfor theproblem,gainabetterunderstandingof theissuesandlearnbroadanddetailedtechnicalskills. Perhapsmostimportant,thisservesasastartingpointwhenstudentsarestuck.

We now describesomeof theseexamples.For thefirst semestersimulationproblem,weprovideonesimulationfor anecosystemandanotherfor a lift (elevator)simulation.Thelatter

- 10-

in availableonlineandis alsoincludedin theprintedresourcebooksothatit canbeusedin thetutorials. In addition,sincethisis thefirst largeproblem,thefirst semesterdevotessometutorialandlaboratorytime to taskswhich helpstudentslearnhow to learnfrom examples.Therearegetting-startedexplorationactivitiesaswell astasksin exploringthestructureof aprogramandseeinghow togetthebigpicture. In tutorials,activitiesinvolvestudyof thedetailsof theprintedexamples,wherethis involveslearninghow to learnnew programmingtechniques,idiomsandlanguagefeaturesfrom examples.Theexamplesupportingthesemester2 informationsystemstaskmanagesstudentgradesfor auniversity.

Onlineresourcesincludea self-assessmentsitewherestudentscantry out taskswhich aretypicalof thestandardthey mightexpectin theexaminations.Thesitealsohasseveralexampleanswersand assessmentcriteria. If they assessone of theseanswers,they can alsoseeourassessmentandanannotatedform of theexample.

We have written chaptersin thetext andresourcebookson technicalaspectsof program-ming. Theserunthegamutof languageissuesfrom dataandcontrolflow to inheritance,includ-ingwhennottouseit. Therearebroadertechnicalchaptersonaspectslikecodeclichesandsoft-wareengineeringissues.We alsoprovide articleson theotherissuesin problems,for exampleethicsandasymptoticrun-timeestimates.Thereareclasstimediscussionquestionson theseis-suesaswell asthepragmaticsof additionalskills requiredin theproblem,for examplepreparingoralpresentationsandreportwriting.

Academicstaff presentweeklyseminarsonthemajorissuesof theproblem.For example,in the informationsystemsproblem,thesedealwith inheritance,informationsystemsin theindustry, persistantstorageandscalability.

2.4. Assessment

Assessmenthasa critical role in any course. This and the role of criterion referencedassessmenthasbeendescribedby Biggs[3, page68]:

Assessmentin practicehastwo functions: to tell uswhetheror not the leaninghasbeensuccessful,andin conveyingto studentswhatwewantthemto learn…

In a criterion-referencedsystem,theobjectivesareembeddedin theassessmenttasks.So,if studentsfocuson theassessment,they will belearningwhat theobjectivessaythey shouldbelearning

Wehavebeenverycarefulin thedesignof ourassessment.Essentialelementsof ourapproachare:

• criterion-basedassessmentso we statecarefully what we want studentwork to demon-strate;

• equalmarksfor examandpracticalwork sothatstudentsseethatweequallyvaluethese;

• within thepracticalwork, equalmarksfor theindividual andgroupwork sothatstudentsseewe valuebothequallyandsothey aremotivatedto performboth individually andasgroupmember;

• groupassessmentfor tasksassociatedwith groupactivity, suchasgroupplanning,manage-

- 11-

mentandcoordinationandthegroupdemonstrations;

• individualassessmentfor code,accordingto theusualcriteriafor design,correctness,styleanddocumentation;

• barriersin practicalwork wherethe individual mustperformsometasksto an adequatestandardbeforethey may move on, andbeforethey areentitledto the marksearnedbytheirgroup;

• minimum performancerequirementsfor both examinationand practical work so thatstudentscannotpassunlessthey achievethatminimumstandardonboth;

• genericaspectsare assessedon the examinationas well as in the practicalwork bothbecausewewanttoassesslearningin theseareasandwewantstudentstoknow weseethemasimportant.

Muchof ourwork in thisareais relevantto any course.An exampleof themarkingschemeforonepartof theassessmentis includedin theAppendix.

2.5. Staff development

A movetoPBLcallsfor aradicalshift in theroleof teachingstaff. In alargefirst yearclass,thismeansthatintroducingPBLrequiresasignificantinvestmentin staff development.Wehaveaddressedthisin four mainways:

• literatureaboutPBL;

• staff developmentsessions;

• scriptsfor teachingstaff;

• teachingmentors.

Thefirst is theeasiestapproach.However, it haslimited value. First,it is difficult for peopletomakethetimetoreadyetanothersetof papers.Moreimportantly,aswefoundin theearlystagesof usingPBL, thementalshift requiredis hardto achieve from merelyreadingpapers.It is tooeasyto fall backonpreviousteachingandlearningexperiences.

Wehavefoundthatintensivestaff developmentsessionsprovideanexcellentstartingpoint.We run thesefor threedays,with morningsdevotedto classroomsessionsandafternoonsforpracticalactivities. Themorningsessionsaregenerallyattendedby mostof thestaff, includingexperiencedPBL teachers.Thesepeoplecan be spreadthrough the groupsusedfor mostsessions.Lessexperiencedstaff areencouragedto doadditionalwork in theafternoons.

Once the semesterstarts,we continue to support staff with detailed scripts. Thesemapout

• preparationtutorsshoulddofor theirclasses;

• anexperiencedteacher’sassessmentof thelikely concernsstudentswill haveat thisstage,especiallywherethis relatesto problemswe know studentswill have with acceptingthestrangenessof PBL;

- 12-

• tipsfor dealingwith theseproblems;

• how to run eachactivity, in termsof how long eachaspectshouldtake, what studentsshouldachieve in that time, what to do if studentsarenot progressingas the schedulesaysthey should,how difficult studentstypically find theactivity, what resourcessupporttheactivity;

• explanationsfor our designof eachof theactivitiessinceweneedthecommitmentof theteachingstaff andweneedto besurethey appreciatethepurposeof learningactivities.

Thefirst weeksof thefirst semesterhave very detailedscripts,up to eightpageslong. As theweekprogress,the guidelines,suggestmorechoicesfor the teacherto make andthereis lessdetail. By theendof thefirst semester, theoutlinefor aweek’sactivity isusuallytwopageslongandin secondsemester, formalscriptsarenot provided. Staff feedbackindicatesthatthis levelof supportis appreciated,especiallyby thoseteachingfor thefirst time.

Thefinal supportcomesfrom structuringthecourseinto sections,with anexperiencedandcommittedpersonasSectionLeader. This personhasweekly meetingswith the four or fivetutorsin their section.This givesnew staff theopportunityto discussproblemsthat they andtheirstudentsarehaving. Thegroupcandiscusswaysto dealwith these.

3. Trial implementation

Sincetheshift from aconventionalcourseto PBL is soradical,wefirst trialledPBL with asmallgroupof studentsin 1996. Benefitsof thetrial were:

• carefulevaluationof it wasto inform thedecisionto moveto PBL (or not);

• weneededanopportunityto learnhow to runaPBL courseandatrial groupwasbetterforthisthana largeclass;

• when we were readyto move to full implementationof PBL, the trial provided solidanswersto the questionsof studentsand staff who were concernedabout difficultiesthey encountered;

• and,althoughwehadnotanticipatedit, its resultshelpeduswhenwemetchallengesin thefull implementationandwe,ourselves,sometimesquestionedthewisdomof PBL.

Thetrial involvedaninitial groupof 42studentschosenrandomlyfrom a poolof volunteers;itwasstaffedby oneof usandoneregularCS1tutor. It wasstructuredsimilarly to theformatwehavedescribedfor thecurrentcourse,althoughwehaverefinedmany detailsoverthethreeyearsof full implementation.

Our trial implementationwascomplementedby an extensive evaluationundertaken bystaff with professionalexpertisein theevaluationof highereducationcourses.Hereweaddressthreeissues:

• assurancethat the PBL studentswere not less competentat programmingthan maingroupstudents;

- 13-

• evidencethat the additional genericskills that PBL is designedfor were in fact be-ing learnt;

• dataaboutstudents’attitudesandperceptionstowardsPBL, sincethey canhave a markedeffecton learning.

For ethicalreasonsit wasnecessaryto selectthePBL streamfrom volunteers,sowecannotruleoutself-selectionbiasaltogether. However,wecansaythatthePBL groupis typicalof themaingroupin ability,asmeasuredbytheTertiaryEntranceRank(TER),asinglenumbersummarisingtheperformanceof eachstudentontheuniversityentranceexaminations.For themaingrouptheaverageTERwas80.9with standarddeviation9.2. For thePBLgrouptheaverageTERwas81.2with standarddeviation9.6,adifferencethatwasnotstatisticallysignificantat the5%level.

It wasnecessaryto usePascalin thetrial sincethestudentsneededcompetencein it whenenteringsecondyear,sotheexpectedsynergybetweenproblem-basedlearningandobject-orient-edprogrammingcouldnotbeassessed.Ontheotherhand,studentsin bothgroupssatthesameexaminations,andtheseexaminationresultsprovideanexcellentbasisfor anobjectivecompar-isonof theoutcomesof thetwo groups.

3.1. Outcomesof the first semester

Ouranalysisof theresultsof thefirst semesterexaminationfoundnosignificantdifferencebetweentheperformanceof thePBL groupandeitheragroupfrom themainCS1classmatchedon thebackgroundvariablesof academicachievementandpreviouscomputingbackground,orthemainclassasawhole. Whentheresultsof individualexaminationquestionswereanalysed,onequestionwasansweredsignificantlybetterby maingroupstudents(not surprisingly, sincethatquestionrelatedtoaspecificwayof drawingdatastructures,asusedin lecturesandthiswasnotusedin thePBL stream)andonequestionwasansweredsignificantlybetterby PBL students(for no clearreason).Theseresultseffectively answerconcernsthat eliminatinglectureswillreducestudentlearning.

Wenotethattheexaminationwassetby thelecturersof themainCS1class.As onewouldexpect,thismeantthattheparticularapproachesof thoselecturerswasreflectedin thequestionsset. That thePBL studentsachievedthesamelevelsof performanceasthemainCS1classis aquitepositiveoutcomefor thePBL trial.

Onthesecondissue,thelearningof genericskills,wedonothavequantitativecomparisonsbetweenthemainandPBL groups;but in many casesthereis no basisfor suchcomparisons.PBL studentswererequiredtodesign,plan,implement,test,manage,andreportonalargegroupsoftwareproject. No comparabledemandsweremadeof studentsin themaingroup. This is acrucialadvantageof PBL: thatwithout losingany of thetechnicalskills, it makesroomin thecoursefor activities that encouragegenericskills. We would have liked to assessthe genericproblemsolvingskills,asfor example,hasbeendonein thecaseof anintroductoryengineeringcourse[15]. Our limited resourcesmadethisinfeasible.

To addressthethird issue,studentperceptionsandattitudes,we conductedanopen-endedquestionnaireat the end of the first semester, in which we asked studentsto completethestatement‘After onesemesterof computersciencemy attitudeto this courseis …’. Answerswerecodedontwo scales:how thestudentsfelt they wereengagedin meaningfullearning,andtheiremotionalresponse.Theanswerswereanalysedby agraduatestudentfrom theFacultyof

- 14-

Educationatouruniversity,underthedirectionof theauthorfromthatfaculty. ThePBLstudentswerecomparedagainstthematchedgroupfrom themainCS1class.

To analysethe students’responsesfrom the point of view of learning attitudes,theresponseswerecodedin thefollowing classes:

Class Example

A. No meaningfullearning

“I feel I have greatdifficulties in learningthiscourse.I considerthespeedof everythingtoofast.I mightdropout.”

B. As A but hopeful “The work loadisveryheavy andI feeldisheartenedbut I supposeifI keptat it, I’d find it a lot easierto cope.”

C. As A but improving “It isprettygood.I find it difficult sometimesbut onceI haveworkedit out it is enjoyable.”

D. Meaningful “Learningall thetime!ThebestandmostenjoyablepracticalcourseI haveenrolledin sofar!”

Conclusionsmustbecautiousbecauseof thehighpercentageof studentswhofailedto respond.However, asindicatedby thegraphbelow, thelargestproportionof PBL studentswhoansweredwerein themeaningfullearnercategory:

0 20 40 60

Code A

Code B

Code C

Code D

No Data

%of CS1-matched%of PBL

Percentageof class

Thesecondanalysiscodedtheresponsesfor theiremotionalcontent:

- 15-

Class Example

A. Despair “I feelverydepressedandanxiousaboutthiscourse.” “I’m sureI’m goingto failandthereisn’t anythingI candoaboutit.”

B. Dislike “I donot likecomputerscience.” “Disheartened,it wasn’t whatI expectedandIcan’t imagineworkingwith computersany more.”

C.Bored “It is veryboring.” “This courseis veryslow anddull.”

D. Hopeful “SometimesI feel overwhelmedby the work, but I feel betterwhensomethinggoesright.”

E.Positive “Positive, I think I’m going to make it.” “It’ s goodoverall, I feel betterwheneverythingis goingwell.”

F. Great “I love this course,it is my favourite.” “This courseis far more varied andenjoyablethanI wouldhave imagined.”

Theresultsare

0 20 40 60

Code A

Code B

Code C

Code D

Code E

Code F

No Data

%of CS1-matched%of PBL

Percentageof class

Onceagain,weneedto notethelargenumberof non-responses.For theothers,wecanseethatthe morepositive emotionsareexpressedby larger proportionsof the PBL group. Similarly,smallerproportionsof thePBL studentsexpressednegativeemotionalresponses.

In interpretingthesecomparisonsit mustberememberedthat thePBL coursewasa firstprototype.It sufferedfrom severalteethingtroublesthatdid not afflict themature,establishedcourse.Ontheotherhand,staff enthusiasmfor thenew coursewasveryhigh.

3.2. Longer term follow up

Fundingfor thedetailedanalysisof thetrial coveredthefirst semester. Sothemostdetailedanalysesarefor thatsemester. We alsocomparedtheresultsfor thesecondsemesterpracticalexamination.Thiswasathree-hourexaminationduringwhichthestudentsworkedindividuallyat a computerterminalon a previouslyunseenprogrammingproblem.Studentswho failedtheexaminationhadanopportunityto try again later.

Thepercentageof studentspassingthefirst sittingof thepracticalexaminationwas48.5for

- 16-

thePBL group(with averagemarksof 55%andstandarddeviationof 40.5%),and48.1for themaingroup(with averagemarksof 58%andstandarddeviation of 38%). Thedifferencesarenotstatisticallysignificantat the5%level.

We analysedthe longer term performanceof the studentsin the PBL trial. Of the 42studentswhobeganfirst year, 22completedsecondyearstudiesand16studiedthird yearlevelsubjects.With sucha smallsample,oneneedscarein interpretingstatisticalcomparisonswiththemainclass.

3.3. Examination performanceand continuation rates

Analysisof examinationresultsshowsnosignificantdifferencebetweenthetrial groupandthemainclass.

A similarstudyof thecontinuationratesfor thetwo groupsindicateaslightlyhigherreten-tion to secondyearlevel for thePBL groupandthenaslightly lower level for thePBL retentionto third yearlevel. This might beconsistentwith a positive first yearexperience,followedbya morenegative perceptionof thesecondyearunits. Thesewereall taughtin a conventionalformat. Perhapsmoreimportantly, atsecondyearlevel, thePBL trial groupwerepartof a largeclassdominatedbythosewhohadstudiedin theconventionalformat. Lecturerswouldhavehavetendedto bemostresponsiveto thebulk of theclass,perhapsbeinglessawareof thosefrom thePBL trial group.

3.4. Affectivemeasures

In 1999,we undertooka limited studyof the long termaffective aspects.Two groupsofstudentswere invited to completean open-endedquestionnaire.Onegroupwasthe thirteenstudentswho beganfirst yearin 1996andwerein thefourth yearHonoursclassin 1999. Thisincludedtwo studentsfrom the PBL trial group. In addition,5 hadbeenin the first semesterAdvancedclasswhichwasin PBL format. Theremaining6hadbeenin theconventionalcourse.Thesecondgroupapproachedwasthosestudentsfrom the1996first yearclassandstill studyingthird yearlevel unitsin 1999. Thesestudentshadtakenmorethantheminimumtime to reachthird yearlevel. Thismaybeduetoarangeof reasons.For example,somestudiedin combineddegreeswhereit isnormaltostudythirdyearlevelsubjectsin thefourthyearof study. Othershadfailedsubjectsor takentimeawayfrom theirstudies.All suchstudentsweremailed.Responseswerecollectedfrom 6 students,4 from theconventionalcourse,1from thePBL trial groupand1 who hadbeenin theAdvancedfirst semestercoursein PBL format. We cannotclaim thesegroupsasrepresentativeof theclassasa whole. However they arediverse.Also, thestudentswhostudyin theHonoursclassareimportantasfutureresearchersandtopstudents.

Thequestionnaireaskedopenendedquestionslike: ‘What do you rememberaboutyourfirst yearof ComputerScience’.‘What doyou think werethepositiveaspects?’‘What doyouthink werethenegativeaspects?’

Therewerecleardifferencesin thecharacterof theanswersfrom thosewho hadstudiedin PBL formatcomparedwith thosein theconventionalcourse.For thosein theconventionalcourse,answersfocusedon thespecificlanguageandlectures.

I rememberlearningPascal.That is not to sayI rememberPascal

- 17-

Fairly well pacedexceptfor tracingbyXXX,pointersbyXXXandfilesbyXXX

They recalledthepositiveaspectsin thesameterms,citingsaspectslike learningprogrammingskillsasapositiveexperience.Negativeaspectswerevariedbut involvedmachineresourcesandspecificlecturers.However, onerecalledthatthey “didn’t getto know many people”.

The studentswho had studied in the PBL format had broaderanswersincluding as-pectslike “friendship” and“group discussion”aswell asworking on “challengingproblems”,“self-directedlearning”andthe “group work”. Somecommentedon the self-pacedandself-directedlearningasapositiveaspect.For example,

learningthingsbymyselfis much betterthanbeingspoonfedby lecturers.

Anotherrecalledthepositiveaspectsas:

anythingPBL….it wastheonlystuff I rememberedandit wasfun

it reallywasa lot of fun….it wasa rapid introductionto programmingandmorewaslearnt in compScithanin anyof myothersubjects

Groupwork wascited with both positive and negative comments.Studentsliked the small,helpfulandfriendly environmentof PBL. Negativecommentsincluded

difficult dueto lazyteammembers

[being] heldback/sloweddownbythegroup

On the other hand,one memberof the trial group had extremely negative memories,withcommentslike:

weweretaughtnothingandhadto learneverythingaboutPascalourselves…it wastoomuch of a burdenespfor peoplewhocameto uni with nopreviousprogrammingexperiencenor knowledgeof a computer

without no oneto eventeach you thebasicsat least,yougaveup after a while …ahopelesssituation

whatever I learnt I learnfromotherstudentsor learningmyself.

and this studentflatly statedthat therewasnothing positive about their first year computerscienceexperience.

Many PBL coursesreporta minority of studentsexpressingthis view. Suchcommentscorrectly identify someaspectsof PBL. Unfortunately, they also indicatea failure to helpstudentsseethe advantagesof this style of learning. They also indicatethe needfor betterscaffolding for students.

Negativeaspectsof PBL emphasisedthelackof guidelinesandfeedback,lackof structureandtheuncertaintyastowhatexactlywasexpected.Othercommentsincludedhavingproblemswith thestructuredexam.

A largely consistentpictureemergesfor the studentswho hadstudiedin the semester1PBL formatAdvancedclass.They likedtheintellectualchallengesandopen-endednessof PBL.

- 18-

They identifiedgroupwork asusefulandpleasurablebut alsoasasourceof frustration.

4. Experiencesfr om full implementation

In threeyearsof full implementation,we have refined the coursesconsiderably. Onewouldhaveexpectedthiswith any new courses,evenoneswhichonly involvedamoveto anewprogramminglanguage.However, with theradicalshiftsinvolvedwith themove to PBL, therehasbeenmorelearningonourpart.

For example,the assessmentcriteria are particularly important in our PBL courses.Aconventionalcoursecanget by with vagueassessmentcriteria,althoughit is wise for everycourseto makebestuseof clearassessmentcriteriato helpcommunicatethelearningoutcomesrequired.In PBL,with theverygeneralproblemstatementsasa startingpoint,it is all themorecritical togivestudentsaveryclearunderstandingof how they will beassessed.Oneof thewellacknowledgedproblemsthatstudentsreportin PBL is thatthey areunsurehow muchthey needtoknow,how fartoexploretheirproblemsandwhenthey canreasonablystoplearning.Wehaveaddressedthisdifficulty with acarefullycraftedsetof assessmentrequirements.Wecontinuetoimproveaspectslike thisaswegainexperience.

At thisstage,wehaveevaluatedseveralaspectsof thecourses.Wenow reportthese.

4.1. Learning outcomes

Thefollowing setsof graphscomparetheresultsof second-semesterexaminationsin thelastnon-PBLyear(1996)with correspondingresultsin thesecondfull PBL year(1998). Therehasbeenasubstantialimprovementin basicprogrammingcompetence:

Programming1996

%of

stud

ents

0 20 40 60 80 1000.0

0.2

0.4

0.6

0.8

Marks(mean63.0)

Programming1998

0 20 40 60 80 1000.0

0.2

0.4

0.6

0.8

Marks(mean91.1)

The two questionsare list traversal problems;the questionsare different but they are ofsimilardifficulty. Webelieve thatthis improvementis partlydueto our new software,theBlueprogrammingenvironment[12] Cite $kolling1999. It is alsopartly dueto thenew curriculum,whichreturnedprogrammingto thecentreof thesecondsemesterunit of study.

Improvementshave alsooccurredin questionson topicsoften thoughtto be beyond thegraspof theaveragestudent,suchastimecomplexity analysis:

- 19-

Analysis1996

%of

stud

ents

0 20 40 60 80 1000.00.10.20.30.40.5

Marks(mean42.2)

Analysis1998

0 20 40 60 80 1000.00.10.20.30.40.5

Marks(mean81.8)

Thesequestionsagain areof similar difficulty. Another advancedtopic, and a key skill forstudentsprogressingto secondyear, is recursion:

Recursion1996

%of

stud

ents

0 20 40 60 80 1000.00.10.20.30.40.5

Marks(mean37.8)

Recursion1998

0 20 40 60 80 1000.00.10.20.30.40.5

Marks(mean47.0)

Althoughwe clearlyhave morework to do here,the1998recursionquestionasksthestudentsto write a recursivedescentparser, andthusis considerablymoredifficult thanthetreetraversalquestionfrom1996.Theseimprovementshaveoccurredbecausethesetopicsarenow integratedinto thesecondsemesterproblems,whereasin theold unit they wereburiedin lecturespoorlyattendedandperceivedasirrelevantby many students.

4.2. Affectiveaspects

Eachyear, we survey studentsin the middle andendof the first semestercourseandattheendof thesecondsemestercourse.Studentsindicatesatisfactionwith mostaspects.Theseminarsconsistentlyratelesswell thantutorialsandworkshops.We have revisedthecontentandstyleof seminarsover theyearsandstudentsatisfactionhassteadilyimproved.

Surveyshavebeenusedto informrefinementof thecourseandthey haveindicatedasteadyimprovementin studentresponses.Herearesometypical responsesummariesfor 1999. Thescaleis 1= very low, 2 = low, 3= average,4 = high,and5= veryhigh:

- 20-

How muchdid you learnin thiscourse?

1 2 3 4 50%

20%

40%

60%

80%

100%

Responses:461; average:3.7

How interestingdid youfind thiscourse?

1 2 3 4 50%

20%

40%

60%

80%

100%

Responses:460; average:3.5

Looking throughthe free text responses,only a handfulof respondentsareclearly unhappywith PBL or otheraspects.Many studentsgave constructive suggestions.Whenasked ‘Whatwasthe bestthing aboutthis course?’,the mostcommonresponseswere: groupwork; PBL;open-endednessandrealismandchallengeof the work; learningto program;object-orientedprogramming.Whenasked‘How canthis coursebeimproved?’,themostcommonresponseswere: moreandfasterterminals;changeto a realworld language;lightenworkload,especiallyearlyon;dosomethingaboutgroupmemberswhodonotdotheir fair share.

Tutor feedbackhasindicatedevenfurtherimprovementin theunit. For example,onetutorwhohastaughtsince1997commented:

Overall I felt thisyearwentbetterthanpreviousyears.I sawstudentswhowereusingFiles,Lists,HashTablesandall kindsof thingsthistimearound,but lastyearsawverylittle. I alsosawa lot of codere-use.Studentsin myclasswerepulling apart MichaelKoelling’sEarthsimulator,andGaryCapell’selevator simulatorandlearningheapsfromthem.

The most important areas for improvementare the managementof the processaspects,ensuringthat studentswork moresteadilyandreflectontheir learning.

Thenew criterion-basedassessmentneedstuning.

5. Relatedwork

Our adaptationof a quite pure form of PBL to teachingfoundationcomputersciencecoursesseemstobenovel. However, therearemany elementsof othercomputersciencecourseswhichsharesomefeaturesof ourapproach.

For example,abouttenyearsago,therewasaninnovativeproblem-basedlearningcourseinBachelorof InformaticsdegreeatGriffith University[13]. It hadaverydifferentemphasis,withthesocialcontext of computingbeingequalto thetechnicalcore. With time,thiscourseended,dueto variousdifficulties,includingthedepartureof somekey staff membersandantipathy toPBL by others.Wehavetriedto learnfrom thiseffort. Ourtrial wasimportantfor buildingstaffsupport. We have alsodevotedconsiderableeffort to staff developmentboth to improve theteachingandto improvestaff acceptance.

Wehavealsobeenableto learnfrom theapplicationsof PBL to Engineeringprogrammes

- 21-

suchasWoods[17, 18, 19] andmorerecentcaseslikeReevesandLaffey [15]. Althoughmuchof thatwork isnotdirectlyapplicable,it isclosertooursituationthatthelargerscaleuseof PBLin themedicalandparamedicalareas.

Therehasalsobeena ComputerSciencecoursetaughtwith successin an AeronauticalEngineeringuniversity[7]. In thatcase,therelativeconsistency in thestudentpopulationmakesthe coursedesignproperlydirectedat the needsof aeronauticalengineering.This appearstohavehelpedstudentsappreciatetheusefulnessof thismaterial.

More seniorlevel coursescommonlyhave variousdegreesof PBL flavour. In particular,mostComputerSciencemajorsdo a seniorlevel coursein what is often calledSoftwareEn-gineeringor maybecalleda capstoneproject. We emphasisethata basicprincipleunderlyingsuchcoursesisthatthestudentshavelearnedthebasictechnicalskillsneededfor theprojecttask:thegoalof thecourseis to give themtheopportunityto gain practicein integratingthoseskillslearnedin variouscourses.This is quitedifferentfrom PBL wheretheproblemis thedrivingforcefor new integratedlearning,not just theintegrationof existingknowledge. In practice,ofcourse,we find that suchcoursesdo involve considerablelearningof new skills requiredforthetask,aswell asbuilding onexistingknowledge.Suchcoursestypically requireanddevelopgenericskills in communicationandproblemsolving,thoughtheexplicit supportof thatlearn-ing maynot beprovided. In practice,if not in philosophy, theseseniorcourseshave a strongPBL flavour.

Artificial Intelligencehasalsobeentaughtusingproblem-basedlearning[14]. Severalotherrecentinitiativesin ComputerScienceeducation[2, 5, 6, 9, 16] haveelementsof problem-basedlearning.To ourknowledge,however, wearethefirst groupto createanentirelyproblem-basedfirst yearComputerSciencecourse.

6. Conclusions

Weadoptedproblem-basedlearning,not for itsown sake,but becausewesaw thatit wouldfosterthefollowing goals.

An integrated curriculum. The educationalliteraturewarnsagainst compartmentalizedunitsof studythatproducestudentswhocannotintegratethedifferentpartsof theirknowledge.Althougha fully integrateddegreewasbeyondour scope,theearlierconventionalfoundationcourseshadcompartmentsthatboreout the literature’spredictions.Thenew coursesarefullyintegrated,sincestudentsbringall theirknowledgeto bearonsolvinga few largeproblems.

Competencein computerprogramming,usinga modernobject-orientedprogramminglan-guageandenvironment.This is our maintechnicalgoalandtheonestudentsaremostmotivat-ed to achieve. We have madethis goalcentralby structuringthe unitsasa sequenceof largeprogrammingproblems,andrequiringourstudentsto usethe‘Blue’ programminglanguageandenvironment.

Ability to enhanceprogram quality by using formal methods.This goal suffered fromcompartmentalizationin theold units. We have integratedit with programmingby adoptingaprogramminglanguagewith featuresto supportit, usingthemroutinelyourselves,andrequiringourstudentsto doso.

Ability toanalysetherunningtimeof programs,andtoproduceprogramswith lowrunningtimes.Thiswasanothervictim of compartmentalizationin theold units. Oneof our problems

- 22-

requireshandling large quantitiesof data,and so studentsmust masterthesetechniquesiftheir programsareto beefficient. Wesupporttheir learningwith tutorialexercises,a text bookchapter, andaseminar.

Ability to userecursion. Recursionis an important conceptualtool and programmingtechnique,andtraditionallya difficult topic for novice programmers.Oneof our problemsisinherentlyrecursive,sothatstudentsmustmasterrecursionin ordertosolveit. Wesupporttheirlearningwith severaltext bookchapters,seminars,andtutorials.

Independenceandinitiative. Theseareimportantgenericgoalsbecausetheold unitswerewidely criticizedfor stifling them. Theproblemswe offer arelooselyspecified,leaving plentyof scopefor studentinitiative. Studentsdiscover they canlearnby themselves,usinga rangeof resources.They areaidedin learningto do thisbecauseof thePBL planningstructuresandtutorialswhichdevelopmetacognitiveskills.

Critical thinking and problem solving. Theseare crucial to effective programming,especiallyat thehigherlevelsof analysisanddesign.We ensurethatstudentsencounterthesehigherlevelsby having largeprojectsrequiringcarefulanalysisanddesign.Oneof our tutors’mostimportantrolesis to guidestudentsin learningtheseskills asthey work on theproblems.Wealsoprovidetutorialsto helpstudentsdevelopgenericproblemsolvingskills.

Theability to work in a group. This is importantfor employers;successfulgroupsalsoincreasestudents’confidenceand initiative. All our problemsafter the first four weeksaregroupproblems.Wesupportgroupsby identifyingspecificrolesfor groupmembers,providingclasstime andguidelineson groupmanagement,monitoringgroupplanningandprogressandassigningmarksfor groupmanagementandreflectionongroupprocesses.

Communicationskills. Thisisanothergoalhighlyvaluedbyemployers.Studentsworkingin a groupnaturallylearnto communicatewith oneanother. At the endof eachproblemthestudentsgivea demonstration,duringwhicheachstudentmustspeak,anda writtenreport. WealsorequireappropriateEnglishcommentarywithin thestudents’programs.

Planning. Real-world programmingprojectsarevery large,andmostfailuresareduetopoormanagementratherthantechnicalproblems.To exposeourstudentsto theseissues,wesetlargeproblems(7–10weeks’work by a groupof four or morestudents)andassessgroupandindividualplanningaswell asproduct.

PBL fostersgenericskills suchasgroupwork - alsonecessaryfor a full appreciationofobject-orientedprogramming- planning,problemsolving,independentlearning,researchskills,writing, andoralpresentation.TheseareUniversitygoalsandalsohighly valuedby employersin thecomputingindustry. PBL allowsstudentsto achieve far morein their programmingthanwaspossiblewithin thesmall,individualassignmentsof theold units;andit virtually eliminatesplagiarism,a long-standingproblemin theconventionalcourse.

7. Acknowledgements

Theintroductionof Problem-BasedLearningfor our largefirst yearclassesrequiredmanyimportantcontributions.TherehavebeenUniversityandFacultygrants:TeachingQualityGrant1995; StudentProgressionAssistanceSchemeAward 1997-8;and Information TechnologyCommitteeGrant1998-9.Many departmentalstaff havebeeninvolved;particularsupportwasprovidedby ProfessorJohnRosenberg andDr. MichaelKölling.

- 23-

Appendix A. Examplesof teachingmaterials

A.1. A criterion basedmarking scheme

Themarkingschemebelow is from thefirst semesterResourceBook,1999. We actuallyusethismarkingschemetodothemarking,andit is therein theworkbookfor studentstoconsult.Not only doesthis ensurethatstudentsareinformedaboutour intentions,it actsasa powerfulcommunicatorof what it is that we value. It also communicatesthe qualitative differencebetweenthedifferentgradesawarded.

NotethatyoumustearnaPassgradeonProblem1in orderto gaincertification.(And youmusthavecertificationonProblem1beforeyoucanbegin Problem2.)

Criteria for a Passin Problem1

To earnaPassgrade,thesubmissionmusthaveall thefollowing:

• It mustbeginwith asignedstatementthat‘thecodeyouhavesubmittedwasentirelywrittenby you’. (If youneededa significantamountof helpfor any otheraspectof thecode,youshouldexplain thatnearthissignedstatement.)

• Yourcodeshouldbedemonstrablyableto dothejob it is supposedto do:it mustwork.

• Yoursubmissionmustincludeonewholeclass.

• Yourcodemustmakeuseof at leastoneloop.

• It mustmakeuseof at leastoneif-statement.

• It mustmakeof at leastoneLList.

• Eachclassmusthavea commentstatingwhatit does.(It mustactuallydo thiscorrectlytobejudgedworkingandthecommentshoulddescribeaccuratelywhatit does)

• YoumustsubmitaTestreportwhichlist thetestsyoudid toconvinceyourselfthatit workscorrectly:maximumlengthis 100words.

• In your3-hrlabyoumustdemonstratethatyourcodeworksbydoingthesetests(maximumtimefor thisdemois 5minutes).

Additional crieria for the gradePass

At leasthalf of thefollowing musthold:

• Theclassinterfaceshouldstatetheauthorandsourcesof significantaspects(for example,if it is basedon informationfrom anaccountingbook,it shouldgivethereference)

• Theclassandeachroutineshouldhavecommentsexplainingwhatthey do.

• Eachroutineshouldhavecommentsexplainingwhatthey do. It is usuallya goodthing toexplaineachroutinein termsof itsparameters.(eg ‘deposit’acceptsan‘amount’in dollars

- 24-

which is depositedin theaccount,with adepositfeededucted.)

• Namesfor classes,routines,parametersandvariablesshouldbehelpful (eg Accountis agoodnamefor abankaccountclass,Snazziisnot.) Choiceof identifiersshouldfollow thestyleof examplesin thetextbookandin theclassdirectories.

• Thereis anattemptto write preconditionsandpostconditions.

• Layoutmustbeconsistentandclear, with indentationshowing codestructure.

• Thereshouldbehelpfulcommentsthroughthecodeasneeded.

Criteria for a Credit

All therequiredandadditionalcriteriafor aPassplusmostof thefollowing:

• All theBluecontrolstructuresshouldbeappropriatefor thetask.

• Thereshouldnot betediouscode(for example,it is a badideato use30 print statementsif you need30 lines,eachwith thesamestring- asa nascentcomputerscientistyou mustsensethattherehasto beabetterwayto dothis,andbedeterminedto find it.)

• This aspectmeansthat presentationof thecodemeansthe readercanunderstandit withminimaleffort.

• You have acknowledgedsourcesandresourceswhich informedyour work on the code.Althoughyoushouldhavewrittentheclassyourself,youwill nothavedoneit in avacuum.For example,youmighthavemodelledyourcodeonsomeof theexamplesin thetext or inourexamples.Youmighthaveusedvariouslibrary classes,likeLList or random.

• Testingis convincing(within 100word limit), stating:thepurposeof thetest;theinput forthe test;theexpectedoutputor behaviour; observedbehaviour. A tabular presentationisprobablyagoodidea.

Criteria for a Distinction ++

All therequiredandadditionalcriteriafor aPassandCreditplusmostof thefollowing:

• Codeshouldbeclearandsimple.

• Eachroutineshoulddoawell definedtaskthesameonedescribedin itsinterfacecomment),havegoodchoiceof parametersandgoodidentifiers.

• Codedoessomethinginterestingandchallenging.

• MoresophisticatedBlueaspectsused(eg nestedloops,morethanasingleLList)

• Testingin reportanddemoareminimalandelegant.

• Eachtestshouldtestadifferentaspectof theclassandthe‘purposeof thetest’shouldmakethisclear.

- 25-

A.2. A conceptinventory

Task A Concept Inventory

Use this inventory to summarizewhat you have learnt in Task A, and to find gapsin yourknowledgecomparedwith whatisexpected.Youmighalsowishtoreview theconceptinventoryin theProblem1work bookfrom COMP1001. Scoreyourselfagainsteachitemlikethis:

1. I’ veneverheardof it2. I’ veheardof it but know nothingmorethanthat3. I know thiswell enoughto try to applyit4. I know thisandI canapplyit5. I know thiswell enoughto explain it to a friend

Score Notesonwhatto doaboutthis item

InheritanceWhy inheritanceis needed

How to getinheritance

Deferredandredefinedroutines

Polymorphism

Staticanddynamictype

Assignmentattempt(?=)

Inheritanceandcreation

Inheritanceof contracts

superFile handling

Thelifetime of routines,objects,filesTextFileHandleandFileSysHandle

Scalability of softwareWhy scalabilityis important

WorstcaseandaveragecaseO-notation

Calculationswith O-notation

O(1), O(logn), O(n), O(nlogn), O( 2n )Analysingroutines

Effectof algorithmsandlibrary classes

Ethical awarenessIdentifyingusersandethicalissues

Effectof ethicsonsoftwaredesign

34

Page 34 of the COMP 1002Workbookfor July Semester1998,showingthe Task A conceptinventory,a checklist that studentscanuseto ensurethat they havecoveredall theconceptsweexpectthemto by theendof TaskA. For example, few studentswouldhaveencounteredthe?=symbol,andthis inventorycanact asanindicationto themthat they shouldfind andstudyit.

- 26-

References

[1] M. A. AlbaneseandS.A. Mitchell. Problembasedlearning:a review of literatureon itsoutcomesandimplementationissues. AcademicMedicine68, 52–81.

[2] CatherineC.Bareiss.A semesterprojectfor CS1. In Proceedings27thSIGCSETechnicalSymposiumon ComputerScienceEducation, pages310–314, Philadelphia,Pennsylvania,1996.

[3] J. Biggs. What the StudentDoes:teachingfor enhancedlearning. Higher EducationResearch andDevelopment18 (1), 57-75(1999).

[4] D. BoudandG.Feletti. TheChallengeof ProblemBasedLearning. KoganPage, 1991.

[5] CurtisA. Carver, RichardA. Howard,andWilliam D. Lane. A methodologyfor active,studentcontrolledlearning:motivatingourweakeststudents. In Proceedings27thSIGCSETechnical Symposiumon ComputerScienceEducation, pages195–199, Philadelphia,Pennsylvania, 1996.

[6] SuzanneW.DietrichandSusanD.Urban. Databasetheoryin practice:learningfrom coop-erative groupprojects. In Proceedings27thSIGCSETechnical Symposiumon ComputerScienceEducation, pages112–116, Philadelphia,Pennsylvania, 1996.

[7] I. Hirmanpour, T. Hilburn,andA. Kornecki. A domaincenteredcurriculum:analternativeapproachto computing curriculum. In Proceedingsof the 26th SIGCSETechnicalSymposium, pages126–130, 1995.

[8] J.Kay, J.Lublin, G. Poiner, andM. Prosser. Not even well begun:womenin computingcourses. Higher Education18, 511–527(1989).

[9] RaymondP. Kirsch. TeachingOLE automation:a problembasedlearningapproach. InProceedings27thSIGCSETechnical Symposiumon ComputerScienceEducation, pages68–72, Philadelphia,Pennsylvania, 1996.

[10] M. Kölling andJ.Rosenberg. Blue– alanguagefor teachingobject-orientedprogramming.In SIGCSETechnicalSymposiumonComputerScienceEducation, 1996.

[11] M. Kölling. TeachingObjectOrientationwith theBlue Environment. Journal of Object-OrientedProgramming12 (2), 14–23(1999).

[12] M. Kölling. TheBlueLanguage. Journalof Object-OrientedProgramming12 (1), 10–17(1999).

[13] S.E.Little andD. Margetson.A project-basedapproachto informationsystemsdesignforundergraduates. Australian ComputerJournal21, 130–138(1989).

[14] L. Cavedon,J.HarlandandL.Padgham.Problembasedlearningwith technologicalsupportin anAI subject:descriptionandevaluation. In Proceedingsof theSecondAustralianCon-ferenceonComputerScienceEducationConference,pages191–200,Melbourne,Australia,1997.

- 27-

[15] T. C. ReevesandJ M Laffey. Design,Assessment,andEvaluationof a Problem-basedlearningEnvironment in UndergraduateEngineering. Higher Educationresearch andDevelopment18 (2), 233–246(1999).

[16] MassoodTowhidnejadandJamesR. Aman. Softwareengineeringemphasisin advancedclasses. In Proceedings27th SIGCSE Technical Symposiumon Computer ScienceEducation, pages210–213, Philadelphia,Pennsylvania, 1996.

[17] D. R. Woods,J. D. Wright, T. W. Hoffman,R. K. Swartman,and I. D. Doig. Teachingproblemsolvingskills. EngineeringEducation66, 238–243(1975).

[18] D. R. Woods and R. J. Sawchuk. Fundamentalsof chemicalengineeringeducation.ChemicalEngineeringEducation, 80–85(1993).

[19] D. R. Woods. Problem-BasedLearning: How to Gain the Most from PBL. McMasterUniversityBookshop, Hamilton,Ontario, 1994.