ACaseforAnOpenSourceCSCurriculum

TomAndersonh/tAdityaAkella,JeffChase,ArmandoFox,WyattLloyd,DavePatterson,GeoffVoelker

IsThereaProblem?

BoundandMorales,WorkforceTrendsinComputerScience,2016USaddedanaverageof~100KnewjobsinCSperyearsince1994

DoWeNeedThatManyCSGrads?

•  SupplyorDemand?– 60KCSbachelorsdegreesin2015intheUS

•  USadded100KnewCSjobs/year(1994-2015)– Steadystateattritionrequires60Knewhires/year

•  Estimated240KCSjobopeningsinUS•  Largelynotaproblemofstudentinterest– 15%ofenteringUWfreshmenintendtomajorinCS– NotsurprisinggiventhejobmarketfornonCSBAs

HowBiganOpportunity?•  ImpactofsupplyoftechdegreesonmetroGDP–  Perietal.,JournalofLaborEconomics,2015

•  1%increaseintechdegreesinmetroarea–  7%increaseinavgincome,forotherBAholders–  3%increasefornon-collegeeducatedworkers

•  SimilartoestimatedbenefittoDCofAmazonrelo–  $15Bperyearby2030

•  Plusbenefittostudentsthemselves– NewBerkeleyCSgrads2xsalariesofnon-CSgrads

•  Nationwide?1%ofworkforceis1.5M–  10yearsat150Kdegrees/year=>$1T/year

TheImpactofTechnology(1980-2014)

TheHeroicProfessor?

•  Writespapers•  Managesgraduatestudents•  Staysuptodateonrecentresearch•  Understandshowstudentslearnbest•  Determinesbestwaytoteacheachtopic•  Developsgreatcustom-builtcourseprojects•  Writestextbookstoshareeducationalknowledge•  Leapstallbuildings

TheReality•  Shortageoffacultyforin-demandtopics– TypicalOSteachertookanundergradOSclass

•  CSteachinglagstechnologyfrontier,badly– Almostnoonehastimeforcoursedevelopment

•  Assignmentsandprojectsaren’twidelyshared–  Itisalotofwork!– Andyou’llseeevenmoreonlinesolutionsets

•  EvenslidesareoftensharedonlyonPiazza– Exceptamongfriends,orbytextbookauthors

•  Fewertextbooksarebeingwritten

PrivateSchoolsAren’tBigEnough

1.  MIT(31%)2.  Caltech(26%)3.  Stanford(16%)4.  CMU(12%)5.  Princeton(10%)6.  Columbia(10%)

18.Chicago(4.6%)19.Yale(4.5%)20.Hopkins(4.5%)21.NYU(4.4%)22.USC(4.2%)23.Northwestern(3.7%)

Privateschoolsintop50:3600totalBAdegrees(2017)Heroicprofessormodelworksforafewstudents.

GraduatingthemostCS/CEBAs(USNewstop50,firstmajors)

GraduatingtheleastCS/CEBAs(USNewstop50,firstmajors)

PublicUniversitiestotheRescue?

PubliccollegesproducethelargemajorityofBAs–  65%of4yeardegreesintheUS–  73%of4yearengineeringandCSdegrees

Tier1researchschoolsaren’tenough–  Top50rankedschoolsinCS(publicandprivate)

producedabout15KCS/CEBAsin2017–  Stillfarshortofworkforceneed

Needsolutionsthatworkinexpensively,atscale,atnon-tier1schools

PublicResearchUniversities

IncreasingtechnicalsophisticationofeconomyHirefacultywhodobothresearchandteaching–  Developknowledgethatpushestheeconomy

forward–  Trainthenextgenerationofstudentsinthat

knowledge

Aneducationopentoallwhocanbenefitfromit–  apathtolevellingincomeinequality(intheory)

TheParadoxofPublicResearchUniversities

•  Thepublicthinkswearebeingpaidtoteach•  Studentsarefundingmostofthecostoftheflagshipstateuniversities

•  Wethinkofourjobasresearch•  Wespendmostofourtimedoingresearch•  Mostofustrytominimizetheamountoftimewespendteaching

PublicSchoolsGraduatingthemostCS/CEBAs(top30publics,firstmajors)

1.  GeorgiaTech(16%)2.  UCSD(11%)3.  Michigan(8%)4.  UCIrvine(8%)5.  UIUC(7%)6.  Minnesota(6%)

GraduatingtheleastCS/CEBAs(top30publics,firstmajors)

25.OhioState(3%)26.Arizona(3%)27.Washington(3%)28.Utah(2.9%)29.UCSantaBarbara(2.7%)30.UCLA(2.6%)

15%ofincomingUWwanttomajorinCS.Mostpublicschoolsareservingmanyfewerstudentsthantheunderlyingworkforceneed.

TheCRAConsensus

1.  NeedmoreCSBAs2.  FundCSPhDeducationviaNSF3.  Wait6years4.  CSPhDstakefacultyjobs5.  TheyproducemoreBAdegrees6.  Wait6years7.  MoreCSPhDstakefacultyjobs

TaulbeeSurvey

•  Facultyattier1researchschoolsgraduateanaverageof0.3PhDs/year– Fairlyconsistentacrosspublic/private– Alsoacrossdifferentschoolrankings

•  Only30%ofCSPhDstakeacademicjobs⇒ Facultyproduceanaverageof1newassistantprofessoreverydecadeorsoMinusfacultyleavingforindustry

IsNSFtheBottleneck?

•  NSFfundingforCSresearchhasincreased3.5xsince1994– $200M/yearin1994->$712M/yearin2017– ExcludingNSFcyberinfrastructure

•  RoughlyinlinewithincreaseinCSjobsinbroadereconomy– 800Kin1994->2.6Min2015

Recap

•  MassiveunderproductionofCSdegreesrelativetoindustryneed– Largemissedopportunityforeconomicgrowth

•  Tier1schoolsproducingtoofewstudents– Evenlargeincreaseswouldonlymakeadent

•  Needsolutionsthatworkformid-tiercolleges– TypicallynotwhereresearchPhDsgototeach– Butitiswheremiddleandlowincomestudentsgotolearn

EqualityofOpportunityProject

Innovativestudyofanonymizedtaxrecords–  Parentandchildtaxreturns,zipcodes–  Anonymizedcollegestudentrecords

USneighborhoodsaresegregatedbyincome–  Moveapoorchildtoawealthyneighborhood–  Outcomesconvergeatrateof4%/year

Universitiesassegregatedasneighborhoods–  Fewlow/middleincomestudentsattendtier1s–  Thosewhodoendupsimilartowealthykids

CollegeCanBeASocialLeveller

DistributionofParentIncome,Ivy+

NotJustIvy+

TopCSResearchUniversitiesUSstudentsfromtop10%/bottom60%PublicUW 1.2Berkeley 1.3UCLA 1.0UCSD 0.6Michigan 2.9Texas 1.4GaTech 1.8

PrivateStanford 2.8MIT 1.8Princeton 4.2CMU 2.6Cornell 2.5Caltech 4.5

Dataforstudentsbornin1991,includes(most)transferstudentsInternationalstudentsatpublicschoolsarelikelyevenmoreskewed

TopCSResearchUniversitiesMedianFamilyIncome(USonly)

PublicUW $113KBerkeley $120KUCLA $105KUCSD $82KMichigan $154KTexas $124KGaTech $130K

PrivateStanford $168KMIT $137KPrinceton $186KCMU $155KCornell $152KCaltech $146K

Dataforstudentsbornin1991,includes(most)transferstudentsInternationalstudentsatpublicschoolsarelikelyevenmoreskewed

TopResearchUniversitiesPellGrants,allundergrads

PublicUCSD 34%UCLA 34%Berkeley 28%Texas 24%UIUC 21%UW 21%GaTech 15%Michigan 15%Wisconsin 13%

PrivateMIT 17%Cornell 16%Princeton 16%Stanford 15%Caltech 12%CMU 12%

Pellgrantthresholdisroughlymedianincome(duetoARRA)

CollegeChoicesofLowIncomeStudents(HoxbyandAvery2013)

•  80%ofhighlyqualifiedlowincomestudentsareundermatchedinschoolquality– Top4%ofSATscores,highGPAinhighschool– Thatis,verystrongstudents

•  Mostlow-incomestudentsapplyonlytocollegeswithin50milesofhome– Goingawaytocollegeisoftennotanoption

•  Needtoteachstudentswheretheylive

AFewExamples

•  UniversityofCentralFlorida– 38%Pellgrant;37%URM– Graduated480CS+CEmajorsin2017(3.6%)

•  TexasA&MCommerce– 47%Pell;43%URM– Graduated42CSmajors(2.5%)

•  UMassBoston– 40%Pell;31%URM– Graduated37CSmajors(1.4%)

FallacyofEliteProjection

Beliefthatpublicservicesshouldbedesignedtobeattractivetotheelite

Worldclassairports,trainstations,universities,…MeansfewerservicesgetbuiltLeavesmostcitizensbehind

Instead,weneedsolutionsthataredesignedtoworkforeveryone

GeographicDispersion

•  Softwareiseatingtheworld– Becominganessentialelementofeverymajorenterprise

– Ex:manufacturing,agriculture,healthcaremanagement,elections…

•  Geographicallydispersedeconomicbenefitsifwecancreateadispersedknowledgebase– Noteveryonewillmovetothevalley

TypesofTeachingKnowledge(h/t:CMUEberlyCenter)

•  Howtoteachingeneral– Notmyexpertise!

•  Subjectknowledge– Concentratedattier1schools– Whicharehighlyincomesegregated

•  Howtoteachsubject(incontext)– Alsoconcentratedattier1’s,butlessonsmaynotapplytoallaudiences

Coursera/EdX?

•  Successfulmodelforstudentswhoarealreadyself-guidedlearners– Wherevertheylivearoundtheglobe

•  Whatabouteveryoneelse?– Needahumanbeingtohelpproblemsolve– Whyaremystudentshavingdifficultywithaconcept?HowcanIexplainitbettertothem?

•  Notacurriculum– MostoftheeffortatCS1/CS2andspecificskills,partlyduetotheeconomicmodel

FlippedClassrooms?

•  Usingsomeoneelse’svideosatanotherinstitution?– Undercutstheauthorityoftheteacher– Stillneedlocalproblemsolvingandadaptation– MayworkwellforCS1outreachtohighschools

•  Usingsomeoneelse’stextbook/assignments/autogradingscriptsatanotherinstitution?– Thisoftenworksreallywell– Fewerbugs,clearerexpectations,…

TailoredInstruction

•  ControlledexperimentatTexas-Austin(Psych1)–  Lecturevs.lecture+onlinequizaboutpreviouslecture– Quizauto-tailoredtothestudent’scomprehension,basedonpreviousanswers

– Almostnoeffectongradesforhighincomestudents– AboutahalfagradepointdifferenceinGPAforlowincomestudents=>impactonotherclasses!

•  Intheskillsetofeveryinstructor– Outsideofthetimebudgetofalmosteveryinstructor

SIGCSE

•  Isn’ttherealreadyaconferencededicatedtoeducationalissues?– Almostexclusivelyattendedbyinstructors– Verylittlemixingwithresearchfaculty

•  LotsofinnovationforCS1,CS2– Manycompetingalternatives

•  Notasmuchprogressasyougohigherintothecurriculum– Notforalackoftrying

Whatistobedone?•  Solutionsthatareeffectiveatscale•  Atschoolswithoutresearchprograms•  Withthefacultyatthoseschoolsasallies•  Withoutspendingalotofmoney•  Andyouwon’tgetrichorfamous

•  IncreasenumberofCSstudentsnationwide•  ImprovequalityofCSteachingnationwide•  ImproveincomediversityofCSgraduates•  ImproveethnicdiversityofCSgraduates

Whatistobedone?

•  Howdowehelprankedschoolsteachmorestudentsandmoreadvancedmaterial?– Manyalreadyteachingverylargeclasses– Lotsoftimespenttreadingwater

•  Howdowehelpmid-tierschoolsteachmorestudentsandmoreadvancedmaterial?– Oftenteachmultipleclasses/term– Lotsoftimespenttreadingwater

WeHaveaLotAlready

•  Forsometopics,textbookauthorshavealreadyputtogethermostofwhatweneed– Examples:CS:APP,others

•  Forsometopics,projectsinwideuse– Someevenwithautograding

•  Forothertopics,there’sworktodo– Especiallytrueinsystems,imo–  It’salargestepfromlocalusetoglobaluse

•  Whatifwecoordinatedourefforts?

AnOpenSourceCSCurriculum

•  Anentireundergraduatecurriculum–  Roughly20courses

•  Focusoncoursesoftware,notlecturedelivery–  Everyschoolhasteachers– Needtoenablethem,notcompetewiththem– Needrangeofassignmentdifficulty

•  Courseinabox–  Problemsets,programmingmodules,documentation– Automatedgrading,autotuningofdifficulty

•  Opensource:designforchange

OpenSource:LinuxModel

Architectforextensibility–  Ex:addinganewfilesystem,oranewpacket

queuescheduler,oranewdevicedriver–  Allowslargenumberofpeopletocontribute

Corearchitectureteam–  definesAPIs–  decideswhichcommunitycontributionsare

adoptedintomainlinesourcetreeAnyonecanforkandcustomize–  Ethicofadoptionofcommunitycontributions

ACurriculum

•  Hardtomakecoursematerialseffectiveinisolation– Whathavestudentsalreadymastered?

•  Needthesettohangtogetherasagroup– Ananswerto:whatdoIneedtodotolearnCS?– Requirescoordination

•  Enableotherstocontributecontent– Ex:MIThasanumberofP/Fskillscoursesofferedbystudentsforstudents

– Loosenideathatonlyfacultysetstandards

HowManyCourses?

•  CS1•  CS2•  Webprogramming•  Discretemath•  Probabilisticreasoning•  Datastructures•  Softwaredesignandtesting•  Machinestructures•  Systemsprogramming•  Programminglanguages

•  Databasesystems•  Algorithms•  Databaseimplementation•  Operatingsystems•  Distributedsystems•  Computersecurity•  Networks•  AI•  Machinelearning•  Computervision

AtUW,20(quarter)coursestakenbyamajorityofmajors

CaseStudy:DistributedSystems

Grewcoursefrom42to175in3years–  TA’sentirelyself-generated–  Buildadynamicallyshardedlinearizableandhighlyavailablekey-valuestorewithmultikeytransactions

OdditydistributedsystemsdebuggerModelchecked,autogradedassignments–  Studentsself-diagnose

Lecturesandlabsonline– AndsolutionsetsL

CommunityColleges

•  Canweextendthismodeltonon-4yeardegrees?–  Twoyearprograminmath,datascience,andcomputing– Withemployersattheotherend

•  Improvegraduationrates–  Focusteachersonhelpingstudents– Notongrading,assignmentcreation,…

•  Continuingedispartoftheircharter– Notjustforyoungadults

•  Berkeleyisprototypingthis– ¼ofallcommunitycollegestudentsareinCA

WhatAboutCopying/Cheating?

•  We’relosingthebattle–  Postingsolutions,andusingpostedsolutions,isendemic

•  Studentshaveanexittest:theirjobinterview–  Enhancedbyhavingastandardcurriculum!–  Copyingoftentriggeredbylackofhelpandunclearexpectations

•  Reducesalienceofprojectgrades?–  Examscantestwhetherstudentmasteredthematerial

•  Centralizetoolsforcopydetection–  Teachersdon’thaveenoughtimetomakethisapriority

Cost

•  Developinga2-weekassignment,hostedinthecloudwithdocumentationandautograding– Asmallintegernumberofperson-monthsofeffort– Eachcourseneedsfourorfiveofthese,possiblybuildingontopofeachother

•  Pluscommonsoftwareinfrastructure•  AboutthecostofanNSFExpedition,intotalacrossallsubjects

NextSteps

•  Communitybuilding•  Whitepaperexplainingtheeffort•  Workshoptoorganizeaproposal•  Contactmeifyouareinterested

Summary

Wehaveanopportunityforalargestepimprovementincomputerscienceeducationonanationalscale,bytakingtheeffortwearealreadyputtingintocoursesoftware,andorganizingitabitdifferently