Projectacronym: EDSA

Projectfullname: EuropeanDataScienceAcademy

Grantagreementno: 643937

D2.7Lessonslearnedandbestpracticesfromtheproductionoflearningresources

DeliverableEditor: AlexanderMikroyannidis(OU)

DeliverableReviewers:

MishaMatskin(KTH),HuwFryer(SOTON)

Deliverableduedate: 31/01/2018

Submissiondate: 26/01/2018

Distributionlevel: P

Version: 1.0

ThisdocumentispartofaresearchprojectfundedbytheHorizon2020FrameworkProgrammeoftheEuropeanUnion

ChangeLog

Version Date Amendedby Changes

0.1 12/01/2018 AlexanderMikroyannidis

Versionforinternalreview.

0.2 22/01/2018 AlexanderMikroyannidis

Revisedversion.

1.0 26/01/2018 AlexanderMikroyannidis

FinalQA.

EDSAD2.7Page3of11

2018©Copyrightlieswiththerespectiveauthorsandtheirinstitutions.

TableofContents

ChangeLog............................................................................................................................................................................................2 TableofContents...............................................................................................................................................................................3 ListofFigures......................................................................................................................................................................................3 1. ExecutiveSummary...............................................................................................................................................................4 2. Introduction..............................................................................................................................................................................5 3. LessonslearnedfromapplyingtheEDSAvalues...................................................................................................8 3.1 LessonslearnedfromapplyingtheEDSAcurriculumdesignvalues...............................................8 3.2 LessonslearnedfromapplyingtheEDSAcurriculumdeliveryvalues...........................................9

4. Conclusion...............................................................................................................................................................................11

ListofFiguresFigure1:TheEDSAproductionprocessforcurriculaandcourseware.-------------------------------------5 Figure2:ChannelsforthedeliveryofEDSAlearningmaterials-TheEDSAcoursesportal,VideoLecturesandFutureLearn.-------------------------------------------------------------------------------------6

Page4of11EDSAGrantAgreementno.643937

1. ExecutiveSummaryThis deliverable reports the overall lessons learned and best practices acquired from the workconductedwithinWP2.ThemainobjectiveofWP2hasoriginallybeentheproductionofthecurriculaand learning resources of the project in multiple formats, languages and for a variety of learningpurposes and learning contexts. Based on the demand analysis conducted in WP1, as well as theevaluationstakingplaceinWP3,acorecurriculumhasbeendevelopedandrefinedinordertotargettherealneedsofdatascientistsandtheEuropeandataindustry.Thedevelopedcurriculumhasbeensupportedbylearningresourcesdeliveredviaavarietyofpedagogicalmethodsandplatforms.

AspreviouslyreportedinD2.5(M24)andasaresultoftheM18projectreview,thefocusoftheprojecthas been shifted towards addressing the supply of training materials in order to bridge the datascienceskillsgap.Asaresult,WP2hasextendedtheEDSAcoursesportfoliotoincludeawiderrangeofcoursesofferedbyrenownededucationalinstitutionsbothinsideandoutsidetheprojectconsortium.These courses have been selected based on their relevance to the EDSA curriculum and the EDSAdemandanalysis.

EDSAD2.7Page5of11

2018©Copyrightlieswiththerespectiveauthorsandtheirinstitutions.

2. IntroductionThemainobjectiveofWP2wasoriginallytheproductionofcurriculaandlearningresourcesdrivenbythe demand analysis conducted inWP1. In order to address the demand for data science skills, aparticipatory approach was initially adopted by WP2 for the design and production of bespokecurriculaandcourseware,asshowninFigure1.

Figure1:TheEDSAproductionprocessforcurriculaandcourseware.

WP1hasbeenmonitoring trendsacrossEurope inorder to assess thedemands forparticulardatascienceskillsandexpertise,usingautomatedtoolsfortheextractionofdatasciencejobposts,aswellas interviews with data science practitioners. WP1 also established an Industrial Advisory BoardrepresentingamixofsectorstoensurethatprojectactivitiescontinuetomeetchangesinthedemandsondatascienceacrossEurope.

StartingfromtheresultsofthisdemandanalysisandinputfromtheIndustrialAdvisoryBoard,WP2has created relevantdata science curricula tomeet theoutlined trainingneeds.Amultidisciplinarycoursewriting teamhasbeendeveloping inparallel a repositoryof relevantsourcematerials,draftmodules to be placed online, aswell asmaterials forwebinars. The draftmodules have then beeniterativelyrevisedbasedonthefeedbackreceivedfromtheIndustrialAdvisoryBoard,fromtheface-to-facetrainingactivities,aswellasfrommonitoringthemaincommunicationchannelsusedbythecommunitiesofstakeholders.Theanalysedfeedbackhasbeenusedtorestructure,finaliseandpublishthe module content via different educational platforms, including the EDSA courses portal,1

1 http://courses.edsa-project.eu

Page6of11EDSAGrantAgreementno.643937

VideoLectures,2aswellasplatformsofMassiveOpenOnlineCourses(MOOCs)includingFutureLearn3andCoursera4(seeFigure2).

Figure2:ChannelsforthedeliveryofEDSAlearningmaterials-TheEDSAcoursesportal,

VideoLecturesandFutureLearn.

Additionally,LearningAnalyticswereincorporatedintoouronlinedelivery,allowingustocollectdatarelated to the learning experiences of our users, which offered feedback into our curricula design.BasedupontheLearningAnalyticsdataandthefeedbackfromourstakeholders,wereconfiguredandrepurposed modules for different learning contexts, thus initiating new cycles of the productionprocess.

Asaresultoftheproject’sM18review,thefocusofWP2wasshiftedfromtheproductionoflearningresources to the curation of a courses portfolio aggregating awider range of high quality learningresources,eitherofferedbyprojectpartnersorbythirdparties.Thisshiftoffocusaimedatclosingthegap between the demand of data science skills across Europe and the supply of learningmaterialssuitedforofferingtherequiredskillstojobseekers.

TheEDSAcoursesportfoliohasthusbeenextendedtoincludeadditionalcoursesofferedbyrenownedinstitutionsbothinsideandoutsidetheprojectconsortium.Thesecoursesareavailableas:

2 http://videolectures.net 3 https://www.futurelearn.com 4 https://www.coursera.org

EDSAD2.7Page7of11

2018©Copyrightlieswiththerespectiveauthorsandtheirinstitutions.

• MassiveOpenOnlineCourses (MOOCs):Theseareonlinecoursesaimedatunlimitedparticipationand open access on the web. They are available on external MOOC platforms, includingFutureLearnandCoursera.

• Face-to-face courses: These courses are taught face-to-face. Face-to-face learning (or in-personlearning)isanyformofinstructionalinteractionthatoccurs“inperson”andinrealtimebetweenteachersandstudentsoramongcolleaguesandpeers.

• Online courses: These courses are taught online via LearningManagement Systems (LMSs) likeMoodleor Sakai.A subset of these courses consists of self-study learningmaterials available asOpenEducationalResources (OERs),which learners can studyat their ownpace, as there is nopredeterminedstartorenddate.

• Blended courses:These courses are taught in a blendedway (face-to-face and online). Blendedlearningisaformaleducationprograminwhichastudentlearnsatleastinpartthroughdeliveryofcontentandinstructionviadigitalandonlinemediawithsomeelementofstudentcontrolovertime,place,path,orpace.

Themain criteria for the selection of both internal and external courses for inclusion in the EDSAcoursesportfoliohavebeen theEDSAcurriculumand theEDSAdemandanalysis.Courseshavebeenselected based on their potential of addressing the EDSA curriculum topics as well as the trainingneedsofdatascientistsasidentifiedbytheEDSAdemandanalysis.WithregardstocompliancewiththeEDSAdemandanalysis, courseshavebeenevaluated against the recommendationsof the StudyEvaluationReport(D1.4).WithregardstocompliancewiththeEDSAcurriculum,courseshavebeenevaluated against the latest version of the curriculum and the topics it addresses. We have alsoestablished a process formonitoring changes to the demand analysis and aligning our curriculumaccordingly.ThisprocessisdocumentedinDataScienceCurricula3(D2.3).

Linkingthedemandfordatascienceskillswiththesupplyoflearningresourcesthatoffertheseskillsiscrucialforbridgingthedatascienceskillsgap.Towardsthisgoal,EDSAhasdevelopedaninteractivedashboard5thatenablesitsuserstoexploreboththecurrentdatascienceskillsdemandandsupply.The EDSA dashboard enables its users to explore both the current data science skills demand andsupply.Usersofthisdashboardareablenotonlytoexplore thecurrentdemand inthedatasciencemarket,butalsofindlearningmaterialsandtrainingrelevanttotheskillstheywillneedtosecureaspecific job position. Additionally, users are supported in building personalised learning pathways,consistingofcoursesandlearningmaterialsthatwillhelpthemreachtheirlearninggoals.TheEDSAdashboardispresentedinmoredetailintheDemandandsupplyanalysisreport(D1.5).

Theremainderofthisdeliverablereflectsonthe lessonslearnedandbestpracticesgained fromtheWP2work.Inparticular,werevisittheEDSAvalues,whichwereestablishedearlyintheproject,andwereflectuponthesevaluesinlightofthelessonslearnedthroughoutthedurationoftheproject.

5 http://edsa-project.eu/resources/dashboard/

Page8of11EDSAGrantAgreementno.643937

3. LessonslearnedfromapplyingtheEDSAvaluesTheEDSAvalues6were establishedwithin the first fewmonthsof theproject inorder todrive thedevelopment and delivery of the project’s curricula and learning resources. The EDSA values havebeengroundedon thepedagogicaland technologicalexpertiseofprojectpartners,aswellason thelessonslearnedfromthepreviousinvolvementofEDSApartnersinrelevantresearchprojects,suchasEUCLID7andLinkedUp.8

TheEDSAvaluesdefinetheprinciplesthattheEDSAcurriculumandlearningresourcesshouldadhereto.TheEDSAvaluesalsocompriseasetofbestpracticesthatcanbeusedasguidelinesgenerallyforthedevelopmentofonlinecourseware.

In the followingsections,werevisit theEDSAvaluesandreflectonhowtheWP2workwasalignedwiththesevalues,aswellasthelessonslearned.

3.1 LessonslearnedfromapplyingtheEDSAcurriculumdesignvalues§ Industry Aligned: The EDSA curriculum has been designed in accordance with the

expectationsofEU industrial sectors connected todata science, providing industry-standardscenariosandtools.Inparticular,theIndustrialAdvisoryBoardestablishedbytheprojecthascontributed to this value in great lengths, by offering feedback and reshaping the project’scurricula and learning resources. This has allowed our learning resources to reach awideraudienceandtargetspecificindustrysectors.

§ Industry Standard Tools: Our compilation of open source data science tools has offeredlearners experience with tools customary to the industry and their specific sector. Inparticular, our learning resourceshave focusedonwidelyaccepted toolswith a stronguserbasewithintheindustry.Inthisway,theproducedlearningresourceshavebecomerelevanttotheeverydaypracticesofdatascientists.

§ RealData:LearnersutilisingtheEDSAcurriculumhaveaccesstoanumberoflarge-scaleopendatasetstoperformtheirlearneddatascienceskills,enablingreal-worlddatascienceonreal-worlddata.ThisisespeciallytrueforMOOCsandonlinecourseseitherproducedorendorsedbyEDSAthatutiliseawiderangeofdatasetswithrealapplicationsonrealusagescenarios.Likebefore, this approachhas allowedour learningmaterials to appeal to awider industryaudienceastheyintegrateseamlesslywiththeireverydaypractices.

§ Open Design: The EDSA curriculum has been designed from user, research, industry andprofessionalrecommendationsand feedbackhasbeen taken intoaccount fromallacross theEU, ensuring that the curriculummeets the needs of the industry, academia and the widermarket. Asmentioned before, the project’s Industrial Advisory Board has brought togetherrepresentativesfromkeyindustrialandacademicsectorsacrossEuropeandbeyond.Thishasallowed us to collect rich feedback from representatives of the world-wide industrial andacademicdatasciencecommunity,inordertocontributetothetrainingofanewgenerationofworld-leadingdatascientists.

§ ExpertProvision: TheEDSA curriculumhasbeendesignedbyworld-classprofessional andacademic experts indata science. In particular,we have employed experts indifferentdatascienceareasforthedevelopmentanddeliveryofourcurriculaandlearningresources.Inthecases that the required expertisewas not availablewithin the project consortium,we haveidentified and endorsed appropriate learning materials of high quality that are offered by

6 http://edsa-project.eu/overview/edsa-values/ 7 http://www.euclid-project.eu 8 https://linkedup-project.eu

EDSAD2.7Page9of11

2018©Copyrightlieswiththerespectiveauthorsandtheirinstitutions.

organisationsoutsideoftheprojectconsortium.Inthisway,theproject’scourseportfoliohasbeenextendedtoincludehighqualitylearningresourcesdevelopedbothinsideandoutsideoftheproject.

§ Modular: The EDSA curriculum is flexible andadaptable to educator requirements and theneedsof their learners. Inorder to ensure that this is true,wehaveperformedourdemandanalysis across a varietyof industry sectors. In thisway,wehave aimed todeliver learningresourcesthataddressawiderangeoflearningrequirements,aswellasresourcesthatcanbereused and repurposed by releasing them as Open Educational Resources under CreativeCommonslicenses.

§ Transferrable: Skills learned through the curriculumcanbe utilisedacross a rangeof datascienceroles,occupationsandcountriesthroughouttheEU.Asmentionedbefore,theprojecthasdedicatedeffortstoperforminganextensivedemandanalysisacrossEurope.Thisanalysishas identified the data science skills that are currently mostly in demand by the industrythroughoutEuropeandbeyond.Asaresult,theEDSAcurriculumandtheassociatedlearningresourceshavebeendevelopedaroundtheseskills.

§ Concise Learning Goals: All EDSA courses have been aligned with clear learning goalsdepictedbyaspecificaspectofthedatasciencerole.Thishasallowedlearnerstoidentifythecourses that best suit their learning needs and preferences. In addition, learning pathwayshavebeenprovidedtoenablelearnerstonavigatethroughthecontent,selectingwhatisusefulto them.Learnersmayfollowthese learningpathways,or furthercustomiseandpersonalisethem,aswellasreflectontheirlearningprogress.

§ AddressingtheWholeDataValueChain:Datascientistsaremadeawareofthetechniquesand stages of the whole data science value chain through the use of easily understandablenarratives.ThisisespeciallytrueinthefreeandopenonlinecoursesofferedbyODI,suchasthe module “Finding Stories in Data”,9 where clear and concise narratives and storytellingdrivethewholelearningexperience.

3.2 LessonslearnedfromapplyingtheEDSAcurriculumdeliveryvalues§ Multilingual:TheEDSAcourseshavebeendeliveredacrossanumberofEuropeanlanguages

to extend their reach and enable others to use the EDSA curriculum. This value has beenfollowedtoacertainextent,asitrequiresadditionalresourcesdedicatedtothetranslationandcustomisation of learningmaterials for different countries and learning contexts. It becameapparentquiteearly in theproject, that theresourcesavailablewithin theEDSAconsortiumwould allow primarily for the production of English language resources. However, theconsortium has published a substantial part of its learning materials as Open EducationalResources,thusallowingthemtobetranslatedandrepurposedbythecommunity.

§ Multimodal:TheEDSAcourseshavebeenprovidedinanumberofmodestosuitskilllevelsand format preferences, such as MOOCs, eBooks and slide decks. As mentioned in theIntroductionsectionof thisdeliverable, theEDSA coursesportfoliospansacrossavarietyofpedagogicalmodelsandemploysdifferentdeliverychannelsandformatsinordertoaddressdifferentlearningcontextsandaudiences.TheEDSAcoursesalsocoveralltypesofpedagogies,from the traditional face-to-face pedagogical model, to the more recent trends in onlineeducation(MOOCsandblendedlearning).

§ Multi-Platform:EDSAhasutilisedawiderangeofplatformsinordertoremainaccessibleandavailabletoalargebodyofdatasciencelearners.WehaveprimarilyusedVideoLecturesandFutureLearn,whichisthelargestEuropeanMOOCplatformfoundedbyTheOpenUniversity,

9 http://courses.edsa-project.eu/course/view.php?id=52

Page10of11EDSAGrantAgreementno.643937

inordertomaximiseoutreachanduptakeoftheEDSAlearningmaterials.Asaresult,wehaveappealed to thousands of learners worldwide that have enrolled and attended the EDSAMOOCs.

§ Cutting-EdgeQuality: The EDSA learningmaterials have been subject to a series of designiterationsthatencapsulatethelatestresearchandprofessionalpractice,priortotheirlaunch.This has been true for thematerials developedwithin the project,which have followed theiterative production process described in the Introduction section of this deliverable.Additionally,externallearningresourceshavebeenevaluatedbeforebeingincorporatedintotheEDSAcoursesportfolio,basedontheirpotentialofaddressingtheEDSAcurriculumtopicsaswellasthetrainingneedsofdatascientistsasidentifiedbytheEDSAdemandanalysis.

§ ReflectiveandQuantified:TheEDSA learningmaterialshavebeendeliveredwithdataandanalyticsinmind,providinglearnerswithquantifiedmeasuresandanalyticstoreflectontheiraptitude, skills and strengths. Learning Analytics have been incorporated into the onlinedeliveryofEDSAcourses,allowingthecollectionofdatarelatedtothelearningexperiencesoflearners. Additionally, self-assessment exercises and quizzes have been incorporated in theEDSAcourses,thusenablinglearnerstomonitorandreflectontheirlearningprogress.

§ Hands-On: The EDSA course materials have been delivered in a way to emphasise aconstructivisthands-onapproach,meaningfullyapplyingknowledgetorealtoolsanddata.Asmentionedbefore,theEDSAlearningresourceshavefocusedonwidelyacceptedtoolswithastronguserbasewithintheindustry.Inaddition,thecourseseitherproducedorendorsedbyEDSAhaveutilisedawiderangeofdatasetswithrealapplicationsonrealusagescenarios,thusappealingtoawiderindustrialaudience.

EDSAD2.7Page11of11

2018©Copyrightlieswiththerespectiveauthorsandtheirinstitutions.

4. ConclusionThisdeliverablehasreportedtheoverall lessonslearnedandbestpracticesacquiredfromtheworkconductedwithinWP2. Inparticular,wehave revisitedand reflectedupon theEDSAvalues for thedesignanddeliveryof curricula and courseware.TheEDSAvalues compriseaset of guidelinesandbestpracticesthatcanbereusedacrossdifferentcontextsrelatedtodesigninganddeliveringonlinecourseware. The EDSA values can be therefore regarded as a sustainable outcome of the project,having the potential to inform future initiatives relevant to the production and delivery of onlinecoursewarefordatascienceorotherdata-relatedfields.

Inretrospect,designingcurriculaandcoursewarefordatasciencehasbeenaninherentlydifficulttaskfacing a number of challenges, most notably the speed at which this field is changing. Increasingamounts of data lead to challenges arounddata storage and processing, not tomention increasingcomplexity in finding the useful story from that data. New computing technologies rapidly lead toothersbecomingobsolete.Newtoolsaredevelopedwhichchangethedatasciencelandscape.Thesealloccuratsucharapidpacethatteachingdatasciencerequiresanagileandadaptiveapproachthatcanrespondtothesechanges.InthecontextofEDSA,wehavecarriedoutrevisionsguidedbytheEDSAvaluestothecurriculumandtheassociatedlearningresourcesthroughoutthedurationoftheproject,inordertoreflectthemostup-to-dateneedsofthedatasciencecommunity.