erasmus plus 2015 sector skills alliances · deliverable 6.2 – evaluation and monitoring plan...
Post on 27-Jun-2020
1 Views
Preview:
TRANSCRIPT
ERASMUSPLUS2015SECTORSKILLSALLIANCES
AGREEMENTNo.2015–3212/001–001
PROJECTNo.562634-EPP-1-2015-IT-EPPKA2-SSA
DeliverableNumber: 6.2
TitleofDeliverable: EvaluationandMonitoringPlan
WPrelatedtotheDeliverable: 6
DisseminationLevel:(PU/PP/RE/CO)*: PU
NatureoftheDeliverable:(R/P/D/O)**: R
ActualDateofDeliverytotheCEC: 16/06/2017
PARTNERresponsiblefortheDeliverable: UVa-Education
WPstartingmonth M16
WPendingmonth M19
PartnerContributor(s): SI4LIFE,UVa-Tech,Uva-Nurses,OMNIA,VE-II,AGE,GGALERY,AGE
PartnerReviewer(s): SI4LIFE,UVa-Tech,Uva-Nurses
*DisseminationLevel:
PU=Public
CO=Confidential,onlyformembersoftheAlliance(includingCommissionServices).
PP=Restrictedtoexternalsubjectsinconfidentialmode(includingCommissionServices)
RE=RestrictedtoagroupspecifiedbytheAlliance(includingCommissionServices).
**NatureofDeliverables:
R=Report
P=Prototype
D=Demonstrator
O=Other
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 2of73
1 ABSTRACTThisdocumentillustratestheevaluationprocesssettledintheCARESSproject,addressingtwofacetswithintheproject:theevaluationoftheframeworkandguidelines,aswellastheevaluationoftheCaressPilotsandtheirresponsivenesstotheneedsofthejobmarket.Moreover,thedocumentdefinesandintegratesalist of criteria and indicators thatwill beused in T6.1, and in T6.3 tomake theevaluationprocessmorereliable.Throughthedocument,wewillpresentsomeexamplesofbasic instrumentsthatwehavedevelopedfortheevaluationofthequalityandtheefficacyoftheCARESSpilotsinFinland,ItalyandSpain(WP4).Pilotswill be evaluated in their threemain dimensions: blended learning, e-learning and non-formal/informallearning,throughtheuseoftheCARESSVirtualCommunityofPractice(VCP).This document also presents an “evaluation toolkit” of basic instruments that have been developed forpilots’evaluation inthethreecountries involved.Theseassessmenttoolscanbeusedbefore,duringandafter the implementationof theaforementionedpilots, tocollectvaluabledata,whichwillbehelpful forformativeandsummativeevaluationpurposes.AsthedesigndocumentsforthePilotsarenotbeingreleasedalready,weneedtotakeintoaccountthoseevaluation instrumentsareprovisional, andmay suffer someminormodifications tobetter fit thePilot’sdesign.Thepilotdesignerscanusethe“evaluationtoolkit”astheyarepresentedhere,translatethem,ordevelopsimilartoolsbasedonthese,accordingtotheirspecificneeds.
2 KEYWORDS:Qualityassessmentstrategies,evaluationandmonitoringplan,CARESSEUframework,Pilots
3 LISTOFBENEFICIARIES(PP-RE)/PARTICIPANTS(PU-CO)***
Ben.No. BeneficiaryName ShortName Country
1 Si4Life–ScienzaeImpresaInsiemeperMigliorarelaQualitàdellaVitas.c.r.l.
Si4Life Italy
2 RegioneLiguria LiguriaRegion Italy
3 Ggallerys.r.l. GGallery Italy
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 3of73
4 I.T.C.“VittorioEmanueleII-Ruffini” VE-II Italy
5 AGE-Platform AGE Belgium
6 OMNIA OMNIA Finland
7 FinnishNationalBoardofEducation FNBE Finland
8 Super SUPER Finland
9 AyuntamientodeValladolid AYTO Spain
10 UniversidaddeValladolid UVA Spain
11 SociedaddeGeriatríayGerontologíadeCastillayLeón SGGCYL Spain
12 AssociazionePoloTecnicoProfessionaleProfessioniVita Pro.ViHub Italy
13 AziendaRegionaleSanitariaLigure ARS Italy
14 RoyalCornwallHospitalsTrust RCHT UK
15 UnitedKingdomHomecareAssociation UKHCA UK
16 NestorPrimecareServicesLtd–AlliedHealthcare AlliedHealthcare UK
***ListofBeneficiaries
IncaseofdisseminationlevelPUorCOpleaseindicateallthepartnersinvolvedinthisDeliverable.
IncaseofdisseminationlevelPPpleaseindicatethenamesoftheothersubjecttowhomthedeliverableisdevoted
IncaseofdisseminationlevelREpleaseindicatetherestrictedgroupofpartners.
4 VERSIONHISTORYandAUTHORS
VERSION PRIMARYAUTHOR VERSIONDESCRIPTION DATECOMPLETED
1 SaraVillagráSobrino(UVa)EstefaníaArribasLeal(UVa)
Deliverablestructureanddraftcontent
26/04/2017
2 SerenaAlvino(SI4LIFE)
FirstdraftfeedbackQualityindicatorsT.6.1Criteriaandindicatorstablestructure
05/05/2017
MaríaJoséCao,MaríaJoséCastroAlija,JoséMaríaJiménez,MiriamIdrissiCao(UVa-Nurses)
FirstdraftfeedbackDescriptionofSpanishPilotcontext
26/05/2017
JuanIgnacioAsensioPérez(UVa-Tech)
FirstdraftfeedbackVCPanalyticsindicators
17/05/2017
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 4of73
3 SaraVillagráSobrino(UVa)EstefaníaArribasLeal(UVa)
ModifytheannexesAddPilots’contextdescriptionIntroduceTableofqualityassessmentindicators
02/06/2017
4 SerenaAlvino(SI4LIFE) Reviewseconddraft 03/06/2017
5 FrancescaFerrando(Ggallery) Trackingindicators 05/06/2017
6 BarbaraMazzarino(SI4LIFE)MartaRomagnoli(SI4LIFE)DanieleMusian(SI4LIFE)SerenaAlvino(SI4LIFE)
Task6.1DescriptionandindicatorsReviewofVCPevaluationandindicatorsFinalreview
14/06/2017
7 SaraVillagráSobrino(UVa)EstefaníaArribasLeal(UVa)
Finalintegrationandapproval 16/06/2017
5 TABLEOFCONTENTS1 ABSTRACT.....................................................................................................................................2
2 KEYWORDS:..................................................................................................................................2
3 LISTOFBENEFICIARIES(PP-RE)/PARTICIPANTS(PU-CO)***.......................................................2
4 VERSIONHISTORYandAUTHORS................................................................................................3
5 TABLEOFCONTENTS....................................................................................................................4
6 INTRODUCTION:STRUCTUREANDAIMSOFTHEDOCUMENT....................................................5
7 EVALUATIONCONTEXTANDMETHODOLOGY.............................................................................7
7.1 EvaluationoftheCARESSframeworkandWizard/guidelines(T.6.1)...............................................10
7.2 Evaluationofthepilots(T6.3)...........................................................................................................11
7.3 EvaluationtaskswithinCARESSproject:generaloverview...............................................................15
8 CONTEXTANDEVALUATIONMETHODINT6.3..........................................................................24
8.1 Descriptionofthetimelineandtheevaluationinstruments............................................................29
8.2 Whowillcollectthedata?.................................................................................................................31
8.3 DraftoftheevaluationtoolkitinT.6.3..............................................................................................33
9 QualityAssuranceandOverallProjectEvaluation.....................................................................34
10 BIBLIOGRAPHY...........................................................................................................................35
11 ANNEX........................................................................................................................................36
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 5of73
11.1 Annex1:Guidelinestoperformtheobservations(Draft)...............................................................36
11.2 Annex2:Guidelinestocompletetheteacher’sReflectiveDiary(Draft)........................................37
11.3 Annex3:Guidelinestoperformtheteacher’sinterview(Draft)....................................................38
11.4 Annex4:Aquestionnairetoevaluatethetrainingmaterialsbythestudents(Draft)...................39
11.5 Annex5:QuestionnairetoevaluatethetrainingmaterialsbyTeachers(Draft)............................43
11.6 Annex6:Students’initialquestionnaire(Draft)..............................................................................48
11.7 Annex7:Students’finalquestionnaire(Draft)...............................................................................49
11.8 Annex8:Teacherfinalquestionnaire(Draft)..................................................................................53
11.9 Annex9:Questionnaireevaluatingteachers’Pilotdesign(Draft)..................................................58
11.10 Annex10:GuidelinestocompletetheStudent’sReflectiveDiary(Draft)....................................65
11.11 Annex11:Student’posttrainingquestionnaire(Draft)...............................................................66
11.12 Annex12:Student’posttraininginterview(Draft).......................................................................67
11.13 Annex13:TrackingreportingCARESSe-learningplatform...........................................................68
6 INTRODUCTION:STRUCTUREANDAIMSOFTHEDOCUMENTThe CARESS Evaluation and Monitoring Plan is a document that describes how we will monitor andevaluate theCARESSpilot courses carriedoutwithin theCARESSproject, and the criteria and indicatorsthatshouldbeusedtoevaluatetheCARESSEUframeworkandguidelines.Afterwards,thosecriteriaandindicatorswillbeusetoevaluatetheresponsivenessofthetrainingprovidedwithintheCARESSprojecttotheneedsofthejobmarketanddrawdownacomprehensivefinalevaluation.Aswe can see in figure 1, whereas the ultimate purpose of Task 6.1 is to gather feedback from pilots’designerstovalidateandprovetheuseoftheCARESSEUframeworkandtheguidelinesdeliveredinT.3.1and3.2,task6.2isfocusedontheevaluationplanandmonitoringofthepilotsanditsfunctioning.Besides,weprovideadescriptionof theprocedurethatwill follow,aswellasasetof indicatorstoassesstheEUframework and the guidelines for the design of the pilots. Finally, a description of the instruments andguidelinesthatwillbeusedtoassessthedesignandtheenactmentofthepilotsisprovided,withtheaimof assessing the responsiveness of the training developed to the jobmarket needs according to home-healthcareprofessional’sperspective. According to the aforementioned ideas, a comprehensive final evaluation of the pilots, linked with theresponsiveness to jobmarketneeds, (i.e., task6.3),will alsobeevaluated following specific,measurableindicatorsdefinedinthisdocument.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 6of73
Figure1.Relationshipsbetweenevaluationtasks
Animportantfirststeptowardsthataimisthedescriptionofthescopeandthemethodofdatacollectionand analysis that will follow. The evaluation plan will clarify the procedure whereby the informationgatheredfromthepilot'sdesignhavebeenusedtoassesstheEUframeworkandtheguidelinesaswellasabouthowthepilot’sexperienceshavebeendevelopedinFinland,ItalyandSpain.Asaconsequence,wewanttoknow,towhatextenttheactivitiesandtasksdevelopedinthenationalpilotsarelinkedwiththeintended learning outcomes. Moreover, we want to illuminate whether the pilots were operatingcoherentlytotheprogram’sdesign.Additionally,theprocessevaluation,alongwithoutputandshort-termoutcomeinformation,willhelptoclarifywhetherpilotsrespondtotheintendedCARESSprojectobjectives.It isworthnoting that theevaluation topicshavebeendefined taking intoaccount the listof short-termresultsand longtermoutcome indicators thatwereexplicitlymentioned in theCARESSprojectproposal.These indicators are specific for eachactivity tobeevaluated, and correspond to thedifferent aimsandgoalsregardingthefunctionalitiesoftheelementtobeassessed.Someoftheindicatorscanbetrackingbycompiling thedata requested (e.g. that the learningmaterials arewriting in two languages, thenationallanguage(Italian,FinnishandSpanish)andEnglish,whileothersmayneedthedevelopofaparticulartool.Nevertheless, all of them are linked with the quality standard for the whole project, connecting theevaluation with the assessment of the project quality. Thus, quantitative and qualitative indicators aredescribedinsection7.3.This report is structured as follows. Section 7 provides a general description of the context and theevaluation methodology guided by the CSCL-EREM evaluation framework (Jorrín-Abellán, 2013) withinCARESSproject.Besides,clarifieswhatitisthescopeoftheevaluationattendingtothedifferentscenariosandobjectives,withasectiondevotedtotheevaluationofEUframeworkandguidelinesandanothertotheevaluationofthenationalPilotsandtheposttrainingresponsivenessofthejobmarket.Likewise,weshowedageneraloverviewof theevaluationprocesswithin theCARESSproject.Also in this section,wepoint out the evaluation objectives linked to criteria and indicators according to the status of theassessmenttoolsandthetimewheretheevaluationwillbecarriedout.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 7of73
Then,section7.2explainshowtheevaluationofthequalityandtheefficacyofthepilot’scourses inVETwillbeperformedinItaly,SpainandFinland.Ontheonehand,weprovidedadescriptionoftheevaluationtimelineand theevaluation instruments thatare supposed tobeused forgatheringevidenceduring thedesignandtheenactmentofthepilot’scourses.Ontheotherhand,wehighlightpartner’sresponsibilitiesandrolesofCARESSpartnerswithrespectontheevaluationtasksthatwillbecarriedout.Finally, section 11, the annexes, draws down a draft of the “evaluation toolkit”: a set of assessmentinstrumentsandguidelinesdevelopedwith theaimofhelpingCARESSpilots'designers togatherdata toassess theusefulnessof theEU frameworkand theguidelines todesignVETpathways.Moreover,aboutthepilots'qualityandefficacyandresponsivenesstojobmarketduringandposttraining.It is worth notice that the evaluation toolkit showed just a first draft of assessment instruments (i.e.,observations,questionnaires,interviews,etc.)thathastheaimofguidingevaluatorsduringtheevaluationtasks. The ideas provided in the “evaluation toolkit” could suffer modifications to better adapt theinstrumentstotherealneedsofthecontextandthe informants (i.e., teachers,students,VETexperts)oftheevaluationtasksperformed.
7 EVALUATIONCONTEXTANDMETHODOLOGYThe evaluation context and scope is divided attending the two different scenarios. First of all, theevaluationoftheusefulnessoftheEUframeworkandguidelines(theWizardtool)duringthedesignofthepilotslearningdesign,describedinsection7.1.Secondly,theevaluationofthenationalpilotsandthejobmarketresponsivenessafteraperiodofsix-monthposttraining,describedinsection7.2.Withtheaimofclarifying the scopeof theevaluation taskswithinCARESSproject, section7.3will showa tablewith theassessment objectives linked with criteria, sub criteria and indicators according to the status of theassessmentinstrumentsaswellasthetimewheretheevaluationwilltakeplace.
DuetotheinherentcomplexityofevaluationitselfandthedifficultyofhealthcarescenarioscarriedoutindifferentplacesacrossEurope, it is convenient tousea theoretical framework toguide thecollectionofevaluationdata,itsanalysisanditsinterpretation.Specifically,theevaluationofCARESSprojectfollowstheEREM model (Evaluand-oriented Responsive Evaluation Model) (Jorrín-Abellán, 2009). The EREM is aframework that relies on a responsive evaluation approach, empowering the idea of conductingevaluationsfocusedonthephenomenatobeevaluated(evaluand)ratherthaninthefieldofexpertiseoftheevaluators(Stake,2004).Itisworthnotingthatanevaluandcanbedefinedastheentity,theprocess,theactivity,ortheproductbeingevaluated.Besides,theEREMtriestobeaframeworkforhelpingintheevaluation of Computer Supported Collaborative Learning programs, innovations, learning and teachingstrategies,educationalresourcesaswellasresearchprojects.This evaluationmethod follows an interpretative research perspective that does not pursue statisticallysignificant results or generalizations. Rather, it aims at a deeper understanding of the particular
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 8of73
phenomenaunder study. Inour case [1] theusefulnessofCARESSEU framework tooland thewizard tohelpteachersduringthedesignofthepilotsand[2]thequalityandefficacyofthelearningcoursescarriedoutthroughthepilotsperformedinItaly,SpainandFinlandacrossCARESSproject.Conductinganevaluationthat is responsivetotheneedsofdifferenthealthcarepractitionerswillensurethat evaluative information ismeaningful andmore likely to be used for programme enhancement andimprovement.Accordingtotheaforementionedideas,wethinkthatEREMcouldbeausefulframeworkwhichcanhelpusin the creation of an evaluation design suitable to the particular necessities derived from the differentcontexts in where the pilots will take place. Furthermore, the responsive evaluation model has beenimplementedinvariouspolicysectors,includingthefieldofhealthcare:
● To the evaluation of clinical skills assessment and training (CSAT) programme at the Faculty ofMedicine(Curran,Christopher,Lemire,Collins&Barret(2003).
● Palliativecareprogramsandunits(Abma,2001).● Elderlycare(Koch,1996)andnursingcurricula(Koch,2000).● Rehabilitationprogramsforpsychiatricpatients(Wadsworth,2001).● SupportedEmploymentprogramsformentallyhandicapped(Widdershoven&Sohl,1999).
InthecontextofCARESSproject,theevaluationhastheaimofprovidingevidenceaboutthequalityandtheefficacyofpilotsandtheirresponsivenesstotheneedsofthejobmarketandabouttheusefulnessofCARESS EU tool to help practitioners during the design of VET courses. Therefore, this evaluation has athreefoldevaluandthatincludes:thequalityandefficacyofpresencecoursesinhomecare,theCARESSe-learning platform and Virtual Community of Practice acceptability and the usefulness of EU CARESSframework tool and their guidelines to support teachers during thedesignsof learningpathways inVETscenarios.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 9of73
Figure:1GraphicalrepresentationofCARESSpilots’evaluationdesign
Aroundtheevaluandconcept,theframeworkstructurestheevaluationdesignintothreedifferentfacets:perspective (why the evaluand is evaluated), covering themain goal pursued andother significant openquestions;ground(wheretheevaluandisevaluated),gatheringtheinformationaboutthecontextsandtheparticipants;andmethod(howtheevaluandisevaluated),showingthedatagatheringtechniquesandthedocuments that support the conclusions. Figure 1 illustrates a general representation of the EREMframeworkincludingtheevaluand(inthecoreofthefigure)andthedifferentfacetsaroundtheevaluand.ThisgraphicalrepresentationhasbeenobtainedusingtheCSCL-EREMwebtool1.TheperspectivefacetisorganizedaroundthedefinitionofIssues.Anissuecanbeunderstoodasatension,atroublingchoice.Oneissuenaturallyarisesfromtheobjectivesofthepresentevaluation.Itisstatedasfollows:[Issue1]HowtheCARESSEU framework tooland thewizardcanhelp teachersduring thedesignof thepilots?[Issue 2] Can training courses based on competences and active methodologies be helpful to improvenurses/OSSandpracticalnursesclinicalpracticeandtransversalskillsinelderlycare?
1 EREMwebtool:https://pandora.gsic.uva.es/cscl-erem
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 10of73
7.1 EvaluationoftheCARESSframeworkandWizard/guidelines(T.6.1)Accordingwiththeproject,twoofthemainresultsofCARESSprojectare:
• AEuropean framework forvocationaleducationandtraining (VET) in the fieldofhomecare, thatreflects the professions and occupations actually involved in domiciliary care, thecompetencies/skills required foreachprofession,andtherelated learningoutcomes (referring toECVET);
• Guidelines supporting different stakeholders working in the field of homecare in the use theEuropean framework itself for designingVET curriculawhich take into account contextual needsandalreadyexistinginitiatives.
Framework andGuidelines are delivered inWP3 andwill evolve during the lifespan of the project. Thisprocedure of progressive refinement requires scheduling different evaluation slot following thedevelopmentoftheresults.
Thedesignofthethreenationalcurriculafortheimplementationofthepilots(WP4)isthefirsttestbedfortheFrameworkandguidelinesandcorrespondtothefirstvalidationslot (M21),aswellas thereleaseoftheirfinalversioncorrespondtothelastevaluationperformed(M32-M34).Theadditionalevaluationstepswillbescheduledintask6.1.
Asdescribed inD3.2.1and inD3.3.1 tosupport thecollaborativedevelopmentof theFrameworkand itsusability, intermsofprovidingstructured,organizedandsystematized information inaneasilyaccessiblemanner,aweb-basedsolutionhasbeenimplemented.Thischoicewillallowfor:
• asimpleanduser-friendlyaccess(throughqueries)tospecificdatacollectedduringthecontextualanalysis;
• a simple and user-friendly review of information, even in a bottom-up approach, involvingrepresentativesofstakeholdersfromdifferentcountries;
• aprogressiveintegrationofinformationthroughoutthelifespanoftheproject.
ObjectivesoftheFrameworkevaluationaretoassess:
1. Theusabilityofitsfunctionalities(asidetothesupportoftheWizard/Guidelines);2. Therepresentativenessoftheinformationcontainedintheframeworkand3. Theusefulnessinthedesignofcompensativelearningmodules.
Details about the related indicators are provided in Table 3. Specific Questionnaires and Interviews toevaluatetheefficacyandusabilityoftheFrameworkaswellasfortheGuidelines/WizardwillbedesignedinT6.1alsoincludingtheresultsofthepreliminaryevaluationoftheframeworkusabilityperformedatM14andtherelatedquestionnaire(D3.1.1).
SincetheFrameworkhasbeenimplementedasaweb-basedplatform,theguidelinessupportingdifferentstakeholdersintheuseoftheFrameworkhasbeenreleasedinT3.2asanonlineWizard.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 11of73
The Wizard is a user-friendly interface allowing for a personalized "step-by-step" access to thefunctionalities of theweb-based framework. In this way different stakeholderswill be supported in theaccess,use,andnavigationoftheFrameworkandinperforming3mainactivities,asdescribedinD3.2.1byfourUseCases:
• Case0:Registration/AccessPersonalization• Case1:Frameworkconsultation(queries)• Case2:Frameworkintegration(dataentry)• Case3:InstructionalDesignofcompensativemodules
Case0andCase1hasbeen implemented in its first releaseandwillbeevaluated following the timelinethatwillbeprovidedinTask6.1.
Case 2will be implemented by the end of the second year of the project, in order to allow frameworkreview/integrationinthethirdyear.Beforeitsrelease,itwillbetestedasplannedinT6.1.FeedbacksfromuserswillbecollectedbyM32inordertopossiblyrefinefeaturesofCase2forthefinalrelease.
Case3isrelatedtothesupportwhichtheWizardcouldprovidetothedesignofnationalcurriculastartingfromtheirrealcontext,takingintoaccountcoursesandinitiativesalreadyimplementedinthecountryandspecificlocalrulesandcertifications.
Specificguidelinesfor“CARESSFramework-supported” instructionaldesignhavebeendelivered inD3.3.2intermsof“DesignPatterns”,i.e.“recurrentsolutions”whichcouldbeusedbyVETinstructionaldesignersand teachers when approaching the challenge of designing a Compensative Learning Module to fill aspecific“competencegap”thataffectsacertainHHCP(seefordetailsD3.3.2).
CARESSDesignPatterns,togetherwiththeCARESSIDTemplate(seeD3.3.2)willbeatthebaseof"Case3"andwillbeintegratedintotheWizardinitsfinalrelease.BeforeimplementingthesedesigntoolsintotheWizard,ithasbeendecidedtoevaluatethemwiththeCARESSpilotdesignersduringT3.4(PilotDesign);tothisend,boththeDesignPatternsandtheCARESS IDTemplatehavebeenusedbythepilotdesigners inT3.4toformalizeandsharethedesignofnationalpilotsFeedbacksgatheredfromthistestingbeusedtorefinetheinitialpatternsandtemplatewhichwillbeimplementedintheWizardinterface
7.2 Evaluationofthepilots(T6.3)Nationalpilotswillbedesignedandimplementedinthreecountries,targetingdifferentHCCPs.
1. SpanishPilotcontextdescription:ThedegreeofNursing, inordertoachieveallofthecompetenciesprovidedwillberequiredtocomplete240ECTS (eachECTS,byagreementof theGoverningCouncilof theUVa,willhaveanequivalenceof25hours), which will be distributed as follows (Distribution in accordance with the guidelines of the RD1393/2007):
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 12of73
· 60ECTSforbasictraining,84ECTSforthecompulsorysubjects· 6ECTSforelectivecourses· 84ECTSforthepracticum· 6ECTSforthecompletionofadegreedissertation
TotalcreditsECTS: 240
Typeofcourse
BasicTraining 60
Compulsory 84
Elective 6
Externalinternships 84
Degreedissertation 6
Table1.ECTScreditsinnursingcareerinSpainaccordingtoeachtypeofcourse
For 2016/2017 there are 57 universities in Spain with Faculties of Health Science, Faculties of Nursing,FacultiesofMedicineorUniversitySchools standingaDegree inNursing,with4750newopeningsbeingoffered. Nevertheless,when considering specifically the homecare assistance of the elderly, several skillgapshavebeenfound.Theseskillgapsareanobstacleandmaketheinvolvementofthenurseswiththisquestiondifficult.Theseskillgapsare:
● Socio-emotionalneeds. ● Administrativeoperationsandsanitarydocumentation. ● Communication. ● Relationswiththeworkequipment. ● Environmentandriskpreventionintheworkplace. ● TraininginEthics. ● Basicknowledgeaboutsocialandhealthservicesorganizationsandnetworks. ● Basicknowledgeoflegalframeworkandhumanrights. ● Useofinformationandmonitoringtools. ● Useofcomputersystemsforsocialparticipation. ● Useofcomputersystemsforhealthmonitoring. ● Informationtocustomersaboutreliableservicenetworks. ● Competenciestocoordinateotherprofessional’stasks. ● Competenciesformentalhealthassessment. ● Competenciesforuserempowerment. ● Supportindepressionordespaircases.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 13of73
● Theneedofsupporttooperatetechnologicaldevicesforhomehealthmonitoring. ● Theneedtosupportusersintheself-managementofhis/hermentalhealth. ● Maintainsocialandfamilyrelations. ● Solvingdailyproblemsorknowingwheretoseekforhelp. ● Significantimprovementincoordinationcapacitytoplanactions. ● Meetings with health teams, and keeping in touch with other professionals who also deal with
eldercare. ● Reportsontheactivitiescarriedout. ● PersonalAssistancePlanning.
Asaconsequence,thecoursewillbeorientatedtoallthegraduatednurseswith6EQFLevelthatmayhavetheinterestinimprovingtheirknowledgeandskillsinelderlyhomecare.Forthiscourse,previouspracticalexperienceonthefieldwillnotberequired.Finally,inrelationtotheICTcompetenciesbeingfamiliarwiththeuseofanonlineplatformwillbeadvisable.
2. ItalianPilotcontextdescriptionItalianpilotwilltargetOSS(SocialHealthOperators),whichareHealthandSocialOperatorswhoperformasubsidiaryroleinnursingcare(EQF4).Atthemoment,characteristicsandlearningobjectivesofcoursesforbecomingOSSareless-structuredandless-defined.
3. FinlandPilotcontextdescriptionFinland pilot will target practical nurses (EQF4). They must command value competence, occupationalethics,toleranceandproblem-solvingskills inthefield, inadditiontonursingandcareskills;skillsgaponthesetransversalcompetenceshasbeendetectedinFinland.
According with the project, the implementation of the training courses has three different spaces orapprenticeshipareas:BlendedlearningEachVETpathwaydesignedwillbeimplementedinablendedlearningsituationinItaly,Spain,andFinland.Table2 illustrates the location inwhich thepilotswill be carriedout, the target, aswell as thepartnersinvolved.
Blendedcourse
Location Target Coordinator
Genoa(Italy) Healthsocialoperators
VittorioEmanueleII.Professionalschool.
Valladolid(Spain) Nursingstudents FacultyofNursingandMedicine.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 14of73
Espoo(Finland) Practicalnurses OMNIA,multi-sectoreducationproviderandregionaldevelopmentcentre
Table2.Blendedcourseslocation,targetandCARESSpartnercoordinator
Presence learning should be based on different methodologies (such as education in the workplaceenvironment,school/workturnover,etc.)andstrategies(i.e.,casestudies,criticalincidents,problem-basedlearning,etc.).E-learningSpecific learning activities will be set up in the e-learning course, fostering learning with differentmethodologies(i.e.,casestudy,criticalincidents,roleplaying,etc.).Learning activities will be supported by the presence of both technical and didactic tutors: an emailticketingsystemwillbeimplementedfortechnicalquestions,whileeducationalissueswillbecoordinatedandmanagedwiththesupportofqualifiedexpertsfromeacheducationalsetting.Tosupporte-learningdelivery,ane-learningplatformwillbedevelopedwithMultilingualOpencontents.Informal/nonformallearning:CARESSVirtualCommunityofPracticeThe CARESS Virtual Community of Practice Platform will be integrated with the e-learning platformdeveloped, so as to facilitate the transition of students from formal to informal/non-formal learningactivities.Thegoalof theevaluation is tosupport theCARESSresearchprocess,byknowinghowthedesignof thepilots has been done, how the pilots’ training has been carried out and whether this training programanswers to the purpose of the Caress project. Thus, CARESS pilots will be conducted with the aim ofovercomingthegapsbetweenjobmarketandlearningprogramsfordifferenthealthcarepractitioners.Thecourseswill be evaluated attending its learning design, specific goals, learning outcomes, the quality ofmaterialsandactivities,aswellasteachers’andstudents’satisfaction.Tobeevaluated,eachnationalpilotwill havea referenceperson to collect the informationandmanageevaluationissues.Thispersonisintendedtobelocatedineachtrainingcentre.Themainkeyissueswheretheevaluationwillbefocused,arethosedescribedintheCARESSproject.Theaimoftheevaluationprocessistocollectdata(fromaqualitativeandquantitativepointofview)toanalysethosefactorsandcontributetothepurposeoftheproject.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 15of73
7.3 EvaluationtaskswithinCARESSproject:generaloverviewInordertobettercontextualisetheevaluationwithoutlosingsightoftheobjectivesofthis,wehavemadeup a table summarising the aims, criteria, sub-criteria and their link with indicators and evaluationinstruments. Moreover, we provide some insight into the status of developing of different evaluationinstrumentswithin CARESS project. Thus, it can see how some instruments are showed as drafts in thesectionofannexesinthisdocumentmeanwhileothers’arestillunderdeveloping.Resuming the different phases of the evaluation process, that will take into consideration the differentcomponents of the CARESS project. The partners will work to evaluate
• (T6.1)theFrameworkandGuidelinesaredeliveredinWP3• (T6.3)thepilotstudies,thoughtthecollaborationwiththeWP8thatwillprovidetheindicatorsof
thequalityassuranceoftheproject• theVirtualCommunityofPractice(T6.3)thatwillbeevaluatedbothduringthepilotactivitiesand
afterthem,monitoringitsefficacyandsustainmentafterthelaunchphase.As described in D3.6 the VCP design process include several steps. To promote the sustainability of theCARESSVCPafterthelaunchperiod,theuserwillbeinvolvedinnewrolesandresponsibilities.TheuseroftheVCPcouldbe involvedassumingdifferentroles includingVCPMemberandVCPSeniorMember(VCPMemberpromotedbytheModeratorortheTechnicalAdministratortobemoderatorofaspecificgroupwithin the community). During the launch phase of the VCP, that will take place during the pilots, thestudentsinvolvedwillbeenrolledintheVCPasVCPmembers.Aftertheendofthepilots,tomaketheVCPself-sustainablethestudentswillbe invitedtoassumetheroleofVCPseniormembersandstimulatetheinvolvementandtheengagementoftheVCPmembers.Thesuccessindicatorsaredesignedalsotodetectthe efficacy of the VCP to be self-sustainable through the involvement of the VCP members in thecoordinationoftheworkofthesubject.The indicators marked in bold (see Table 3) are those collected from the short and long term CARESSoutcomeindicators.ThischartisintendedtobeaguidetothedifferentelementstobetakenintoaccountwithintheevaluationtasksinWP6.
NB:IndicatorsinboldaretheSUCCESSINICATORSoutlinedintheprojectproposal
EVALUATIONOBJECTIVES
CRITERIA SUBCRITERIA INDICATORS EVALUATIONINSTRUMENTS
STATUS TIMINGofEVALUATION
FRAMEWORK
EFFECTIVENESS(T6.1)
Supporttothedesignof compensativelearningmodules
Usefulness of the dataincludedintheframeworktosupportID
- Feedbacksfrompilotdesigners- Quality level of pilots’ IDdocuments
Annex9
Questionnaires andinterviews targeting pilotdesigners tobedefined inT.6.1
DraftD.6.2 During/after pilotsdesign
- OverallqualityevaluationoftheItalianpilot
- Overall quality evaluation ofSpanishpilot
- Overall quality evaluation ofFinnishpilot
Annexes4,5,6,7,8,10,11,
12
Draft.
D6.2
Atthebeginning/endofpilots
Usability of theframeworkfunctionalities (asidetothesupportoftheWizard)
UsabilityofFreeSearch Positive feedback fromrepresentatives of VET providersaboutclarityofusers
Annex9
Questionnaires andinterviews involving usersrepresentatives to bedefinedinT.6.1
DraftD6.2
During/after pilots’design
For final refinement(M34)
Usability of dataintegration
Questionnaires andinterviews involving usersrepresentatives to bedefinedinT.6.1
To bedeveloped
During/after pilots’designFor final refinement(M34)
Representativeness Representativeness of EUcountries
- Number of EU Member Statesrepresented in the framework(>75%)
- Coverage of occupations in thefield of homecare defined byESCO(>75%)
Frameworkanalysis Firstrelease(M15)
Finalrelease(M34)
Compliance with EUstandards/tools
- CompliancewithECVET- CompliancewithEQAVET
ECVET/EQAVET Expertsevaluation
To bedeveloped
Firstrelease(M15)
Finalrelease(M34)
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 17of73
EVALUATIONOBJECTIVES
CRITERIA SUBCRITERIA INDICATORS EVALUATIONINSTRUMENTS
STATUS TIMINGofEVALUATION
WIZARDEFFECTIVENESS
(T6.1)
Supporttothedesignof compensativelearningmodules
Effectiveness of DesignPatterns
- Feedbackfrompilotdesigners- Quality level of pilots’ IDdocuments
Annex9
QuestionnairesandinterviewstargetingpilotdesignerstobedefinedinT.6.1
Draft.6.2 During/after pilotsdesign
Effectiveness of theCARESSIDTemplate
- Feedbackfrompilotdesigners- Quality level of pilots’ IDdocuments
Annex9QuestionnairesandinterviewstargetingpilotdesignerstobedefinedinT.6.1
Draft.6.2 During/after pilotsdesign
Effectiveness of Wizardfunctionalitiesfor ID(Case3)
- Feedback of representatives ofend-users
Interview/questionnairetargeting representativesofend-users
To bedeveloped
After the first release ofCase3
For final refinement(M34)
WizardUsability Usability ofRegistration/Personalization
(Case0)
- Feedbackofpilotdesigners- Positive feedback ofrepresentativesofVETprovidersaboutclarityofuse
- CompliancewithEQAVET
Questionnaires andinterviews involving usersrepresentatives to bedefinedinT.6.1
To bedeveloped
After the first release ofCase0
For final refinement(M34)
Usability of Free Search(Case1)
- Feedbackofpilotdesigners- Positive feedback ofrepresentativesofVETprovidersaboutclarityofuse
- CompliancewithEQAVET
Interview/questionnairetargeting representativesofend-users
To bedeveloped
After the first release ofCase1
For final refinement(M34)
Usability of dataintegration(Case2)
- Feedbackofpilotdesigners- Positive feedback ofrepresentativesofVETprovidersaboutclarityofuse
- CompliancewithEQAVET
Interview/questionnairetargeting representativesofend-users
To bedeveloped
After the first release ofCase2
For final refinement(M34)
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 18of73
Usability of Support toInstructional Design (Case3)
- Feedbackofpilotdesigners- Positive feedback ofrepresentativesofVETprovidersaboutclarityofuse
- CompliancewithEQAVET
Interview/questionnairetargeting representativesofend-users
To bedeveloped
After the first release ofCase3
For final refinement(M34)
Representativeness Compliance with EUstandards/tools
- CompliancewithECVET ECVETExpertsevaluation To bedeveloped
Firstrelease(M15)
Finalrelease(M34)
Representativeness of EUcountries
- 4 languages: English, Italian,FinnishandSpanish
Finalrelease(M34)
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 19of73
EVALUATIONOBJECTIVES
CRITERIA SUBCRITERIA INDICATORS EVALUATIONINSTRUMENTS
STATUS TIMINGofEVALUATION
QUALITYOFPILOTS–(T6.3)
PresenceLearning
Students’performance
The participation andstudents’performance.
- Achievementleveloflearningoutcomes defined accordingtoECVET
Annexes 1, 2, 3,7,8 10,11,12
Draft.6.2 During/ at the end ofpilotsenactment
Students’satisfaction
The participation andstudents’performance.
- Achievementleveloflearningoutcomes defined accordingtoECVET
Annex7,8,10
Draft6.2 During/ at the end ofpilotsenactment
Impact of thelearningprogram
Learning outcomes ofthe pilots fit bettermarketneeds.
- Achievementleveloflearningoutcomes defined accordingtoECVET
- Responsiveness to jobmarketneeds
Annexes8,9 Draft6.2 During/ at the end/six-months posttraining impact, ofpilotsenactment
Quality of learningmaterial
Quality of the learningdesignsandmaterials
- The quality of trainingmaterials
Annexes4,5 Draft6.2 During/ at the end ofpilotsenactment
Teachers’competencies
Teacher’s competenciesandskills.
- Competence andeffectivenessofteachers
- Cost-effectiveness- Presence ofmultiple trainingmethodologies
Annex2,3,9,10 Draft6.2 During/ at the end ofpilotsenactment
Teachers’effectiveness
Teacher’s competenciesandskills.
Annex7,8,11,12 Draft6.2 During/ at the end ofpilots enactment andsix-months posttrainingimpact
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 20of73
EVALUATIONOBJECTIVES
CRITERIA SUBCRITERIA INDICATORS EVALUATIONINSTRUMENTS
STATUS TIMINGofEVALUATION
QUALITYOFPILOTS(T6.3)
e-learning
Students’performance
The participation andstudents’performance.
- Learningoutcomesdefined ininstructional design documentsareaddressed
- Platformstatistics.- Functionalities of the platform
allow for implementing all theactivities envisaged by theinstructionaldesigndocuments
Analysis of data log filesthat will be trackedautomatically by thesystem
Annex1,2
Annex7,8,10
Draft6.2 During/ at the end ofpilotsenactment
Students’satisfaction The participation andstudents’performance.
- Positive feedbacks of usersaboutclarityandusability
Annex7,10 Draft6.2 At the end of pilotsenactment
Impact of thelearningprogram
Rateof job findingafter6months post training oftargetedstudents
- Learning outcomes of the pilotsfitbettermarketneeds.
- Career improvement/jobfinding of practitionerstargetedbythepilot.
- Responsivenesstojobmarketneeds
Annex11,12 Draft6.2 Six-months post trainingimpact
Quality of learningmaterial
Quality of the learningdesignsandmaterials.
Non-formal and InformalLearning.
- Positive feedbacks of usersabout clarity and usability oflearningmaterials
Annex4,5 Draft6.2 During/ at the end ofpilotsenactment
Teachers’competencies
Teacher’s competenciesandskills.
Sameaspresencelearning Annex2,3,9,10 Draft6.2 During/ at the end ofpilotsenactment
Teachers’effectiveness
Teacher’s competenciesandskills.
Sameaspresencelearning Annex7,8,11,12 Draft6.2 During/ at the end ofpilotsenactment
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 21of73
E-learning platformusability
The uploaded contents(documents or othermultimedia content), theuseofaccessingresources(e.g.,watchingthisorthatvideo);uploadingcontents(this or that document);the active participation incertain communities orvirtual meetings (postingmessages, joining ateleconference).
- Opencontentsavailableforfreeonthee-learningplatform- Platformstatistics
Analysis of data log filesthat will be trackedautomatically by thesystem
SeepossibleindicatorsinTable4.
During/ at the end ofpilotsenactment
QUALITYOFPILOTS-(T6.3)
CommunityofPractice
Members’performance
Platformstatistics
- Uploadedcontents- The use of accessing
resources- The active
participation incertain communitiesorvirtualmeetings
- NumberofpractitionersinvolvedintheVCP;- Level of activity, i.e. level ofinteractionsamongmembers- Connections with professionalassociations;- Connections with final and endusers
Analysis of data log filesthat will be trackedautomatically by thesystem
Annex7,10
SeepossibleindicatorsinTable4.
Draft6.2
Attheendofpilots
M34
Members’satisfaction
The effectiveness of VCPthrough the membersatisfaction
- The meeting of the community’sinitialobjectives.
- The added value provided to thetarget as a group, in terms ofnew/enhanced organization, tools,opportunities derived from thebelongingtotheVCP
Annex7,10
Draft6.2 Attheendofpilots
M34
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 22of73
2VCPMemberpromotedbytheModeratorortheTechnicalAdministratortobemoderatororaspecificgroupwithinthecommunity
Seniors members2activityeffectiveness
The scaffolding providedby seniors members andmoderators
Presence or not of the followingqualitycriteria:
- Senior members provide withincentives to increase theparticipation
- Senior members promotemotivating situations within theVCPtoincreaseparticipation
- Senior members provide withusefultools
- Senior members provide withnetwork situations andopportunities to increase andmotivatethe internalandexternalparticipation
- Interactions and networks arestimulatedbothatanationallevel,with the support of nationalpartners,andatinternationallevel
- Inter-professional exchanges andcontacts are being fosteredthrough theVCPparticipation andthecreationofinterest’sgroup
- The senior member promotes,motivateandmoderatethegroupshim/herhaveassigned
Interview/questionnairetargeting representativesofend-users
To bedeveloped
M34
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 23of73
EVALUATIONOBJECTIVES
CRITERIA SUBCRITERIA INDICATORS EVALUATIONINSTRUMENTS
STATUS TIMINGofEVALUATION
QUALITYOFPILOTS-(T6.3)
CommunityofPractice
VCP Platformeffectiveness
The scaffolding providedby platformstructure/functionalities
Presence or not of the followingqualitycriteria:
- Theplatformprovidesuseful toolsenhancing participation andnetworking
- The platform allows for aneffective management ofinteractions and networks both ata national level at internationallevel
- The platform allows for aneffective management of inter-professional exchanges andcontacts
Attheendofpilots
M34
VCP platformusability
Positivefeedbacksofusersaboutplatformusability
- Functionalitiesoftheplatformallowfor implementing all the activitiesenvisaged by the instructionaldesigndocuments.
Questionnaire/
Interview.
To bedeveloped
During/ at the end ofpilotsenactmentandsix-monthsaftertraining
Effectiveness ofcompetencerecognitiontools
Effectiveness ofDiary/portfolio
- Achievement level of learningoutcomes defined according toECVET- Thebenefitsprovidedtoindividualssuch as learning/job/networkingoccasions derived from thebelongingtotheVCP
Annex11,12 Draft6.2 During/ at the end ofpilotsenactmentand sixmonthsaftertraining
Effectiveness of tracking/self-reporting
Achievement level of learningoutcomes defined according toECVET
Analysis of data log filesthat will be trackedautomatically by thesystem
To bedeveloped
During/ at the end ofpilotsenactmentandsix-monthsaftertraining
Table3.GeneraloverviewofevaluationtaskswithinCARESSproject
8 CONTEXTANDEVALUATIONMETHODINT6.3Withtheaimofassessingtheevaluand,threestudiesinauthenticscenarioswillbecarriedoutfromMay2017untilFebruary2018.ThesestudieswereformalisedusingtheEREMevaluationmodel(seesection7)andinvolvedrealendusers,likeeducatorsandstudents.Weaimtoadeeperunderstandingoftheparticularphenomenaunderstudy, inourcase,thedesignandtheenactmentofthreenationalpilotsinVETtofosterstudentsofnursing,OSSandpracticalnurse’sskillsandcompetences inelderlyhomecare.Besides, thisevaluationproposalwill showhowteachers supportlearningactivities in face to face,on-lineandnon-formal and informal courses, andallows todetect thebenefitsandlimitationstoassessthequalityandtheefficacyoftheenactedpilots.The aforementioned issue 2 (Can training courses based on competences and active methodologies behelpfulto improvenurses/OSSandpracticalnursesclinicalpracticeandtransversalskills inelderlycare?)willbeexploredthroughinformativequestionsgroupedintoseveraltopics,bothdefinedbyusatthestartoftheevaluation.Alsoemergenttopicswillbetakenintoaccountwhilegatheringandanalysingdata,ina“progressivein-focusapproach”(Stake,2010).Thesetopicscanbeillustratedbythefollowing:Topic1:InstructionaldesignoftheVETcourses:toevaluatethedesignofthecourse,weaimtoanalysetheinstructionaldesigndocumentsforformal,non-formalandinformallearningaccordingtothenationalVET pathways targeting Social Health Operators, practical nurses and registered nurses. Thus, in thisparticularway,weneedtoknowwhetherthecontentsarewell-suitedtothecourse’sobjectives.Also,wewillgatherinformationabouttheinstructionalstrategiestobeused,aswellastowhatextentthedesignofthecoursesacrosscountries isgoing to followtheEQAVET indicators.Finally,wewill collect informationaboutstudents’ impressionsaboutthedesignofthecourse, their learningexpectationsand if thedesignandthecontentsofthecoursearewellsuitedtotheneedsoftheworkmarket.Topic 2: Participation and students’ performance: we pretend to gather information regarding theenactment of the pilots. We will ask students about the course contents, activities, methodologiesemployedaswellastheirperceptionsabouttheexperienceandsatisfactionwiththecourse.Furthermore,wewillfocusonunderstandingifthecoursesperformedhavecontributedinsomewaytoovercometheirlearninggaps.Wealsowillaskaboutthecourseutilitytocovertheneedsperceivedintheirtraining/job.Ontheotherhand,wewillgatherdataabouttheteachers’perceptionsofthepilotcarriedout.Topic3:Teacher’scompetenciesandskills,wherewepretendtoputemphasisinevaluatingspecificissuessuchastheadequateknowledgeonthesubjectshowedbyteachers/tutors,theclarityoftheexplanations,theirlevelofpreparationofatopic,theirinteractionswithstudentsregardingquestionsandissuesraisedduringthetraining,etc.Topic4:Qualityandusefulnessoftrainingmaterials:Qualityandusefulnessoftrainingmaterials:thatwillbe used in the training courses from the participants and the teacher's point of view in the learningexperiences.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 25of73
Topic5:NonformalandinformallearningthroughCARESSVCP:avirtualcommunityinhealthcarerefersto a “group of people (and the social structure that they collectively create) that is founded ontelecommunication with the purposes of collectively conducting activities related to health care andeducation” (Demiris,2006,p.179). Suchactivitiescanbesupportedbyaplatformaimedatallowhealthprofessionalstodiscussonelderlycareissuesandproblems,sharingdocumentsaswellasconsultingwithexpertsandsustainingrelationshipsbeyondface-to-faceevents.CARESSVCPplatformwill supportnon-formal and informal learning recognitionaccording to threemaintypesofevaluationtoolsandfunctionalities(seeD.3.6):a)Usertrackingfunctionalities:wewillgather informationcomingfromtheanalysisofdata logfilesthatwillbetrackedautomaticallybythesystem.Thisway,weobtaineddataforevaluationpurposesfromthefinalusersoftheCARESSe-learningplatformandVCPusers.Itisworthnoticingthatdatacouldbeexportedinstandardformatswiththeaimofmakingitaccessibletothepilots’teacherstohelpthemintheidentificationofcompetencerecognition.Apotentialtrackingdatacouldbetheuploadedcontents(documentsorothermultimediacontent),theuseofaccessingresources(e.g.,watchingthisorthatvideo)andtheactiveparticipationinparticularcommunitiesorvirtualmeetings(e.g.,postingmessages,joiningateleconference).Table 4 illustrates some potential tracking data that could be obtained from the use of the e-learningplatformaswellasfromtheVCP.Furthermore,annex13providesadetaileddescriptionaboutthetrackingindicatorsthatcanbeobtainedfromtheCARESSe-learningplatform.
Trackingindicators VirtualsettingVirtual Community ofPractice
e-learningplatform
Uploadedcontents(documents,multimedia,etc.) x Accessingresources(numberofvisits,lastaccessed)
x x
Postingmessages x xJoiningateleconference x Activitysubmissions xNºofEventsubscriptions x Enrolledstudents x xBadgesachieved x Create/Joinagroup x Inviteuserstoagroup x Nºofsearchings x Nºofinvitationstojoinagroup x Nºofcreateddiaries x Nºuploadedcontents(documents,multimedia,etc.)inthegroupmoderated
x
Accessingresources(numberofvisits,lastaccessed)tothegroupmoderated
x
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 26of73
Nºofusersthatjoinedagroup x Nºofteleconferencecoordinatedinagroup x NºMessagesPostedinagroup x NºActivitysubmittedinagroup x NºEventsubscriptedinagroup x
Table4.TrackingdatafromVCPandCARESSe-learningplatform
b)VCPexperiencediary/portfolio: is anorganised collectionofmaterials that presents and verifies skillsandknowledgeacquiredthroughexperience.AccordingtoCedefop,2015,“Itisofparticularrelevanceforvalidatingnon-formalandinformallearningbecauseitallowstheindividualcandidatetocontributeactivelytothecollectionofevidenceandalsooffersamixofapproachesstrengtheningtheoverallvalidityoftheprocess”(p.61).Aportfoliomightincludedocumentssuchasabstracts,performanceappraisals,referencefromcurrentandpastemployers,etc. Inthecontextofthisevaluation,anexperiencediary/portfoliowillbeused toallowstudentsdescribehis/her thoughtsandexperienceswithin theVCP (e.g. lessons learnt,discussions,createdcontacts,activityreflections,etc.)AproposalofthistoolwillbepresentedinD.5.1.2“IntegrationofVCPplatformwithe-learningplatform”.Thistoolwillbeusedforevaluationpurposeswiththeaimofcollectingdatafromfinalusersabouttheirpersonalexperiencesrelatedtoelderlycareinanon-formalandinformalcontextaswellastoassesstheircompetenceacquisition.TheVCPexperiencediarywillallowforapersonaldescriptionoftheuserexperienceasamemberoftheVCP.TheVCPmembercouldfill-inperiodicallyadiarywherehe/shecoulddescribehis/herownexpertiseintheVCP(e.g.,activities,discussions,lessonlearnt,createdcontacts,etc.).Theexperiencediarycouldbeprovidedwith questions or suggestions to encourage the textual description of the personal experienceandthenewknowledgeachieved.Thechronologicalversioningoftheresultingdiaryshouldbetrackedandpresentedinaformattoallowtobeanalysedforevaluationpurposes.Besides,itispossibletocreatesometemplatesintheVCPthatmaymakeiteasiertodescribetheuser’spersonalexperience.Also,thesoftwarethatwewillusefortheplatformallowstheissuingof“badges”totheparticipantswhencarrying out specific tasks in the platform. This can be used for generating an “activity report” of theparticipantthatcanbecomplementedwithinformationprovidedbytheparticipantthroughtheirreflectivediary, information that couldbe labelled tohelp furtheranalysis. In addition, the categories let theuserfilter the entries depending on different categories. These categories could describe just personalexperiences or could be aimed at competencies recognition. The use of a blog-like activity (where theparticipants would provide links to, e.g., the artefacts created during their interaction with the VCPplatform)couldhelptoanalyseaparticularsuccessindicatorfortheVCP.c)VCPusers’acceptanceandusage:forevaluationpurposes,wewillusetheTechnologyAcceptanceModel(TAM)(Davis,1989)toassessabouttheusersCARESSVCPperceivedusefulnessaswellastheirperceivedease of use. The Technology Acceptance Model was developed to predict and explain, informationtechnology acceptance and usage. Themain topics to be look at according with thismodel are systemquality, service quality, and perceived usefulness, perceived ease of use and user satisfaction with thesystem.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 27of73
Moreover, we will use the “Think Aloud Protocol (TAP)” as a research strategy commonly used inpsychologicalandeducationalresearch(Lewis,1982;Charters,2003).TAPprotocolisdevotedtoinvolvingparticipantsthinkingaloudmeanwhiletheyareperformingasetofspecifiedtasks.Participantsareaskedtosaywhatevercomesintotheirmindastheycompletethetask.Thismightincludewhattheyarelookingat, thinking, doing, and feeling. This gives observers’ insight into the participant's cognitive processes(rather than only their final product), to make thought processes as explicit as possible during taskperformance. To provide evidence of the user's acceptance and usage of the CARESS VCP partnersresponsible for the evaluation issues will need to design a process, which takes into account someconcerns,byselectingasuitabletask.Accordingtotheformalresearchprotocol,allverbalizationswillbetranscribedandthenanalysed.It is worth notice that mix method will be used as a strategy for collecting information from differentsourcesanddifferentmoments.The finalaim is tohaveamorecompletepictureof thePilotsworkandtheir strengthsandweaknessesasaneducationaltoolformanagingtheCARESSprojectgoalofovercometheskillmismatchinhealthcareprofessionals.Figure 3. Connections between data gathering techniques, evaluation happenings and topics.
According to the aforementioned ideas, the EREM highly encourages to use mixed data gatheringtechniques and multiple informants in order to obtain different perspectives about the evaluand, thusenriching theevaluationprocess (Green,Benjamin&Goodyear, 2001). Specifically, in this evaluationwe
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 28of73
will combinequantitativeandqualitativedatacoming from4 typesof informantsandsources (teachers,students, CARESS Virtual Community of Practice and CARESS e-Learning platform), using different datagathering techniques such as interviews, questionnaires, log file analysis, observations and reflectivediaries,aswecanseeinFigure3.Then,evidencecollectedfromdifferentsourcesand informantswillbeanalysedfollowingatriangulationapproach (Guba, 1981), in order to reach the global conclusions of this evaluation, increasing thetrustworthinessofthefindings.Likewise,wewillperformananalysisofthecontentofthedocumentsprovidedbythepilots’instructionaldesigners, the transcriptions of the interviews, the reflective diaries, the observations of the pilots’enactment and the open questions coming from the questionnaires. A quantitative analysis will beperformedusingdatacomingfromtheLikertquestionnairesaswellasfromthelogfilesobtainedfromtheCARESSVCP,asitisillustratedinFigure4.
Figure4.GraphicrepresentationofthedatagatheringandanalysistechniquesflowduringtheevaluationofCARESSpilotsinVET.
Thesupportingtechnologiesthatwillbeusedduringfacts(e.g.teacherdesignsandenactmentsoftheVETpilots)willincludeNVIVOqualitativeanalysissoftware(QSR,2008),MSExcelforquantitativeanalysisandGoogleformsforweb-basedquestionnaires.
8.1 DescriptionofthetimelineandtheevaluationinstrumentsBeforewedescribetheevaluationmethodologythatwilltakeplace,wethinkitisimportanttogobrieflyovertheevaluationphasesandprovidedageneraloverviewoftheassessment instrumentsthatwillbeused within the evaluation of CARESS pilots in VET. Furthermore, in this section, we identify a set ofevaluationresponsibilitiesthatmightbeassumedbyCARESSpartnerswithdeclaredeffortinWP6.AsitisillustratedinFigure3,theCARESSevaluationofVETpilotswillbedividedintotwomainmoments.Afirstevaluationhappeningcalled[H1]:Design,wherewewilllookatteachers’coursedesigns,thatwillinformtheDesignofthePilots.Thisphaseispriortothepilots’launch.Asecondevaluationhappeningcalled[H2]:Enactment,whilethePilotsarealreadycarriedout.Inturn,itisdividedinthreemainmoments:
a) The beginning of the Pilot. The evaluation team will collect a student questionnaire, a courseobservationandateacherinterview
b) Midtermof the Pilot. During this period, a classroomobservationwill take place for each Pilotcourse.Also, a TAP (ThinkingAloudProtocol) questionnaire regarding theuseof theE-learningplatformandtheVCPwillbefilledbytheevaluators.
c) FinalofthePilot.Wewillcollectthetrainingmaterials,teacher’sfinalreflectionsdiary,students’and teachers’ final questionnaire, and a final teacher interview. Additionally, we will collect astudentdiaryregardingtheusetheVCP.
Figure 5 illustrates a third evaluation happening [H3]: Post training, in which we will collect data toevaluate the impact after six-month training. This way, we will perform a student's interview and aquestionnaire with the aim of assess the impact of the pilots training considering the professionals’satisfaction,jobimprovementsandcareersteps.
Figure5.Connectionsbetweendatagatheringtechniquesandtopicsduringpost-trainingevaluationphase.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 30of73
Through all those moments, the teachers and students’ log files will be collected to evaluate the e-learning module and the VCP use. Furthermore, statistics will also derive from the online communitythroughwhichteacherswillbeabletocommunicateandinteract,inordertohaveadescriptiveaccountofparticipants’contributions.Table5summarizesthedatagatheringtechniques,informants,topicsandtimeswhentheinformationwillbecollectedaccordingtoFigure3and4.
DATAGATHERINGTECHNIQUES
INFORMANTS TOPICS HAPPENINGS
[Diary1]
Teachers Design H2:EnactmentFinal
[Diary2] Students VCP H2:EnactmentFinal
[Art1][Art2]
Design H1:Design
QualityofTrainingMaterials H2:EnactmentFinal
[Quest1] Students Design H2:EnactmentBeginning
[Quest2][Quest3]
StudentsTeachers
Participationandstudent’sperformanceTeachers’competencyandskillsQualityoflearningmaterials
H2:EnactmentFinal
[Quest4] StudentsTeachers
VCP H2:EnactmentFinal
[Quest5] Students Posttrainingimpact H3:Sixmonthsposttraining
[Ob1][Ob2]
TeachersStudents
Participationandstudents’performanceTeachers’competencyandskills
H2: Enactment Beginning andMidterm
[Int1][Int2]
Teachers
DesignQualityofLearningmaterialsParticipationandstudent’sperformanceTeachers’competencyandskills
H2: Enactment Beginning andFinal
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 31of73
[Int3] Students Posttrainingimpact H3:Post-training
[LogFiles] TeachersStudents
VCPe-learningplatform
H2: Enactment Beginning,Midtermandfinal
[TAP] StudentsTeachers
VCP H2:EnactmentMidterm
Table1.Evaluationinstruments,informantsandhappenings
8.2 Whowillcollectthedata?Therewillbeonepersonresponsibleforeachpresencelearningcourse(VittorioEmanueleII,OMNIAandthe Nurses School of UVA) in charge of collecting the data required using the evaluation instrumentsprovided in this document, according toa time schedule theevaluation coordinatorswill beproviding.The e-learning log files will be collected by GGallery, along with the VCP’s feeding. One responsiblepartnerfromGGalleryandSI4Lifewillbethereferencecontactforevaluationpurposesofthee-learningevaluationandtheVCPs.Eachpersoninchargeoftheevaluationineachcountrywillberesponsibleofprovidingthedatacollectedbytheevaluationinstruments,sotheevaluationcoordinatorscananalyseandwritethereportwiththefinal results. Besides, to facilitate the analysis, we will ask the partners to send the data required inEnglish.Furthermore,membersofAGEwillmakeatleasttwoobservationsofthepilots’enactmentandtheywilltransfertheresultstothecoordinatoroftheevaluationworkpackage.All the instruments showed in the appendix are an initial proposal. The primary structure will bemaintained,althoughtheinstrumentsmayberefinedoradaptedwiththehelpoftheinvolvedpartners’feedback,andoncethecontext’scharacteristicswheretheevaluationwillbetakenwillbebetterknown.Table 6 illustrates the responsible partners according to the different evaluation tasks identifiedwithinthisworkpackage.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 32of73
Evaluationtarget Responsiblepartners Tasks
Presence learning:designandenactmentofCARESSVETpilots.
One person responsible foreach presence learningcourse:
·-VE-II·-OMNIA·-NursesSchoolatUVa
-Collaboration in the definition and refinement of thedatagatheringinstrumentsavailableintheappendixofthisreport.-Translate the evaluation instruments into thecorrespondinglanguages(Spanish,ItalianandFinnish)-Completetheevaluationtoolkit-Sendthe information inEnglishtothecoordinatorsofthisWP.
On responsible person byAGEplatform
-Performat leastoneobservationof theenactmentofeachpilot-Sendthe information inEnglishtothecoordinatorsofthisWP.
One responsible person byOMNIA
-Identifyappropriateindicatorstoevaluatethetrainingmaterials from the pilots according to EQAVETframework.
e-learningcourses One responsible person byOMNIA
-Identifyappropriateindicatorstoevaluatethetrainingmaterials from the pilots according to EQAVETframework
One responsible person byGGalleryOne responsible person bySI4LIFE
-Identify useful indicators coming from the analyticsobtainedfromtheuseofthee-learningplatform.-Report the data coming from the e-learning platformanalyticstothecoordinatorofthisWP.
Virtual Community ofPractice(VCP)
One responsible person byGGallery.OneresponsiblebyUVa-Tech.One responsible person bySI4LIFE.
-Identifyuseful indicatorscoming fromtheanalyticsofthe VCP to assess the final user’s performance withinthisvirtualspace.-Report the data coming from the analysis performedbythecoordinatorofthisWP.
Table2.Evaluationresponsibilities
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 33of73
8.3 DraftoftheevaluationtoolkitinT.6.3The resources available for the evaluation will be the instructional designs and the trainingmaterials.Also,wewilllookatthefinaleducationalplatform,whereweexpecttocollectdataabouttheeducationaland pedagogic process, along with the pilots’ designs. The teacher’s actions and reflections, and thestudents’actionswillprovidedataandmaterialtoevaluatemeaningfulinputs.Themainelementsdesignedfortheevaluationkitare:
•Guidelinestoperformtheobservations•Student’sandteacher’squestionnaires•Guidelinestoperformthestudentsandteachers’interviews•Guidelinestocompletethestudent'sandteacher’sdiaryreflection•Questionnairestoevaluatethetrainingmaterials
Thoseelementswillbeapplyingtoin-personcourses,thee-learningplatformandthevirtualcommunitiesofpractice.Thefollowinginstrumentsaredesignedtobeusedfordatacollectionpurposes.Thetopicstobeanalysedforevaluationpurposesare:
● InstructionaldesignoftheVETcourses.Thisdesignwillbeinformedmainlythroughtheteacher’scoursedesign,theanalysisofthetrainingmaterialsandthefinalteacher'sreflectionsaboutthedesign and the enactment of the pilot. Some aspects of this topic may be enriched by datarecorded for other topics, like the participation and students’ performance, the teachercompetenceandskillsandthequalityoftrainingmaterials.
● Theparticipationandstudents’performance.Thistopicwillbeilluminatedthrough:
○ Thestudents’previousskillsandmotivation○ Thecontentandstructureofthelearningcourses○ Theoverallstudents’satisfactionandtheimpactofthelearningprogram
Wewillgatherdatathroughstudentsandteacher’squestionnairesatthebeginningandtheendof theenactmentphase.Also,datawillbecompletedwith reflectionscollectedwhenanalysingthequalityoftrainingmaterialsandtheteachersandstudentslogfiles,regardingtheuseofthee-learning platform and the VCP. The courses’ activities and knowledge acquisition will beevaluatedthroughobservations,teacherinterviewsaswellasbytheemploymentoftheteacherreflectivediary.
● Teacher’s competencies and skills. This evaluation topic will be observed through courseobservations in the classroom, but also analysing the student’s participation and performance.Teacher interviews, final students’ questionnaire, classroom observation and teacher reflectivediarywillbethemaintoolsfordatacollectingforthisevaluationtopic.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 34of73
● The quality of training materials. Directly linked with the design of the course, final students’
questionnaire,teacherinterview,questionnairesforcourseproducts’evaluationandlogfiles,willprovideuswithevidenceaboutthisissue.
● NonformalandInformalLearning.Thiswillbeevaluatedthroughtheteachers’andstudents’log
files regarding theuseof e-learningplatformand theVCP, the student's’ diary collectedat theendoftheenactmentandtheuseofaThinkAboutProtocol(TAP)withtheaimofevaluatingtheusabilityandacceptabilityoftheVCP.Regardingthistask,thefinalevaluationtoolswillbefullydevelopingoncetheinformationabouttheVCPhavebeencompletedandallthefunctionalitiesandpotentialdatatobeextractedforevaluationpurposeswouldbeanalysed.
● Post-trainingimpact.ThiswillbeevaluatedsixmonthsafterthecompletionofthePilotandwill
helpustoassesstheimpactofthetraining.Wewilluseastudentpost-trainingquestionnaireandaninterviewtogetdatatoilluminatethistopic.
9 QualityAssuranceandOverallProjectEvaluationAccording toRIGA conclusions (June, the22ndof 2015), andaspartof quality assurance systems, it isimportant to establish continuous information and feedback loops in VET systems based on learningoutcomes.ThatwouldincludetheuseofinformationonVETgraduateemployabilityandacombinationofdata on learning, labour market entry, and career. Moreover, the use of outcomes of VET should besystematically used as part of quality assurance systems. As the RIGA document states, to delivervocational curricula, would be necessary to track learners after completing their training, through theidentificationofadequatemeasures,inordertoprovide“feedbackloops”.Therehavebeenalready someexperiencesabout tracking students (e.g. thedevelopmentofGraduateTrackingSysteminCroatia,2013-2016.In2013,theCroatianMinistryofEducationbeganapilotprojectwith theaim to collectexistingdataongraduate students,employedandunemployedpersons). Thesetrackingcouldbedonecombiningadministrativedatawiththeresults,sometimeseasiesttoget,throughtargetedsurveys,feedbacksheetsorcourseevaluations,face-to-facecontactorfocusgroupinterviews.Trackingcanbecomplementedwithothermeasuressuchas interviews, focusgroupmeetings, studentfeedbacksheets,etc.Tracking Learners’ and Graduates’ Progression Paths (2012), states “Tracking should lead to theenhancementofteachingandlearning,andbelinkedtostudentservices.Relevantresultswouldalsobemade available to staff and students, prospective students, policy makers and employers” (pg. 51).Employment ratesofgraduateswithinaspecifiedperiodaftergraduationandstudent feedbackon thequalityofstudyprogrammesandfacilitiescouldprovidefeedbacktoimproveandconfirmthequalityofVETprograms.TheinformationcollectedfromthedifferenttrackingactivitiesdesignedwithintheCARESSVirtual Community of Practice could be combined and used in this “feedback loop”. Thatwill allowussystematicallymonitorthestudentslearningoutcomesanditseffectiveness.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 35of73
Thismonitoringwill take place during the early period after the conclusion of the Pilots and over thetargeted students, during a period of six months. Those students will be tracked using the e-learningplatformandVCPdata,togetherwithstudents’interviewsandquestionnaires.Thefollowinganalysiswillallowustoprovidetheprojectwiththisfeedbacktocontributetotheimprovementofthequalityofthehomecareprogram.
10 BIBLIOGRAPHY
● Abma,T.A.(2001).Evaluatingpalliativecare.Facilitatingreflexivedialoguesaboutanambiguousconcept.Medicine,HealthCareandPhilosophy,4(3),261–276.
● Cedefop(2015).Europeanguidelinesforvalidatingnon-formalandinformallearning.Availableathttp://www.cedefop.europa.eu/en/publications-and-resources/publications/4054 . Last visited2017/04/27.
● Charters, E. (2003). The use of think-aloudmethods in qualitative research an introduction tothink-aloudmethods.BrockEducationJournal,12(2),68-82.
● Curran,V.,Christopher,J.,Lemire,F.,Collins,A.,&Barrett,B.(2003).Applicationofaresponsiveevaluationapproachinmedicaleducation.MedicalEducation,37(3),256-266.
● Demiris,G.(2006).Thediffusionofvirtualcommunities inhealthcare:conceptsandchallenges.Patienteducationandcounselling,62(2),178-188.
● Gaebel,M.,Hauschildt, K.,Mühleck, K. and Smidt,H. (2012) Tracking Learners’ andGraduates’Progression Paths. TRACKIT. European University Association. Available athttp://www.eua.be/Libraries/publications-homepage-list/EUA_Trackit_web.pdf?sfvrsn=2 Lastvisited2017/05/10.
● Greene, J.C.,Benjamin, L.,&Goodyear, L. (2001). Themeritsofmixingmethods inevaluation,7(1),25-44.
● Guba,E.G.(1981).Criteriaforassessingthetrustworthinessofnaturalisticinquiries.EducationalTechnologyResearchandDevelopment,29(2),75-91.
● Jorrín-Abellán,I.M.,&Stake,R.E.(2009).DoesUbiquitousLearningCallforUbiquitousFormsofFormalEvaluation?:AnEvaluandorientedResponsiveEvaluationModel.UbiquitousLearning:AnInternationalJournal,1(3).71-82.
● Koch, T. (1996). Implementation of a hermeneutic inquiry in nursing: Philosophy, rigour andrepresentation.JournalofAdvancedNursing,24,174–184.
● Koch,T.(2000).“Havingasay”:Negotiationinfourthgenerationevaluation.JournalofAdvancedNursing,31(1),117–125.
● Lewis,C.,&Mack,R. (1982). Learning tousea textprocessingsystem:Evidence from“thinkingaloud”protocols.InProceedingsofthe1982conferenceonHumanfactorsincomputingsystems(pp.387-392).ACM.
● Wadsworth,Y.(2001).“Becomingresponsive-andsomeconsequencesforevaluationasdialogueacrossdistance”.NewDirectionsforEvaluation.92.SanFrancisco:JosseyBasspp.45–58.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 36of73
● Widdershoven,G.A.M.,&Sohl,C.(1999).Interpretation,actionandcommunication:Fourstoriesabout a supported employment program. In T. A. Abma, Telling tales. On narrative andevaluation.AdvancesinProgramEvaluation(6)(pp.109–130).Connecticut:JAIPress,109–130.
● RIGA conclusions, on a new set ofmedium-termdeliverables in the field ofVET for theperiod2015-2020, as a result of the review of short-term deliverables defined in the 2010 BrugesCommuniqué. Available athttp://ec.europa.eu/dgs/education_culture/repository/education/policy/vocational-policy/doc/2015-riga-conclusions_en.pdf.Lastvisited2017/05/10.
● Stake,R(2004).Standards-basedandresponsiveevaluation.ThousandOaks,CA:Sage.● Stake,R.E.(2010).QualitativeResearch:StudyingHowThingsWork.NewYork,NY:GuilfordPress
11 ANNEX11.1 Annex1:Guidelinestoperformtheobservations(Draft)
Date:___________________Location:_________________________
Instructor(s)evaluated:________________________________________
Course:_____________________________________________________
Class:________Numberofstudentspresent:_____________________
Observer:___________________________________________________
Instructions: This is an open-ended tool. The observer should take notes regarding the activity that isimplementedduring the lesson. Someof the areasonwhich theobserver should focus arementionedbelow.Feelfreetomodifytheseareasaccordingtotheprogressofthelesson/activity.(a)Abouttheteacher
1. Describetheteachingmethodthattheteacheruses.
2. Makeareferenceontheteachermainskillsthatareusedduringthelesson/activity.
3.Describetheclassroom’sorganization.
4. Observeandtakenotesregardingtheflowofthelesson/activity.
5. Describeteacher’sflexibilityindealingwithunexpectedsituations.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 37of73
6. Indicateteacher’sfeedbacktothestudents.
(b)Aboutstudents
1. Howdostudentsuseandinteractwiththeteacher’spedagogicalmaterials?
2.Describetheiractivities.
3.Describetheirbehaviour.
4. Describethestudents’skillsthataredevelopedduringthelesson.
Time Data(Whathappensnow?) Interpretations/Comments
00:00
00:05
00:10
00:15
00:20
11.2 Annex 2: Guidelines to complete the teacher’s Reflective Diary(Draft)
Country:
Teacher:
Course:
Starttime:
Endtime:
Date: Namethreethingsthatimpressedyouthemostaboutthislesson:_______________________________________________________________________________________________________________________________________________________________________________________________________________
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 38of73
Namethreedifficultiesthatyoufaced: _______________________________________________________________________________________________________________________________________________________________________________________________________________ Whathasworkedwellduringthelesson? _______________________________________________________________________________________________________________________________________________________________________________________________________________ Additionalcomments: _______________________________________________________________________________________________________________________________________________________________________________________________________________
11.3 Annex3:Guidelinestoperformtheteacher’sinterview(Draft)Annex3A.InitialInterview. 1.Whatisyourmainmotivationregardingtheteachingofthiscourse?
2.Areyoupleasedwiththeinstructionaldesignprovided?
3.Doyouthinkitwillbeeasytoimplementtheinstructionaldesign?
4.Whatisthebiggestdifficultyyoufaceregardingthedesignofthispilotcourse?
5.Whatdoyoubelieveisthebiggestadvantageofthisthepilotcourse?
6.Haveyouanypreviousexperienceinteachinghealthcare?
7.Doyouthinkyourcompetencesandexperiencearesufficienttoteachthistrainingcourse?
8.Whatisyourmaingoalregardingtheteachingofthispilot?
9.Howfamiliarareyouwithevaluatingyourstudentsintermsoftheirlearningoutcomes?
10.Inyouropinion,whichwillbethedifficultiesthatthestudentswillfaceduringthepilotcourse?Name3
11.Whatdoyouthinkarethestrongpointsofthelearningdesign?Andtheweakpoints?
12.Isthereanythingelseyouwouldliketosharewithus?
Annex3B.FinalInterview.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 39of73
1.Didyouconsiderthetrainingpilotcoursehelpyoutofillsomeneedoftraininginthefieldofhomecareeducation?
2.Wereyoupleasedwiththeinstructionaldesignprovidedduringthetrainingcourse?
3.Wasiteasytoimplementtheinstructionaldesign?
4.Whatwasthebiggestweaknessofthepilotcourseenactment?
5.Whatwasthebiggestadvantageofthepilotcourseenactment?
6.Didthestudentsseemtobehappy/excitedduringthecourse?
7.Describethereactionsofthestudents.Whatdidtheysayduringandattheendofthecourse?Whatdidtheycommenton?
8.Howwouldyouevaluateyourstudentsintermsoftheirlearningoutcomes?
9.Whichwerethedifficultiesthatthestudentsfacedduringthepilotcourse?Name3.
10.Whatdoyouthinkthatcouldgobetter?
11.Whatwouldyouchangeifyoucouldimplementthepilotcourseforasecondtime?
12.Isthereanythingelseyouwouldliketosharewithus?
11.4 Annex 4: A questionnaire to evaluate the training materials bythestudents(Draft)
Student/Evaluator:_________________________________Date:_____________________________________________Instructions: Grade the items below on a scale from 1-5. To what degree does the training materialsdemonstrate the following? Please, take in account the training materials regarding the in-persontraining,e-learningandVirtualCommunitiesofPractice.Q:Towhatextentdoyouagreewiththefollowingcriteriaregardingtheproductofthepilotcourse? Criteria Completely
disagree Partiallydisagree
Neitheragreeordisagree
Partiallyagree
Completelyagree
1 2 3 4 5
Content
Coursehaveacurrentcontent.
Coursehaveanaccuratecontent.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 40of73
ContentsupportsrelevantknowledgeinsocialandhomecareinVET.
Scope(range)anddepthoftopicsareappropriatetostudent’sneeds.
Thelevelofdifficultyisappropriateforintendedaudience.
Content materials consider andintegratework-basedlearning.
Contentmaterialshelpstudentstoapplypracticalknowledgeinworkbasedlearningenvironments.
Instructionaldesign
Instructional goals and learnerobjectivesareclearlystated.
Materialsaresuitableforawiderangeoflearning/teachingstyles.
Learningmaterialspromotesstudentengagement.
Learningmaterialsfostergroupinteraction.
Learningmaterialsencouragesstudentcreativity.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 41of73
Learningmaterialsallows/encouragesstudentstoworkindependently
Learningmaterialsaresuitableforintendedpurpose.
Materialsarewellorganizedandstructured.
Materialshaveunity/congruency.
Technicaltermsareconsistentlyexplained/introduced.
Adequate/appropriateassessment/evaluationtoolsareprovided.
Technicaldesign
Learningmaterialsarevisuallyattractive.
Illustrations/visualsareeffective/appropriate.
Charactersize/typefaceisappropriate.
Studentscaneasilyemploytheresource.
Thelearningmaterialsarewellexplainedandinaclearlanguage.
Learningmaterialsarepresentedinadifferentformat(i.e.videos,documents,webpages,software,openeducationalresources,ICTtools)
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 42of73
Inclusion/socialconsiderations
Thelearningmaterialstakeintoaccountdifferentwaysoflearningbythestudents.
Learningmaterialstakeintoaccountstudentspreviousknowledgeandlearningoutcomes.
Learningmaterialsarewellsuitedtothestudents’diversity.
Learningmaterialsestablishdifferentdifficultygradesaccordingwiththediversityofstudents.
Workoriented
Learningmaterialsopportunitiestodevelopentrepreneurialmindsets.
Learningmaterialsintegrateopportunitiesofknowinglabourmarketneedsandskills.
Learningmaterialsarerelevantforthestudent’sfuturejobsopportunities.
Educational/multimediaandmanagementaspectsrelatedtoe-learningandVCPmaterials
Thedesignisuser-friendly.
Thetextisclearandeasytoread
Thesoftware/webpages/videosarelogicallyorganized
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 43of73
Navigationisclearandconsistent
Thescreenisuncluttered
Navigationisuser-controlled
Littlehelpisrequiredtoworkindependently
Thegraphicsandsoundarewellused
Thegraphicsarenotoutdated.
Graphicsandaudioarenotdistractingelements
Thankyouforyourparticipation!
11.5 Annex 5: Questionnaire to evaluate the training materials byTeachers(Draft)
Teacher/Evaluator:__________________________________ Date:______________________________________________ Instructions: Grade the items below on a scale from 1-5. To what degree does the training materialsdemonstrate the following? Please, take in account the training materials regarding the in-persontraining,e-learningandVirtualCommunitiesofPractice. Q:Towhatextentdoyouagreewiththefollowingcriteriaregardingtheproductoftheworkshop?
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 44of73
Criteria Completelydisagree
Partiallydisagree
Neitheragreeordisagree
Partiallyagree
Completelyagree
1 2 3 4 5
Content
Coursehaveacurrentcontent.
Coursehaveanaccuratecontent.
ContentsupportsrelevantknowledgeinsocialandhomecareinVET.
Scope(range)anddepthoftopicsareappropriatetostudent’sneeds.
Thelevelofdifficultyisappropriateforintendedaudience.
Content materials consider andintegratework-basedlearning.
Contentmaterialshelpstudentstoapplypracticalknowledgeinworkbasedlearningenvironments.
Instructionaldesign
Instructionalgoalsandlearnerobjectivesareclearlystated.
Materialsaresuitableforawiderangeoflearning/teachingstyles.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 45of73
Learningmaterialspromotesstudentengagement.
Learningmaterialsfostergroupinteraction.
Learningmaterialsencouragesstudentcreativity.
Learningmaterialsallows/encouragesstudentstoworkindependently.
Learningmaterialsaresuitableforintendedpurpose.
Materialsarewellorganizedandstructured.
Materialshaveunity/congruency.
Technicaltermsareconsistentlyexplained/introduced.
Adequate/appropriateassessment/evaluationtoolsareprovided.
Technicaldesign
Learningmaterialsarevisuallyattractive.
Illustrations/visualsareeffective/appropriate.
Charactersize/typefaceisappropriate.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 46of73
Studentscaneasilyemploytheresource.
Thelearningmaterialsarewellexplainedandinaclearlanguage.
Learningmaterialsarepresentedinadifferentformats(i.e.videos,documents,webpages,software,openeducationalresources,ICTtools)
Inclusion/socialconsiderations
Thelearningmaterialstakeintoaccountdifferentwaysoflearningbythestudents.
Learningmaterialstakeintoaccountstudentspreviousknowledgeandlearningoutcomes.
Learningmaterialsarewellsuitedtothestudents’diversity.
Learningmaterialsestablishdifferentdifficultygradesaccordingwiththediversityofstudents.
Workoriented
Learningmaterialsopportunitiestodevelopentrepreneurialmindsets.
Learningmaterialsintegrateopportunitiesofknowinglabourmarketneedsandskills.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 47of73
Learningmaterialsarerelevantforthestudent’sfuturejobsopportunities.
Educational/multimediaandmanagementaspectsrelatedtoe-learningandVCPmaterials
Thedesignisuser-friendly.
Thetextisclearandeasytoread.
Thesoftware/webpages/videosarelogicallyorganized.
Navigationisclearandconsistent.
Thescreenisuncluttered.
Navigationisuser-controlled.
Littlehelpisrequiredtoworkindependently.
Thegraphicsandsoundarewellused.
Thegraphicsarenotoutdated.
Graphicsandaudioarenotdistractingelements.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 48of73
Thankyouforyourparticipation!
11.6 Annex6:Students’initialquestionnaire(Draft)InformantAge:_______________________________________________________Previousleveloflearning:________________________________________________Instructions:Please complete thisquestionnaireby responding to the topicsbelow. Your feedbackwillassistus inevaluating thequalityof theCARESSpilot course.This informationwillbe kept confidentialandusedonlyforthepurposeofevaluatingtheCARESSpilotcourse.Note:Beforefillingthisquestionnaire,youhavetoreadthelearningprogramorbefamiliarwithit.1.WhyhaveyoudecidedtoattendthiscoursePilot?
�Iwascurious.�Itwasmandatory. �Itcouldbeusefulformycareer. �Other(pleasespecify):________________________________________ 2.Aretheobjectivesofthetrainingactivitiescleartoyou? �Yes �No �Idon’tknow Q Towhatextentdoyouagreewiththefollowingstatements?
Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neither agree nor disagree 4= Partially agree 5=Completelyagree
3. Thecoursegoalsareclear 12345
4. Thereareknowledge,skillsandcompetencesinhomecarethatcanbeusefultoimprovemyfuturejoboptions
12345
5. I find the courseprogram interesting to improvemyactual knowledge, skillsandcompetences.
12345
6. I am able to express doubts and concerns about the skills needed in homecare.
12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 49of73
7. Iamabletoworkinteamwithotherstudenttoreachthelearningobjectivesthroughtheproposalactivities
12345
8. Ifindthelearningstrategiesmotivatingandattractive 12345
Pleasewriteinyourpersonalresponses 9.Doyouhaveapreviouslearningexperienceinhealthcare?Incaseyouansweredaffirmatively,what
kindoflearningwas?(presential,on-line,blended…)
10.Whatdoyouexpecttolearnfromthispilotcourse?
Thankyouforyourparticipation!
11.7 Annex7:Students’finalquestionnaire(Draft) Instructions:Pleasecompletethisevaluationbyrespondingtothetopicsbelow.YourfeedbackwillassistusinevaluatingthequalityoftheteacherPilotCourse.ThisinformationwillbekeptconfidentialandusedonlyforthepurposeofevaluatingtheCARESSpilotcourse. a)ContentandstructureofthePilot
Q Towhatextentdoyouagreewiththefollowingstatements? Pleaseusethefollowingscale: 1=Completelydisagree 3=Neitheragreenordisagree 2=Partiallydisagree4=Partiallyagree5=Completelyagree
1. Thegoalsofthecoursewerecleartoyou. 12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 50of73
2. Courseactivitieswerewellorganized. 12345
3. Courseactivitiesallowedinteractiveparticipation. 12345
4. Courseactivitiesenhancedyourpreviousknowledgeregardinghomecare.
12345
5. Courseactivitiesenhancedyourknowledgeregardingjobopportunitiesinhomecare.
12345
6. Themethodologychosentakesinaccountthestudentsdiversityandpossibledifficultiestobesolved.
12345
7. Therewasenoughtimetocompletethecourseactivities. 12345
8. Courseactivitiesweresuitableforthegoalsofthetraining. 12345
9. Coursecontentwasrelevanttoyourfuturepractice. 12345
10. Thecoursematchedyourexpectations. 12345
b)Teachers-Tutors Q Towhatextentdoyouagreewiththefollowingstatementsregardingyourtutors?
Pleaseusethefollowingscale: 1=Completelydisagree 3=Neitheragreenordisagree 2=Partiallydisagree 4=Partiallyagree 5=Completelyagree
11. Thedescriptionofthecourse’sactivitieswasfullyunderstandable
12345
12. Teacherwasprovidinguswithguidancethroughouttheprogram.
12345
13. Teacherwasprovidinguswithfeedbackthroughouttheprogram.
12345
14. Teacherhadadequateknowledgeonthesubject 12345
15. Teachercommunicatedwellwithyou 12345
16. Teacherwaswellprepared 12345
17. Teacherdevotedenoughtimetoanswertoyourquestionsandissues
12345
18. Teacherguidedyouwell,throughoutyourlearningexperience 12345
c)Students’experience
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 51of73
Q Towhatextentdoyouagreewiththefollowingstatements? Pleaseusethefollowingscale: 1=Completelydisagree 3=Neitheragreenordisagree 2=Partiallydisagree4=Partiallyagree5=Completelyagree
19. IfeeltheCARESSprojectgoalsareinterestingandIconsidermyselfmotivatedtoknowmoreaboutit.
12345
20. Iwouldliketoincorporatethenewknowledge,skillsandcompetenceslearnedtomypractice.
12345
21. Thelearningdesigntakesinaccountmypreviousexperiences. 12345
22. Ifeltlessonsareeasytoputinpracticeinafuturejobsituation. 12345
23. Iwasabletocollaboratewithmyclassmatesduringthetraining.
12345
24. Theknowledge,skillsandcompetencesshowedduringthetrainingwereputincontext.
12345
25. Ihadtheopportunitytoaskquestionsandconcernsabouthomehealthcarenewknowledge,skillsandcompetences.
12345
26. Iwasabletodemonstratemyunderstandingbypreparingpresentations/writingreports,etc.
12345
27. Iwasabletoanalysed,criticized,comparedandcontrasted,throughthelearningactivities.
12345
28. Iwasabletoprovideconstructivefeedbacktomyteacherandclassmates.
12345
29. Iwasabletoreceiveconstructivefeedbackfrommyteacherandclassmates.
12345
30. Ihadtheopportunitytoreflectonmyownlearning. 12345
31. Iwasabletosharelearnedknowledge,ideasandlearningmaterialsthroughtheVCP
12345
32. IwasabletomakeonlinecontactwithotherstudentsandprofessionalsthroughtheVCP
12345
33. Iwasabletoshareideas,doubtsand/orexperienceswithfinalandendusers
12345
d)Overallsatisfaction
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 52of73
Q Towhatextentdoyouagreewiththefollowingstatementsregardingyoursatisfaction? Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree 5=Completelyagree
34. Theoverallorganizationoftheprogram 12345
35. Thecontentoftheprogram 12345
36. Thecourseactivities 12345
37. Thetrainingresourcesandmaterialsgivenbytheorganizers 12345
38. Thecommunicationwithinteacherandotherstudents 12345
39. Iwaspleasedattendingthiscourse 12345
40. Iwaspleasedwiththefacilitiesprovidedduringthecourse 12345
41. Iwaspleasedwiththematerialsprovidedduringthecourse 12345
42. Iwasprovidedwithsatisfactorysupportwhileacquiringthenewlearning.
12345
43. Iwaspleasedwiththelearningmethodologyusedduringthecourse
12345
44. Ifeelthemethodologyusedwasadaptedtomylearningspreferencesandstyleoflearning
12345
45. IwaspleasedwiththeuseandpossibilitiesoftheVCP 12345
e)Impact Q Towhatextentdoyouagreewiththefollowingstatements?
Pleaseusethefollowingscale: 1=Completelydisagree 3=Neitheragreenordisagree 2=Partiallydisagree 4=Partiallyagree5=Completelyagree
46. Iwillusethelearningacquiredinhomecareinmyfuturepractice.
12345
47. Ifeelthatthisnewlearningisusefulformyfuturejobpractice 12345
48. Thisnewlearningiseasytoapplyinmyfuturejobpractice 12345
49. Theapplicationofthislearningwillimprovemyjobopportunities
12345
50. Itwouldbegoodtoextendtoothercolleaguesthelearnings 12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 53of73
acquiredinthisPilotcourse.
f)Pleasewriteinyourpersonalresponses 51.Haveyourobjectivesforthecoursebeenmet?Whatwerethese?
52.Whatdidyoulikeabouttheformatofthecourse?Nameatleastonething.
53.Whatdidn'tyoulikeorcouldbeimproved?Nameatleastonething.
54.Othercomments
Thankyouforyourparticipation!
11.8 Annex8:Teacherfinalquestionnaire(Draft) Instructions:Pleasecompletethisevaluationbyrespondingtothetopicsbelow.YourfeedbackwillassistusinevaluatingthequalityoftheteacherPilotCourse.ThisinformationwillbekeptconfidentialandusedonlyforthepurposeofevaluatingtheCARESSpilotcourse.
a)Satisfactionandgoalachievement
Q Towhatextentdoyouagreewiththefollowingstatements? Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 54of73
1. Thestructureofthelearningactivityhavebeensatisfactory a.12345
2. Thelearningdesignmatchedthegoalsforthepilot 12345
3. Courseactivitiesallowedinteractiveparticipation 12345
4. ThelearningactivitiesandthelessonsmetthePilot’sobjectives 12345
5. Thelearningactivitiesandthelessonsfocusedonthedevelopmentandassessmentoftheskillgapsinhomecare
12345
6. Courseactivitieswereeasytofollow 12345
7. Therewasenoughtimetocompletethecourseactivities 12345
8. Courseactivitiesweresuitableforthegoalsofthetraining. 12345
9. Coursecontentwasrelevanttoyourstudents’futurepractice 12345
10. Thecoursematchedyourexpectations 12345
b)Teachingperformance Q Towhatextentdoyouagreewiththefollowingstatementsregardingyourteacher
performance? Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 55of73
11. Ihadanadequateknowledgeontheskillgaps 12345
12. Thestudentscommunicatedwellwithme 12345
13. Thelessonsandactivitiesplannedwerewellprepared 12345
14. Ihaveenoughtimeduringthelessonstoanswerthestudentsquestionsandissues
12345
15. Thetimeforthelearningobjectiveswasenough 12345
c)Overallsatisfaction Q Towhatextentdoyouagreewiththefollowingstatementsregardingyoursatisfaction?
Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree 5=Completelyagree
16. TheoverallorganizationoftheCARESSPilot 12345
17. Thecontentofthepilot 12345
18. Thecourse’activities 12345
19. Thetrainingresourcesandmaterialsused 12345
20. Thecommunicationbetweentheteacherandstudents 12345
21. IampleasedIteachthiscourse 12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 56of73
22. Iampleasedwiththefacilitiesprovidedduringthecourse 12345
23. Iampleasedwiththeresourcesprovidedbythelearningcentreduringthecourse
12345
24. IwasprovidedwithsatisfactorysupportwhileteachingthePilot 12345
d)Impact Q Towhatextentdoyouagreewiththefollowingstatements?
Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
25. Iwillusethisteachingmaterialinthefuture 12345
26. Ifeelthatthislearningonskillgapsisbeenusefulformystudents 12345
27. Thisnewlearningiseasytoapplyinmyfuturejobpractice 12345
28. Theapplicationofthislearningwillimprovemystudentsjobopportunities
12345
29. Itwouldbegoodifothercolleaguescouldteachaboutthisknowledge,skillsandcompetences.
12345
e)Pleasewriteinyourpersonalresponses
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 57of73
30.Haveyourobjectivesforthecoursebeenmet?Whatwerethese?
31.Whatdidyoulikeabouttheformatofthecourse?Name3things.
32.Whatdidn'tyoulikeorcouldbeimproved?Name3things.
33.Othercomments
Thankyouforyourparticipation!
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 58of73
11.9 Annex9:Questionnaireevaluatingteachers’Pilotdesign(Draft)Teacher/Evaluator:__________________________ Date:______________________________________ Centre:____________________________________ Instructions:Pleasecompletethisevaluationbyrespondingtothetopicsbelow.YourfeedbackwillassistusinevaluatingthequalityoftheteacherPilotCourse.ThisinformationwillbekeptconfidentialandusedonlyforthepurposeofevaluatingtheCARESSpilotcourse.1.Pilot’spreparation
Q TowhatextentdoyouagreewiththefollowingstatementsregardingthePilot’sPreparation?
Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
1. Thedocumentstodesignthepilotwereclearandenoughtohelpmedesignthepilot.
12345
2. TheCARESSframeworkhelpedmeinidentifytheknowledge,skillsandcompetencestobereach.
12345
3. Iwasabletoworkinteamwithotherteacherstoreachthelearningobjectivesthroughtheproposalactivities.
12345
2.Objectives
Q TowhatextentdoyouagreewiththefollowingstatementsregardingthePilot’sObjectives? Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completely
agree
4. Objectivesaremadeavailableinavarietyofareasinthecourse(withineachindividuallearningunitormodule)
12345
5. Objectivesareclearlywrittenattheappropriatelevelandreflectdesiredoutcomes.
12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 59of73
6. Objectivesarewritteninmeasurableoutcomes(studentsknowwhattheyareexpectedtobeabletodo)
12345
3.Methodology Q Towhatextentdoyouagreewiththefollowingstatementsregardingthemethodology?
Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
7. Themethodologywaschosentomatchthegoalsaccordingwithmypreviousexperienceinteachinghealthcarers.
12345
8. Themethodologytakesintoaccountthestudentsdiversityandpossibledifficultiestobesolved.
12345
9. Thechosenmethodologyusesavarietyofappropriate mechanisms(contentmodules,linkstoexternalresources,and/ormultimedia,etc.)
12345
4.Contentpresentation Q Towhatextentdoyouagreewiththefollowingstatementsaccordingtothecontent
presentation? Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
10. Contentispresentedindistinctlearningunitsormodules 12345
11. Contentflowsinalogicalprogression 12345
12. Contentispresentedusingavarietyofappropriate mechanisms(contentmodules,linkstoexternalresources,and/ormultimedia,etc.)
12345
13. Contentissequencedandstructuredinamannerwhichenableslearnerstoachievethestatedgoals
12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 60of73
14. Purposeoflearningactivitiesisclearlypresented. 12345
15. Theselectedtoolforeachactivityisappropriateforeffectivedeliveryofthecontent.
12345
16. The description of the units reflects the knowledge, skills andcompetencesassociated.
12345
5.Instructionalstrategies Q Towhatextentdoyouagreewiththefollowingstatementsregardingtheinstructional
strategies? Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
17. Itisclearhowtheinstructionalstrategieswillenablestudentstoreachcourseobjectives
12345
18. Coursedesignincludesguidanceforlearnerstoworkwithcontentinmeaningfulways
12345
19. Technologiesareusedcreativelyinwaysthattranscendtraditional,teachercentredinstruction
12345
20. Learnershavetheopportunitytogiveanonymousfeedbacktotheinstructorregardingcoursedesignandcoursecontentbothduringcoursedeliveryandaftercoursecompletion
12345
21. LearningactivitiesandotheropportunitiesaredevelopedtofosterStudent-Studentcommunicationand/orcollaboration.
12345
22. LearningactivitiesandotheropportunitiesaredevelopedtofosterStudent-Instructorcommunicationand/orcollaboration.
12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 61of73
23. LearningactivitiesandotheropportunitiesaredevelopedtofosterStudent-Contentinteraction.
12345
24. ThelearningactivitiesincludedonmyteachingdesignareeasilyusedintheVCPenvironment(provisional)
12345
25. TheIDtemplateshavebeenusefulregardingthedesignofthelessonplan(provisional)
12345
26. TheIDtemplateallowedmetohaveanoverallideaofthePilotcontentandhelpmetoidentifyrelevantinformationtopurposealessonplan.(provisional)
12345
27. TheILDEtemplateshavehelpedmetobetterdesigntheteachinglessonsaccordingtothelearningoutcomes,havinginmindthegoals,knowledge,aswellastheskillsandcompetencesthatshouldbeacquiredbythestudents.(provisional)
12345
28. TheILDEtemplatescreatedcouldbeusefulforfurtherVETcourses.(provisional)
12345
6.Assessment
Q Towhatextentdoyouagreewiththefollowingstatementsregardingtheassessment?
Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree 3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
29. Assessmentactivitieshave“facevalidity”(i.e.,theyappeartomatchthecurriculumandareexplainedusingappropriatereadinglevelandvocabulary)
12345
30. Higherorderthinkingisrequired(e.g.,analysis,problem-solving,etc.) 12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 62of73
31. Assessmentsaredesignedtomimicauthenticenvironmentstofacilitatetransfer
12345
32. Assessmentactivitiesoccurfrequentlythroughoutthedurationofthecourse,andtheinstructorprovidesmeaningfulfeedbackinatimelymanner.
12345
33. Multipletypesofassessmentsareused(researchproject,objectivetest,discussions,etc.)
12345
34. Thereareopportunitiesforstudentself-assessment,providingfeedbackthatallowsstudentstoseekadditionalhelpwhennecessary
12345
9.CompliancewithEQVET
Q TowhatextentdoyouagreewiththefollowingstatementsregardingEQVET?
Pleaseusethefollowingscale:1=Completelydisagree 3=Neitheragreenordisagree 5=Completelyagree 2=Partiallydisagree 4=Partiallyagree
35. TheteacherhasbeentrainedinthepurposeandgoalsofthisPilot(regardingdesign,teachingmethods,evaluation,etc.)
12345
36. Theparticipantshavebeenchosenaccordingtothetypeofprogramme 12345
37. Stakeholdersfromtheareahavebeeninvolvedduringtheteachingprocess. 12345
38. Learningmaterialshavebeenadapted,consideringvulnerable/disabledstudentsinthegroupclass.
12345
39. Thestudents’participationrateincludesvulnerablegroups(earlyschoolleavers,highesteducationalachievement,migrants,personswithdisabilities,lengthofunemployment)
12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 63of73
10.QualitylevelofPilot’sIDdocuments Q Towhatextentdoyouagreewiththefollowingstatementsregardingthequalitylevelof
Pilot’sIDdocuments? Pleaseusethefollowingscale: 1=Completelydisagree 3=Neitheragreenordisagree 2=Partiallydisagree 4=Partiallyagree 5=Completelyagree
40. TheCaressIDTemplatehavebeenusefulregardingtheteachingperformance.CaressIDdocumentshavebeenusefulregardingtheteachingperformance.
12345
41. TheIDdocumentshelpmereflectingaboutdifferentteachingstylesandlearningstrategiesthatcanbeusedinmylessons
12345
42. TheIDdocumentsareflexibleenoughtobeadaptedthroughtheteachingperformance.
12345
43. TheIDdocumentshelpmeinreflectaboutmethodology,skillsandtransversalcompetences.
12345
44. TheIDdocumentsreflectthemacroandmicro-design. 12345
11.UseofCaressFrameworktodesignthePilot Q TowhatextentdoyouagreewiththefollowingstatementsregardingUseofCaress
FrameworktodesignthePilot? Pleaseusethefollowingscale: 1=Completelydisagree 3=Neitheragreenordisagree 5=Completelyagree 2=Partiallydisagree 4=Partiallyagree
45. Tobecompleted 12345
46. Tobecompleted 12345
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 64of73
47. Tobecompleted 12345
48. Tobecompleted 12345
49. Tobecompleted 12345
12.UseofWizardinhelpingtodesignthePilot Q TowhatextentdoyouagreewiththefollowingstatementsregardingtheuseofWizardin
helpingtodesignthePilot? Pleaseusethefollowingscale: 1=Completelydisagree 3=Neitheragreenordisagree 5=Completelyagree 2=Partiallydisagree 4=Partiallyagree
50. Tobecompleted 12345
51. Tobecompleted 12345
52. Tobecompleted 12345
53. Tobecompleted 12345
54. Tobecompleted 12345
13.Pleasewriteinyourpersonalresponses
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 65of73
55.HaveyourobjectivesforthedesignofthePilotbeenmet?Whatwerethese?
56.Whatwillyouimprove?
57.Othercomments
Thankyouforyourparticipation!
11.10 Annex 10: Guidelines to complete the Student’s ReflectiveDiary(Draft)
School:
Student:
Course:
Starttime:
Endtime:
Date: Namethreethingsthatimpressedyouthemostaboutthee-learning/VCPsessions._______________________________________________________________________________________________________________________________________________________________________________________________________________ Namethreedifficultiesthatyoufacedduringtheactivitiesperformed:
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 66of73
_______________________________________________________________________________________________________________________________________________________________________________________________________________ Whathasworkedwellduringthesession? _______________________________________________________________________________________________________________________________________________________________________________________________________________ Namesomelessonlearntduringthee-learning/VCPsession. _______________________________________________________________________________________________________________________________________________________________________________________________________________ Wereyouabletodiscusstheemergedissueswithotherstudents/professionalprofilesduringthee-learning/VCPsession(understanding“session”asafullperiodofusebetweenloginandlogout,withthedevelopofoneormoreactivitieswithintheplatform)? _______________________________________________________________________________________________________________________________________________________________________________________________________________ Wereyouabletocreatecontactswithotherstudents/professionalprofilesduringthee-learning/VCPsession? _______________________________________________________________________________________________________________________________________________________________________________________________________________ Additionalcomments: _______________________________________________________________________________________________________________________________________________________________________________________________________________
Thankyouforyourparticipation!
11.11 Annex11:Student’posttrainingquestionnaire(Draft)Instructions: Please complete this evaluation questionnaire by responding to the topics below. YourfeedbackwillassistusinevaluatingthequalityoftheCARESSpilotcourse.ThisinformationwillbekeptconfidentialandusedonlyforthepurposeofevaluatingtheCARESSpilotcourse.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 67of73
Q Towhatextentdoyouagreewiththefollowingstatements? Pleaseusethefollowingscale: 1=Completelydisagree2=Partiallydisagree3=Neitheragreenordisagree4=Partiallyagree5=Completelyagree
1. The learning achieved through the pilot helpedme to understandtheneedsofthejobmarket.
12345
2. Thelearningachievedthroughthepilothelpedmetobetteraddressmyeffortsinlookingforajob.
12345
3. I could apply to awide range of job offers thanks to the learningachievedthroughthepilot.
12345
4. Itwas able to show and talk about the learning achieved throughthepilotduringajobinterview.
12345
5. IbelievemyjobopportunitiesarebetterafterthePilot. 12345
6. I was able to prove my new learning knowledge, skills andcompetencesaftercompletepilotlearning.
12345
7. Iwouldrecommendthiscoursetootherprofessionals. 12345
Pleasewriteinyourpersonalresponses
8.Howwasyourjob/jobsearchingexperienceafterthePilot?
Thankyouforyourparticipation!
11.12 Annex12:Student’posttraininginterview(Draft)
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 68of73
1.HowwasyourexperienceofjobsearchingafterthePilot? 2.Wereyouabletogointoanyspecificjobselectionthankstothelearningachieved? 3.Doyoubelieveyourpossibilitiesoffindingajobhasbeenincreasedthankstothisexperience? 4.DoyouthinktheVirtualCommunityofPracticehelpyoubygivingyouadviceoraddressingyour jobsearch? 5.DoyoufindtheVirtualCommunityofPracticeusefultomeetotherprofessionals,keeponprofessionalupdateorfindsupportandhelpinyourprofessionalcareer? 6.Didyourecommendthiscoursetootherfellows?Why? 7.Inyouropinion,areyoubetterpreparedfromthejobmarketsinceyoucompletedthecourse? 8.Doesthejobmarketopportunitiesreflecttheneedofthenewlearningacquired?
Thankyouforyourparticipation!
11.13 Annex13:TrackingreportingCARESSe-learningplatform
TRACKINGPROGRESS
The tutor can check all the work the users have been doing through several tracking methods andthereforeprovidingthemwithafeedbackonit.
HereyoucanfindsomeofthesolutionsthatMoodleofferstocheckthejobcarriedoutbyeachuser.
• LOGS
LogsinMoodleareactivityreportsandtheyareavailableatsiteandcourselevel.
Youcanseewhatpagesthestudentsaccessed,thetimeanddatetheyaccessedthem,theIPaddresstheycamefromandtheiractions(view,add,update,delete).
Actually,thelogreportpagecontainsactivelinkswhichenableyoutoaccessastudent'sprofilepageoraspecificpagethatthestudentwascheckingatacertaintime.TheIPaddresslinkprovidesanestimateofthestudent'slocation
• ACTIVITYREPORTS
Acourseactivityreportshowsthenumberofviewsforeachactivityandresourceandanyrelatedblogentries.Itcanbeviewedbymanagers,teachersandnon-editingteachers.Ifactivityreportsareenabledforacourseinthecoursesettings,eachcourseparticipantcanaccessthereportsfortheircontributions,suchasforumpostsorassignmentsubmissions,logsandstatisticsreports.
• PARTICIPATIONREPORT
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 69of73
Some examples of participation reports are Forum View: (View Discussion, Search, Forum, Forums,Subscribers)andForumPost:(AddDiscussion,AddPost,DeleteDiscussion,DeletePost,MoveDiscussion,PrunePost,UpdatePost).
Anextremelyuseful featureof theparticipation report is theoption to sendamessage toall studentswho have not completed a certain action. Other options with selected users are to add a note or toextendenrollment.
• STATISTICS
If statistics are enabled for the site, graphs and tables for the user activity may be generated. Thestatisticsgraphsandtablesshowhowmanyhitstherehavebeenonvariouspartsofthesiteatadifferenttime.Theydonot showthenumberofusers though.Theyareprocesseddailyat thetimeyouspecify.Youmustenablestatisticsinordertoseeall.
Furthermore,thetutorhasvariousmethodstocheckthejobcarriedoutbytheuser.
ThegraderreportcollectsitemsthathavebeengradedandcomefromthevariouspartsofMoodlethathavebeenassessedandallowsyoutoviewandchangethemaswellassortthemoutintocategoriesandcalculate totals in variousways.When you add an assessed item in aMoodle course, the grade bookautomatically creates space for the grades itwill produce and also adds the grades themselves by thetimetheyaregenerated,eitherbythesystemorbyyou.
Thegradesareinitiallydisplayedasrawmarksfromtheassessments.Therefore,theywilldependonhowtheyhadsetthoseupe.g.anessayoutof36willappearashowevermanyrawmarksthatstudentgot,notapercentage(althoughthiscanbechangedlater,seebelow).
Noticethatvariousdefaultoptionsforthegradebookaresetatasystemlevelbytheadministratorandcanbemarkedasbeingoverreadablebyyou,orfixed.Thismeansthattheoptionswillnotalwaysbesetupinthesamemannerforeveryuserwhentheyseethegradereportforthefirsttime.
• COURSEGRADESETTINGS
Coursegradesettingsdeterminehowthegradebookappearstoallparticipantsinthecourse.
Course grade settings are found in Settings > Grade Administration > Course grade settings or via thegradebookSettingstab.
ThedefaultcoursegradesettingsaresetbyanadministratorinSettings>Siteadministration>Grades.
• Graderreportpreferences
Teacherscansettheirpreferencesforthegradereportviathegradebookcalled:'Mypreferences'.Thesesettingswillapplytoallcoursesforthatteacher.Theteachermaychangethematanytime.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 70of73
Thedefaultgradereportpreferencesaresetbyanadministrator inSettings>Siteadministration>Grades>Reportsettings>Gradereport.
FromMoodle 2.4.2 onwards, the default number of students per page (grade report students perpage)cansafelybeincreasedwithoutanydatalosswhengradinglargenumbersofstudents.
• Siteadministrationsettings
Defaultvaluesforallgradesettingscanbesetbyanadministrator.Theeasiestwaytodothisistologinasadminandthenbrowseacoursegradebookandfollowthe'Changedefaults'links.
The following additional grade settings can be found in Settings > Site administration > Grades >Generalsettings.Thesettingswillaffectallgradebooksusedbyallteachers.
• GradedRoles
Gradedrolesarethetypeofuserthatwillappearinthegradebook.Mostofthetime"student"istheonlyoneneeded.
• USERPROFILEREPORT
Thedefaultuserprofilereportsettinghasjustoneoption,'UserReport',unlessacustomuserreporthasbeenaddedtothesite.
Teacherscanviewtheuser reports forallorsomeselectedstudents in thecourse inSettings>Gradesadministration>Userreport.
• INCLUDEDSCALESINAGGREGATION
Scalescanbeincludedasnumbersinallaggregatedgradesacrossallgradebooksinallcourses.Changingthissettingwillforceallaggregatedgradestoberecalculated.
If this is checked by default, then all custom scaleswill, therefore, have a corresponding value that isassignedbasedonthenumberofoptionsinthescale(i.e.threevalueswillhavevaluesof1,2,3)andthiswillbeincludedinthegradeaggregation.
• Enablepublishing
Gradepublishingsettings:
GradepublishingisawayofimportingandexportinggradesviaaURLwithoutbeingloggedintoMoodle.Gradepublishing isforadministratorsonly.Security implicationsshouldbethoroughlyexaminedbeforenon-adminusersareallowedtoreleasegrades.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 71of73
Grade publishingmay be enabled by checking the grade publishing box. If enabled, administrators areprovidedwithgradeexportpublishingsettingsineachcoursegradebook.
• Recovergradesdefault
Whenmanually enrolling a student in a course, there is an option called 'Recover user's old grades ifpossible',butthischeckboxiseasytomiss.Enabling'Recovergradesdefault'resultsin'Recoveruser'soldgradesifpossible'beingtickedforeverycourse.
• Unlimitedgrades
Teachers can enter grades over 100% directly in the grade book if the unlimited grades setting areenabled.
Gradescanbedistributedintogradecategories.Agradecategoryhasitsownaggregatedgrade,whichiscalculatedfromitsgradeitems.Thereisnolimittothelevelofnestingofcategories.Thatbeingsaid,eachgradeitemmaybelongtoonlyonecategory.Also,allgradeitemsandcategoriesbelongtoatleastone,permanent category: the course category. A grade calculation is a formula used to determine grades,based)onothergradeitems.NotethatthisisnotthesameasCalculatedquestiontypes.
Calculations for the grade book follow the pattern of formulas/functions in popular spreadsheetprograms.Theystartwithanequal (=)signandusecommonmathematicaloperatorsand functions. toproduceasinglenumericaloutput.Thisoutputisthenusedasthecomputedvalueforthegradeitemyouareediting.
• Scales
Scalesareawayofevaluatingorratingastudents'performance.Moodleoffersastandardsetofnumericscales.Itisalsopossibletocreatecustomscales,whichcanbeavailableonthesite.Forinstance,youcangivethestudentawordorsmallphraseasawayofratingorgiveastudentfeedback.
ScalesmaybeusedinForums,GlossariesandAssignmentsforratingand/orgradingastudent'sactivity.
• Outcomes
Outcomes are specific descriptions of what a student has demonstrated and understood at thecompletionofanactivityorcourse.Eachoutcomeisratedbysomesortofscale.
Basically, outcomes are similar to sub-components of a grade. A grade is an assessment of the overallperformanceandittmayincludetests,participation,attendance,andprojects.Outcomesassessspecificknowledgelevelsthroughaseriesofstatements,whichmaybecodedwithnumbersorletters.Thus,anoverallgradecanbegivenforacoursealongwithstatementsaboutspecificcompetenciesintheformofoutcomes.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 72of73
• Gradeletters
Gradeletters,alsocalledlettergrades,aresymbolsusedtorepresentarangeofgrades.Forexample,"A"couldbeusedtorepresentgradesof80%andabove,"B"torepresentgradesbetween70and80%,"C"torepresentgradesbetween50and70%,andsoon.Alternatively,youcouldhave"Pass"forgradesabove50%and"Fail"forgradesbelow50%.
Whenatutorrealizesthatauseriseithernotveryactiveorisonthevergeofabandoningthecoursehecan take action by stimulating the user through a series of utilities on the platform itself. The Surveymodule is a course activity that provides a number of verified survey instruments, including COLLES(Constructivist On-Line Learning Environment Survey) and ATTLS (Attitudes to Thinking and LearningSurvey),whichhavebeenfoundusefulinassessingandstimulatingtheonlinelearningprocess.Teacherscanusethesetogatherdatafromtheirstudentsthatwillhelpthemlearnabouttheirclassandreflectontheirownteaching.Currently,Moodleonlyoffersspecifictypesofsurveys(futureversionswillenableyoutocreateyourown).
The available surveys have been chosen as being particularly useful to evaluate online learningenvironmentsthatuseaconstructivistpedagogy.Theyareusefulto identifycertaintrendsthatmaybespreadingamongparticipants.
• COLLES-ConstructivistOn-LineLearningEnvironmentSurvey
TheCOLLEScomprises24statementsdividedintosixscales,eachishelpfultoaddressakeyquestiononthequalityoftheon-linelearningenvironment:
• Relevance:Howrelevantison-linelearningtostudents'professionalpractices?• Reflection:Doeson-linelearningstimulatestudents'criticalreflectivethinking?• Interactivity:Towhatextentdostudentsengageon-lineinrelevanteducativedialogue?• TutorSupport:Howwelldotutorsenablestudentstoparticipateinon-linelearning?• PeerSupport:Isthesupportprovidedon-linebyfellowstudentssensitiveandencouraging?• Interpretation:Dostudentsandtutorsmakegoodsenseofeachother'son-linecommunications?
AnewknowledgeisasocialConstructivism,whichportraysthelearnerasanactiveconceptualizedwithinasocially interactivelearningenvironmentandit isactuallybasedonunderpinningthedynamicviewoflearning. Social Constructivism is an epistemology or a way of knowing where learners work togetherreflectivelytoco-buildnewunderstandings,especiallywhenitcomestothemutualinquirygroundedintheirpersonalexperience.
Essentialtothiscollaborationisthedevelopmentofthestudents'communicativecompetence,thatistheability to engage in an open and critical discourse with both teacher and peers. This discourse ischaracterized by an empathic orientation to build a reciprocal understanding and a critical attitudetowardsexaminingunderlyingassumptions.
Deliverable6.2–EvaluationandMonitoringPlan
562634-EPP-1-2015-IT-EPPKA2-SSA CARESSProject 73of73
COLLES has been designed to enable you tomonitor the extent to which you are able to exploit theinteractivecapacityoftheWorldWideWebtoengagethestudentsindynamiclearningpractices.
• ATTLS-AttitudestoThinkingandLearningSurvey
The theory of 'ways of knowing', originally from the field of gender research (Belenky et al., 1986)providesuswithasurveytooltoexaminethequalityofdiscoursewithinacollaborativeenvironment.
TheAttitudesTowardsThinkingandLearningSurvey(ATTLS)isatooldevelopedbyGalottietal.(1999)tomeasuretheextenttowhichapersonisa'connectedknower'(CK)ora'separateknower'(SK).
People with higher CK scores tend to find learning more enjoyable and are often more cooperative,congenialandmorewillingtobuildontheideasofothers,whilethosewithhigherSKscorestendtotakeamorecriticalandargumentativestancetolearning.
Studies have shown that these two learning styles are independentof eachother (Galotti et al., 1999;Galottietal.,2001).Additionally,theyareonlyareflectionoflearningattitudes,notlearningcapacitiesorintellectualpower.
top related