1

Title:

TheDevelopmentandEvaluationofMini-GEMs–Short,Focused,OnlineE-LearningVideosin

GeriatricMedicine

AbbreviatedTitle

Mini-GEMs:OnlineGeriatricsEducationalVideos

2

ABSTRACT

Background

Mini-GEMs are short, focused, e-learning videos on geriatric medicine topics, hosted on

YouTube,whicharetargetedatjuniordoctorsworkingwitholderpeople.Thisstudyaimed

toexplorehowtheseresourcesareaccessedandused.

Methods

Weanalysedtheviewingdatafrom22videospublishedoverthefirst18monthsoftheMini-

GEMproject.WeconductedafocusgroupofUKjuniordoctorsconsideringtheirexperiences

withMini-GEMS.

Results

TheMini-GEMswereviewed10,291timesover18months,equatingto38,435minutesof

totalviewingtime.Theaverageviewingtimeforeachvideowas3.85minutes.

Learners valued thebrevity and focusednatureof theMini-GEMsand that theywatched

them in a variety of settings to supplement clinical experiences and consolidate learning.

Watchingthevideosledtoanincreaseinself-reportedconfidenceinmanagingolderpatients.

Conclusion

Mini-GEMscaneffectivelydisseminate clinical teachingmaterial toawideaudience. The

videosarevaluedbyjuniordoctorsduetotheiraccessibilityandeaseofuse.

3

4

INTRODUCTION

Theworld’spopulationisageing–by2050twobillionpeoplewillbeagedover60years[1].

Doctorsmustreceivespecifictrainingtomeetthehealthcareneedsofthisgroup.National

survey data from theUnited Kingdom in 2013 found themedian timedevoted to formal

teaching about geriatric medicine was only 55 hours out of a five year programme[2].

Harnessinginnovativeteachingmethods,suchastechnology-enhancedlearning(TEL),may

helptoaddressthistrainingdeficit[3].

TELisincreasinglyusedandacceptedinGeriatricMedicine.Computer-aidedlearning(CAL)in

core Geriatric Medicine topics, used as part of a blended learning approach, has been

associated with improved student examination performance[4]. The Portal of Geriatrics

Online Education (POGOe) provides access to an array of TEL resources[5]. Emerging

technologiessuchastheserepresentaparadigmshiftinmedicaleducation.Amongstmedical

educators there is increasing support for this concept of high-quality, free, open-access

‘Meducation’(FOAM)[6].

Theapplicationofsocialmedia,suchasYouTubeandTwitter[7],isincreasinglybeingusedin

delivery of medical education[8]. Mobile learning, using internet-enabled devices, has

potentialtoimprovethereachofmedicaleducationduetowidespreaddeviceownership[8]

coupledwithawillingnesstousetechnologytoaccesscontent[9]. It iscrucial thatclinical

teacherswhousethesetechnologiesarebothcreativeandcriticalintheirimplementation.

Considering ‘pedagogybefore technology’emphasises rationalapplicationof technologies

withinprovenpracticesandmodelsofteaching[10].

5

Againstthisbackgroundwedevelopednovelgeriatricmedicineteachingresourcesoptimised

formobilelearning,distributedviasocialmediaanddesignedtobebrief.MiniGeriatricE-

LearningModules(Mini-GEMs)areshort,focused,onlinevideoslideshowsaimedatjunior

doctorswhocareforolderpeople.Wedescribeheretheirdevelopmentandevaluation.

6

METHODS

DevelopmentofMini-GEMs

Aim

Mini-GEMs were developed with to provide an educational resource around Geriatric

MedicinethatfollowedtheprinciplesofFOAM.Theintentionwastobuildalibraryoftopics

with key points, that could be accessed easily and quickly by busy clinical learners. The

primarytargetaudiencewasjuniordoctors,butthematerialswerecreatedwiththeintention

thattheycouldbeofinteresttoawiderangeofclinicalstaffwhoworkwitholderpatients.

Style

PechaKucha(Japanesefor‘chit-chat’)isapresentationformat,developedintheearly2000s,

that specifies speakers use 20 slides, for 20 seconds each[11].Wehypothesised that this

formatwouldfitwellwithmobileopportunisticlearning.

TheoreticalConsiderationsInformingDesign

ThepredominantlearningtheoriesthatinformedthedesignoftheMini-GEMswerecognitive

load[12] andmultimedia design theories. These consider processing of audio, visual and

textual information[13] and describe the capacity ofworkingmemory to be limited,with

learningbecomingmoredifficultifthecognitiveloadofataskexceedsthislimit[14].Mini-

GEMswere designed tominimise cognitive load by restricting information on each slide,

ensuringslidelayoutwasminimalistandprovidingtimeforlearnerstoattendtocontent.

7

TargetAudience

Mini-GEMsweredesignedforjuniordoctors.EachMini-GEMisdesignedtocoveraspecific

topic relevant to clinical practice (the Mini-GEMs library is available to view at

aeme.org.uk/mini-gems).

SoftwareandHostPlatform

WechosetohosttheMini-GEMsonYouTube,basedonitspopularityandbroadcompatibility

with internet-connecteddevices,Nologin isrequired,andtherearefewbandwidthissues

withmoderndevicesindevelopedcountries.

Authors&Process

Mini-GEMauthorswereinitiallydrawnfromtheAssociationofElderlyMedicineEducation

(AEME)–anon-profitorganisationseekingtoadvanceeducationofhealthcareprofessionals

caring forolderpatients. Initial topicswereselectedamongst thegroup,basedonclinical

interestsofauthors.Oncetheformathadbeenlaunched,andthelibraryofvideosbeganto

grow,AEMEwerecontactedbyGeriatriciansthroughouttheUKwhowerekeentocreate

theirownMini-GEM.TopicswerejointlyagreedbetweenauthorsandtheAEMEcommittee,

toensureabreadthofcontentthatwouldbeclinicallyrelevanttojuniormedicalstaffworking

witholderpatients.Theelectronicnatureofthematerialmaderemoterecordingofcontent

feasible.TheprocessofcreatingaMini-GEMissummarisedbelow:

1. Authors design their slideshow presentation on their own computer, following

standardisedformattingguidelinesprovided

2. Theslidesareinternallypeer-reviewedbytwo‘editors’atAEMEtoensureclarityand

accuracy-followinganyrequiredrevisions,thepresentationisagreedwiththeauthor.

8

3. Theauthorrecordsnarrationforeachslideusingeithervoicerecordingsoftwareon

theirsmartphoneoracomputermicrophone.

4. Theslidesandaudioarereformattedcentrallytoensureuniformityofstyle,andare

thencombinedtoformavideoslideshowwhichisuploadedtoYouTube.

EvaluationofMini-GEMs

Mini-GEMs were evaluated using a synthesis of objective- and participant-orientated

evaluation[15]. Wewerenotseeking todemonstrate thatwatchingMini-GEMs improved

objective knowledge, but rather to describe if, how and why this format of educational

resourcewouldbeutilisedbyclinicians.

Theaimoftheobjectives-orientatedevaluationwastodeterminetheuptakeofMini-GEMs

and viewer characteristics. Data was collected using YouTube’s built-in analytic software

whichprovidedinformationonthenumberanddurationofviews,geographicallocationof

viewers,anddevicesused.

Theaimoftheparticipant-orientatedevaluationwastoexplorehowusersengagedwithMini-

GEMs,andtheirattitudestowardstheresourceshavingdoneso.Thiswasdoneusingafocus

group.Participantswereattendeesat‘GeriatricsforJuniors’-aUKnationalconferencefor

junior doctors interested in geriatric medicine. Delegates were sent invites before the

conferenceaskingthemtojointhefocusgroupiftheyhadusedMini-GEMs.Participantswere

selectedonafirst-come,firstservedbasis.ThefocusgroupwaschairedbyAG,aconsultant

geriatricianindependentoftheMini-GEMproject.Thediscussionfollowedasemi-structured

topicguide(Appendix1),whichwasgeneratedtoexplorehow,whyandwhenusersaccessed

9

theMini-GEMsandtodescribetheirexperiencesof,andattitudestowards,them.Thetopic

guide was allowed to develop freely during the focus group as areas of interest arose.

Discussionsweredigitallyrecordedandmanuallytranscribed.

Thematicanalysis followedan interpretativephenomenologicalapproach[16],aswewere

looking to explore and describe participants’ experience of, and interaction with, the

educationalmaterial. Transcriptswere initially reviewedandcoded independentlyby two

researchers (MG, JF), with the aim of identifying patterns and themes in the data. The

researchersthenmettodiscussthecodingframeworksproduced.Consensuswasreachedby

mergingsimilarcodesanditerativejointreviewofthedatauntilafinalthematicstructure

wasagreed.Potentiallycontradictoryevidencewassoughtandconsideredagainstemergent

themes(negativecaseanalysis),aprocessrecognisedasbeingcriticaltorigorousanalysis[17].

Ethics

Ethicalapprovalwasnotrequiredforthisstudy,asparticipantswerealleitheremployedby

oraffiliatedwiththeNHS.Thiswasconfirmedbythe localResearchEthicsCommittee.All

participantssignedaconsentformpriortothefocusgroup,outlininghowthedatawouldbe

gathered,storedandused.

10

RESULTS

YouTubeAnalytics

YouTubeanalyticsweredownloaded18monthsafter the initialMini-GEMwaspublished.

Theseshowed22videospublishedby15authorswithacombinedtotalof10,291views,and

38,435minutes of viewing time. Contentwas accessed from 110 countries (76%UK). An

exampleofaudienceretentiondatafromYouTubeanalyticsisshowninFigure1.Thisistypical

ofaudienceretentionpatternsseenacrosstheMini-GEMslibrarywithsignificantaudience

losswithinthefirsttwentyseconds,followedbyamoregraduallossthereafter.Only30%of

initial viewerswere stillwatchingat thevideo’send.Theaverageviewingdurationwas3

minutesand44seconds,equatingto54%oftheavailablevideo.Fullviewingandretention

dataforalltheMini-GEMsisshowninTable1.

FocusGroupData

Thefocusgroupconsistedofsix juniordoctors,withbetween1and4yearspostgraduate

experience,andonefinalyearmedicalstudent,fromacrosstheUK.

Thematic analysis of transcripts identified three main themes, with no significant

disagreementsbetweentheresearchers:

• WhylearnershadchosentouseMini-GEMs.

• HowMini-GEMswereincorporatedintoexistinglearningframeworks.

• TheperceivedimpactofMini-GEMsonclinicalwork.

11

WhyuseMini-GEMs?

Participants identified that Mini-GEMs were easily accessible on mobile devices and

contrastedthiswithpreviousexperiencesofTEL:

Theeaseofaccessingthem…forsomeonewho’sgonetodifferenthospitalswhohas

hadtodoonlinemodules…itwasanabsolutepleasuretoliterallypressit,itwason

YouTube,IhaditonmyiPad,itwassosimple,ratherthanputtinginfifteenmillion

passwords...Ipressedplayonceanditworked.(Participant#2).

TheshortdurationoftheMini-GEMswasattractivetojuniordoctorsandenabledthemto

undertakead-hoclearningnotpracticablewithlengthierresources:

IknowthatIhavefiveminutesfreewithaquickcupofteaandthatIcanplayit.But

that if it’s an hour it’s more daunting and I’m like ‘do I have time to do that’?

(Participant#6)

Ithinkthethingaboutthembeingshortisthatitisnotdauntingtogoandsitdown

whenyou’vecomeinfromalongdayatworkandyou’retired.(Participant#3)

Thereliablenatureoftheformat,whereviewerswereawareofthepre-specifieddurationof

thecontent,wasalsoappealing,sincethe‘risk’ofwastedtimewasminimised:

12

Youcangetdrawnintothinkingsomethingmightbeusefulandthenrealisingit’snot

quitewhatyou’relookingfor.Whereasthesearepositiveintwoways–they’reshort

enoughsothatyoucouldwatchthewholefiveminutes,andevenifyouthoughtthat

wasn’tusefulit’sonlyfiveminutes.(Participant#7)

Mini-GEMsfocussedoncommonlyencounteredscenariosforhealthcareprofessionalscaring

forolderpeople–viewersspokeabouttheclinicalrelevanceofthecontent,whichtheyfelt

helpedfacilitatetranslationofacquiredknowledgeintopractice:

Becausetheyrelatetopatientsyouseeallthetimeitseemsmorelikelythatit’llstick

inyourhead.(Participant#1)

WhatIreallylikeaboutthemmostisthattheyteachyouhowtoapproachpatients.

Sotheygiveyouanapparatus,asystem,anattitudeorastructuretotakewithyou,

sothatwhenyoudogoandseethesepatients….you(are)intherightframeofmind.

(Participant#5)

ParticipantshighlightedthatMini-GEMsdidnotprovideanexhaustiveoverview.Although

furtherworkwasrequiredtolearnin-depthaboutasubject,participantsspokeaboutthisin

positiveterms:

It’sashort,conciseversionofsomethingthat’sgivingyouconceptsratherthanfirm

detailsof it.So ifyoufindsomethingwhereyouthinkhangon, I’mnotquitesure I

knowthedetailsofthat,youknowyoucangoandlookitup.(Participant#3)

13

HowareMini-GEMsbeingused?

Participantsdescribedvariouscontextswheretheyusedthevideos.Theseincluded:priorto

geriatricmedicineattachments;tocomplementrevisionforpostgraduateexaminations;to

aidreflectionfollowingclinicalexperiences;andasanaidememoirpriortoseeinganolder

patient.

Iknewwhatthetopicswere,andthenIgotaskedtoseesomeoneonthewardwitha

fall.SoIthought‘OK,because[there’s]notthatmuchofarush,I’vegotfiveminutes

orwhateverI’llwatchitfirsttogivemyselfabitofarefresher….(Participant#1)

Participants suggested that the Mini-GEMs felt similar to ward-based clinical teaching

deliveredbypeersorseniorcolleagues.

Itwasn’tinapatronisingway,itwasinacomfortingway,almostlikeyouwerebeing

taughtbyoneofyourregistrars,(Participant#2)

ParticipantsseemedtoattachcredibilitytoMini-GEMsbecausethefocusoftheteachingwas

similar to informal ward based teaching by senior colleagues but also because senior

colleagueswithinthespecialtycouldbetrustedtoknowwhattheyweretalkingabout:

Whatgivesthesecredibility?(AG)

Iguessbecausethey’re…donebyregistrarsandconsultantsworkinginthespecialty.

(Participant#1)

14

Andwhydoyouassumethattheyknowwhattheyaretalkingabout?(AG)

Youhavetotrustseniorstoteachanditjustkindofcomesdown(tothat).(Participant

#1)

WhatistheimpactofMini-GEMsonclinicalpractice?

Participants reported that Mini-GEMs increased their confidence in dealing with clinical

problems:

[The Mini-GEMs have] given me more confidence in what I previously thought. I

wouldn’tsaytherewasanythingground-breakingontheonesthatIlookedatbutthey

mademefeelabitmorehappy,canIsay,withwhatIwasalreadydoing.(Participant

#2)

Mini-GEMs helped reinforce good clinical practice and in some cases participants felt

empoweredtoactmoreautonomouslythantheyhaddonepreviously.

Ithinkalotofpeopleare,youknow,dealingwithpatientsontheirownmostofthe

timeandyes,youhaveaccesstoaseniororaregistrar,butyouknow,youwantto

makesurethatyou’redoingsomethingforthemyourself.(Participant#5)

In addition to clinical knowledgeand skills, therewasa suggestion thatMini-GEMsmight

inspire viewers to challenge sub-optimal attitudes and approaches to the care of older

patients.

15

Iwouldlovetoplaythemtosomeconsultantsthatyou’reontakewithandjustthink

actually,thethreeminutesyoujustspentwiththatpatientisnotanadequateamount

of time… ‘Oh, it’s just a UTI’ or ‘Oh it’s a fall’, kind of simplifying things that are

common in the elderly but beneath them have a whole multitude of causes.

(Participant#7)

Participants reported a willingness to promote and share the resources amongst junior

colleagues and medical students, including those working outside the field of Geriatric

Medicine:

Italkedtomyhousemateaboutthem,sheisinterestedinGeneralPractice–thereare

some similarities… shewas quite interested that therewere these snippets online.

(Participant#6)

I think they’d be useful to show…to some of my colleagues in surgery who feel

medicallyunsupported…asasortofsuccinctsurvivalguide.(Participant#7)

DisadvantagesofMini-GEMs?

Werevisitedthetranscriptsseveraltimeslookingforaccountsofnegativeaspects,or

limitationsoftheMini-GEMs.Whilstitwasclearthatthefacilitatorsoughttoelicit

these,participantsdidnotrespondbyclearlystatinglimitationsoftheformat.

16

DISCUSSION

OurstudydescribesthetheoreticalframeworkunderpinningthedevelopmentoftheMini-

GEMsandanevaluationusingrobustmeasuresofuseandusability.Aseparatequalitative

evaluationhelpeddevelopamoredetailedunderstandingofhowandwhyusersaccessed

Mini-GEMsandhowtheybenefittedfromdoingso.Themainfindingsare:

• Mini-GEMs have beenwidely accessed, both in terms of numbers and geographic

distributionofviewership

• Anaverageof36%ofallviewersfollowedtheMini-GEMsthroughtotheirend

• Junior doctors valued them because of accessibility, brevity, simplicity and the

credibilityofpresentersandthematerialpresenteds.

TheMini-GEMswereviewedonmobiledevicesandsharedandaccessedviasocialmedia.

Theconsistentstyleandformatcreatedasenseofbrandfamiliarityamongstusers,enabling

themtoplanhowtoincorporatethemintotheirlearning.Hostingconcisecontentonareadily

accessibleforumincreasedconvenienceforthelearners.

Ithasbeenarguedelsewherethatpromotingaccessibilityofconcise,digestiblecontentmay

leadto‘superficiallearning’,withfailuretointernaliseknowledgeandthatthismightrender

learnersreliantonrevisitingthecontent[18].Againstthis,whilstviewersacknowledgedthat

the brevity of Mini-GEMs limited the amount of content that could be addressed, they

reported that theyused themini-GEMsasa supplement to, rather than replacement for,

othereducationalresources,thusallowingdepthlearningthroughreinforcement.

17

TheMini-GEMs were used in a variety of ways. They were used to facilitate knowledge

acquisition during exam revision, before starting elderly care placements and for

consolidationafterclinicalencounters.Mini-GEMsalsoprovidedlearnerswithframeworks

and schemata for the evaluation of an older patient they recognised as generalisable to

broaderpractice.

Mini-GEMsmayhave influenced learners’attitudes towardsolderpeople.Oneparticipant

describedadesiretochallengesub-optimalattitudestocareofolderpeoplewithfrailty.Her

commentssuggestedhigher-levelreflectiononthelearningandthedevelopmentofinsight

that practice could be improved. Negative attitudes about older patients have been

recognised,evenwithindoctorswhohaveexpressedaninterestingeriatricmedicine[19]–

moreworkisneededtoexploreeducationalstrategiesthataddresstheseattitudes.

Mini-GEMs were used ‘on the job’, enabling ‘reactive learning’ - that is unplanned but

intentionallearningwhich“takesplacealmostspontaneouslyinresponsetorecent,current

or imminent situations”[20]. The use of learning videos as refreshers in theworkplace is

established for procedure-based content[21] but we had not anticipated that Geriatric

Medicine resources, which are less protocol-driven, would be used in this manner. This

demonstrates thepotential forTEL tosupplantorcompliment themoretraditionalquick-

referencetextbooksfrequentlyused.

WatchingaMini-GEMmaybeconsideredpassivelearningduetolackofuserinteractionwith

the video. It has been suggested that medical students may prefer passive learning

18

experienceswhenfatigued[22].Tostrengthenthesocio-collaborativeelementof learning,

Twitter-based discussion related to the content of newly uploadedMini-GEMs has been

encouraged. Combining theMini-GEMswith other learning resourcesmay producemore

effectivelearning-forexample,the‘flippedclassroom’approachmaybeapotentialwayto

integrateTELandface-to-facelearning[23].

Analysingaudienceretentiondataenabledustounderstandusers’viewinghabits.Theinitial

lossofviewersislikelytorepresentviewerswhorapidlyrealisedthattheyhadnointerestin

thesubjectmatter,orwhohadmistakenlyaccessedthecontent.However,themoregradual

declineincumulativeviewingtimeoverthelengthofeachvideosuggeststhatevenforthose

viewerswhocontinuewatchingbeyondtheinitialintroduction,interestwaslostovertime.

Furtherwork is required todetermine theoptimumduration for this styleof educational

resource.

There is potential for resources such as the Mini-GEMs to be used to support clinical

educationinthecareofolderadultsoutsidethespecialtyofGeriatricMedicine,whichmay

be an important aspect of improving standards of care for all older patients within the

healthcaresystem.

Limitations

Caution is neededwhen interpreting data derived from YouTube’s proprietary software -

there are challenges associated with over-interpreting multiple variables gleaned from a

relatively small cohort. In addition, the truemeaningof someof these variablesmust be

19

caveated.A‘view’,forexample,maynothaveactuallyconstitutedatrueviewing.Wehave

norecordofwhatthelearnerwasdoingorthinkingatthattime[24].

Similarly,thedemographicsofindividualviewersisunknown-Itisnotpossibletodetermine

whether the viewers were clinicians or interested members of the public, thus making

interpretationofviewerretentionstatisticschallenging.Insistinglearnersregistertoenable

contenttobeaccessedmayfacilitateprofilingofusers,butdoingsowouldaddabarrierto

accessthatmaydetersomeusersandiscontrarytotheprinciplesofFOAM[6].Theeasewith

whichviewerscouldaccessthevideoswasoneofthekeystrengthsoftheformathighlighted

bythefocusgroup.

Our focus group participants were all UK-based, and were recruited at a conference on

geriatric medicine. As delegates at such a conference, they may already have an innate

enthusiasism for the specialty, and their viewsmaynot be representative of thebroader

community of clinicians accessing the Mini-GEMs, who may have different international

perspectivesandmaybeapproachingthematerialwithlessenthusiasmforthespecialty.

Weacknowledgethatthisevaluationdoesnotprovideevidenceofknowledgeacquisition.

This was by design, sincemeta-analysis level evidence already exists demonstrating that

onlinee-learningisassociatedwithsignificantknowledgegains[25].Instead,theaimwasto

evaluateperceivedstrengthsandweaknessesof the format in the ‘realworld’ (i.e.why it

works)andtoexploreutilisationoftheresource(i.e.howitworks).Cooketal.describedhow

suchresearchquestions(formingso-called‘clarificationstudies’),arerarelyconsideredand

20

howitiscrucialthattheyareaddressedifthescienceofmedicaleducationreseachistobe

advanced[26].

ItisimportanttorecognisethatinsightsgeneratedfromYouTubeanalyticsandasinglefocus

group will be limited in terms of generalisabilty to a wider audience. However, these

representmoredetailedevaluationsthancommonlypresentedinarticlessharinginnovative

e-learningpackages(choosesomeparticularlysuperficialPoGoEarticlesfromJAGShere–get

yourownback!)andtheydeliversomeimportantinsightsintoboththeputativeimpactof

theMini-GEMsand suggest somepossibleareas forboth future investigationandproject

development.Ideally,thethematicframeworkthathasemergedfromtheworkundertaken

so far should be further explored throughmore focus groups, and questionnaire surveys

attached to future Mini-GEMs. It would also be interesting to explore the potential for

multiprofessionaland interspecialtyuseof theseresources,andhowtheymight integrate

withmoreformalstructuredtraining.

21

CONCLUSION

TheMini-GEM format provides an effective way of disseminating free, concise, focused,

clinicalteachingmaterialrelatingtocaringforolderpatientstoawideaudience.Thevideos

werevaluedbyjuniordoctorsduetotheiraccessibility,easeofuseonavarietyofdevices,

their perceived credibility and limited duration. Mini-GEMs were viewed in a variety of

settingsasanadjuncttootherlearningresourcesandledtoimprovedconfidenceincaring

forolderpatients.Furtherworkisneededtoexploretheoptimumdurationofthevideosto

maximisetheirpotentialaseffectiveeducationalresourcesforavarietyofclinicalstaffthat

workwitholderpatients.

22

REFERENCES

1. World Population Prospects: The 2012Revision. (Accessed 13th June 2015, athttp://esa.un.org/wpp/Excel-Data/population.htm.)2. Gordon AL, Blundell A, Dhesi JK, et al. UK medical teaching about ageing isimproving but there is still work to be done: the Second National Survey ofUndergraduateTeachinginAgeingandGeriatricMedicine.AgeandAgeing2014;43:293-7.3. Oakley R, Pattinson J, Goldberg S, et al. Equipping tomorrow's doctors for thepatientsoftoday.AgeandAgeing2014;43:442-7.4. Daunt LA, Umeonusulu PI, Gladman JRF, Blundell AG, Conroy SP, Gordon AL.Undergraduateteachingingeriatricmedicineusingcomputer-aidedlearningimprovesstudentperformanceinexaminations.AgeandAgeing2013;43:721-4.5. RamaswamyR,LeipzigRM,HoweCL,SauvigneK,UsiakC,SorianoRP.ThePortalofGeriatricsOnlineEducation:A21st-CenturyResourceforTeachingGeriatrics.JournaloftheAmericanGeriatricsSociety2015;63:335-40.6. ShawG.BreakingNews:Don'tCallItSocialMedia:FOAMandtheFutureofMedicalEducation.EmergencyMedicineNews2013;35:1-30.7. Forgie SE, Duff JP, Ross S. Twelve tips for using Twitter as a learning tool inmedicaleducation.MedicalTeacher2013;35:8-14.8. NolanT.Asmarterwaytopractise2011.9. Ellaway RH, Fink P, Graves L, Campbell A. Left to their own devices: Medicallearners’useofmobiletechnologies.MedicalTeacher2014;36:130-8.10. Beetham H, Sharpe R. Rethinking Pedagogy for a Digital Age: Designing andDeliveringE-Learning.London:Routledge;2007.11. Beyer AM. Improving Student Presentations: Pecha Kucha and Just PlainPowerPoint.TeachingofPsychology2011;38:122-6.12. Lau KHV. Computer-based teaching module design: principles derived fromlearningtheories.MedicalEducation2014;48:247-54.13. Sandars J, Patel RS, Goh PS, Kokatailo PK, Lafferty N. The importance ofeducationaltheoriesforfacilitatinglearningwhenusingtechnologyinmedicaleducation.MedicalTeacher2015;(earlyonline):1-4.14. Young JQ, Van Merrienboer J, Durning S, Ten Cate O. Cognitive Load Theory:Implicationsformedicaleducation:AMEEGuideNo.86.MedicalTeacher2014;36:371-84.15. Cook DA. Twelve tips for evaluating educational programs. Medical Teacher2010;32:296-301.16. BunnissS,KellyDR.Researchparadigmsinmedicaleducationresearch.MedicalEducation2010;44:358-66.17. BarbourRS.Checklistsforimprovingrigourinqualitativeresearch:acaseofthetailwaggingthedog?BMJ2001;322:1115-7.18. Wallace S, ClarkM,White J. ‘It's onmy iPhone’: attitudes to the use ofmobilecomputingdevicesinmedicaleducation,amixed-methodsstudy.BMJOpen2012;2.19. FisherJM,HuntK,GarsideMJ.GeriatricsforJuniors:Tomorrow’sGeriatriciansorAnotherLostTribe?JournaloftheRoyalCollegeofPhysiciansofEdinburgh2014;44:106-10.20. ErautM.Non-formallearningandtacitknowledgeinprofessionalwork.BritishJournalofEducationalPsychology2000;70:113-36.

23

21. ToppsD,HelmerJ,EllawayR.YouTubeasaPlatformforPublishingClinicalSkillsTrainingVideos.AcademicMedicine2013;88:192-7.22. YavnerSD,PusicMV,KaletAL,etal.Twelvetipsforimprovingtheeffectivenessofweb-basedmultimediainstructionforclinicallearners.MedicalTeacher2015;37:239-44.23. MoffettJ.Twelvetipsfor“flipping”theclassroom.MedicalTeacher2015;37:331-6.24. EllawayR.Scholarshipinanageofbigdata.MedicalTeacher2013;35:613-5.25. CookDA,LevinsonAJ,GarsideS,DuprasDM,ErwinPJ,MontoriVM.Internet-BasedLearningintheHealthProfessions:AMeta-analysis.JAMA.2008;300(10):1181-1196.26. Cook DA, Bordage G, Schmidt HG. Description, justification and clarification: aframework for classifying the purposes of research in medical education. MedicalEducation2008;42(2):128-133