Stories of Flipping:

+ Data Matters

Alan DixTalis & University of Birmingham

http://alandix.com/

about me …

University ofBirmingham

Tiree

Tiree Tech Wave22-26 October 2015

today I am not talking about …• intelligent internet interfaces• visualisation and sampling• situated displays, eCampus,

small device – large display interactions• fun and games, virtual crackers,

artistic performance, slow time• creativity and Bad Ideas• modelling dreams and regret

and the emergence of self

…

… or even lots of lights

http:/www.hcibook.com/alan/projects/firefly/

I am talking about …

Flipping– costs of online and reuse of MOOCs– five shades of flipping– learning analytics and the academic life

Data Matters (if time)– long tail of small data, REF, open data islands, etc.

five shades of flipping

loads of experience …

early adopteruse of web in teaching

&teaching about learning

technology

textbookauthor

online material

even a MOOC

but never flipped

why flip? R&D

reuse MOOC materials

Talis lighthouse trial

why flip? R&D

reuse MOOC materials

Talis lighthouse trial

online HCI courseran early 2013to gain experience

with ‘MOOCs’and reusable

materials

Human–ComputerInteraction

HCIcourse – content

talk-over slides video+ additional resources

HCIcourse – experience

low-quality videois still a lot of work

attrition:1000s of interest100+ formal sign up2 completed

HCIcourse – legacy

loads of videos

course now hostedat OER siteinteraction-design.org

why flip? R&D

reuse MOOC materials

Talis lighthouse trial

lighthouse pilot

universal player

micro-analytics … individual course resource student

why flip? pedagogy

Prosbetter use of face-to-face time

greater student autonomy

more flexible learning

etc., etc., etc., …

Conslots more work visibiity & control

will they do it?

panic!!!!!!

why flip? pedagogy

Prosbetter use of face-to-face time

greater student autonomy

more flexible learning

etc., etc., etc., …

Conslots more work visibiity & control

will they do it?

panic!!!!!!

autonomous learning?

http://de.wikipedia.org/wiki/Erasmus-Programm#/media/File:Erasmus_party.JPG

starting small …

Autumn 2014 course (& 2015 starting now)mix of UG3 & MScportion of course (4 weeks)mixing video with face-to-face

Spring 2015 coursemasters students onlysingle session

different mixes

basics + integrationpreparatory videos on ‘basics’ followed by integrative lecture (chalk & talk!)

2 fully flippedvideos followed by discoursive F2Fvideos followed by group discussions

2 part & partall material on video, some also taught in class N.B. noticable attendance fall-off when told in advance!

70 students

20 students

… a quick dip into the past

p-learning and e-learning

p-learning(physical learning)

• lectures

• tutorials and labs (20-30)

• small groups tutorials ?

• individual tutorials

p-learning e-learning

• lectures web

• labs ?

• tutorials bulletin boards & chat

• individual one-to-one email

translationnot transliteration

deconstruction and reconstruction

p-learning

e-learning

deconstruction reconstruction

deconstructionfind the real objectives

• lecture• information, motivation, demonstration

• group tutorial• collaboration, individual feedback

• lab+ personal experience, physical materials

reconstruction

• take delivery ecologye-learning: web, CD-ROM, video, email, webcam, bulletin boards, chat, streaming

video/audiop-learning: weekend course, monthly evening meeting, summer camp, paper

materials and booksm-learning: PDA, mobile phone, WAP, SMS, 3G

• match with objectives e.g. information -> web good

motivation? … face to face sessionstutorial feedback?

online costs

reuse and online ’content’

online ‘content delivery’:senior mgt pressure since 1990sprincipally for cost saving!

reuse:LOs, SCORM, Tin-Can APIwe all know it’s good in HE use still limited

Jorum

https://pixabay.com/en/headstone-cemetery-grave-graveyard-312540/http://iwantmyanime.deviantart.com/art/Stork-Commission-180796355

every one loves a MOOC(well they did in 2013!)

but what does it cost?

effort: Glasgow University FutureLearn

two courses:Right vs Might

360 hours academic + 800 hrs learning technologist (development only)2.5 hours of video + supporting resources656 participants (first run)

Genomics 2236 hours academic (development only)6 hours video + supporting resources747 participants (first run)

Source: Building and Executing MOOCs: A practical review of Glasgow’s first two MOOCsJ. Kerr, S. Houston, L. Marks, A. Richford (2015)

comparison

• MOOCs– 400 hours development time per hour video– 700 participants per run (time amortised)– £29 statement of participation (~15% takeup)

• Traditional classroom– 2–4 hours preparation per hour lecture– 50-200 students per lecture (time repeated)– £9000 fees (for ~ 200-300 hours lectures)

bottom line

MOOCs vs classroom

10 times as many students

100 times the effort

1/30 payment / student–hour

other estimates?

$39K to $325K per MOOC$74-$272 per completer

Source: Resource Requirements and Costs of Developing and Delivering MOOCs. Hollands and Tirthali (2014)

Udacity ~ $200K per courseEdX ~ $250K design + $50K per run

Source: Why MOOCs Aren't So Cheap ... for Colleges. Fiscal Times (2013)

High quality video ~ $4K per hour (1)

~ $2.5K–10K per minute (2)

Source: (1) MOOCs: Expectations and Reality Hollands and Tirthali (2014)

(2) What does a corporate web video cost? Fox (2010)

benefits

brand awareness (overseas student recruitment)

development consultancy (platform providers)

democratisation of education (… but who pays?)

sustainable?

reuse in face to face?

amortise over online and F2F delivery

MOOC materials:– self-contained units– learner-centric design

? Issues of level (most MOOCs pretty intro)

small is beautiful

video length:often suggested 4 mins or even 2 minssmaller resources improve engagement

(Ferriday)

we saw 10 mins OK but 20 mins too long

=> need better ways to create, edit and manage smaller videos

small things matter

sharing portions not just whole videos=> added end as well as start times to Player

need better audio fade-in – fade-out

simple sequencing

units vs narrative?

learning analytics

learning analytics ….

not just traffic lights!

http://en.wikipedia.org/wiki/Traffic_light#/media/File:LED_Traffic_Light.jpg

analytics – who read/viewed whattypically about 1/3 watch everything, 1/3 some, 1/3 none at all!

used stats to ‘encourage’ students in class

N.B. did not look at individual student analytics

students did not seemphased by this level of analytics

analytics – how much

journal paper PDFrecommended reading

most students just read beginning

in class explained structure of paper

world

organisationalsocial & political

context

directinteraction

data visualisation

visual analytics the big picture

?decision

action

processing

simple model: actors, agents and events

individualresources structures

& courses

repository

?

??

?

academiclife

studentlife

learning supportsystems

creation& reuse

delivery

peerinteraction

community ofpractice

feedback

tutor – studentinteractions

analytics

analytics and action

action

??

?

recognise issues

current coursefuture course

allowMacawber management

analyticsvisualisation

automatic

drivers capability

value

careerdevelopment

resources

time

course materialscommunication

time frames for learning analytics

days and hoursemail, during lectures and labs, student meetings, gaps

weekpreparing for teaching, exercises

months/mid-semesterreporting points, staff meetings, cohort/student progress

end of semester/term/yearexams, exam boards, course review,

start of semester/term/yearpreparing for new courses or re-runs, rollover!

yearsnew courses, professional development, appraisal, promotion

main points so far

many different ways to use online materials– fully flipped, remedial, extension, …

online is costly– reuse essential

… but may need new toolslearning analytics

– new opportunities from detailed LA… but must fit with academic work

patterns

data matters

REF analysis

REF data

loads in the public domain … especially computing

– ACM area codes, Morris’ sub-area profiles

citation analysis and metrics– bad for assessment (volume, special cases)

… but good to validate assessment

Citation Analysis

my own analysis (all public domain data)large apparent sub-area and institutional biasfactor of five to ten!may be due to SP11’s unusual methods

HEFCE’s reportgender differences in Computingfemail staff 30% less likely to get 4*… may be implicit bias from the other effects

long tail of small data

Big Dataeveryone is talking about it

Twitter, Google, Facebook, NSA, universities, … and funding

Big Data does it with MapReduceSemantic Data does it with RDF

the long tail

size ofdata set

a few very large data setse.g. Twitter, streams,Open Govt., OS, geonames, dbpedia the small data of ordinary life:

from local bus timetables to squash club league tables

stories of small data …

Walking Wales

Learning analytics

Open Data Islands and Communities

Musicology