Current challenges for educational technology research

Martin Oliver

What I’m going to cover…

• Technology as revolution• Technology as cause• The myth of learners• Technology as solution• Hope…?

• Technology as revolution

In the film "Groundhog Day", the protagonist is forced to experience the events of a single day over and over again. He is free to act in any way he chooses, but whatever he does the day always finishes in the same way.

Part of the fascination of this predicament is the awful familiarity of this experience: so often one feels caught in a flow of events which will unfold in an entirely predictable way.

People who have been involved over any length of time with educational technology will recognise this experience, which seems characterised by a cyclical failure to learn from the past. We are frequently excited by the promise of a revolution in education, through the implementation of technology. We have the technology today, and tomorrow we confidently expect to see the widespread effects of its implementation. Yet, curiously, tomorrow never comes. We can point to several previous cycles of high expectation about an emerging technology, followed by proportionate disappointment, with radio, film, television, teaching machines and artificial intelligence.

Revolutions

• Multimedia (CD roms)• Hypertexts• World Wide Web• Virtual Learning Environments• Wikis, blogs• Social networks, virtual worlds, games,

twitter…

Revolutions or cycles?

• Proof of concept (needed!)• Replication• Replication in disciplinary contexts

…increasingly “me too” studies

very few, “yes, but” or “no, not really” studies

What do I see getting published?

• Lots of (positive) case studies• A fair amount of studies of organisational

development (inc. academic/staff development)

• Some studies of design/development• Some quasi-experimental studies (quantitative but still

case-based)• A few multi-site comparative studies (qualitative and quantitative)• Some national surveys• Some theory, position papers etc

So how should we be looking at this?

• Do ‘medical’/“agro-botanical” studies better

(Alsop & Tompsett, 2007)

1 Can a new drug be shown to have an effect? Effect

2 Ditto, with an RCT in a selected population that takes drug properly?

Efficacy

3 Ditto, for a typical population that behaves normally Effectiveness

4 Any previously unobserved outcomes post release? Side effects

…or…?

• But…– Still dose/response assumption– Assumes aggregation is unproblematic

• Do something different– Design research– Thesis/antithesis/synthesis theory building– Something else…?

None of it makes any difference anyhow…

• http://www.nosignificantdifference.org/

• The phrase refers to the innumerable quasi experimental “media comparison” studies […] These studies have shown, in other words, that technology-based courses, when compared to those taught in the classroom, do not result in a statistically significant difference in student performance or educational efficiency. From print-based correspondence to courses taught via radio, television, and the Web, the use of new media in each case was not found to result in a statistically significant improvement in educational efficiency.

• Technology as cause

The problem with dose/response models

• “black box” trials – “does it work?”– Attribution is the object of study– Causation is problematic– Explanation is unlikely

• What happens if you open the “black box”…?

Why the medical model never worked anyhow

…analyses [that] comprise an attach on the philosophical principles behind the evidence-based Medicine movement. [… It] undermines the assumption that results from random controlled trials can be applied across varying context without considering the social and structural specifics of the context in which medicine is practiced. […] ‘technological guidelines’ can be problematic if they are posited to be universal while the practice they are meant to guide is very place and culture specific.

(Johnson & Berner, 2010: 76-7)

Perhaps we should be studying practices?

• Social studies of science and medicine– It’s not just “stuff” acting on “stuff” unproblematically– Scientific knowledge is socially constructed– But saying it’s all just talk misses out the hands,

bodies and so on of medical practice

…so how can we bind them back together?

• Mol’s ‘Praxiology’ – the study of practices– Talk, action and their ongoing negotiation

Why are there (apparently) no successes?

• Activity theory– Activity– Action– Operation

• Routine, breakdown and repair• Tacit knowledge• Anything that works, we become unable to

talk about

How does something “work”?

• Studying teachers (HE) starting to use a virtual learning environment

I’m looking for some kind of contribution, any contribution, I look for basically and if I don’t get that then I know there’s probably something wrong. It’s when people are chipping in their bits and then all of a sudden it goes quiet. That’s the danger sign. You do pick up on odd stuff like that – its just transferring what you normally do in normal situations to a virtual environment.

• Activity (Strategy) – no real change…to responsibilities, values, role

• Action (Tactics) – some differences…changes to tools used to ‘look’ for participation,

changes to pace and time of ‘looking’

• Operation – completely different…scan the room, listen for pauses…

…click to generate list of contributors, look for 0’s…

What this reveals is how the move to teaching online renders the role of the teacher both the same and different simultaneously. The purpose and strategic direction may remain unchanged, but the methods of achieving this alter in significant ways.

(Price & Oliver, 2007: 24)

• The myth of learners

Who’s practicing what, exactly?

• Prensky, Tapscott, Oblinger & Oblinger– We know our learners, and they’re not us

(even down to how their brains work)

“rather than being empirically and theoretically informed, the debate can be likened to an academic form of a 'moral panic’” (Bennett et al)

We don’t know our learners…

…but at least we’re trying.

• UK-based studies of learner experiences of e-learning– JISC learner experience projects– ELESIG.ning.com– Institutional studies: what are our learners like?

• National/international studies– In the UK, it’s not so simple, although there are trends

(Eynon, 2009; Jones et al, 2010)– In South Africa, there’s sacrifice to get there and

there’s important pockets of non-engagement (Czerniewicz et al, 2009)

…and what are they doing?

• Not what the headlines suggest…– Selwyn’s “state of the actual”– Creativity isn’t the norm (Luckin et al 2009);

only a few children do ‘interesting’ things like media production

– Not using social networks in a collective wisdom, ‘connectivist’ sense; instead, gossip, banter and ‘identity work’ (Selwyn, 2009)

…or not using it at all? Average age of social networker is 35-44

Open and flexible learning?

• Well intentioned, but flexibility can cause as well as solve problems

(Holley & Oliver, 2010)

• Learners struggling with new barrier• Unfamiliar expectations• New cultural norms to engage with

• Charles – ‘ideal’ student – colonisation of home space for study, with notes “as far as you can see”; no mention of family, constraints; purchases resources “even when they’re not essential”

• Kwame – non-engaged – struggled, but tried to avoid bothering those “in authority” (merely another resources to Charles); struggled to use shared access machines; knew answers but not how to communicate them; eventually helped when he finds a friend

Those with social advantage find it easier to take advantage of new opportunities; advantage can be perpetuated, not eroded, by introducing new forms of learning and teaching

• Refocusing on practice, not technology:

People aren’t using it because it’s technology; they’re using it because they see it as useful in the context of something they’ve chosen to do

• Technology as solution

Back to determinism…

• Given all this complexity, why have we spent so much on this?

• The appealing myth of technology as a solution

• A policy desire?– More technology means more learning– I want to improve education, but it’s tough– But I can buy technology…– So I can buy improvements to education

Not just a caricature…

• New labour policy frames education as an economic endeavour (not necessarily cultural or political)

• Technology is presented as being able to make systems more efficient

• Policy drives…– Purchase (but now saturation)– Some innovation (but the exception not the norm)– Scapegoating of teachers for non-use (it’s not that just

buying tech didn’t work, honest…)

(Saima Rana)

A massive influence on research too

• Funding from JISC, The Higher Education Academy– Proof of concept, implementation projects,

quality enhancement (and learner experience)

(This explains a lot about the technological emphasis/patterns of publication earlier…)

– Even medical model fails: roll-out isn’t a (fundable) research issue

“To use the words of educational technologist Rob Koper […] this research tends not to be “theory-oriented,” but rather “technology-oriented” in character. E-learning research, Koper (2007) explains, is not focused on “predicting or understanding events [in] the world as it exists” (p. 356); it instead seeks to “change the world as it exists” (p. 356; emphasis added). E-learning or technology-oriented research, in other words, attempts “to develop new technological knowledge, methods, and artifacts” for practical ends or purposes (p. 356). It is this applied, practical, and technological research that Koper (2007) says is ideally suited to e-learning.”

• Knowledge-constituative interests– Technical (prediction and control)– Practical (mutual and self understanding)– Emancipatory

(New ALT SIG: Politics, activism and critical theory)

To illustrate some of this…

• Open Educational Resources

OER are teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others. Open educational resources include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge.

How does this fit with Higher Education?

• Echoing moves such as MIT Open Courseware initiative

• In the UK, part of a growing tradition of research and policy– Projects, e.g. OU Learning Design Initiative– Policy about how content should be created

and shared, and how learning should be organised

Dearing report (1997)

• Analysis of technology-related excerpts – Students portrayed as passive, apart from

when choosing a course (described in terms of costs and outcomes), then ‘developed’

– Lecturers not talked about as teachers; expectation of giving up teaching to focus on materials development

– Technology described in terms of access to information

The new interactive media, offering adaptive feedback and student control have the potential to support independent study, but only if fully developed, tested and maintained. […] Many staff would seek to spend some of their time on development of learning materials, because these will enshrine the core of their teaching. […] IT methods must achieve their promise of greater efficiency both by improving the quality of student learning, and by amortising the cost of development over large student numbers.

(Dearing report, Appendix 2)

What constitutes the curriculum here?

• Curriculum as syllabus (a thing to be bought)

• Curriculum as materials (to be consumed)• Some pedagogy (Resource-based

learning)

• Gives no real sense of the experience of learning

The e-University

As the learner progresses through the courseware, there is the opportunity to ask questions by selecting the associated ‘chat’ channel in the toolbar. In response, a chat window opens and the learner is greeted and invited to describe the assistance sought, in text form. The person who answers the questions is part of a call centre and is specifically trained to answer questions about the courseware. […] If the mentor is unable to answer a question, it is referred to a tutor with superior subject expertise, who returns a full answer to the learner by e-mail within a set period.

Not-so-intelligent tutoring?

• Intelligent tutoring systems– Selecting and sequencing of materials for

each learner– Built on a (sophisticated) student model– Big in the 70s/80s, largely abandoned in the

90s as not really worth the effort

(Est. half as good, at best, as tutor support)• Now, available materials ‘personalised’

using learning styles…

Cook recommendation to Denham committee

…a new approach to virtual education based on a corpus of open learning content: the UK must have a core of open access learning resources organised in a coherent way to support on-line and blended learning by all higher education institutions and to make it more widely available in non-HE environments. This needs to be supported by national centres of excellence to provide quality control, essential updating, skills training, and research and development in educational technology, e-pedagogy and educational psychology. All HEIs should be encouraged and helped to exploit virtual education technologies as appropriate to their student’s requirements and their strategies.

Building an effective and competitive on-line learning capacity at both undergraduate and postgraduate level will help meet the changing needs of students and stimulate growth in both higher education and the skills sector. A coherent collection of learning resources can also be exploited to save staff time. Failure to do so will reduce the UK’s ability to exploit e-learning; an aspect of learning and teaching where the UK should aspire to, and gain, a world leading position.

• The UK must have a core of open access learning resources – Why? If they’re open, why not use US ones, etc?

• This needs to be supported by national centres of excellence– Not quite the monolithic e-University, but still about centralisation,

control, and (arguably) preservation of advantage• research and development in educational technology, e-pedagogy

and educational psychology– But not education, ethics, cultural studies… all instrumental

• help meet the changing needs of students– Do we really know what these are?

• Failure to do so will reduce the UK’s ability to exploit e-learning– It’s all about the money

What have we learnt?

• Studies about production and tagging of resources, design of repositories– Relatively little about successful re-use

• Better ways to reify practice– Work of formalisation (e.g. LDSE) and pedagogic

patterns – Relatively little about how teachers have made use of

these

• Processes: of excellent value to researchers, but perhaps of less impact for teachers

• Like Cuban’s studies of technology in schools:– Belief in the power of technology– A moral imperative to change– An economic argument– Scapegoating: staff have ‘failed’ to do this

• Systematic separation of course production from academic role– A power grab in the name of efficiency– Can be traced back to instructional design and “so

called subject matter experts (SMEs)”

What could we learn?

• Millen’s studies of course reading lists– The politics of curricula

• Small-scale work about the co-ordination of course texts by ‘exemplary’ VLE users– Design as performance– Agility and adaptivity vs. transparency and

encoding; practice, spaces and reification• Where are the studies of how teachers

work with textbooks?!

What could we do?

• Cape Town Open Education DeclarationThis […] combines the established tradition of sharing good ideas with fellow educators and the collaborative, interactive culture of the Internet. It is built on the belief that everyone should have the freedom to use, customize, improve and redistribute educational resources without constraint. [… P]art of a worldwide effort to make education both more accessible and more effective.

These resources include openly licensed course materials, lesson plans, textbooks, games, software and other materials that support teaching and learning. They contribute to making education more accessible, especially where money for learning materials is scarce. They also nourish the kind of participatory culture of learning, creating, sharing and cooperation that rapidly changing knowledge societies need.

However, open education is not limited to just open educational resources. It also draws upon open technologies that facilitate collaborative, flexible learning and the open sharing of teaching practices

Hype, hope and disappointment

• Revolutions as repetitive cycles• Difficulty of shifting naïve accounts of the

influence of technology• Little (funded) interest in critical,

emancipatory work

• Where now?

No funding - is there a bright side?

• UK cutbacks marginalising technology in education– As a very ‘new labour’ thing, is it realistic to expect it

to be supported?

• Less research in the area… but will the research that does happen be better as a consequence?– Tapping into established concerns– Working through other disciplines (any discipline is

better than none!)

• Consolidation?– Less innovation, more valuing (and understanding) of

established practice?– More scope to synthesise research and genuinely

address gaps, not just follow fashions?

• Less instrumentalism?– If it’s no longer ‘the solution’, perhaps we can think

differently about it?– Growing interest in critical engagement – can this be

sneaked in…?

Fin.