The Practicalities of Using Research EvidenceKey messages from an LSRN workshop held on 14th November 2014

PurposeA workshop organised by the Learning and Skills Research Network, as one of its bi-annual series, focused on the use of evidence in practice. Practitioners and intermediaries from different parts of the sector shared experiences and developed thinking about the use of research in their various contexts.

BackgroundThere is a growing recognition that relatively little research actually gets used in developing practice or policy. This is a global, not a parochial trend – it is being experienced in many branches of public service in many countries. Post - 16 education in the UK is no exception. Fortunately the problems of using research knowledge have themselves begun to be studied in recent years. As a result we are now aware of key factors in using evidence effectively. For example an OECD study1 highlighted the value of third-party brokers acting between researchers and research-users and a study for LSDA2 emphasises the importance of communicating research outcomes in different modes for different audiences.

Evidence about evidence useA 2014 study from NFER3 reviews research about using evidence at classroom level. One of its authors Julie Nelson outlined its key messages at the workshop, three of which are summarised here.

1. Research evidence is one of several contributors to evidence-informed practice as shown is this diagram.

1 OECD (2007) Evidence in Education: linking research and policy. p26. 2 Nutley, S., Percy –Smith, J. and Solesbury, W. (2002) Models of Research Impact: a cross sector review of literature and practice. London. LSRC. Available online at http://www.tlrp.org/rcbn/capacity/Activities/Themes/Impact/LSDA_models_of_research_impact.pdf3 Nelson, J. and O’Beirne, C. (2014). Using Evidence in the classroom: What Works and Why? Research Summary. Slough. NFER. Available online at http://www.nfer.ac.uk/publications/IMPA01/IMPA01researchsummary.pdf

2. Professional researchers, practitioner-researchers and practitioners in general each have roles to play in the effective use of evidence at classroom level. Several distinct approaches can be identified in linking research and practice, as illustrated in this diagram

3. Both practitioner enquiry at local level and academic4 research are needed if actual practices are to be influenced by high quality evidence. They have different roles to play in the research ecosystem and potentially there can be synergy between the two. The following diagram illustrates this:

4 There is no adequate term to describe research undertaken by professional researchers in research-focussed organisations. The term ‘academic’ is used here to cover all such research whether it is undertaken in ‘academia’ or not.

4. Key messages from Julie Nelson’s presentation include: System –wide change is needed for effective use of evidence Socially interactive approaches appear more beneficial than linear ones For practitioner enquiry

- think first of purpose, then method, then producer- there is scope for synergy with academic research through innovation and

implementation

What can be doneDiscussion led to a number of ideas about what can be done to encourage and support the use of research evidence – about how we think about it and what we do about it.

Thinking

It’s really no use waiting for initiatives from government or its agencies. Very few still exist, those that do have little money and policy has shifted anyway towards self-help. Let’s encourage colleagues to stop counting beans; data needn’t be collected just for accounting to external bodies; it can be used to increase knowledge about how to enhance learning as well.The sector as a whole is accumulating considerable datasets, such as the ILR which could be combined with other data from, for example, UCAS, Ofsted, the census and longitudinal cohort studies, for research purposes – it is hoped the new BIS VET Research Centre will address this.

Practical suggestions

1. The approach of many practitioners who have put evidence to use is quite pragmatic: just make a start, decide what knowledge you need then frame your enquiry accordingly. Armed with evidence from other people can convince people to authorise a local enquiry; them armed with your own and others’ evidence you have better chance of getting buy-in.

2. When you get to discuss evidence-based change with decision-makers, know your stance – where to give way and what to stand firm on. In the prevailing culture, if data is perceived as primarily for audit purposes, use such data as a starting point for enquiries; it shows you are taking the other culture seriously. Bear in mind when advocating an evidence-informed change that there’s usually some uncertainty around the outcome of research (if there weren’t would it necessary to do the research?). This may present a degree of risk for a manager in terms of credibility or resource allocation and needs to be openly acknowledged.

3. People from organisations that are routinely using evidence suggest that time and space need to be freed up for professional development. To persuade managers it helps to build on existing structures such as CPD days and to address acknowledged problems in the institution, perhaps using existing data where possible. Existing approaches to professional development may be modified to favour practitioner enquiry over external courses; there is some evidence that this has more impact.

4. Some organisations have set up an R&D working group successfully. For some purposes it may better to use the concept of ‘practitioner enquiry’ rather than research, to overcome suspicions about the word.

5. Some colleagues have found that devising projects that use information technology in interesting new ways can attract otherwise reluctant teachers. An example given at the workshop concerned use of augmented reality on mobile devices for bricklaying students. Some have found that social media are playing a part, particularly in bringing teachers together to discuss innovative ideas. Twitter is being used to organise “teachmeets”, in which people meet informally in twilight time in a pub or cafe.

6. There are sources of support for practice-based projects. In particular: the Practitioner Support Programme organised by the Education & Training Foundation has offered a number of fellowships over recent years. These include opportunities for initial training plus ongoing support from professional researchers. The new FE Trust for Leadership also offers fellowships for individuals and grants for organisations to study leadership-related issues.

Challenges1. In practice, of course, there are usually barriers to be overcame in making greater use of

research evidence. The outcomes of research by their very nature are uncertain; many planners and budget holders feel uncomfortable coping with this. Some organisations are creating a culture in which data collected for audit purposes are also used to understand patterns through research and enquiry. But for many this has yet to develop. Indeed some may fear that research-based interventions can be disruptive.

2. Practical challenges also arise in getting staff together to collaborate, share and learn from one another. For organisations with many part-time staff, for example in adult or prison education this can be particularly hard to organise.

3. There can also be ‘conservative’ tendency amongst practitioners, accustomed as they are to habitual ways of working and too little time for reflection. One study quoted at the workshop implied that, were Ofsted to be abolished, most staff would want to continue operate according to its model.

4. A large-scale national challenge is to influence the nature of academic and government sponsored research, so that it is more likely to address issues of importance for practice. Experience in heath and other services suggests that practitioners have an important role to play in helping shape the agenda for research. Could the new VET Centre proposed by the McLouglin Commission and the new Vocational Education Research Centre being commissioned by BIS build such an approach into their designs?

Favourable conditions

For research and enquiry to flourish at provider level, these and other challenges need to be met, one way or another. Practical suggestions put forward at the workshop have been described above. For these to prove effective conditions need to be favourable. People from many organisations reported that gaining the support of the senior leadership team is crucial if research is not only to take place, but also to improve the lot of learners. Leadership teams need to be confident enough to take risks. It is advisable for those making proposals to make sure their case is backed up, or at least informed by, evidence.

At the same time there is some evidence that bottom-up initiatives, involving practitioners in the specification, design and delivery of projects can result in greater impact. The suggestion is

that this helps by combining explicit knowledge from a study with tacit knowledge about what works on the ground. Where funding is related to student achievement efforts to improve learning outcomes can also help with the bottom line.

To release the potential of practitioners engaged in research or enquiry, space needs to be created for people to search, read and reflect. Some suggestions about achieving this are offered above. An approach known as ‘joint practice development’ is found by many practitioners to be helpful; it addresses issues of importance in practice collaboratively. By sharing concerns at a workshop it becomes clear that many problems are not just faced by the individual but are shared by colleagues and perhaps others across the sector.

Whichever approaches are adopted for research and enquiry need to be tailored to suit local circumstances; there are no standard approaches to using evidence, no silver bullet - maybe ‘silver pellets’ as one wag expressed it.

ExamplesThe points brought out above were based on discussion of a number of evidence-informed initiatives described at the workshop by their leaders. Details of these are given in the associated workshop booklet. Key points about some of them are given below:

Motivating vocational students (Coleg Meirion-Dwyfor)

Hearing from Chinese teachers at a conference about their research into motivating academic students to address practical problems, the lecturer was inspired to return to college to trial a reversed approach

Vocational/practical students were encouraged to take a conceptual approach

Using a ‘clinical interview’ research method, students working on numeracy problems were asked to give a commentary on their thought processes

The work was described at staff meetings and staff in other vocational areas joined in

Creative approaches to offender learning (HMP Dovegate)

Involvement in an early phase of the LSIS RDF programme trialling a creative approach to increase motivation of learners has proved so productive that the college leader has continued with the SUNCETT and now EMCETT (blended learning) schemes

The initiatives have been expanded across the team taking in first tutors and now learning support assistants in collaboration with a university

The increased motivation of staff and learners has resulted in a move from a grade 3 Ofsted to a grade 2 for teaching and learning

Initial Assessment (Barking & Dagenham College)

The focus was on an important, pressing problem that was affecting many parts of the college

Information was collected about existing practices and new possibilities

An evidence-informed case for change was built up and put to the senior team

An alternative procedure was designed and then piloted

The pilot scheme was evaluated

The college was able to move forward with the knowledge and understanding gained from testing

Developing a research culture (Birmingham Metropolitan College)

A study by a senior member of staff studied the key components of the teaching role

As a result a common time slot was dedicated to staff development

A Professional Development Centre was set up at each campus

A developmental approach was taken to lesson observation, in which leading practitioners played the observer role rather than line managers

Training youth leaders (Walsall College)

An international EU-funded project to develop skills and knowledge of professionals and residents in communities affected by gang and extremist activity

A practitioner enquiry strand is built into the programme in order to develop an evidence-using culture

The cycle of enquiry and intervention will go from: Research question → literature search → intervention design → trial

Andrew Morris 3rd December 2014