impact by design . stephen roper. isbe 2014

Impact by design

Stephen RoperStephen.roper@wbs.ac.uk

Impact is …• Impact is according to the Economic and Social Research Council (ESRC):

– ‘…the concept of impact in the social sciences applies to all sectors … It embraces economic and societal impact in the sense of direct and often quantifiable economic benefits; wider social impacts that will benefit society more generally such as effects on the environment, public health or quality of life; and impacts on government policy, the third sector and professional practice’.

• And occurs through a range of different routes - The 5 ‘I’s…

– Impact through world class social science research– Impact through skilled people– Impact through world class infrastructure– Impact through international leadership– Impact through partnerships

• And it matters because research can change the world and improve lives but also – for us – as research quality and impact metrics will (probably) have equal weight in the next REF.

Mapping

Engagement

Build evidence

Communicate

Impact

Impact by design…

Aim today:

To share some experienced based reflections on impact and how we can design impact in from the outset of a project.

Four key elements ->

Impact by design – 1: Understanding the user community • Consider conducting a ‘stakeholder

review’ or ‘stakeholder mapping exercise’

• Discussions with potential users to identify ‘interest’ and ‘influence’. (Also promotes awareness too).

• Subjective of course but provides basis for targeting relationship development and dissemination

• Used very effectively by Henry Overman at LSE Centre for Spatial Economics Influence

Interest

Impact by design -2: Engaging the user community

• Increasingly ESRC and other research councils are looking to see user engagement in project definitions. Will not define focus of work but may determine priorities or contingent issues.

• Early interaction may also create the potential for partnering and help with issues of access where this is a problem.

• How? Lead users. Reference Groups. Irish cheese example.

• Research is about knowledge production but can be: – Mode 1 – the traditional,

investigator-initiated, curiosity-led and discipline-based approach

– Mode 2 is problem-focused and interdisciplinary. Limoges (1996:14-15) wrote –

– 'research carried out in a context of application, arising from the very work of problem solving and not governed by the paradigms of traditional disciplines of knowledge’.

• What do you engage in?

Impact by design -3: Building the evidence

• Evidence needs to be:

• (1) Robust ….– Are samples large and representative ?– Are survey methods/measures robust? – Are analysis methods rigorous?

(2) Significant …– Statistical significance is not enough. Provide

evidence of both statistical significance and economic and social importance

• (3) Relevant – Does it address policy question?– Is it available in a timely enough manner?

• Assessing the evidence we can provide needs a reflexive approach. PhD example

• Evidence and evidence-based policy

• Internationally, the ‘movement for evidence based policy’ has re-emphasised the value of evidence in policy design and development. Lunn (2013, p. 99), for example, quotes the US Coalition for Evidence-Based policy as follows:

• • ‘… to increase government

effectiveness through the use of rigorous evidence about what works’ [because] ‘… a concerted government effort to build the number of these proven interventions, and spur their widespread use, could bring rapid progress to social policy similar to that which transformed medicine’.

Impact by design – 4:Communication

• Who reads academic papers apart from us? Few people!

• Lessons from the ERC experience:

– No substitute for F2F interaction – Blackberries are poor with attachments – One page good, four pages bad– Communication needs to be targeted (and

invested in)– Think beyond the paper or the website– Always think about the story

• Different types of dissemination or knowledge transfer exist all with different advantages – perhaps key point is that we need different ‘channels’ for different user groups

• The UK-US Ambition Gap

Source: Perkmann and Walsh (2007)

Impact by design - 4: Communication channels

Impact by design – 5:Evidencing impact

• Is very often difficult! No simple answers.

• Key issues, reflecting those in standard policy evaluations are:– identifying and estimating effects on outcomes– establishing causality and attribution – being clear about contingent influences/barriers to impact – Providing convincing evidence

• No clear methodology except documentary or narrative approach so important to keep good notes and maintain contact with potential users

Mapping

Engagement

Build evidence

Communicate

Impact

Impact by design…

Thank you ….