use academic research_by_public_sector_ cherney_gl_conf_2013
DESCRIPTION
Dr Adrian Cherney from the University of Queensland presented some recent findings from an ARC Linkage project investigating the utilisation of social science research in policy development at the Where is the evidence conference 2013 in Melbourne, 11 November 2013. More information on this project which has many aspects to it is available at http://www.issr.uq.edu.au/ebp-homeTRANSCRIPT
Use of academic social research by public officials: exploring preferences and
constraints that impact on research use Adrian Cherney
ARC Linkage project: “The U9lisa9on of Social Science Research in Policy
Development and Program Review” LP 100100380
Other team members: Brian Head, Paul Boreham, Michele Ferguson, Jenny Povey, Jenny Van der Arend and Jenny Bell.
Where is the evidence conference 2013: Recognising the value of grey literature 11th November 2013
State Library of Victoria, Melbourne
Presenta9on outline
• Background to ARC linkage project. • Preferences, access, accessibility and workplace culture & evidence-‐based policy.
• Results from public sector survey. • Concluding remarks.
ARC Linkage Partners
• Four governments: Federal, QLD, NSW, VIC. • Line departments in human services (educaPon, family/community services, public health).
• Central agencies plus ‘knowledge specialist’ agencies.
• Nine funding partners plus another 12 supporPng agencies.
Data sources for ARC Linkage Project
• Targeted survey of Australian social scienPsts n=693 • Targeted survey of policy relevant personnel in 21 agencies n=2084
• Interviews with a selecPon of academic respondents n=100
• Interviews with a selecPon of policy personnel n=125
Public sector agencies and evidence-‐based policy (EBP)
• Perceived gap between research producPon and uptake by governments.
• Understanding access / accessibility important to closing this gap.
• Preferences and organisaPonal context impacts on choices to seek out and access research-‐based knowledge.
!"#$%&'(&)*%+,-%.&/01&2"34-,-2"4%5&-+&.637%8& 93%:;& <%3,%+4&=1>&414"$&<??@&
A1;&1>&.4">>&.637%8&
5-.43-#64%5&41&
B%.21+.%&3"4%&C&
!"#$%&'()('*+,#--(..(#/+0,#--#/1234'56+ 78+ 9:;;+ !+<88+ !!78:88+=%.'"34(3/+>%"23%+#?+@'3'(.'(&.+0,#--#/1234'56++ 99;+ <8:AB+ CC9+ 9A:DE+
F"23.%"*+0,#--#/1234'56+ <9E+ D:A+ !+B88+ !+E8:CD+G2H3"'-2/'+#?+'52+!"(-2+I(/(.'2"+3/$+,3J(/2'+0,#--#/1234'56+ <B+ 8:7C+ !!E88KB88+ KKKKKK+
G2H3"'-2/'+#?+L3-(4(2.M+N#%.(/OM+,#--%/('(2.+P+Q/$(O2/#%.+=??3(".+0,#--#/1234'56+ 9D9+ <9:8A+ <<<D+ 99:78+
G2H3"'-2/'+#?+R$%&3'(#/M+R-H4#*-2/'+3/$+S#"TH43&2+U243'(#/.+0,#--#/1234'56+ ;;+ B:99+ !+<988+ !+C:EE+V%22/.43/$+N234'5+ <<9+ D:EC+ A<7+ <9:9E+
V%22/.43/$+G2H3"'-2/'+#?+,#--%/('(2.+ <88+ B:;+ W#'+T/#1/+ KKKKKK+
V%22/.43/$+G2H3"'-2/'+#?+R-H4#*-2/'M+R&#/#-(&+G2)24#H-2/'+P+Q//#)3'(#/+ CE+ E:D+ <78+ BD:7E+
V%22/.43/$+G2H3"'-2/'+#?+'52+!"2-(2"+3/$+,3J(/2'++ <;+ 8:;7+ 78+ E8:88+
V%22/.43/$+F"23.%"*++ <E+ 8:79+ W#'+T/#1/+ KKKKKK+
V%22/.43/$+G2H3"'-2/'+#?+R$%&3'(#/+3/$+F"3(/(/O++ C8+ E:E7+ 9E8+ E8:BE+
W@S+G2H3"'-2/'+#?+R$%&3'(#/+3/$+,#--%/('(2.++ 7D+ E:<9+ EAD+ <7:B7+
W@S+F"23.%"*++ B<+ <:AC+ 9CC+ <B:;8+
W@S+G2H3"'-2/'+#?+!"2-(2"+3/$+,3J(/2'++ DD+ 9:7B+ W#'+T/#1/+ KKKKKK+
W@S+G2H3"'-2/'+#?+L3-(4*+3/$+,#--%/('*+@2")(&2.++ <DB+ C:EA+ DB;+ 9;:<8+
X(&'#"(3/+G2H3"'-2/'+#?+!43//(/O+3/$+,#--%/('*+G2)24#H-2/'++ 9;+ <:EB+ <8;+ 9D:AE+
X(&'#"(3/+G2H3"'-2/'+#?+R$%&3'(#/+3/$+R3"4*+,5(4$5##$+G2)24#H-2/'++ E;B+ <;:BE+
Q/('(34+(/)('3'(#/+E89EY+'3"O2'2$+
"2-(/$2"+'#+;E;+&2/'"34+.'3??++
!"<9:C8+
X(&'#"(3/+G2H3"'-2/'+#?+N%-3/+@2")(&2.++ <89+ B:A8+ W#'+T/#1/+ KKKKKK+
X(&'#"(3/+G2H3"'-2/'+#?+!"2-(2"+3/$+,3J(/2'++ D8+ 9:B8+ <98+ B9:D8+
X(&'#"(3/+F"23.%"*++ DB+ 9:DA+ W#'+T/#1/+ KKKKKK+
!14"$& DEFG& 'EE& & &
3.9
14.4 14.6
5.3
3.8
23.6
28.6
5.9 7.7
38.0
43.1
11.2
0 %
5 %
10 %
15 %
20 %
25 %
30 %
35 %
40 %
45 %
50 %
Senior execuPve Manager Policy officer Data analyst
Figure 1: Gender and current posiPon
Male Female
6.0 4.5
34.2
16.2
39.2
0 %
5 %
10 %
15 %
20 %
25 %
30 %
35 %
40 %
45 %
Year 12 Advanced Diploma/Diploma
Bachelor Degree Graduate Diploma/Graduate
Cer9ficate
Postgraduate Degree
Figure 2: Level of educaPon
2.5
19.3
23.6
27.1
18.1
8.1
1.3
0 %
5 %
10 %
15 %
20 %
25 %
30 %
Less than a year
1-‐5 years 5-‐10 years 10-‐20 years 20-‐30 years 30-‐40 years 40+ years
Figure 3: Years in the public service
93.1
82.5
72.1
71.6
69.6
63.9
63.8
56.0
53.7
52.1
48.6
37.4
0 % 10 % 20 % 30 % 40 % 50 % 60 % 70 % 80 % 90 % 100 %
Internal agency Staff
Federal government agencies
Other state government agencies in your state
Professional or industry associaPons
University researchers
Comparable state government agencies in other
Interest groups
InternaPonal organisaPons
Private consultants
News media
Think Tanks
Local government
Figure 4: Level of importance placed on informaPon available from parPcular sources to inform decision-‐making:
Very important/important
89.5
44.2
41.4
32.8
32.7
27.5
26.3
22.0
20.6
17.5
12.5
12.0
0 % 10 % 20 % 30 % 40 % 50 % 60 % 70 % 80 % 90 % 100 %
Internal agency Staff
Other state government agencies in your state
Federal government agencies
Professional or industry associaPons
News media
University researchers
Interest groups
Comparable state government agencies in other
Private consultants
InternaPonal organisaPons
Local government
Think Tanks
Figure 5: In the last 12 months, how oien have you consulted with each source of informaPon listed below: Very oien/Oien
68.0
47.5
45.2
43.0
35.6
32.4
28.1
0 % 10 % 20 % 30 % 40 % 50 % 60 % 70 % 80 %
You would prefer to use search engines on the web (e.g. Google)
You would rather consult a work colleague about sourcing relevant arPcles or reports
You don’t know how to use these databases
You have not requested access because such a resource would not be relevant to your role
You don’t have access from your work staPon
You can't download full-‐text versions of academic arPcles and and reports from these databases
The department, agency or unit you work for does not subscribe to any electronic bibliographic
databases
Figure 6: If you don’t access bibliographical databases, is it because: (n = 868)
What might be driving these behaviours?
• Trust in the source? • Accessibility? • Convenience? • Skill sets of individuals? • Few partnership opportuniPes with knowledge producers?
• Access to infrastructure? • Nature of public policy-‐making? • Workplace culture?
55.6
52.3
35.7
35.6
19.3
15.9
14.6
11.4
0 % 10 % 20 % 30 % 40 % 50 % 60 %
There is not enough Pme in the day or week to read relevant research studies
There is limle opportunity to build relaPonships with researchers outside the public service
My department has no formal processes to translate academic research into policy
I lack sufficient decision-‐making power to ensure policy is based on research
The use of research evidence is a low priority of my unit
I do not have the necessary skills to interpret results from staPsPcal analyses
Staff are not encouraged to use research evidence
I lack experPse in how to apply the results of research studies
Figure 7: Accessing and using research evidence in day-‐to-‐day duPes: Strongly agree/Agree
Impact of organisa9onal factors • LogisPc regression model to examine the relaPve strength of
various organisaPonal factors on reported levels of research use.
• Dependent variable measure of research use was based on quesPons asking respondents whether in the last 12 months they had used academic products or outputs to understand policies and programs in their field.
• This measure of research uPlisaPon was divided into a dichotomous variable (0 = Don’t consult academic research and 1 = consult academic research).
32.9 35.3
38.6 38.3
17.9 16.4
10.6 10.0
0 %
5 %
10 %
15 %
20 %
25 %
30 %
35 %
40 %
45 %
In the last 12 months, I have used journal arPcles and books produced by academics to
understand policies and programs in my field
In the last 12 months, I have used research reports produced by academics to understand
policies and programs in my field
Figure 8: ConsultaPon of academic research
Frequently Occasionally Rarely Never
Independent variables – factors influencing research use
• Items related to organisaPonal ethos and culture e.g. there is not enough Pme in the day or week to read relevant research studies; the use of research evidence is a low priority of my unit; staff are not encouraged to use research evidence.
• Judgements about skill levels, and impediments to access (difficulty in accessing full text versions of academic reports).
• EducaPonal level and posiPon.
!"#$%&'(&)"*+,-.&/01"*+/23&,2&+4%&5.%&,6&"*"7%0/*&-%.%"-*4&1-,75*+.
!"#$%&'()*+"#$),-(%,($./(0'/(%1("#"2/*)#(&/'/"&#.(+&%20#$'((
( ( 345(67(1%&(822'(9"$)%(
(:( ;<( =%>/&(
822'(9"$)%(
?++/&(
@#"2/*)#(&/'/"&#.(&/'0A$'("&/(#%,')2/&/2(&/A/B",$(CD(*D(#%AA/"-0/'(
EFG4HHH( IEFE3J( KFGL( MFKK( MF4E(
9/'/"&#.()'()*+%&$",$(),(*D(+&%1/'')%,"A(1)/A2( EFGNHHH( IEFE3J( KFGG( MFE3( MFNG(
="#O(&/'/"&#.('O)AA'( EFE3( IEFELJ( EF3N( KFE3( KFMG(
="#O("(&/'/"&#.(#0A$0&/( PEFQQHH( IEFKKJ( EF4L( EFGM( EF3E(
R%A)#D(*"O),-(C"'/2(%,('%0,2(/B)2/,#/( PEFE3( IEFKEJ( EFG4( EF3K( KFKK(
S)11)#0A$D("##/''),-(10AA($/T$( EFUNHHH( IEFE3J( KF43( KF3E( MFMG(
@2B",#/2(2)+A%*"( EFQN( IEFQGJ( EFUL( KFNK( MF3M(
V@(( EFN4( IEFMUJ( EF3N( KF4G( MFUQ(
W&"20"$/(2)+A%*"( EFGKH( IEFM3J( KFKU( MFEN( QF43(
R%'$-&"20"$/(2/-&//( EF3NHHH( IEFMGJ( KF4K( MF4U( NFQU(
;/,)%&(/T/#0$)B/( PEFKM( IEFMGJ( EF4M( EFL3( KF4M(
X","-/&((( PEFNQHH( IEFK4J( EFNL( EFU4( EFLG(
S"$"(","AD'$( PEFG3HHH( IEFMKJ( EFQE( EFNU( EFU3(
Y/'('$"11(A),O),-($%(&/'/"&#./&'( EFQM( IEFKGJ( EF33( KFQL( KF3M(
6%,'$",$( PNF3LHHH( IEFGUJ( EFEE( EFEK( EFEQ(
8C'/&B"$)%,'( K33U( ( ( ( (
R'/02%(9M( EFMQL( ( ( ( (
=%-(A)O/A).%%2( PGUKFGL4( ( ( ( (
#.)M(I21ZMEJ( NGUFM4( ( ( ( (
Standard errors in parentheses * p < 0.05, ** p < 0.01, *** p < 0.001 The results reported in Table 4 provide support for the argument that an overall organisational ethos
and professional culture that value research has a bearing on the uptake of academic research among
policy personnel, well above any perceived deficits in individual skills. As stated the positive
relationship observed between experiencing difficulties in accessing articles and reports and the
uptake of academic research is somewhat unusual. This can perhaps be explained by the fact that
respondents who are actively engaging academic research and are using it to inform their work
Conclusion: Public sector use of research evidence & EBP
• Convenience and expediency mamer a great deal. • Physical or electronic availability important – but not overwhelmingly so.
• Day-‐to-‐day pressures and constraints reinforce various pamerns of informaPon seeking.
• OrganisaPonal ethos and professional culture help to generate behaviours that promote research use.