managing risk in outsourced clinical trials
DESCRIPTION
The State of Clinical Outsourcing: Managing Risk in Outsourced Clinical Trials Interactive Discussion Based on Data from Avoca’s 2013 Industry Research DIA Forum June 24, 2013TRANSCRIPT
The State of Clinical Outsourcing Managing Risk in Outsourced Clinical Trials Interactive Discussion Based on Data from Avoca’s 2013 Industry Research DIA Forum June 24, 2013
2
Panel Members
Denise Calaprice-Whitty (Forum Chair) Senior Consultant, The Avoca Group Andy Lee Senior VP, Deputy Head of CSO Platform, Genzyme, a Sanofi Company Mitchell Katz Executive Director, Medical Research Operations, Purdue Pharma L.P. Alistair Macdonald Chief Operating Officer, INC Research
3
Today’s Discussion
Presentation of Data Highlights from Avoca 2013 Industry Survey, and Interactive Discussion
Avoca 2013 Industry Survey Topic: Management of Risk in Outsourced Clinical Trials
• Part One: Risk Sharing Models • Part Two: Risk Assessment • Part Three: Risk Management
4
2013 Avoca Research: Questions Explored
• Risk-sharing models: What kinds of models are most often used? Under what circumstances? What are the magnitudes of the incentives/penalties? Have they been successful? In what ways? What have been the downsides?
• Risk assessments: How often are these conducted, and by whom? Are they just qualitative, or also quantitative? On what types of risks do they focus? Do stakeholders generally agree about risks? Have these approaches been successful? In what ways?
• Risk-based management approaches: To what tasks have such approaches been applied? Have they been successful? In what ways? What have been the downsides? How has this work been viewed by regulators?
5
2013 Avoca Research Respondents
237 Respondents
• Sponsors: 113 Respondents from 70 companies – Approximately half in top 20 in terms of revenue
• 66% Pharma • 26% Biotech • 5% Device • 3% Combination/Other
• Clinical Service Providers: 124 Respondents from 66 companies – 60% in top 20 in terms of revenue
• 86% CROs • 11% Niche • 3% Laboratories
6
Part One: Risk Sharing
7
Types of Risk-sharing Between Sponsors and Providers
• Provider bonuses for achieving milestones or targets
• Provider penalties for missing milestones or targets
• Guarantees of continued work/revenue stream in exchange for provider commitments
• Provider stake in outcome of study (e.g., company stock)
8
Prevalence of Risk Sharing: Sponsors
0% 5%
10% 15% 20% 25% 30% 35% 40% 45%
Stra
tegi
c pa
rtne
rshi
ps/
allia
nces
(N
=28)
Pref
erre
d pr
ovid
er
rela
tion
ship
s (N
=43)
Tras
acti
onal
re
lati
onsh
ips
(N=3
7)
Bonuses Penalties Guarantees of continued work/revenue Provider stake in outcome
For each of the types of outsourcing relationships listed, which of the risk-sharing models listed has your company used?
9
N=33
Magnitude of Bonuses and Penalties
3% 3%
27%
61%
6% >20% of contract value
11-20% of contract value
6-10% of contract value
2-5% of contract value
<2% of contract value
Sponsor Data: When bonuses are used, what is generally the magnitude of the
maximum possible bonus?
3% 3%
26%
61%
7% >20% of contract value
11-20% of contract value
6-10% of contract value
2-5% of contract value
<2% of contract value
Sponsor Data: When penalties are used, what is
generally the magnitude of the maximum possible penalty?
N=31
10
Experience with Risk-sharing by Sponsors
54%
35%
14%
41%
56%
41%
6%
9%
45%
0% 20% 40% 60% 80% 100%
Guarantee of continued work/revenue stream in exchange for provider commitments
Provider bonuses for achievement of milestones/targets
Provider penalties for failure to achieve milestones/targets
Primarily positive A mix of positive and negative Primarily negative
N
54
57
58
To date, my experience with each of the below risk-sharing outsourcing models has been…
11
Experience with Risk-sharing by CROs
58%
46%
24%
40%
48%
47%
2%
7%
29%
0% 20% 40% 60% 80% 100%
Guarantee of continued work/revenue stream in exchange for provider commitments
Provider bonuses for achievement of milestones/targets
Provider penalties for failure to achieve milestones/targets
Primarily positive A mix of positive and negative Primarily negative
N
61
51
53
To date, my experience with each of the below risk-sharing outsourcing models has been…
12
Sponsor Data: All Respondents Overall, how satisfied are you with the work that has been done for you by Clinical
Service Providers (including but not limited to CROs)?
5%
50%
60%
71%
100%
50%
30%
24%
10%
0% 20% 40% 60% 80% 100%
>10% of contract value
6-10% of contract value
2-5% of contract value
<2% of contract value
Very satisfied Generally satisfied Neither satisfied nor dissatisfied Generally dissatisfied Very dissatisfied
N
2
10
21
2
Magnitude of the maximum possible penalty
13
Comments on Experiences
• “These relationships require absolute clarity in roles and responsibilities and up-front planning assumptions that we have not always had, which has caused difficulties along the way.”
• “The inclusion of provider bonuses and penalties has had a detrimental affect on study team morale. OK when the target has been reached outright, but lots of 'extra' negotiating when a negative outcome.”
14
Comments on Experiences
• “In my opinion a CRO doesn't perform better only because they have a bonus or a penalty. The bonus is a way to show them appreciation at the end but never a motivational factor. Aligning with the CRO at the beginning on deliverables and sharing risks and pitfalls before signing the contract will increase the trust level and consequently the commitment.”
15
Comments on Experiences • “CRO must have complete decision-making authority on the
operational aspects of trial execution (minimal handoffs/decisions by sponsor). CRO must be compensated for a thorough feasibility assessment and allowed to select the suppliers they believe will deliver (without sponsor imposing non-producing investigators in the mix). Also, sponsor company must be willing to provide both the penalty as well as the reward. In my experience, penalties were never applied because there were always mitigating factors (sponsor impacted operational success due to slow response or non-response to needs or due to sponsor mandate of key requirements- such as SOPs, KOLs, etc.).”
16
Part Two: Risk Assessment
17
39%
19%
26%
13% 3%
N=31
Prevalence of Systematic, Formal Risk Assessments in Outsourced Trials
38%
18%
16%
14%
14% For >75% of outsourced clinical trials For 51% - 75% of outsourced clinical trials For 25% - 50% of outsourced clinical trials For 1% - 24% of outsourced clinical trials Never
N=50
Sponsors CROs
Most sponsors and clinical service providers reported that a systematic risk assessment process was used for the majority of
their clinical trials.
18
Sponsor Use of Systematic Risk Assessment Processes: Is there a difference based on size of company?
50%
32%
13%
21%
19%
15%
13%
15%
6%
18%
0% 20% 40% 60% 80% 100%
Yes
No
For >75% of clinical trials For 51% - 75% of clinical trials For 25% - 50% of clinical trials
For 1% - 24% of clinical trials Never
N
16
34
Top 20?
19
N=63
CRO Involvement in Systematic Risk Assessment
32%
47%
16%
5%
Sponsor Data:
For outsourced clinical trials, to what extent is the CRO generally involved in
the systematic risk assessment?
Provider Data:
To what extent is your company generally involved in the systematic risk
assessment?
48%
39%
5% 8%
N=66
20
What Risks are Assessed?
0%
10% 20% 30% 40% 50% 60% 70% 80% 90%
Pati
ent
enro
llmen
t ri
sks
Vend
or p
erfo
rman
ce
risk
s
Dat
a qu
alit
y ri
sks
Oth
er t
imel
ine
risk
s
Site
com
plia
nce
risk
s
Cost
ris
ks
Clin
ical
tri
al s
ubje
ct
safe
ty r
isks
Dru
g/de
vice
sup
ply-
rela
ted
risk
s
Risk
s to
rig
hts
of
clin
ical
tri
al s
ubje
cts/
ethi
cs
Oth
er
Sponsor (N=70) Provider (N=75)
When a systematic risk assessment is performed for projects conducted by your company, which of the following risks are typically formally assessed?
21
N=68
Risk Assessment: Qualitative or Quantitative?
43%
9%
41%
7% Primarily qualitative
Primarily quantitative
Approximately equal mix across trials It depends
Is the risk assessment process primarily qualitative (e.g. risks identified and discussed), or quantitative (e.g. probabilities and consequences of specific risks
estimated quantitatively)?
32%
15%
53%
Primarily qualitative
Primarily quantitative
Approximately equal mix across trials
N=66
Sponsors CROs
22
Part Three: Risk Management
23
How often does your risk assessment process lead to the introduction or
refinement of proactive measures designed to reduce risk?
24
Reducing Risk: Sponsor Data
51%
50%
34%
34%
31%
29%
29%
28%
28%
24%
12%
15%
31%
28%
19%
20%
31%
21%
12%
22%
26%
25%
27%
34%
40%
39%
36%
32%
47%
41%
12%
10%
8%
4%
10%
11%
4%
19%
12%
13%
0% 20% 40% 60% 80% 100%
Review plan for performance data
Capture of performance data (i.e. not clinical trial data)
Changes in numbers of sites selected
Training additions/enhancements
Personnel additions/enhancements
Protocol amendments
Procedural additions/enhancements
Changes in locations of sites selected
Decisions regarding CROs/other vendors used
Addition or removal of specific sites
For >50% of clinical trials For 25% - 50% of clinical trials For 1%-24% of clinical trials Never
N 43
48
49
50
42
44
45
47
49
46
How often does your risk assessment process lead to the introduction or refinement of each of the following proactive measures designed to reduce risk?
25
Reducing Risk: Provider Data How often does your risk assessment process lead to the introduction or
refinement of each of the following proactive measures designed to reduce risk?
47%
45%
38%
34%
31%
23%
21%
18%
16%
14%
22%
19%
32%
36%
35%
33%
27%
33%
19%
13%
26%
30%
30%
23%
33%
35%
48%
43%
56%
50%
4%
6%
7%
2%
9%
5%
8%
9%
24%
0% 20% 40% 60% 80% 100%
Review plan for performance data
Capture of performance data (i.e. not clinical trial data)
Procedural additions/enhancements
Changes in numbers of sites selected
Training additions/enhancements
Changes in locations of sites selected
Addition or removal of specific sites
Personnel additions/enhancements
Protocol amendments
Decisions regarding CROs/other vendors used
For >50% of clinical trials For 25% - 50% of clinical trials For 1%-24% of clinical trials Never
N 46
47
44
44
46
43
44
40
43
38
26
N=50
Use of Risk-related Information to Determine Oversight of CROs
24%
22%
20%
14%
20% For >75% of clinical trials
For 51% - 75% of clinical trials
For 25% - 50% of clinical trials
For 1% - 24% of clinical trials
Never
Sponsor Data
27
Satisfaction with CROs
9% 73%
56%
9%
41%
9%
4%
0% 20% 40% 60% 80% 100%
For >50%
For <50%
Very satisfied Generally satisfied Neither satisfied nor dissatisfied Generally dissatisfied Very dissatisfied
How often do your teams use risk-related
information to determine the level
and/or type of sponsor oversight that you will
employ for your CRO partners?
N
23
27
by Use of Risk-related Information to Determine Oversight
28
Satisfaction with Risk Assessment and Management Performance
Scale of 1 to 5; 1=Very dissatisfied, 5=Very satisfied
29
Opportunities to Enhance Understanding
38%
13%
9%
23%
54%
36%
31%
23%
45%
8%
10%
9%
0% 20% 40% 60% 80% 100%
Executive Management
Middle Management
Project Management
Very strong understanding Good understanding Fair understanding Poor understanding
N
13
39
11
How would you rate your understanding of best practices in risk assessment and management in clinical trials? (Sponsor Data)
While executives generally reported a strong understanding, operational teams may benefit from more training.
30
Are risk-based approaches leading to greater efficiency and higher quality?
14%
32%
38%
34%
23%
6%
25%
28%
0% 20% 40% 60% 80% 100%
more efficient use of resources for your company and/or
your CRO partner?
increased quality?
Yes Sometimes No Don't know/too soon to tell
N
65
65
In general, have your risk assessment and management approaches resulted in…
31
Thank you!
www.theavocagroup.com
32
Definitions Included in Avoca Survey
Risk is defined as "the combination of the probability of occurrence of harm and the severity of that harm" (ISO/IEC Guide 51). RISK ASSESSMENT AND MANAGEMENT (taken from FDA Guidance for Industry Q9: Quality Risk Management). Risk assessment: A systematic process of organizing information to support a risk decision to be made within a risk management process. It consists of the identification of hazards and the analysis and evaluation of risks associated with exposure to those hazards. Risk management: The systematic application of quality management policies, procedures, and practices to the tasks of assessing, controlling, communicating, and reviewing risk.