emea performance indicators pre-authorisation · dossier presentation q nc c 3 q nc c 3 item v2000...
TRANSCRIPT
1
EMEA Performance Indicators Pre-Authorisation
Bo AronssonEMEA
EMEA-EFPIA Info Day 2009
2
Contents
• Analysis Period• EMEA questionnaires• Results• Summary• Work in progress (Predictors of outcome)• Conclusions
3
Analysis Period
• EMEA Data Set: All applications with outcome between 1 January 2003 and 31 December 2008– Period of questionnaires follows annual reporting
to Management Board– Source:
• Questionnaires to (co-)rapporteurs • Scientific Memory Database
• EFPIA Data Set: October 06-October 08
4
EMEA QuestionnairesEMEA Questionnaires• Two versions have been used in 2003-2008
• “Old” version, implemented in 2000
– 10-point Scale (0 dissatisfied to 10 satisfied)
• “New” version implemented in 2007– Keep some of the same domains from “old” questionnaire
– Includes new domains (e.g., Scientific Advice)
– 5-point Likert Scale (1 agree to 5 disagree)
– Note: validation ongoing
• Questionnaires administered after day 80• Average scores between Rapporteur/Co-rapporteur, per product
• Exclusion of duplicates
5
EMEA Questionnaires Compared
5Yes-RMP/PVP
1Yes-Communication
4Yes-Scientific Advice
-3YesSPC, PL, Labelling
-3Q NC CStudy Reports
-3Q NC CSummary
-2NC COverview
11Q NC C3Q NC CEvidence-Data/Design3Q NC C 3Q NC CDossier Presentation
No.V2007No.V2000Item
Q= quality, NC=non-clinical, C=clinical, CPh=clinical pharmacology; CE=clinical efficacy, CS=clinical safety, PVP=pharmacovigilance plan.
6
Data Set 2003-2008 (N=209)• Allows to explore 2 domains and Parts of
Dossier– Presentation of the Dossier for Q, NC and C – Evidence (Data/Studies) included in the dossier for
Q, NC and C
209443643333221No. Questionnaires ("new" +
"old")
74636186928970Compliance (%)281705950363630No. Outcomes
31301No. "new" quest.
178143543333221No. "old" quest.
Total200820072006200520042003Year
7
Product Characteristics (N=209)
40.6785Scientific Advice
75.12157PositiveOutcome
13.8729Other
7.1815V
11.0023N
23.9250L
17.7037J
7.1815C
5.2611B
13.8829AATC
26.7956Orphan StatusPercentFrequency
8
Questionnaire Results
•All scores converted to 10-point Scale (0 dissatisfied to 10 satisfied)
9
Presentation of Dossier (N=209)
Quality Non-clinical Clinical
02
46
810
Sco
re
10
Evidence by Module (N=209)
Quality Non-clinical Clinical
02
46
810
Sco
re
11
Presentation B
y Time (N
=209)
0.001.00
2.003.004.00
5.006.007.008.00
9.0010.00
1-Jan-03
1-May-03
29-Aug-03
27-Dec-03
25-Apr-04
23-Aug-04
21-Dec-04
20-Apr-05
18-Aug-05
16-Dec-05
15-Apr-06
13-Aug-06
11-Dec-06
10-Apr-07
8-Aug-07
6-Dec-07
4-Apr-08
2-Aug-08
30-Nov-08
Outcom
e
Score
Quality
Non-clinical
Clinical
Poly. (Q
uality)P
oly. (Non-clinical)
Poly. (C
linical)
12
Evidence by M
odule by Time (N
=209)
0 1 2 3 4 5 6 7 8 9 10
1-Jan-03
1-May-03
29-Aug-03
27-Dec-03
25-Apr-04
23-Aug-04
21-Dec-04
20-Apr-05
18-Aug-05
16-Dec-05
15-Apr-06
13-Aug-06
11-Dec-06
10-Apr-07
8-Aug-07
6-Dec-07
4-Apr-08
2-Aug-08
30-Nov-08
Outcom
e
Score
Quality
Non-clinical
Clinical
Poly. (Q
uality)P
oly. (Non-clinical)
Poly. (C
linical)
13
Evidence by Orphan (N=209)
No Orphan
24
68
10
Non-clinical
No Orphan
02
46
810
Quality
Sco
re
No Orphan
24
68
10
Clinical
14
Is the Score Associated with Outcome and Clock-stop?
15
Success by Outcome Year (N=209)
IsPos 0 1
PERCENT
0
10
20
30
40
50
60
70
80
90
100
YearFinal2003 2004 2005 2006 2007 2008
16
Clinical Evidence versus Outcome (N=209)
Neg. Pos.
24
68
10
Data Clinical
Sco
re
17
Average Score and Clock-stop (N=209)
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
0 200 400 600 800
Clock-stop (Days)
Ave
rage
Sco
re(a
ll Pr
esen
tatio
n/Ev
iden
ce fo
r Q, N
, C)
18
Summary• Majority satisfaction
– No new time trends– Orphan status associated with lower satisfaction with
Evidence for all modules (and Presentation, data not shown)
• Satisfaction with Clinical Evidence associated with outcome and clock-stop
• Need to improve compliance with questionnaire• Future
– Further validate new questionnaire and explore new domains (work in progress)
– Predictors of Outcome (work in progress)
19
Predictors for Outcome - SA
Work in progress
20
Proportion of Proportion of MAAsMAAs that received SA (by that received SA (by outcome year)outcome year)
38%
47%
56%
0%
10%
20%
30%
40%
50%
60%
2006 (n=50) 2007 (n=59) 2008 (n=70)
SA given
21
Distribution of Scientific Advice over eligibility Distribution of Scientific Advice over eligibility --20082008
Eligibility with/without SA 2008
6 5
16
11
1
1
3
18
5 40
0
5
10
15
20
25
30
35
Biotech MandatoryIndication
Orphan New ActiveSubstance
Signif icantInnovation/Patients
interest
WHO Generic
No SA
SA
22
Did the Company follow SA?
• 31 “new” questionnaires in study period– 16 with SA given– 6/16 (35%) show poor
compliance according to Rapporteurs (score <5)
• Is SA or compliance to SA related to outcome?
Compliance
PoorGood
23
Size of company, success rate and Compliance with SA
36%29%50%56151+***
63%34%72%5321-150**
84%46%89%83Top 20 largest*
Compliance with SA
Proportion with SA
Success rate
Number applications
Pharmasize
*Top 20 largest (n=83) defined as being among the 20 largest companies **21-150 (n=53) defined as being among the 21 – 150 largest companies ***151+ (n=56) defined as not being among the 150 largest companies based on Total revenues 2005 according to Scrips Pharmaceutical League Tables 2006.
Regnstroem et al., (in manuscript)
24
Predictors of Outcome
Data on ranking was only available for 148 applications (work in progress)Stepwise logistic regression. Compliance, retrospectively assigned in Regnstroem et al. (in manuscript)
78.3111.1759.593SA & Compliant vs. (No SA or Not Compliant)
0.7440.1220.301Major Objection on RCT (No vs Yes)
1.9011.161.485Clinical Evidence (0-10)
3.361.0791.904Company Size (1: 151+; 2: 21-150; 3: Top 20 largest)
95% WaldConfidence Limits
Point EstimateEffect
Odds Ratio Estimates
25
Conclusions
• Most important factors associated with outcome – Compliance with Scientific Advice– Company Size– Rapporteurs’ satisfaction with Clinical
Evidence submitted– Major Objections on the Lack or
Randomised Controlled Trials
26
Acknowledgments
• Analysis– Francesco Pignatti– Franz Koenig– Jan Regnstroem
• Data Management– Esther Cozar Calvente– Nadia Kresse– Monica Simeoni