An agency of the European Union

SYMPOSIUM PROTECT: The Challenges and Successes What difference will PROTECT make

to regulatory practice?

Xavier Kurz

ICPE, 2015

1

Disclaimer

The views expressed in this presentation are my own views and

may not be understood or quoted as being made on behalf of the

European Medicines Agency or one of its Committees or working

parties.

I have no conflicts of interest to declare.

In this presentation

• Best evidence in the context of

regulatory decision-making

• PROTECT: some successes

• PROTECT: challenges

– Impact of PROTECT on research

– Impact of PROTECT on regulatory

practice

• Conclusions

2

Best evidence in context

• EMA seeks to optimise the benefit/risk profile of medicines

• This involves many complementary initiatives to support knowledge

management and decisions on benefit/risk

• Bottom line is ensuring we are effective in doing this and we are doing

it as efficiently as possible

3

4

• The EU network manages or has direct or indirect access to potentially highly

relevant data, information or knowledge e.g.:

– Safety data from clinical trials (SUSARs in EudraVigilance)

– Suspected adverse reaction reports from marketed use (EudraVigilance data)

– Published literature

– Electronic healthcare records (e.g. THIN, IMS, BIFAP, CPRD)

– Access to networks that have data, and methodological expertise e.g. the

European Network of Centres for Pharmacoepidemiology and

Pharmacovigilance (ENCePP)

– Patient registries

– Research projects

Best evidence in context

5

• The EU network manages or has direct or indirect access to potentially highly

relevant data, information or knowledge e.g.:

– Safety data from clinical trials (SUSARs in EudraVigilance)

– Suspected adverse reaction reports from marketed use (EudraVigilance data)

– Published literature

– Electronic healthcare records (e.g. THIN, IMS, BIFAP, CPRD)

– Access to networks that have data, and methodological expertise e.g. the

European Network of Centres for Pharmacoepidemiology and

Pharmacovigilance (ENCePP)

– Patient registries

– Research projects

Best evidence in context

Pharmacoepidemiological Research on Outcomes of

Therapeutics by an European ConsorTium

To strengthen the monitoring of benefit-risk of medicines in Europe by developing

innovative methods

to enhance early detection and assessment of adverse drug reactions from different data

sources (clinical trials, spontaneous reporting and

observational studies)

to enable the integration and presentation of data

on benefits and risks

Started 1st September 2009, finished 30 June 2015 6

Clinical trials Observational studies

Electronic health records

Spontaneous ADR reports

Risks

Benefit-risk integration and representation – WP5

Signal detection WP3

Benefits

Reproducibility studies

WP6

Training and education

WP7

Signal evaluation WP2

Data collection from consumers – WP4

PROTECT work programme

7

8 By June 2015, 52 publications and many others in preparation

PROTECT:

Successes

www.imi-protect.eu

Presentation title (to edit, click View 9

Benefit-risk integration and representation

www.PROTECTBenefitRisk.eu

10

11

12

GOOD JOB – WORKED

WELL!

BUT…

ULTIMATE JUDGE OF SUCCESS IS WHETHER

THE RESEARCH RESULTS (OUTPUTS) ARE

CONVERTED INTO OUTCOMES FOR

INNOVATION AND PUBLIC HEALTH

Source: Angela Wittelsberger. ADVANCE 3rd General Assembly meeting,

18-19 September 2014

Translation of outputs into outcome

Project

Output Output Output

Output = Short-term result

-product, service, knowledge, e.g.

Database, software, biomarker...)

-Paper, patent, ...

Outcome

Outcome = Long-term result/impact

-Social and economical impact of an

output after (successful)

implementation

-Where possible quantitative

measurement (e.g. costs saved,

QALYs gained, times shortened,...)

13

14

PROTECT: Challenges

What will be the impact of PROTECT?

15

• IMI1 7th Call (2012) Incorporating real-life clinical data into drug

development (GetReal)

This work will draw on the experience gained in the IMI PROTECT project. The

PROTECT project will deliver a standardized approach to the analyses of risk of

events, in order to increase the consistency of observational studies of drug

effects.

• IMI1 7th Call (2012) Developing a framework for rapid assessment of

vaccination benefits and risks in Europe (ADVANCE)

ADVANCE activities will capitalize on the benefit-risk appraisal, integration and

representation for drugs as performed in the PROTECT project.

Benefit-risk monitoring: In WP5 of the PROTECT project, benefit-risk

methodologies for drugs have been appraised and tested - this knowledge will be

utilized in ADVANCE and extended to the vaccine area.

Potential impact on research

16

• IMI1 9th Call (2013) WEBAE – Leveraging emerging technology for

pharmacovigilance

Relevant projects: WP4 of the PROTECT project is relevant to the proposed Call

Topic.

• IMI2 5th Call (July 2015) - Patient perspective elicitation on benefits and

risks of medicinal products, from development through the entire life

cycle, to inform the decision-making process by regulators and health

technology assessment bodies

In particular the activities of this action would strongly build on the project

started by IMI PROTECT WP5 (Benefit-risk integration and representation)/ WP6

(Replication studies) that finished in Q1 2015. http://www.imi-

protect.eu/index.shtml).

Potential impact on research

17

Potential impact on regulatory practice

• When are results matured enough to form a basis to implement

changes in regulatory or clinical practice?

• To what extent should results/recommendations be systematically

validated, scrutinised and peer reviewed in the scientific

community before their implementation?

• Should there be a trade-off between timing of implementation and

scientific replication/validation?

• Which outputs should be prioritised for implementation?

• Are results acceptable to potential users? Will they actually use

them?

Conceptual framework for impact and feasibility assessment

Survey and test of this framework with PROTECT outputs

18

• On 28 May 2015, survey of 264 participants of the Final PROTECT Symposium

(18-20 February 2015)

• Two EMA panels convened on signal detection and pharmacoepidemiology

• Questionnaire developed to assess potential impact and feasibility of the

implementation of the selected recommendations and outputs

• Links to reference documents, publications or presentations provided on each

selected output.

• Survey participants asked to evaluate at least three outputs based on their

expertise

• Survey: 133 evaluations received from 40 participants

• EMA Panel: 97 evaluations received from 14 participants

Objectives of survey: To develop and test on the PROTECT results a

conceptual framework for the review of the potential impact and

feasibility of outputs of regulatory science projects and the

prioritisation of their implementation into regulatory practice

19

Criteria for output evaluation

20

Final PROTECT

outputs selected for

evaluation

Outputs 1 (Inventory of drug utilisation data), 4 (Comparison of covariate adjustment methods), 6 (Evaluation of disproportionality analyses), 9 (Grouping of existing adverse drug reaction terminologies), 11 (Subgrouping and stratification of signal detection), 17 (Recommendations on methodologies for B-R integration and representation) and 19 (Repository of training material) are those that are clearly considered as being ready for implementation.

21

16

53

8

11

14

10 3

9

1

10

12

9

4 4

7

2

6

104

1

5

2

6 2

2

8

1

811

7

2 2

1

1

1

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Do you consider that this output is currently ready for implementation?

Yes No

22

6

1

3

4

3

8

6 2

9

0

8

1 10 0 0

1

0

1

0

0

5

2

0

1

1

3 1

1

6

1

6 84 2 2

1

0

0

00%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Do you consider that this output is currently ready for implementation? - Regulators

Yes No

10

4

0

4

8

6

41

0

1

2

0

1

9

4 4 6

2

5

0

0

5

2

1

4

1

31

1

2

0

2

3

3

0 0 0

1

0

1

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Do you consider that this output is currently ready for implementation? Other respondents

Yes No

Main difference: other respondents (mostly industry)

consider that methods for benefit-risk integration and

representation are ready for implementation.

23

Output 1

Output 2

Output 3

Output 4

Output 5

Output 6

Output 7

Output 8

Output 9

Output 11

Output 12

Output 13

Output 14

Output 15

Output 16

Output 17

Output 18

Output 19

Output 10

Low High

Low

High

Feasibility

Imp

act

Im

pact

Feasibility

24

Feasibility Feasibility

Im

pact

Im

pact

Regulators Other

participants (majority from

industry)

Better scores to outputs related to methods in pharmacoepidemiology and signal detection (1, 4, 5, 6, 7, 8, 9, 19).

Better scores for impact given to outputs related to methods for benefit-risk integration and

representation (14, 16, 17, 18) and for feasibility to PE and SD (1,

4, 9, 11, 15).

25

Discussion of survey results

• Method tested with non-random sample of symposium

participants

• Limited sample size

• Differences in scoring of outputs between regulators and other

participants may reflecting expertise and work priorities

• Visual representation potentially useful for prioritisation

• Assessment criteria, survey methods and scoring matrix to be

further developed.

26

Conclusions: Challenges and successes

Successes:

• Extensive work programme executed in full

• Many publications and outputs

Some of them already implemented

• PROTECT results used as starting point for other research projects

Challenges:

• What will be the actual impact on regulatory practice? And on

innovation and public health?

• How to identify and prioritise those results with the greatest

potential impact?

• How to implement these results?

• How to measure the actual impact on

Process – Behaviors - Outcome

Thank you for your attention

Xavier.Kurz@EMA.Europa.eu

European Medicines Agency

30 Churchill Place • Canary Wharf • London E14 5EU • United Kingdom

Telephone +44 (0)20 3660 6000 Facsimile +44 (0)20 3660 5555

Send a question via our website www.ema.europa.eu/contact

Further information

Follow us on @EMA_News