![Page 1: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/1.jpg)
Systematising the process : the role of PSMs in informing model structure
Jim ChilcottTechnical Director, ScHARR-TAG
Suzy PaisleyDoH Research Scientist in Evidence Synthesis
Paul TappendenSenior OR Analyst
Consensus Working Group on the Use of Evidence in Economic Decision Models MRC HSRC Workshop 26 September 2005
![Page 2: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/2.jpg)
Before we start…
![Page 3: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/3.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Today’s presentation
Define ‘structuring’ within the modelling process
Describe current practice– Evidence used– Tasks undertaken
Discuss ‘What role problem structuring methods (PSMs)?’
![Page 4: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/4.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Defining model structuring
Disease and treatment pathway
Model scope Modelling method
![Page 5: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/5.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:HTA Guidelines
HTA guidelines (n=32)
None 17 (53%)
Description 2 (6%)
Justification 4 (13%)
Sensitivity analysis 5 (16%)
Not available 4 (13%)
![Page 6: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/6.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Modelling guidelines (Philips et al.,2004)
Statement of problem
Justification of approach
Scope / perspective Structural
assumptions Consistency
Strategies / comparators
Model type Time horizon Health states Cycle length Parsimony
![Page 7: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/7.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Reported (Cooper et al., 2005)
HTA monographs (n=42)
Diagram of model 26(62%)
Model structure
and assumptions discussed 5 (12%)
![Page 8: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/8.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Observed
8 case study projects Description model structuring process Short, semi-structured interviews Examination of corresponding study
reports
![Page 9: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/9.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Evidence consulted
Subject Clinical literature
Clinical expert (one-off)
Clinical expert (ongoing)
Existing models
1 2 3 4 5 6 7 8
TOTAL 8 (100%) 1 (13%) 5 (63%) 8 (100%)
![Page 10: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/10.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Reporting
Subject Methodology justified Health states /pathways justified?
Source of structural assumptions reported?
1 Report not yet available
2 3 4 5 6 7 8 TOTAL 5 (71%) 2 (29%) 3 (43%)
![Page 11: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/11.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Summary of themes
Model structure amended according to data availability
Clinician input (1st order validation) Existing models consulted as 1st step Iterative structuring process Complexity of structure shifted both ways Very limited reporting process of development MSc dissertation provided most transparent
reporting
![Page 12: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/12.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Evidence used
Clinical expertise Published evidence Existing models
Natural history Treatment pathway Modelling method Effectiveness
evidence
![Page 13: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/13.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Tasks undertaken
Iteration Consultation Evaluation and comparison Feedback and clarification Decisions regarding simplicity,
complexity, feasibility and inconsistency
![Page 14: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/14.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Why PSMs?
Lack of transparency Data modelling vs problem modelling Validity of model (unquestioned
question) Issues that ‘can’t be modelled’ ‘Not qualified to comment’
![Page 15: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/15.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Why PSMs?
Data led models vs evidence informed models
Use of expert opinion Implicit use of PSM methods Observed usefulness of action research
process
![Page 16: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/16.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
PSMs / MSMs
Messes Problems
SODA
SSM
SCA
Robustness Analysis
Drama Theory
Decision Conferencing
?
Decision analysis
Markov
Queuing theory
Simulation
Game Theory
![Page 17: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/17.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Characteristics of PSMs
Techniques to facilitate exploration / definition of a problem situation
Group based Uncovering options Enabling group action Accepting uncertainty Managing conflict Facilitator - analyst
![Page 18: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/18.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
SODA
Individual interviews =>
Merge maps =>
Group workshops =>
Commitment to action
Cognitive mapsGroup mapExpand/refine map Group ownership
![Page 19: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/19.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
SSM
Enter situation Take action to improve theconsidered problematic problem situation
Define possible changes which areboth desirable and feasible
Express the problem situation
Compare models with the real worldReal world
Systems thinking about the real world
Formulate root Build conceptual models definitions of the systems named in
the root definitions
![Page 20: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/20.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Pros Problem structuring, not problem
solving Gathering information, generating ideas Retain flexibility in early stages Demonstrate understanding of
complexity, diversity and inconsistency Prioritising and simplifying
![Page 21: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/21.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Cons
Methods development required Adaptation to HTA context New skills Ownership Facilitation of group decision making Reproducibility
![Page 22: Systematising the process : the role of PSMs in informing model structure](https://reader036.vdocuments.site/reader036/viewer/2022070406/5681415c550346895dad374b/html5/thumbnails/22.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Developing a mixed method
‘Soft’ qualitative structuring process
‘Hard’ analytic
methods
Qualitative model structuring process to inform quantitative decision-making process.