mdic maturity model...• cmmi® - for official record • mdic - for ecosystem ‘opt-in’ and...
TRANSCRIPT
![Page 1: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/1.jpg)
MDIC Maturity Model
George Serafin, Vizma Carver, Becky Fitzgerald,
Cisco Vicenty
October 26, 2016
![Page 2: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/2.jpg)
2
Quality System Model Benefits
Implementing a standard, recognized maturity model across the medical device industry, could produce
a wide range of benefits.
![Page 3: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/3.jpg)
3
Maturity Model Team
Regulators Industry Experts
Deloitte Project
Management
Support
POC, Panel and Pilot,
Prepare Team Leadership
Vizma Carver
Carver Global
Health Group
Becky Fitzgerald
Two Harbors
Consulting
Cisco Vicenty
FDA
Chris Brown
FDA
Bleta Vuniqi
FDA
George Serafin Deloitte
Jodie Bastian
Stryker
Mark Rutkiewicz
Innovize
Ann
Lee
Bob Kenney
Veeva
Chavonne Hoyle
CMMI
Hans Beinke
Siemans
Michael Crader
Medtronic
Nicole
Schumacher
Nick Premnath
Deloitte
Workgroup Leadership
Jason Spiegler
Siemans
Dan Torrens
CMMI
Jeff Fecho
St. Judes
Patterson Shafer
Deloitte
Doug Reimer
JnJ
Sarah Deegan
Johnson & Johnson
Pat Baird
Baxter
Al Crouse
CVRx
Rahmat Muhammad
Deloitte Peter Shearstone
Sysmex
Vipul Sheth
Medtronic
Justin Grant
JnJ
Stephen Zielinski
Medtronic
Kimberly Paris
B Braun Medical
Doug Grindstaff
CMMI
George Zack
Two Harbors
Consulting
Jim Shaver
Two Harbors
Consulting
Kathyrn Tate
CMMI
Diarmuid Flavin
Creganna
James Dabbs
Medtronic
Sonia Carbajal
JnJ
![Page 4: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/4.jpg)
• An internationally recognized
framework
• Flexible and agnostic to the project,
product, or services lifecycle
• A representation of mature capabilities
• A measurement mechanism: baselining tool (gap analysis)
benchmarking tool (ratings potential)
• Describes an improvement path
4
What is the
Capability Maturity Model Integration
(CMMI®)?
![Page 5: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/5.jpg)
Continuous Improvement
Data-driven Decision making
Useful Process Infrastructure
Plan your work; Work your plan
Work happens
5
Maturity Levels 5
1
2
3
4
Initial
Managed
Defined
Quantitatively Managed
Optimizing
![Page 6: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/6.jpg)
Use of the Standard CMMI® Appraisal Methodology for
Process Improvement (SCAMPISM) to demonstrate the
CMMI® framework to the MDIC and FDA Stakeholders
6
MDIC Pilot Appraisal
![Page 7: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/7.jpg)
CMMI® SCAMPI Classes
Provides full flexibility: DISCOVER
• Appraisal process
• Model scope
• Organizational scope
B
C
A
Provides less flexibility: AFFIRM
• Appraisal process
• Model scope
• Organizational scope
• Gather evidence for all practices
Least flexible: PROVE
• Appraisal process
• Model scope – required elements for
rating
• Organizational scope – sampling rules
• Evidence gathered and considered with
affirmations
Maturity level rating only possible for SCAMPI A
![Page 8: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/8.jpg)
Process & Product QA
(PPQA)
Organizational Performance Management (OPM)
Quantitative Project Management (QPM)
Technical Solution (TS)
CMMI® Basics
Reqts Mgmt (REQM)
Project Planning (PP)
Project Monitoring
& Ctrl (PMC)
Supplier Agreement
Mgmt (SAM)
Config Mgmt (CM)
Measurement & Analysis
(MA)
Product Integration
(PI)
Verification (VER)
Validation (VAL)
Requirements Development
(RD)
Risk Mgmt (RSKM)
Organizational Process
Definition (OPD)
Organizational Process Focus
(OPF)
Decision Analysis & Re-solution (DAR)
Organizational Training (OT)
Integrated Project Mgmt
(IPM)
Organizational Process Performance (OPP)
Causal Analysis & Resolution (CAR)
Level 3
Level 4
Level 5
Level 2
8
![Page 9: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/9.jpg)
Generic Goal 2:
Institutionalization
Protects the organization’s work practices in times of stress
GG 2 Practices
“Institutionalization”
Set Policy
Plan the Process
Provide Resources
Assign Responsibility
Train People
Objectively Evaluate
Adherence
Review Status with Higher
Level Management
Monitor and Control the
Process
Identify and Involve
Relevant Stakeholders
Control Work Products
9
![Page 10: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/10.jpg)
How is CMMI® Different from other quality
system models?
10
![Page 11: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/11.jpg)
11
Compare the components
CMMI® QSR Others
• “What are the differences?”
Compare the implementation
CMMI® begins with a broad question & uses Appraisals to evaluate
• “Who are your clients internal and external, and what are their requirements”
QSR begins with the specific question & uses Audits to evaluate
• “What medical device products do you make, and how do you manage the process”
Other standards based begin with a prescriptive approach & use Audits to evaluate
• “What medical devices and components do you make, and how do you execute the specific measures and activities to ensure within thresholds?”
CMMI® relationship to
QSR and others
![Page 12: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/12.jpg)
12
CMMI and Regulatory
CMMI (Certified Coach)
Regulatory & Notified Bodies
(Rules of Engagement/Enforcement)
![Page 13: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/13.jpg)
13
CMMI - Six Sigma - ISO
CMMI (What)
Six Sigma (How)
ISO (Prescriptive tolerance)
![Page 14: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/14.jpg)
We want to see it work!!!
14
Proof of Concept
![Page 15: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/15.jpg)
15
Take away:
Open dialogue = higher level of trust transparency into issues
Native CMMI®
Proof of Concept Results
Impact
Take notice REQM SP1.1
![Page 16: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/16.jpg)
16
Trust
Cirque du Soleil
Core ingredient implementing CMMI®
and desired outcome for quality devices
www.twicial.com
www.tnlp.valuecenter.com
![Page 17: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/17.jpg)
How would it work?
17
![Page 18: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/18.jpg)
18
Maturity Model
• CMMI® - SCAMPI C SCAMPI A
Competencies
• Project Management
• Safety
• Effectiveness
• Reliability
• Patient Satisfaction
• Usability
• Availability
• Compatibility
Advances Analytics - VAC
Metrics
• Pre-Production – Total # changes / Total # of projects
• Production - Total # units mfg RFT / Total # units started
• Post Production – Index
Case for Quality (CfQ)
![Page 19: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/19.jpg)
19
Maturity Model
• CMMI® - SCAMPI C SCAMPI A
Competencies
• Project Management
• Safety
• Effectiveness
• Reliability
• Patient Satisfaction
• Usability
• Availability
• Compatibility
Advances Analytics - VAC
Metrics
• Pre-Production – Total # changes / Total # of projects
• Production - Total # units mfg RFT / Total # units started
• Post Production – Index
Case for Quality (CfQ)
![Page 20: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/20.jpg)
20
MDIC trusted
Medical Device Ecosystem
![Page 21: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/21.jpg)
Has anyone done this before?
Is there proof that there is value?
21
![Page 22: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/22.jpg)
22
Take-a-Ways
• Bottom up, not Top Down
• The right data metrics, not more data metrics!
• It’s a journey of continuous process improvement, not driven to attain a
certification or rating
• Listen and learn - NOT Audits or tests
ROI
• 108% to less than 12% post-release support costs (dev cost: post-release)
• Predictive staffing levels to support launch
• 510k clearance from 6 months to 2 months
• Employee staff morale improve as their issues were heard and addressed
Panel of
CMMI® experts “I believe the potential of this initiative to change our industry is huge.”
![Page 23: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/23.jpg)
Need more info?
How would medical device data be included?
23
![Page 24: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/24.jpg)
24
Proof of Concept and Pilots De-Identified
POC
![Page 25: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/25.jpg)
25
Native CMMI® POC
vs
MedDev CMMI® Pilot #1
![Page 26: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/26.jpg)
26
Survey results
Pilot #1 & #2 - 90% interviewees responded
![Page 27: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/27.jpg)
27
Survey results
![Page 28: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/28.jpg)
28
Survey results
Range of hours:
1-25
![Page 29: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/29.jpg)
Table Legend
D The intent of the practice is absent or poorly addressed. Goal achievement is unlikely.
P The intent of the practice is partially addressed. Goal achievement is threatened.
S The intent of the practice is adequately addressed. Goal achievement is supported.
NY The practice has not yet been deployed.
Not Applicable
29
Interpreting maturity &
culture of an organization
![Page 30: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/30.jpg)
30
Tool to drive Best Practice
development
![Page 31: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/31.jpg)
31
Meta Data
Company / Parent Organization {Note: single selection}
X-Small (1-40 people, 0-$4M in sales)
Small (30-100 people, $5M-$49M in sales)
Medium (80-300 people, $50M-$499M in sales)
Large (300-3000 people, $500M-$5B in sales)X-Large
Size {Note: single selection} Instantiations/ projects {note: multi-select for each instantiation}
X-Small (1-40 people, 0-$4M in sales) Research
Small (30-100 people, $5M-$49M in sales) Manufacturing - HW
Medium (80-300 people, $50M-$499M in sales) Manufacturing - SW
Large (300-3000 people, $500M-$5B in sales) Product Design-HW
Division(s) within Corporation Product Design-SW
Test - bench - mechanical
Test - pre clinical
Test - clinical trial
Laboratory - Diagnostics
Labeling
Sterilization
Distributor
Other: wite in field
Organizational Unit
![Page 32: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/32.jpg)
What does it take?
32
![Page 33: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/33.jpg)
33
Our hours:
150 cum hrs Appraisal Team
each Class C style
MedDev Appraisal
Excluding travel, pre-appraisal prep
Pilot LOE
Range of hours
1-25
Average per participant:
8.25
8-12 participants
Total cumulative
80 hours
![Page 34: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/34.jpg)
What was the workgroup’s experience?
34
![Page 35: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/35.jpg)
What’s next?
35
![Page 36: MDIC Maturity Model...• CMMI® - for official record • MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning • 30 encounters: Appraisals and Check Points](https://reader030.vdocuments.site/reader030/viewer/2022040602/5e988b269254564ee86d8440/html5/thumbnails/36.jpg)
36
• Data for Dec 2017: • Appraisal Collector and repository
• CMMI® - for official record
• MDIC - for Ecosystem ‘Opt-In’ and ‘De-identified’ workgroup planning
• 30 encounters: Appraisals and Check Points
• On-going feedback from participants
• Training • Industry on the Best Practices
• Specialized MedDev Lead Appraisal and Appraisal Team Member (CMMI®)
• Memorandum Of Understanding • Criteria & triggers: more support, regulatory action, voluntary/involuntary
exit
• Communication Plan
• FDA Recognition of Ecosystem participation
• Process Quality: conflict resolution
Proposed Implementation
Program Development