omop cdm: the validation approach · pre-etl tools • white rabbit – profiles source data and...
TRANSCRIPT
![Page 1: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/1.jpg)
OMOP CDM: The Validation Approach
Jon Duke, MD Director, Center for Health Analytics and Informatics
Georgia Institute of Technology
![Page 2: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/2.jpg)
Disclosure
• Have received prior or current funding from Celgene, Merck, Bayer, and Janssen.
• No COI related to today’s presentation
![Page 3: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/3.jpg)
“Loss of fidelity begins with the movement of data from the doctor’s brain to the medical record.”
-- Clem McDonald, MD Director, Lister Hill Center for Biomedical Informatics National Library of Medicine, USA
![Page 4: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/4.jpg)
What is Validation?
• OHDSI views validation as a holistic set of processes necessary to achieve the highest quality reproducible evidence from diverse data sources
![Page 5: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/5.jpg)
Software Validation
Methods Validation
Clinical Validation
Data Validation
Validation Components
![Page 6: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/6.jpg)
Enabling Validation
• OHDSI enables validation by the community through tools, workgroups, shared data discovery, and collaboration platforms
• A research network within OHDSI may enforce given validation processes as appropriate for that network’s objectives
![Page 7: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/7.jpg)
Software Validation
Methods Validation
Clinical Validation
Data Validation
Are the data completely captured with plausible values in a manner that is conformant to agreed structure and conventions?
Data Validation
![Page 8: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/8.jpg)
![Page 9: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/9.jpg)
Data Validation
• How do we ensure preservation of source data into the CDM?
• How do we ensure ETL conventions are followed?
• How do we ensure vocabulary mappings are correct?
• How do we detect issues in the underlying data?
![Page 10: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/10.jpg)
Data Validation Tools
• OHDSI has a set of tools to support data quality and CDM conformance both before and after ETL
• Pre-ETL – White Rabbit / Rabbit in a Hat – Usagi
• Post-ETL – ACHILLES
![Page 11: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/11.jpg)
Pre-ETL Tools
• White Rabbit – Profiles source data and highlights patterns in
source values (eg. variability, frequency)
• Rabbit in a Hat – Provides consistent mechanism for documenting
ETL processes for each dataset to OMOP CDM
![Page 12: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/12.jpg)
Pre-ETL Tools
• Usagi – Tool to support vocabulary mapping including mapping of
local codes to standards – Exposes quality and density of fully mapped vs partially
mapped vs unmapped content
![Page 13: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/13.jpg)
Post-ETL Data Validation
• ACHILLES is OHDSI’s standardized data characterization, quality, and CDM conformance package
• ACHILLES performs checks on every domain and every concept within the dataset
• OHDSI recommends running ACHILLES following ETL and subsequent refreshes
![Page 14: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/14.jpg)
Example Quality Checks
• Demographics
• Date inconsistencies
• Data drop off
• Data outside observation periods
• Check for certain types of data
Full List at ohdsi.org
![Page 15: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/15.jpg)
Example Conformance Checks
Full List at ohdsi.org
![Page 16: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/16.jpg)
Post-ETL Validation
• Use ACHILLES visualizations to review anomalies
![Page 17: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/17.jpg)
Post-ETL Validation
• Use ACHILLES visualizations to review anomalies
![Page 18: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/18.jpg)
Post-ETL Validation
• Use ACHILLES visualizations to review anomalies
Diabetes reimbursement change in UK
![Page 19: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/19.jpg)
Checks Can Emerge from Community
![Page 20: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/20.jpg)
ETL Conventions
• In addition to the previously mentioned tools, ETL conventions are evolved by a public working group known as THEMIS – e.g. how to handle multiple birth dates, conflicting
genders, events after death
• Any group of sites with data in the OMOP model (eg, a collaborative network) can require a common set of conventions
![Page 21: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/21.jpg)
Vocabulary Validation
• The majority of OMOP vocabulary content is external and used without modification – Integration testing and all scripts publicly available
• Some mappings created manually – Eg. ICD-9-CM to SNOMED, ICD-10 to SNOMED,
local drug forms – 2nd-level review by independent coding manager
• Quality and change evaluation scripts run prior to each release
• OHDSI Community review
![Page 22: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/22.jpg)
Rapidly Reviewable
![Page 23: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/23.jpg)
Community Vetted
![Page 24: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/24.jpg)
How does use of a standard terminology affect analysis results?
![Page 25: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/25.jpg)
Changing language may change your code list, which may change your cohort
depending on the disease…
![Page 26: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/26.jpg)
…but in practice, running an estimation analysis using source vs. standard vocabulary yields similar results
![Page 27: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/27.jpg)
More Data Quality Questions?
Check out our detailed FAQ at
tinyurl.com/ohdsi-faq
https://github.com/OHDSI/CommonDataModel/wiki/Frequently-Asked-Questions
![Page 28: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/28.jpg)
Software Validation
Methods Validation
Clinical Validation
Data Validation
![Page 29: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/29.jpg)
Methods Validation
Data Validation
Software Validation
Clinical Validation
Does the software do what it is expected to do?
Software Validation
![Page 30: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/30.jpg)
Software Validation • OHDSI creates two types of software
– ETL software to transform source data to CDM – Analytics software that performs analyses on CDM
• Software should follow best-practice development guidelines – Unit Testing – Integration Testing – Functional Testing – Acceptance Testing
• The OHDSI community’s computer scientists and IT professionals test their code and then have it evaluated by the community
![Page 31: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/31.jpg)
Methods Packages at github.com/ohdsi
![Page 33: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/33.jpg)
Meeting Compliance Requirements
![Page 34: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/34.jpg)
Meeting Community Requirements
![Page 35: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/35.jpg)
Transparent Updates in Each Release
![Page 36: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/36.jpg)
Want to know more about an OHDSI software package?
• Just Google it! It’s easy to find!
![Page 37: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/37.jpg)
Software Validation
Methods Validation
Clinical Validation
Data Validation
![Page 38: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/38.jpg)
Data Validation
Methods Validation
Software Validation
Clinical Validation
To what extent does the analysis conducted match the clinical
intention?
![Page 39: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/39.jpg)
Clinical Validation
• Cohort definitions should be explicit and well-communicated for collaborator review prior to cohort generation
• Once cohorts have been generated, they must be validated in a standardized fashion
![Page 40: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/40.jpg)
Cohort Definition Review
• OHDSI tools are designed to make clear and computable exactly what has been specified in a cohort definition
![Page 41: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/41.jpg)
ATLAS www.ohdsi.org/web/atlas
Create and Share via UI (even highly complex definitions)
![Page 42: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/42.jpg)
Readable View
![Page 43: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/43.jpg)
Shareable Object
![Page 44: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/44.jpg)
Validate Generated Cohort
• Once the cohort has been generated, need to assess whether it indeed reflects the intended exposure or outcome of interest
• OHDSI tools provide support for both statistical and clinical review in a standardized fashion
![Page 45: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/45.jpg)
Atlas Generates Comprehensive Cohort Statistics
![Page 46: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/46.jpg)
Atlas Supports De-identified Patient Level Chart Review
![Page 47: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/47.jpg)
Standardized Manual Validation
![Page 48: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/48.jpg)
Standardized Manual Validation
• Each cohort can be annotated using a custom set of questions
• The ‘question set’ is a shareable object that can be sent in conjunction with the cohort definitions themselves, expediting validation
• Can automate calculation of phenotype performance measures with goal of expediting generation of cross-site performance metrics
![Page 49: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/49.jpg)
Software Validation
Methods Validation
Clinical Validation
Data Validation
![Page 50: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/50.jpg)
Data Validation
Validation Components
Methods Validation
Software Validation
Clinical Validation
Do the estimates generated in an analysis measure what they
purport to measure?
![Page 51: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/51.jpg)
Methods Validation
• Methods Diagnostics – Is the planned statistical method valid given the
data?
• Empirical Calibration – Are there systematic errors that require
calibration of the results?
![Page 52: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/52.jpg)
Keppra and Angioedema
Request identified by OHDSI community member March 2016
![Page 53: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/53.jpg)
Levetiracetam and Angioedema
Extended for 2 additional sites Paper submitted 4/2017, Published 8/2017
![Page 54: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/54.jpg)
Leverage OHDSI CohortMethod
• Leveraged OHDSI CohortMethod R package
• Code tested at 2 sites prior to study start
• All code posted on GitHub
![Page 55: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/55.jpg)
Propensity Score Distribution
Covariate Balance
Package Provides Diagnostic Checks
![Page 56: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/56.jpg)
Covariate Balance
![Page 57: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/57.jpg)
Empirical Calibration
• Uses negative controls (conditions known not to be associated with the exposure) to calibrate p-values
This study is well calibrated!
![Page 58: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/58.jpg)
Generates Results and Supporting Documents
![Page 59: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/59.jpg)
“Many eyes make all bugs shallow.”
![Page 60: OMOP CDM: The Validation Approach · Pre-ETL Tools • White Rabbit – Profiles source data and highlights patterns in source values (eg. variability, frequency) ... Pre-ETL Tools](https://reader033.vdocuments.site/reader033/viewer/2022060716/607c1282ca6b3534d731cc59/html5/thumbnails/60.jpg)
Validation Requires Many Eyes
• Every aspect of validation is improved dramatically by transparency, openness to challenge, and engagement of a community that is passionate about making it better
• As OMOP adoption continues to grow rapidly from industry to academia to government (VA, DoD, CDC, FDA), our eye-count goes up and quality and innovation will continue to grow even further