Improving the Clinical Quality Measure Testing and Certification Process

HIMSS13March 4, 2013

Agenda

• Cypress v2 Overview• Feedback on CQM Testing and Certification• Improvements to Certification Process– Jacob Reider, Office of the National Coordinator

for Health Information Technology (ONC)– Saul Kravitz, MITRE– Steve Emrick, National Library of Medicine– Bob Dolin, Lantana Consulting Group

• Open Discussion

• Meaningful Use (MU) Stage 2 Clinical Quality Measure (CQM) Testing and Certification Tool

– ONC championed activity

– MITRE designed and stewarded free open source software

• Enables repeatable testing that an EHRs’ ability to calculate Eligible Professional (EP) CQMs, Eligible Hospital (EH) CQMs, and Quality Reporting Document Architecture (QRDA) XML is accurate and aligned with MU Stage 2

1. Capture and Export 2. Import and Calculate 3. Electronic Submission

Cypress Overview

Status of Cypress and MU Stage 2 CQMs

• Cypress released to Authorized Testing Labs (ATLs) – V2.0.0 released 12/21/2012– V2.1.0 released 2/28/2013

• ONC’s 2014 HIT Certification Program operational January 2, 2013

• ATLs have successfully certified vendors using the new CQM testing procedure and Cypress

• We are receiving feedback from vendors and ATLs– 125+ discussions on the public open source email list– 200+ issues on ONC’s Jira tool

• We are improving the certification tools and process based on your feedback

Types of Feedback Collected

• eCQM Logic• Value Sets• QRDA-I Test Data (Capture/Export)• QRDA-III Test Data (Compute)

eCQM Logic Feedback

• Many questions regarding proper interpretation of MU2 eCQMs

• Questions about specific eCQMs, e.g.:– Measure X doesn’t reflect the clinical guidelines

• General Questions, e.g.:– How to interpret specific occurrence idioms?– How are episode of care eCQMs computed?

Response to eCQM Feedback

• Short-Term Actions– V1.1 of eCQM Implementation Guide on CMS Website

– eCQMs updated as part of 2013 annual update

• Longer-Term Actions– Publication of test data for use by vendors to

demonstrate intended behavior for each measure

– Improvements to logic syntax

– Incorporation of automated testing in MU3 eCQM development process

Value Set Authority Center (VSAC)

• Provides Value Sets that support MU Stage 2 CQMs– Requires UMLS Metathesaurus License for access to GUI and

download files

• Received lots of useful feedback and input from users, e.g.:– How can I obtain pre-packaged value sets for all eCQMs?– Can’t you sort these in various ways – CMS ID, QDM category?– How can I filter my search results by code system (SNOMED CT,

RxNorm, etc), in the user interface?– How do I find out about updates to value sets or the VSAC?

Value Set Authority Center (VSAC)

Consumption

Authorship Web page population for human consumption

Integration with Authoring Tool(s) to support value set authorship

Web download / web services for machine consumption

Free-standing, web-based value set authoring tool(s)

API

Phase 1 Phase 2

API

Controlled Value Set Repository & Validation Engine

Value Set / Measure AuthorsConsensus Organization(s)

NLMONCConsensus Org(s)

NLM VendorsProvidersPublic Health Orgs

Curation Delivery

Value Sets – Downloadable Resources Table

CMS ID Value Set NameQDM Category

Packages available for EP, EH, and EP+EH measures

Value Sets – Code System Filter

Show only value sets containing RxNorm codes

Statin prescribed at discharge

VSAC: Ongoing Work

• Development of VSAC authoring environment– Improve quality of value sets– Improve authoring productivity

• Value set harmonization– Gather ‘purpose’ of all value sets as structured metadata– Will provide ‘semantic boundaries’ of value sets, which could

be helpful on determining overlap of value sets, new code appropriateness, etc.

• Gathering of non-MU value sets – e.g., Route of Administration – Could eventually be used in some MU context

QRDA-I Certification Feedback

• “Excessive” data entry burden• Test data incompatible with clinical

applications• QRDA-I specification issues• Many spurious validation errors and warnings

Response to QRDA-I Certification Feedback

• Near Term (Following 2/28 release)– Additional QRDA-I implementation guidance– Revise test procedure to reuse Logic Test Deck for

QRDA-I Testing• Initial focus on syntactic correctness only• Elimination of MANY warnings

• Longer Term (With 4/26 release)– Software changes to check that QRDA-I files are within

upper and lower bounds of information content– Minor revision to test procedure

QRDA-III Certification Feedback

• Comments on the challenge of working with HQMF files

• Questions about correctness of eCQM interpretation

• Spurious validation errors on QRDA-III output

Response to QRDA-III Certification Feedback

• Feedback to HL7 HQMF revision process• Software fixes• Feedback to eCQM update process

Changes to QRDA-III Test Deck

• For EH and EP Certification:– Moved dates off of January 1st

– All ‘active’ states have a duration– Tried to use widely supported codes– Avoid unprescribable medications*– Moved all events inside of business hours– Moved all events inside encounters

• For EH Certification:– Reduced length of stay

Cypress Timeline

• Version 2.0 – 12/21/2012• Version 2.1.0 – 2/28/2013

– Addresses all resolved QRDA-I validation issues– Supports shift to use of Logic Test Decks for QRDA-I Certification– Updates to EH and EP test decks based on feedback– Support ATL testing of multiple eCQM releases with a single Cypress

installation

• Version 2.2.0 – 4/26/2013– Support for ‘dynamic’ logic testing for all measures– Improved QRDA-I testing based on Logic Test Decks– Support for 2013 EH Update, versioned value sets

• Updated Test Decks – EH – Early May 2013– EP – Early July 2013

Communications Channels

• Cypress Email Lists– talk@projectcypress.org - public e-mail list

– announcement@projectcypress.org - broadcast list

– Details on http://projectcypress.org/faq.html

– ONC Testing and Certification Team ONC.Certification@hhs.gov

• ONC Jira for Issue Tracking– General CQM Tracker https://cqm-issue-tracker.atlassian.net/browse/

CQM

– Cypress-specific Trackerhttps://cqm-issue-tracker.atlassian.net/browse/CYPRESS

• VSAC updates listserv – Go to https://list.nih.gov/cgi-bin/wa.exe?A0=nlm_vsac_updates

– Click on ‘Subscribe or Unsubscribe’

– Enter your name, e-mail, and click ‘Subscribe (NLM_VSAC_UPDATES)’

QUESTIONS AND FEEDBACK AND FEEDBACK

21

BACKUP

Capture and Export Current Concept of Operations

Import, Calculate and Electronic SubmissionConcept of Operations

Release Notes for Cypress V2.1.0Test Deck Changes

• Measurement period updated from 2010 to 2012, ending Dec 31, 2012 at 23:59• Code provided for each data element trigger logic in a consistent manner across all

measures• Added code description mouse overs to the HTML patient records (RxNorm

descriptions and all other code sets)• Length of stay for encounters minimized for EP• Patient data with the status of "active" given a duration in the test deck• EP and EH events occur during business hours• EP and EH Test deck events occur during encounters• EP GP_Peds, E immunization end time fixed• EH Newborn, A and Newborn, B birthdate and delivery times brought into sync• Value sets added to the Cypress bundle to allow coordinating value set & measure

releases• EH Newborn, A/B and OB A/B updated admit and discharge times on encounter to

match start/end times

Release Notes for Cypress V2.1.0Cypress Software Changes

• Quality Measure Engine:– Dynamic Support for CV measures in EH and EP– Bundle Import code Moved to Health Data Standards library

• Measures:– Reloaded measure definitions from CMS to fix issue with measure population criteria ids

• Cypress:– Added support for multiple bundles to be loaded simultaniously– Fixed double quoate bug in

resources/schematron/qrda/qrda_cat_1/measure_specific/d78ce034-8288-4012-a31e-7f485a74f2a9.sch

– Removed xsi:type='CE' checks from generic Cat I schematron rules– Updated schematron rule for CONF:7618– to remove check for unit='1' – Added ability to import bundle from within the application– Updated QRDA Cat I test patient generator to eliminate duplicate entries