2017 Audit Guidance: Preparation, Experiences, and Lessons LearnedNPA Quality Symposium

Friday, June 9th

Objectives

Discuss preparing for the audit

Review and discuss the work of the Audit Task Force Record Layout (Universe) templates

Risk Assessments

Operational Guide

Discuss the tracer methodology and how it applies to the CMS audit

Discuss two audit experiences

2

CMS PACE Conference July 8, 2016

Goal in restructuring PACE audit: Make PACE audits more outcomes-based

Focus on access and the participant experience

Reduce the administrative burden of PACE organizations

Drive improvements in the quality of care for participants

3

Outcome Focused Audits

Measuring a clinical outcome against well-defined standards set on the principles of evidence-based medicine in order to identify the changes needed to improve the quality of care.

Purpose is to highlight the discrepancies between actual practice and standard in order to identify the changes needed to improve the quality of care.

Determine Audit Focus

Set Criteria and Standards

Data CollectionData Analysis and Implementation

of Changes

Improvements –Monitor and Maintenance

4

Audit Elements

Audit Elements (with Naming Conventions)

Service Delivery Requests, Appeals and Grievances (SDAG) Did the PO appropriately process service delivery requests, appeals and

grievances?

Clinical Appropriateness and Care Planning (CPAP) Did the PO develop and document an appropriate plan of care for the

participants?

Personnel Records (PER) Do personnel have appropriate licensure, were OIG exclusions checks

performed, were background checks completed, evidence of competency evaluation, health records.

Onsite Review (ON) Observe 3 to 5 participants (1 who receives care from home and one who

receives care at the center), transportation vehicle and emergency equipment.

Quality Assessment (QA) Did the PO develop and/ or implement an effective, data driven quality

assessment and performance improvement program? Did the PO ensure that the appropriate staff were involved in the

development and implementation of QAPI activities?

5The PACE Audit Process and Data Request Document and all Attachments can be found: https://www.cms.gov/Medicare/Compliance-and-Audits/Part-C-and-Part-D-Compliance-and-Audits/PACE_Audits.html

Pre-Audit Document Submissions

Pre-Audit Disclosed Issues of Non-Compliance PO to provide a list of all disclosed issues of non-compliance that are

relevant to the audit elements.

Use Excel template (Pre-Audit Issue Summary) provided by CMS

Attachment III included with Audit Process and Data Request protocol sent 4/11/2017

Disclosed issue: one that is disclosed to CMS prior to the receipt of the audit engagement letter.

6

Pre-Audit Document Submissions

Pre-Audit Universe uploads to HPMS

Requires specific naming conventions of documents as specified in Appendix N (sent to PO through HPMS after audit engagement letter)

SDAG (Service Delivery Requests, Appeal, and Grievance)1. [PO Name]-[SDAG]-[SDR]--[Universe]-[Version Number]

(e.g. OnLok-SDAG-SDR—Universe-Version 1)

2. [PO Name]-[SDAG]-[AR]-[Universe]-[Version Number]

3. [PO Name]-[SDAG]-[GR]-[Universe]-[Version Number]

CPAP (Clinical Appropriateness and Care Planning)1. [PO Name]-[CPAP]-[LOPMR]-[Universe]-[Version Number]

2. [PO Name]-[CPAP]-[OCU]-[Universe]-[Version Number]

PER (Personnel Records)1. [PO Name]-[PER]-[LOP]-[Universe]-[Version Number]

ON (Onsite Element) No universes will be submitted for this element. Participants will be selected from the CACP list

of participants.

QA (Quality Assessment)1. [PO Name]-[QA]-[QAIR]-[Universe]-[Version Number]

7

Pre-Audit Document Submissions

Pre-Audit Universe Questions Grid Upload to HPMS

Use Excel template provided by CMS (PO Questions for CMS)

Appendix V sent to PO through HPMS after audit engagement letter

Naming convention required [PO Name]-[Questions for

CMS]-[Date]

(e.g. OnLok-Questions for CMS-04112017.xlsx)

Question # Element

Universe/Table/

Record Layout

(ifapplicable)

PACE

Organization (PO)

Questions

CMS'

Response

1

2

8

Pre-Audit Document Submissions

PACE Supplemental Questions Attachment II provided with Audit Process and Data Request

protocol sent 4/11/2017

Grievance information

Emergency medications readily available

Emergency and disaster preparedness training

Staff vaccinations

Driver communication

EMR information (access remotely?)

Service delivery request definition and policy

9

Responding to Documentation Requests

During the Audit

Sample Case Supporting Documentation Uploads to HPMS PO must upload all sample case supporting documentation

requested during the audit to HPMS by selecting:

Naming conventions and instructions detailed in Appendix N

10

Audit Findings and Corrective Action

CMS will determine if each condition cited is: An Observation – 0 points

Corrective Action Required (CAR) – 1 point

Immediate Corrective Action Required (ICAR) – 2 points

All points related to CARs and ICARs will be added then divided by the number of audit elements (5 audit elements) tested to determine the PO’s overall PACE audit score.

11

Development of the NPA Audit Materials

Began with an initiative involving MI PACE programs.

Through the NPA Regulatory Compliance Quality Subcommittee, an Audit Task Force was developed with participants from each of the CMS regions.

Purpose of developing the audit materials: Assist POs in collecting and submitting data universes required for

revised CMS audits beginning 2017;

Support POs’ compliance efforts and ability to identify compliance issues in real time and in advance of audits;

Support POs’ efforts in identifying opportunities for system improvements resulting from analyses of the data universes;

Identify where the data universe elements overlap with HPMS PACE Quality Data Level I reporting requirement;

Assist EHR vendors to understand the audit process and related data requirements; and

Support CMS’ efforts to assure consistency in audit practices.

NPA Operational Guide should be used concurrently with CMS's Programs of All-Inclusive Care for the Elderly (PACE) Audit Process and Data Request document issued April 11, 2017

Although CMS has responded to specific questions related to the templates for the audit data universes, the templates, risk assessment tools and the operational guide referenced in this presentation have not been reviewed or approved by CMS. NPA disclaims all liability with respect to these materials. Members use them at their discretion.

Audit Data Universe Record Layout Templates

The 7 data universes are related to the 5 audit elements:1. Service Delivery Requests, Appeals and Grievances;

2. Clinical Appropriateness and Care Planning;

3. Quality Assessment;

4. Personnel Records; and

5. Onsite Review

7 record layouts to be used in providing the data universes to CMS:1. Service Delivery Requests (SDR);

2. Appeal Requests (AR);

3. Grievance Requests (GR);

4. List of Personnel (LOP);

5. List of Participant Medical Records (LOPMR);

6. Quality Assessment Initiatives Records (QAIR); and

7. On-call Universe (OCU)

13The NPA Record Layouts, Risk Assessment, Opertaional Guide, and recent recorded webinar may be found: http://www.npaonline.org/member-resources/compliance

Audit Data Universe Record Layout Templates

The 7 record layout templates have the following features: For ease of use, the templates include the description of each data

element and the number of characters allowed within each field.

Additional reference information for POs’ consideration (e.g. definitions for subjective data, list of dementia diagnoses that correspond with HCC51 and HCC52, etc.).

Dropdown responses for many data fields to facilitate data entry.

The Service Delivery Requests, Appeal Requests, and Grievance Requests templates include an “analytics” function that identifies potential compliance problems.

The Appeal and Grievance Requests templates include HPMS PACE Quality Data Level I to avoid maintaining similar data in multiple places.

Data integrity functions provided by CMS to ensure the data entered are consistent with CMS specifications.

Password: 2017Audit

14

Risk Assessment Tools

To assure ongoing compliance with critical PACE regulatory requirements and to assist POs in assessing their compliance in the 5 audit areas, the following 5 Risk Assessment tools were created:

Service Delivery Requests Risk Assessment

Appeal Requests Risk Assessment

Grievance Requests Risk Assessment

Clinical Appropriateness and Care Planning Risk Assessment

Personnel Records Risk Assessment

15

Risk Assessment Tools

Each Risk Assessment exists within an Excel workbook consisting of an “Audit Totals” worksheet and 20 numbered spreadsheets.

Each of the 20 numbered spreadsheets refers to an individual audit observation (i.e. a service delivery request, an appeal, a grievance, a participant medical record, or a personnel record).

After completing a workbook for a probe sample of 20 observations (random or targeted), the “Audit Totals” worksheet will help to identify areas of overall compliance/noncompliance.

Each risk assessment tool contains a set of compliance standards with a corresponding reference to the applicable PACE regulation.

Each audit observation requires a scoring mechanism and provides a space to make comments specific to noncompliance observed.

16

Impact Analysis

Impact Analysis: identifies who was subjected to and in what ways an issue of non-compliance effected the outcome (i.e. identifying the consequences of non-compliance).

15 CMS templates provided with Audit Process and Data Request protocol sent 4/11/2017

Appeal template

Grievance template

Personnel template

Root cause template

Service delivery template

10 Clinical appropriateness templates

PO must upload all IAs requested during the audit through HPMS

Naming conventions and instructions provided in Appendix N (sent to PO through HPMS after audit engagement letter)

17

Tracer Methodology

Tracer methodology uses organizational information to follow the experience of care, treatment, or services.

Allows for the identification of performance issues in one or more steps of a process or interfaces between processes.

Individual Tracers (participant focused) Designed to trace the care experience of a participant in the PACE

program.

Analyzes the PO’s system of providing care, treatment or services using actual participants to assess compliance to standards.

Participants selected are typically high risk or medically complex

System Tracers (process focused) Trace a process or system within the PO (use individual tracer

information).

Evaluates the system or process, integration of processes, coordination and communication among disciplines and departments within the system/process.

Relies upon the use of quality data in performance improvement.

18

Tracer Methodology

Involves talking with multiple staff, the participant, and caregivers to learn details about the individual experience.

Identify gaps or risk points that could affect quality or safety of care.

Learn from individuals directly involved in providing or receiving services about how the process actually works.

Evaluate the following: Compliance with standards and evidence based principles.

Consistent adherence to policy and implementation of procedures.

Communication within and between departments, disciplines, and services/providers.

Staff competency for assignments and workload capacity.

The physical environment.

19

Purpose of Tracers

Retrospective Learn more about why a process didn’t work or was successful.

Prospective Evaluate a process identified as problematic, determine current

practice around new regulatory standards, evaluate high risk participants or processes with poor outcomes.

20

How to Apply a Tracer

What would you like to know more about in your program?

What worries you?

What keeps you awake at night?

What does your data show as potentially problematic? Clinical outcomes

Grievances

PAC feedback

Contracted provider feedback

21

Planning Tracers

Assemble the team – determine who should be on the team related to skills, experience, expertise, etc.

Assure objectivity.

Identify a team leader to manage the process.

Provide the necessary guidance and training for tracer team.

Communicate with leadership and personnel.

Determine the goal of the tracer: Is it driven by a level II or significant event?

Confirm practices of key policies and systems?

Identify key practices that increase the success or effectiveness of a process?

Preparing for strategic program growth or development?

Evaluating outcome of recent performance improvement efforts?

22

Performing the Tracer

Identify the pool of applicable participants and select participants for tracer

Random or targeted selection

Trace processes of care not clinical appropriateness

Review applicable policies and procedures

Review applicable standards, regulations, evidence based practices, etc.

Create tracer tool to ensure interrater reliability and standard application of observations and interviews –similar to the audit risk assessments.

Provides a means of documenting the process.

23

Creating a Tracer Tool

Participant Name

Date grievance was made

Date grievance was resolved

Universe Compliance Standard

Grievance RequestsA PO must determine the timeframe for resolving grievances in their internal policies and procedures. §460.120

Scoring Key 0 = Noncompliant 1 = Partial Compliance 2 = Full Compliance Blank = Not Applicable

Score Comments (Required for any score of 1 or0):Possible Actual

A §460.120(b) Upon enrollment, was the participant given written information regarding the grievance process? 0

B §460.120(b)Was the participant given written information regarding the grievance process annually after enrollment? 0

C§460.120(c)(

3)

Is there documentation showing that the grievance was resolved timely and consistent with the program's policy? (If the response is "2", skip question D. 0

D N/A If no, was the root cause determined and addressed? 0

E§460.120(c)(

2)Is the participant's grievance documented and does documentation show that all issues were addressed? 0

F §460.120(e) Is there documentation showing that the participant was notified of the grievance outcome? 0

G §460.120(d) Is there documentation showing that the program continued to furnish all required services to the participant during the grievance process? 0

Total 0 0

24

Performing the Tracer

Interview staff involved in the process you are tracing Maintain an open tone

Listen attentively

Restate and clarify responses

Ask open ended questions

Interview staff directly involved in the process – not the manager

All interview questions and responses should be documented

All interviewees should be asked the same questions

Interview applicable participants

Review medical records

Debrief as a tracer team and review findings

Organize and analyze and summarize findings

Communicate findings to leadership and staff

Develop and implement improvement plans

25

Questions Questions before we transition to the discussion of audit experience and lessons?

26