fy 2019-20 medi-cal specialty mental health external ... and... · 1. fy 2019-20 medi-cal specialty...
TRANSCRIPT
1.
FY 2019-20 MEDI-CAL SPECIALTY MENTAL HEALTH
EXTERNAL QUALITY REVIEW
GLENN MHP FINAL REPORT
Behavioral Health Concepts, Inc. 5901 Christie Avenue, Suite 502 Emeryville, CA 94608
[email protected] www.caleqro.com 855-385-3776
Prepared for:
California Department of
Health Care Services (DHCS)
Review Dates:
August 6, 2019
- 2 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
TABLE OF CONTENTS
INTRODUCTION .................................................................................................... 5
MHP Information ............................................................................................................. 5 Validation of Performance Measures .............................................................................. 6 Performance Improvement Projects ................................................................................ 6 MHP Health Information System Capabilities .................................................................. 6 Validation of State and MHP Beneficiary Satisfaction Surveys ....................................... 6
Review of Recommendations and Assessment of MHP Strengths and Opportunities .... 6
PRIOR YEAR REVIEW FINDINGS, FY 2018-19 ..................................................... 8
Status of FY 2018-19 Review of Recommendations ....................................................... 8
Key Recommendations from FY 2018-19 ....................................................................... 8
PERFORMANCE MEASUREMENT ....................................................................... 15
Health Information Portability and Accountability Act (HIPAA) Suppression Disclosure: ...................................................................................................................................... 17
Total Beneficiaries Served ............................................................................................ 18 Penetration Rates and Approved Claims per Beneficiary.............................................. 18 High-Cost Beneficiaries ................................................................................................. 22
Psychiatric Inpatient Utilization ...................................................................................... 22 Post-Psychiatric Inpatient Follow-Up and Rehospitalization .......................................... 23
Diagnostic Categories ................................................................................................... 24
PERFORMANCE IMPROVEMENT PROJECT VALIDATION ................................. 25
Glenn MHP PIPs Identified for Validation ...................................................................... 25 Clinical PIP—Intensive Home-Based Services (IHBS) .................................................. 25 Non-clinical PIP—Timely Access .................................................................................. 26
INFORMATION SYSTEMS REVIEW ..................................................................... 31
Key Information Systems Capabilities Assessment (ISCA) Information Provided by the MHP .............................................................................................................................. 31 Telehealth Services ....................................................................................................... 33
Summary of Technology and Data Analytical Staffing................................................... 34 Current Operations ........................................................................................................ 35 The MHP’s Priorities for the Coming Year ..................................................................... 35 Major Changes since Prior Year ................................................................................... 36
Other Areas for Improvement ........................................................................................ 36 Plans for Information Systems Change ......................................................................... 36 Current EHR Status....................................................................................................... 36
Personal Health Record (PHR) ..................................................................................... 37 Medi-Cal Claims Processing ......................................................................................... 38
CONSUMER AND FAMILY MEMBER FOCUS GROUP(S) .................................... 40
CFM Focus Group One ................................................................................................. 40
- 3 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
PERFORMANCE AND QUALITY MANAGEMENT KEY COMPONENTS............... 42
Access to Care .............................................................................................................. 42 Timeliness of Services .................................................................................................. 43 Quality of Care .............................................................................................................. 44 Beneficiary Progress/Outcomes .................................................................................... 46
Structure and Operations .............................................................................................. 47
SUMMARY OF FINDINGS ..................................................................................... 49
MHP Environment – Changes, Strengths, and Opportunities ....................................... 49
FY 2019-20 Recommendations ..................................................................................... 53
ATTACHMENTS ................................................................................................... 55
Attachment A—On-site Review Agenda ........................................................................ 56
Attachment B—Review Participants .............................................................................. 57 Attachment C—Approved Claims Source Data ............................................................. 60
Attachment D—List of Commonly Used Acronyms ....................................................... 61 Attachment E—PIP Validation Tools ............................................................................. 64
- 4 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
LIST OF TABLES AND FIGURES
Table 1: MHP Medi-Cal Enrollees and Beneficiaries Served, by Race/Ethnicity Table 2: High-Cost Beneficiaries Table 3: MHP Psychiatric Inpatient Utilization Table 4: PIPs Submitted by MHP Table 5: PIP Validation Review Table 6: PIP Validation Review Summary Table 7: Budget Dedicated to Supporting IT Operations Table 8: Distribution of Services, by Type of Provider Table 9: Contract Providers Transmission of Beneficiary Information to MHP EHR System Table 10: Technology Staff Table 11 Data Analytical Staff Table 12: Primary EHR Systems/Applications Table 13: EHR Functionality Table 14: Summary of CY 2019 Short-Doyle/Medi-Cal Claims Table 15: Summary of Top CY 2018 Top Three Reasons for Claim Denial Table 16: Access to Care Components Table 17: Timeliness of Services Components Table 18: Quality of Care Components Table 19: Beneficiary Progress/Outcomes Components Table 20: Structure and Operations Components Figure 1A: Overall Penetration Rates, CY 2016-18 Figure 1B: Overall Approved Claims per Beneficiary, CY 2016-18 Figure 2A: Latino/Hispanic Penetration Rates, CY 2016-18 Figure 2B: Latino/Hispanic Approved Claims per Beneficiary, CY 2016-18 Figure 3A: Foster Children Penetration Rates, CY 2016-18 Figure 3B: Foster Children Average Approved Claims per Beneficiary, CY 2016-18 Figure 4A: 7-day Post-Psychiatric Inpatient Follow-up Figure 4B: 30-day Post-Psychiatric Inpatient Follow-up Figure 5A: Beneficiaries Served, by Diagnostic Categories, CY 2018 Figure 5B: Total Approved Claims by Diagnostic Categories, CY 2018
- 5 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
INTRODUCTION
The United States Department of Health and Human Services (HHS), Centers for Medicare and Medicaid Services (CMS) requires an annual, independent external evaluation of State Medicaid Managed Care Organizations (MCOs) by an External Quality Review Organization (EQRO). External Quality Review (EQR) is the analysis and evaluation by an approved EQRO of aggregate information on quality, timeliness, and access to health care services furnished by Prepaid Inpatient Health Plans (PIHPs) and their contractors to recipients of State Medicaid Managed Care Services. The Code of Federal Regulations (CFR) specifies the requirements for evaluation of Medicaid MCOs (42 CFR, Section 438; Medicaid Program, External Quality Review of Medicaid Managed Care Organizations). These rules require an on-site review or a desk review of each Medi-Cal Mental Health Plan (MHP).
In addition to the Federal Medicaid EQR requirements, the California External Quality Review Organization (CalEQRO) also takes into account the State of California requirements for the MHPs. In compliance with California Senate Bill (SB) 1291 (Section 14717.5 of the Welfare and Institutions Code), the Annual EQR includes specific data for Medi-Cal eligible minor and nonminor dependents in foster care (FC).
The State of California Department of Health Care Services (DHCS) contracts with 56 county Medi-Cal MHPs to provide Medi-Cal covered Specialty Mental Health Services (SMHS) to Medi-Cal beneficiaries under the provisions of Title XIX of the federal Social Security Act.
This report presents the fiscal year (FY) 2019-20 findings of an EQR of the Glenn MHP by the CalEQRO, Behavioral Health Concepts, Inc. (BHC).
The EQR technical report analyzes and aggregates data from the EQR activities as described below:
MHP Information
MHP Size ⎯ Small-rural
MHP Region ⎯ Superior
MHP Location ⎯ Willows
MHP Beneficiaries Served in Calendar Year (CY) 2018 ⎯ 846
MHP Threshold Language(s) ⎯ Spanish
Threshold languages are listed in order beginning with the most to least number of eligibles. This information is obtained from the DHCS/Research and Analytic Studies Division (RASD), Medi-Cal Statistical Brief, September 2016.
- 6 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Validation of Performance Measures1
Both a statewide annual report and this MHP-specific report present the results of CalEQRO’s validation of eight mandatory performance measures (PMs) as defined by DHCS and other additional PMs defined by CalEQRO.
Performance Improvement Projects2
Each MHP is required to conduct two Performance Improvement Projects (PIPs)—one clinical and one non-clinical—during the 12 months preceding the review. The PIPs are reviewed in detail later in this report.
MHP Health Information System Capabilities3
Using the Information Systems Capabilities Assessment (ISCA) protocol, CalEQRO reviewed and analyzed the extent to which the MHP meets federal data integrity requirements for Health Information Systems (HIS), as identified in 42 CFR §438.242. This evaluation included a review of the MHP’s Electronic Health Records (EHR), Information Technology (IT), claims, outcomes, and other reporting systems and methodologies for calculating PMs.
Validation of State and MHP Beneficiary Satisfaction Surveys
CalEQRO examined available beneficiary satisfaction surveys conducted by DHCS, the MHP, or its subcontractors.
CalEQRO also conducted 90-minute focus groups with beneficiaries and family members to obtain direct qualitative evidence from beneficiaries.
Review of Recommendations and Assessment of MHP
Strengths and Opportunities
The CalEQRO review draws upon prior years’ findings, including sustained strengths, opportunities for improvement, and actions in response to recommendations. Other findings in this report include:
1 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validation of
Performance Measures Reported by the MCO: A Mandatory Protocol for External Quality Review (EQR), Protocol
2, Version 2.0, September, 2012. Washington, DC: Author. 2 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validating
Performance Improvement Projects: Mandatory Protocol for External Quality Review (EQR), Protocol 3, Version
2.0, September 2012. Washington, DC: Author. 3 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). EQR Protocol 1:
Assessment of Compliance with Medicaid Managed Care Regulations: A Mandatory Protocol for External Quality
Review (EQR), Protocol 1, Version 2.0, September 1, 2012. Washington, DC: Author.
- 7 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
• Changes, progress, or milestones in the MHP’s approach to performance management — emphasizing utilization of data, specific reports, and activities designed to manage and improve quality.
• Ratings for key components associated with the following three domains: access, timeliness, and quality. Submitted documentation as well as interviews with a variety of key staff, contracted providers, advisory groups, beneficiaries, and other stakeholders inform the evaluation of the MHP’s performance within these domains. Detailed definitions for each of the review criteria can be found on the CalEQRO website, www.caleqro.com.
- 8 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
PRIOR YEAR REVIEW FINDINGS, FY 2018-19
In this section, the status of last year’s (FY 2018-19) recommendations are presented, as well as changes within the MHP’s environment since its last review.
Status of FY 2018-19 Review of Recommendations
In the FY 2018-19 site review report, the CalEQRO made a number of recommendations for improvements in the MHP’s programmatic and/or operational areas. During the FY 2019-20 site visit, CalEQRO reviewed the status of those FY 2018-19 recommendations with the MHP. The findings are summarized below.
Assignment of Ratings
Met is assigned when the identified issue has been resolved.
Partially Met is assigned when the MHP has either:
• Made clear plans and is in the early stages of initiating activities to address the recommendation; or
• Addressed some but not all aspects of the recommendation or related issues.
Not Met is assigned when the MHP performed no meaningful activities to address the recommendation or associated issues.
Key Recommendations from FY 2018-19
PIP Recommendations
Recommendation 1: Measure results of interventions for both PIPs at least quarterly.
This allows the opportunity to adjust the PIP if barriers to success are discovered.
Status: Met
• The MHP runs data quarterly for both the clinical and non-clinical PIPS as is evidenced upon review of the Quality Improvement Committee (QIC) and System Improvement Committee (SIC) meeting minutes.
Recommendation 2: Clinical PIP: Add at least one new intervention and refine the PIP
to reach conclusion this year.
Status: Partially Met
• The MHP added two interventions to the clinical PIP focused on In-Home Based Services (IHBS). These include use of the Child Adolescent Needs and Strengths (CANS-50) outcome tool (July 2018) and staff training (May 2019); however, the newly implemented interventions do not meet the standard of PIP
- 9 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
interventions. Rather, the outcome tool is a measurement tool for PIP indicators, and staff training is focused on staff deficiencies, not beneficiary outcomes.
Recommendation 3: Non-clinical PIP: The PIP would benefit from beneficiary input.
Status: Met
• Beneficiaries were given the opportunity for input on this PIP through the consumer section of the monthly QIC and SIC meetings, and at Consumer Voice focus groups run monthly at the drop-in centers. Feedback from grievances related to timely access are also reviewed each quarter.
Recommendation 4: Re-measurement of percent of improvement achieved needs to
be done quarterly, at a minimum.
Status: Met
• As previously mentioned, data was run quarterly for both PIPs which also included examining the percent of improvement achieved.
Recommendation 5: Add a new intervention to allow for measurement of changes the
MHP has made in clinical assessments being the first service provided and refine the
PIP to reach conclusion this year.
Status: Met
• The MHP clarified that all clinical assessments occur during first appointment sessions and that this has been the process pre-dating the non-clinical PIP on timely access. All remeasurements in the PIP are compared against the baseline measure of the clinical assessment as the first appointment.
Recommendation 6: Consult with EQRO early and often during the continuation of
both PIPs and the design of a new PIP at the clinical PIP’s conclusion.
Status: Partially Met
• The MHP attended PIP technical assistance webinars and attended the CalQIC session on creating a good PIP. The MHP sought limited technical assistance from the assigned CalEQRO reviewer in advance of the onsite review; however, the MHP would have benefited from ongoing consultation with CalEQRO over the last twelve months on its PIPs.
- 10 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Access Recommendations
Recommendation 7: Resolve issue of psychiatry capacity through increase in
telehealth and/or recruitment of onsite psychiatrist.
Status: Met
• The MHP added five more telepsychiatry hours for FY 2018-19. The MHP’s non-clinical PIP focused on psychiatry timeliness and reflects an improvement in timeliness to psychiatric services. The MHP plans to evaluate psychiatrist caseloads to determine those who may be ready to step down to the mild-to-moderate level of care; however, a formal evaluation plan and timeline was not available. The MHP also reports that they have not received a plan of correction to increase psychiatry capacity through its network adequacy submissions to DHCS.
Recommendation 8: Continue discussion with Glenn Medical Center and North Valley Indian Health Center regarding the opportunity for health information exchange (HIE) pilot program collaboration.
Status: Met
• Glenn Medical Center (GMC) and North Valley Indian Health (NVIH) have not expressed interest in collaboration for health information exchange. The MHP continues to work closely with both entities on other projects, including an updated memorandum of understanding (MOU) for delivering crisis services at GMC and partnering with the local American Indian Health Facility as a choice of provider for beneficiaries under the new network adequacy standards issued by DHCS.
Timeliness Recommendations
Recommendation 9: Continue to work with Kings View to resolve psychiatry capacity
issue through increase in telehealth. Continue recruitment for onsite psychiatrist.
Status: Met
• Telehealth hours were expanded with five additional hours per month (for a total of 29 hours per week).
• The MHP continues to recruit for an onsite psychiatrist and is considering hiring a locum provider.
- 11 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Quality Recommendations
Recommendation 10: Design and implement interventions to increase collaboration, transparency, and engagement with beneficiaries, specifically focusing on increasing treatment planning that is in direct collaboration with the beneficiary in the office.
Status: Met
• First appointments are a face-to-face treatment planning session. The MHP also practices the Wellness Team Meeting model for its Full-Service Partnership (FSP) beneficiaries. This model gathers individuals together in support of the beneficiary including service providers, family members, probation, attorneys, and public guardians to develop a treatment plan focused on the beneficiary’s work-life goals and action plan.
• Staff also received training specifically focused on treatment planning (June 2018 and June 2019). Training focused on collaboration, beyond documentation. While there was not an evaluation of the in-person training, the MHP reports a positive impact on its documentation data. Last year, before the training, 15 percent of charts had missing or had late treatment plan signatures, with 18 percent of planned services being provided without a signature. So far this year, 10 percent of charts at chart review had missing or late treatment plan signatures with no planned services being provided without a signature.
Recommendation 11: Research implementation options of standard level of care tool into EHR.
Status: Met
• The MHP researched the Level of Care Utilization System (LOCUS) tool and found it to be an overly extensive tool given the increased documentation and data tracking requirements. Alternatively, the MHP uses the CANS-50 as a level of care indicator for beneficiaries up to age 20. For adults, 21 and older, the MHP uses the Risk and Resiliency Factors Inventory, which produces outcomes, informs treatment planning and quantifies level of care at the start of treatment.
• The MHP amended its medical necessity form to build-in standardized indicators to screen for Specialty Mental Health Services (SMHS) and mild-to-moderate behavioral health levels of care. The amended form incorporates the same indicators used by both managed care plans. Also included in the medical necessity form update is a screening for In-Home Based Services (IHBS) and Intensive Case Coordination (ICC) services for all beneficiaries up to age 20.
- 12 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Recommendation 12: Resolve issue of extracting co-occurring diagnoses this past year.
Status: Met
• The MHP now runs the co-occurring diagnoses report by diagnostic category rather than subunit of treatment. As a result, the MHP can extract co-occurring diagnoses.
Recommendation 13: Evaluate the low (2.4 percent) number of co-occurring diagnoses reported for year before last for accuracy. If incorrect, resolve data issue.
Status: Met
• The MHP changed its methodology from running the data by beneficiaries open to both mental health and substance use disorder (SUD) services, to looking specifically at diagnostic categories. On the mental health side, they are now able to run reports by diagnosis. On the SUD side, they are evaluating beneficiaries via the Addiction Severity Index (ASI). Referrals between SUD and mental health (collocated) are now being made according to American Society of Addiction Medicine (ASAM) criteria. The MHP now reports a co-occurring diagnoses rate of 16 percent.
Recommendation 14: Design and implement staff training for competency in diagnosing co-occurring mental health and substance abuse disorders.
Status: Met
• In partnership with the MHP’s managed care plan partners, the MHP organized a training on co-occurring disorders held during the January 2019 All Staff meeting. In attendance were counselors, clinicians, and case managers. The training included best practices, the recovery model, criteria from the Diagnostic and Statistical Manual of Mental Disorders (DSM–5) and was followed-up by supervision discussion.
• The need for continuous training was identified. The MHP hired two SUD certified counselors. Of note, the MHP anecdotally found that clinicians were reluctant to give diagnoses related to a substance use disorders as they believed that if beneficiaries were applying for Social Security Disability Insurance (SSDI), it could slow the process down for beneficiaries and delay approvals/payments.
- 13 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Outcomes Recommendations
Recommendation 15: Research evidence-based practice outcomes tool to implement
for adult beneficiaries.
Status: Not Met
• The MHP elected to continue the use of the Risk and Resiliency Factors Inventory. Given increasing documentation and tracking demands, the MHP is waiting to see what direction DHCS will pursue for adult outcomes before deciding on another tool.
Foster Care Recommendations
Recommendation 16: Assess staffing capacity issues related to insufficient numbers
of personnel.
Status: Met
• The MHP was able to hire and fill open positions this last fiscal year. Currently they do not have any provider deficiencies as identified in the MHP’s network adequacy certification from DHCS.
Recommendation 17: Retrain staff on coding and service delivery types for ICC and
Intensive Home Based Services (IHBS) counts and conduct a post-training assessment.
Status: Met
• The MHP provided a formal training on coding in May 2019 to all mental health staff. The MHP identified this as an ongoing training need and will continue training on this topic in the future. Coding is also addressed in individual supervision with youth and family providers.
Recommendation 18: Design and implement a system that ensures parents and
caretakers have adequate information regarding their children’s medications.
Status: Met
• The MHP reports that parents and caretakers are offered information every time a child is prescribed or has a change in medication. Information is reviewed with the parent or caretaker by the psychiatrist; further, parents/caretakers acknowledge their receipt of this information through the medication consent process. For foster youth not residing with a parent, the MHP’s psychiatrist ensures that a caretaker is present. The MHP has worked closely with its tele-psychiatry program to ensure that medication consents are documented during any change in medication.
- 14 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Information Systems Recommendations
Recommendation 19: Resolve the issue of extracting co-occurring diagnoses from the EHR.
Status: Met
• This recommendation was addressed previously.
Recommendation 20: Implement action schedules installed in the EHR with
collaboration of EHR vendor.
Status: Partially Met
• The MHP reports that the design of action schedules is cumbersome resulting in significant time constraints on staff and little margin for error correction. The MHP consulted Kings View regarding the issue and were told that the issue will not be modified or considered until the Millennium upgrade is released from Cerner.
• The MHP has opted to continue use of an internal beneficiary list in Excel, with due dates of all documentation for staff, as an alternate to action schedules.
Recommendation 21: Continue work to install Cerner Community Behavioral Health
(CCBH) Promotions 227 and 228.
Status: Met
• As of June 19, 2019, the MHP has installed Promotions 227, 228 and 229 on their EHR and are fully operational.
Carry-over and Follow-up Recommendations from FY 2017-18
Recommendation 22: Continue discussion with GMC and NVIHC regarding the opportunity for HIE pilot program collaboration. (This recommendation is a follow-up to the partially met Recommendation 4, FY 2017-2018.)
Status: Met
• This recommendation was addressed previously.
Recommendation 23: Design and implement interventions to increase collaboration, transparency, and engagement with beneficiaries, specifically focusing on increasing treatment planning that is in direct collaboration with the beneficiary in the office. (This recommendation is a continuation of part of the met Recommendation 3, FY 2017-2018 and is based on the MHP’s response.)
Status: Met
• This recommendation was addressed previously.
- 15 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
PERFORMANCE MEASUREMENT
CalEQRO is required to validate the following eight mandatory PMs as defined by DHCS:
• Total beneficiaries served by each county MHP.
• Penetration rates in each county MHP.
• Total costs per beneficiary served by each county MHP.
• High-Cost Beneficiaries (HCBs) incurring $30,000 or higher in approved claims during a CY.
• Count of Therapeutic Behavioral Services (TBS) beneficiaries served compared to the 4 percent Emily Q. Benchmark (not included in MHP reports; this information is included in the Annual Statewide Report submitted to DHCS).
• Total psychiatric inpatient hospital episodes, costs, and average length of stay (LOS).
• Psychiatric inpatient hospital 7-day and 30-day rehospitalization rates.
• Post-psychiatric inpatient hospital 7-day and 30-day SMHS follow-up service rates.
- 16 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
In addition, CalEQRO examines the following SB 1291 PMs (Chapter 844; Statutes of 2016) for each MHP:4
• The number of Medi-Cal eligible minor and nonminor dependents.
• Types of mental health services provided to children, including prevention and treatment services. These types of services may include, but are not limited to, screenings, assessments, home-based mental health services, outpatient services, day treatment services or inpatient services, psychiatric hospitalizations, crisis interventions, case management, and psychotropic medication support services.
• Performance data for Medi-Cal eligible minor and nonminor dependents in FC.
• Utilization data for Medi-Cal eligible minor and nonminor dependents in FC.
4 Public Information Links to SB 1291 and foster care specific data requirements:
1. Senate Bill (SB) 1291 (Chapter 844). This statute would require annual mental health plan reviews to be conducted
by an EQRO and, commencing July 1, 2018, would require those reviews to include specific data for Medi-Cal eligible
minor and nonminor dependents in foster care, including the number of Medi-Cal eligible minor and nonminor
dependents in foster care served each year. The bill would require the department to share data with county boards
of supervisors, including data that will assist in the development of mental health service plans and performance
outcome system data and metrics, as specified. More information can be found at http://www.leginfo.ca.gov/pub/15-
16/bill/sen/sb_1251-1300/sb_1291_bill_20160929_chaptered.pdf
2. EPSDT POS Data Dashboards:
http://www.dhcs.ca.gov/provgovpart/pos/Pages/Performance-Outcomes-System-Reports-and-Measures-
Catalog.aspx
3. Psychotropic Medication and HEDIS Measures:
http://cssr.berkeley.edu/ucb_childwelfare/ReportDefault.aspx includes:
• 5A (1&2) Use of Psychotropic Medications
• 5C Use of Multiple Concurrent Psychotropic Medications
• 5D Ongoing Metabolic Monitoring for Children on Antipsychotic Medications New Measure
http://www.dhcs.ca.gov/dataandstats/Pages/Quality-of-Care-Measures-in-Foster-Care.aspx
4. Assembly Bill (AB) 1299 (Chapter 603; Statues of 2016). This statute pertains to children and youth in foster care and ensures that foster children who are placed outside of their county of original jurisdiction, are able to access mental health services in a timely manner consistent with their individualized strengths and needs and the requirements of EPSDT program standards and requirements. This process is defined as presumptive transfer as it transfers the responsibility to provide or arrange for mental health services to a foster child from the county of original jurisdiction to the county in which the foster child resides. More information can be found at
http://www.leginfo.ca.gov/pub/15-16/bill/asm/ab_1251-1300/ab_1299_bill_20160925_chaptered.pdf
5. Katie A. v. Bonta: The plaintiffs filed a class action suit on July 18, 2002, alleging violations of federal Medicaid laws, the American with Disabilities Act, Section 504 of the Rehabilitation Act and California Government Code Section 11135. The suit sought to improve the provision of mental health and supportive services for children and youth in, or at imminent risk of placement in, foster care in California. More information can be found at https://www.dhcs.ca.gov/Pages/KatieAImplementation.aspx.
- 17 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
• Medication monitoring consistent with the child welfare psychotropic medication measures developed by the State Department of Social Services and any Healthcare Effectiveness Data and Information Set (HEDIS) measures related to psychotropic medications, including, but not limited to, the following.
o Follow-Up Care for Children Prescribed Attention Deficit Hyperactivity Disorder Medication (HEDIS ADD).
o Use of Multiple Concurrent Antipsychotics in Children and Adolescents (HEDIS APC).
o Use of First-Line Psychosocial Care for Children and Adolescents on Antipsychotics (HEDIS APP).
• Metabolic Monitoring for Children and Adolescents on Antipsychotics (HEDIS APM).
• Access to, and timeliness of, mental health services, as described in Sections 1300.67.2, 1300.67.2.1, and 1300.67.2.2 of Title 28 of the California Code of Regulations and consistent with Section 438.206 of Title 42 of the Code of Federal Regulations, available to Medi-Cal eligible minor and nonminor dependents in FC.
• Quality of mental health services available to Medi-Cal eligible minor and nonminor dependents in FC.
• Translation and interpretation services, consistent with Section 438.10(c)(4) and (5) of Title 42 of the Code of Federal Regulations and Section 1810.410 of Title 9 of the California Code of Regulations, available to Medi-Cal eligible minor and nonminor dependents in FC.
Health Information Portability and Accountability Act (HIPAA)
Suppression Disclosure:
Values are suppressed to protect confidentiality of the individuals summarized in the data sets when the beneficiary count is less than or equal to 11 (*). Additionally, suppression may be required to prevent calculation of initially suppressed data; corresponding penetration rate percentages (n/a); and cells containing zero, missing data or dollar amounts (-).
- 18 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Total Beneficiaries Served
Table 1 provides details on beneficiaries served by race/ethnicity.
Penetration Rates and Approved Claims per Beneficiary
The penetration rate is calculated by dividing the number of unduplicated beneficiaries served by the monthly average Medi-Cal enrollee count. The annual average approved claims per beneficiary (ACB) served is calculated by dividing the total annual Medi-Cal approved claim dollars by the unduplicated number of Medi-Cal beneficiaries served during the corresponding year.
CalEQRO has incorporated the Affordable Care Act (ACA) Expansion data in the total Medi-Cal enrollees and beneficiaries served. Attachment C provides further ACA-specific utilization and performance data for CY 2018. See Table C1 for the CY 2018 ACA penetration rate and ACB.
Regarding the calculation of penetration rates, the Glenn MHP uses a different method than that used by CalEQRO.
Race/Ethnicity
Average Monthly
Unduplicated
Medi-Cal
Enrollees
%
Enrollees
Unduplicated
Annual Count
Beneficiaries
Served
% Served
White 4,488 35.2% 461 54.5%
Latino/Hispanic 6,795 53.3% 287 33.9%
African-American 89 0.7% * n/a
Asian/Pacific Islander 433 3.4% * n/a
Native American 259 2.0% 19 2.2%
Other 683 5.4% 50 5.9%
Total 12,746 100% 846 100%
Table 1. Medi-Cal Enrollees and Beneficiaries Served in CY 2018
by Race/Ethnicity
Glenn MHP
The total for Average Monthly Unduplicated Medi-Cal Enrollees is not a direct sum of the averages above
it. The averages are calculated independently.
- 19 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Figures 1A and 1B show three-year (CY 2016-18) trends of the MHP’s overall penetration rates and ACB, compared to both the statewide average and the average for small-rural MHPs.
CY 2016 CY 2017 CY 2018
Glenn 7.42% 6.58% 6.64%
Small-Rural 7.66% 7.58% 7.71%
State 4.53% 4.52% 4.66%
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
Figure 1A. Overall Penetration Rates Glenn MHP
CY 2016 CY 2017 CY 2018
Glenn $3,751 $3,408 $3,624
Small-Rural $3,845 $3,692 $3,794
State $5,978 $6,170 $6,454
$0
$1,000
$2,000
$3,000
$4,000
$5,000
$6,000
$7,000
Figure 1B. Overall ACB Glenn MHP
- 20 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Figures 2A and 2B show three-year (CY 2016-18) trends of the MHP’s Latino/Hispanic penetration rates and ACB, compared to both the statewide average and the average for small-rural MHPs.
CY 2016 CY 2017 CY 2018
Glenn 4.67% 3.84% 4.22%
Small-Rural 4.77% 4.54% 4.79%
State 3.51% 3.35% 3.78%
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
Figure 2A. Latino/Hispanic Penetration Rates Glenn MHP
CY 2016 CY 2017 CY 2018
Glenn $3,214 $2,598 $3,333
Small-Rural $3,499 $2,886 $3,146
State $5,588 $5,278 $5,904
$0
$1,000
$2,000
$3,000
$4,000
$5,000
$6,000
$7,000
$8,000
$9,000
$10,000
Figure 2B. Latino/Hispanic ACB Glenn MHP
- 21 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Figures 3A and 3B show three-year (CY 2016-18) trends of the MHP’s FC penetration rates and ACB, compared to both the statewide average and the average for small-rural MHPs.
CY 2016 CY 2017 CY 2018
Glenn 59.09% 48.15% 46.34%
Small-Rural 39.90% 40.96% 40.58%
State 47.48% 47.28% 48.41%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Figure 3A. FC Penetration Rates Glenn MHP
CY 2016 CY 2017 CY 2018
Glenn $6,528 $6,723 $7,310
Small-Rural $7,290 $6,553 $6,630
State $9,521 $9,962 $9,340
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
Figure 3B. FC ACB Glenn MHP
- 22 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
High-Cost Beneficiaries
Table 2 provides the three-year summary (CY 2016-18) MHP HCBs and compares the statewide data for HCBs for CY 2018 with the MHP’s data for CY 2018, as well as the prior two years. HCBs in this table are identified as those with approved claims of more than $30,000 in a year.
See Attachment C, Table C2 for the distribution of the MHP beneficiaries served by ACB range for three cost categories: under $20,000; $20,000 to $30,000; and above $30,000.
Psychiatric Inpatient Utilization
Table 3 provides the three-year summary (CY 2016-18) of MHP psychiatric inpatient utilization including beneficiary count, admission count, approved claims, and LOS.
MHP YearHCB
Count
Total
Beneficiary
Count
HCB %
by
Count
Average
Approved
Claims
per HCB
HCB
Total Claims
HCB % by
Total
Claims
Statewide CY 2018 23,164 618,977 3.74% $57,725 $1,337,141,530 33.47%
CY 2018 * 846 n/a $39,863 - n/a
CY 2017 * 842 n/a $40,032 - n/a
CY 2016 * 929 n/a $42,059 - n/a
Table 2. High-Cost Beneficiaries
Glenn MHP
MHP
CY 2018 36 45 10.88 $11,426 $411,351
CY 2017 36 48 14.38 $11,344 $408,385
CY 2016 38 58 10.43 $10,704 $406,761
Table 3. Psychiatric Inpatient Utilization - Glenn MHP
Year
Unique
Beneficiary
Count
Total
Inpatient
Admissions
Average
LOSACB
Total Approved
Claims
- 23 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Post-Psychiatric Inpatient Follow-Up and Rehospitalization
Figures 4A and 4B show the statewide and MHP 7-day and 30-day post-psychiatric inpatient follow-up and rehospitalization rates for CY 2017 and CY 2018.
Outpatient MHP Outpatient State Rehospitalization MHP Rehospitalization State
CY 2017 59% 36% 3% 3%
CY 2018 47% 32% 0% 3%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Figure 4A. 7-Day Post-Psychiatric Inpatient Follow-up Glenn MHP
Outpatient MHP Outpatient State Rehospitalization MHP Rehospitalization State
CY 2017 76% 54% 6% 7%
CY 2018 75% 48% 3% 6%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Figure 4B. 30-Day Post-Psychiatric Inpatient Follow-up Glenn MHP
- 24 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Diagnostic Categories
Figures 5A and 5B compare statewide and MHP diagnostic categories by the number of beneficiaries served and total approved claims, respectively, for CY 2018.
The MHP’s self-reported percent of beneficiaries served with co-occurring (i.e., substance abuse and mental health) diagnoses: 16 percent.
Depression Psychosis Disruptive Bipolar Anxiety Adjustment Other Deferred
MHP CY 2018 33% 7% 5% 5% 30% 12% 5% 3%
State CY 2018 28% 16% 9% 8% 14% 8% 14% 5%
0%
5%
10%
15%
20%
25%
30%
35%
Figure 5A. Diagnostic Categories, Beneficiaries Served Glenn MHP
Depression Psychosis Disruptive Bipolar Anxiety Adjustment Other Deferred
MHP CY 2018 28% 13% 8% 7% 28% 8% 7% 0%
State CY 2018 22% 20% 10% 6% 13% 5% 22% 1%
0%
5%
10%
15%
20%
25%
30%
35%
Figure 5B. Diagnostic Categories, Total Approved Claims Glenn MHP
- 25 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
PERFORMANCE IMPROVEMENT PROJECT
VALIDATION
A PIP is defined by CMS as “a project designed to assess and improve processes and outcomes of care that is designed, conducted, and reported in a methodologically sound manner.” CMS’ EQR Protocol 3: Validating Performance Improvement Projects mandates that the EQRO validate one clinical and one non-clinical PIP for each MHP that were initiated, underway, or completed during the reporting year, or featured some combination of these three stages.
Glenn MHP PIPs Identified for Validation
Each MHP is required to conduct two PIPs during the 12 months preceding the review. CalEQRO reviewed two PIPs and validated one PIP, as shown below.
Table 4 lists the findings for each section of the evaluation of the PIPs, as required by the PIP Protocols: Validation of Performance Improvement Projects.5
Clinical PIP—Intensive Home-Based Services (IHBS)
The MHP presented its study question for the clinical PIP as follows:
“Will increasing the delivery of IHBS services to children, youth, and caregivers who have an open Child Welfare Services (CWS) case improve the family’s resiliency and increase the number of children who are reunified with their family, as indicated by an improved score in the Life Domain Functioning section of the CANS-50?”
Date PIP began: July 2017
Projected End date: July 2019
Status of PIP: Inactive, developed in a prior year (not rated)
5 2012 Department of Health and Human Services, Centers for Medicare and Medicaid Service Protocol 3 Version
2.0, September 2012. EQR Protocol 3: Validating Performance Improvement Projects.
Table 4: PIPs Submitted by Glenn MHP
PIPs for Validation
# of PIPs
PIP Titles
Clinical PIP 1 Intensive Home-Based Services (IHBS)
Non-clinical PIP 1 Timely Access
- 26 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
The goal of this PIP, as stated last year, was is to increase the number of IHBS provided to beneficiaries with open CWS cases, increase beneficiary resiliency as evidenced by scoring on the Risk and Resiliency Factors Inventory, and, thereby, promote an environment that supports permanency and/or a stable living situation. The MHP revised its question to clarify that it aimed to increase reunification through increased IBHS, and to rate the success of the process via the CANS-50, in particular the life domain function. The MHP made few changes. Having implemented the CANS-50 in compliance with the State, the MHP named the outcome tool as an intervention. It also added staff training; however, both of its new interventions are not true clinical interventions with a direct impact on beneficiaries. The CANS-50 is a measurement tool, and staff training on codes and modalities is a process to improve quality and the integrity of data entry. Further, it was not clear why the MHP decided that children need more IHBS. The indicators presented need further explanation as to how they measure performance and are relevant to the study question.
Suggestions to improve the PIP: This PIP is not rated and is categorized as having been developed in a prior year. While there were changes that were recommended last year, it is advised that this PIP formally end, and a new one begins. For this PIP, CalEQRO reviewed study question criteria – that study questions need to be measurable and contain quantifiable goals (percentage or number). Also discussed was the need for the MHP to outline why they think that children need more IHBS, and how it relates to the cause of the identified problem. The indicators presented need further explanation as to how they measure performance and are relevant to the study question.
Relevant details of these issues and recommendations are included within the comments found in the PIP validation tool.
The technical assistance (TA) provided to the MHP by CalEQRO identified several steps of a PIP submission that need further detail. The MHP presented improving outcomes through better family functioning as an idea for its next Clinical PIP. CalEQRO reviewed the different phases of PIP development and the need to do adequate background work to set the PIP up for success. CalEQRO reviewed gathering data to define the scope and depth of a problem, and a barrier analysis to determine the cause(s) of the problem. CalEQRO reviewed the connection between identified barriers and selecting appropriate beneficiary centered interventions, which are likely to have the greatest impact. The MHP was encouraged to consult with EQRO early and often during the continuation of this PIP.
Non-clinical PIP—Timely Access
The MHP presented its study question for the non-clinical PIP as follows:
“Will implementing a triage assessment process help high-need beneficiaries with an urgent need access services within five days; ensure beneficiaries with a routine need
- 27 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
for services access services within 10 days; and beneficiaries requesting psychiatric services receive services within 15 days?”
Date PIP began: September 2016
Projected End date: September 2019
Status of PIP: Completed
The goal of this non-clinical PIP was to implement a triage assessment process to help high-need beneficiaries access services as quickly as possible, as well as ensure all beneficiaries have access to services in a timely manner. Annual External Quality Reviews have shown that in recent years Glenn County did not score well on timeliness to mental health and/or psychiatric services. The MHP identified this as an important area for improvement to ensure proper and consistent access to care for their beneficiaries.
In the initial year of the PIP, the MHP used 7, 14, and 21 days for prioritizing referrals for urgent, moderate, and routine beneficiaries, and have since updated the standards to include the Final Rule timeliness regulations of 15 business to psychiatric and 10 business days to mental health services.
The MHP initially developed interventions to directly impact the access processes and improve timeliness to services for beneficiaries. For this year, the MHP added the expansion of available appointments through hiring a dedicated staff person for assessments and adding assessment slots to supervisor schedules. The MHP also implemented a scheduling process which allowed the front office to schedule follow-ups efficiently.
For 5-day urgent requests for service, at baseline, 42.4 percent met the standard. At the time of remeasurement, 57 percent met the standard, representing an improvement of 9.4 percentage points.
Suggestions to improve the PIP: The PIP could have benefited from beneficiary inclusion on the PIP team. Also, re-measurement should occur quarterly at minimum rather than yearly. Although the initial PIP and problem were identified through data, further evaluation as to the related causes and/or barriers was not completed. Although the study question was measurable, its multipart structure was complicated. Simple, straight-forward questions which identify the problem, intervention and outcome measure might prove more answerable.
Relevant details of these issues and recommendations are included within the comments found in the PIP validation tool.
The TA provided to the MHP by CalEQRO consisted of recommending that the MHP measure results quarterly. Discussion onsite involved the MHP’s plan to end the PIP and begin a new Non-clinical PIP on the topic of improving beneficiary access to Harmony House, the MHP’s wellness center. The MHP was also encouraged to consult with EQRO early and often during the development of new PIPs. Recommendations for
- 28 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
new PIPs include using a simple, straight-forward question which identifies the problem, intervention and outcome measure might prove more answerable; moreover, the new PIP should include more in-depth data evaluation, a barrier analysis, to ensure that interventions are related to problem causes.
Table 5, on the following pages, provides the overall rating for each PIP, based on the ratings: Met (M), Partially Met (PM), Not Met (NM), Not Applicable (NA), Unable to Determine (UTD), or Not Rated (NR).
Table 5: PIP Validation Review
Item Rating
Step PIP Section Validation Item Clinical Non-
Clinical
1 Selected
Study Topics
1.1 Stakeholder input/multi-functional team
NR PM
1.2 Analysis of comprehensive aspects of enrollee needs, care, and services
NR PM
1.3 Broad spectrum of key aspects of enrollee care and services
NR M
1.4 All enrolled populations NR M
2 Study
Question 2.1 Clearly stated NR PM
3 Study
Population
3.1 Clear definition of study population NR M
3.2 Inclusion of the entire study population
NR M
4 Study
Indicators
4.1 Objective, clearly defined, measurable indicators
NR M
4.2 Changes in health states, functional status, enrollee satisfaction, or processes of care
NR M
5 Sampling Methods
5.1 Sampling technique specified true frequency, confidence interval and margin of error
NR NA
5.2 Valid sampling techniques that protected against bias were employed
NR NA
- 29 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table 5: PIP Validation Review
Item Rating
Step PIP Section Validation Item Clinical Non-
Clinical
5.3 Sample contained sufficient number of enrollees
NR NA
6 Data
Collection Procedures
6.1 Clear specification of data NR M
6.2 Clear specification of sources of data
NR M
6.3 Systematic collection of reliable and valid data for the study population
NR M
6.4 Plan for consistent and accurate data collection
NR M
6.5 Prospective data analysis plan including contingencies
NR PM
6.6 Qualified data collection personnel NR PM
7 Assess
Improvement Strategies
7.1 Reasonable interventions were undertaken to address causes/barriers
NR PM
8
Review Data Analysis and Interpretation
of Study Results
8.1 Analysis of findings performed according to data analysis plan
NR M
8.2 PIP results and findings presented clearly and accurately
NR PM
8.3 Threats to comparability, internal and external validity
NR M
8.4 Interpretation of results indicating the success of the PIP and follow-up
NR M
9 Validity of
Improvement
9.1 Consistent methodology throughout the study
NR M
9.2 Documented, quantitative improvement in processes or outcomes of care
NR M
9.3 Improvement in performance linked to the PIP
NR PM
- 30 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table 5: PIP Validation Review
Item Rating
Step PIP Section Validation Item Clinical Non-
Clinical
9.4 Statistical evidence of true improvement
NR NM
9.5 Sustained improvement demonstrated through repeated measures
NR NM
Table 6 provides a summary of the PIP validation review.
Table 6: PIP Validation Review Summary
Summary Totals for PIP Validation Clinical PIP Non-clinical
PIP
Number Met 0 15
Number Partially Met 0 8
Number Not Met 0 2
Unable to Determine 0 0
Number Applicable (AP)
(Maximum = 28 with Sampling; 25 without Sampling) 28 25
Overall PIP Ratings ((#M*2) +(#PM))/(AP*2) 0% 76%
- 31 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
INFORMATION SYSTEMS REVIEW
Understanding the capabilities of an MHP’s information system is essential to evaluating its capacity to manage the health care of its beneficiaries. CalEQRO used the written response to standard questions posed in the California-specific ISCA, additional documents submitted by the MHP, and information gathered in interviews to complete the information systems evaluation.
Key Information Systems Capabilities Assessment (ISCA)
Information Provided by the MHP
The following information is self-reported by the MHP through the ISCA and/or the site review.
Table 7 shows the percentage of MHP budget dedicated to supporting IT operations,
including hardware, network, software license, and IT staff for the past four-year period.
For comparative purposes, we have included similar size MHPs and statewide average
IT budgets per year for prior three-year periods.
Table 7: Budget Dedicated to Supporting IT Operations
FY 2019-20 FY 2018-19 FY 2017-18 FY 2016-17
Glenn 8.16% 4.42% 2.85% 2.78%
Small-Rural MHPs N/A 4.20% 3.70% 3.80%
Statewide N/A 3.40% 3.30% 3.40%
• The MHP IT budget has increased incrementally over the past four years.
The budget determination process for information system operations is:
☐ Under MHP control
☐ Allocated to or managed by another County department
☒ Combination of MHP control and another County department or Agency
- 32 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table 8 shows the percentage of services provided by type of service provider.
Table 8: Distribution of Services, by Type of Provider
Type of Provider Distribution
County-operated/staffed clinics 97.29%
Contract providers 2.71%
Network providers 0.00%
Total 100%*
*Percentages may not add up to 100 percent due to rounding.
Table 9 identifies methods available for contract providers to submit beneficiary clinical
and demographic data; practice management and service information; and transactions
to the MHP’s EHR system, by type of input methods.
Table 9: Contract Providers Transmission of Beneficiary Information to MHP EHR System
Type of Input Method Percent
Used Frequency
Direct data entry into MHP EHR system by contract provider staff
0% Not used
Electronic data interchange (EDI) uses standardized electronic message format to exchange beneficiary information between contract provider EHR systems and MHP EHR system
0% Not used
Electronic batch files submitted to MHP for further processing and uploaded into MHP EHR system
0% Not used
Electronic files/documents securely emailed to MHP for processing or data entry input into EHR system
50% Weekly
Paper documents submitted to MHP for data entry input by MHP staff into EHR system
50% Daily
Health Information Exchange (HIE) securely share beneficiary medical information from contractor EHR system to MHP EHR system and return message or medical information to contractor EHR
0% Not used
- 33 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
• MHP does not utilize contract providers that serve beneficiaries within county service area to provide outpatient services locally in a clinic/program setting.
Telehealth Services
MHP currently provides services to beneficiaries using a telehealth application:
☒ Yes ☐ No ☐ In pilot phase
• Number of county-operated sites currently operational: Two
Identify primary reason(s) for using telehealth as a service extender (check all that apply):
☒ Hiring healthcare professional staff locally is difficult
☐ For linguistic capacity or expansion
☐ To serve outlying areas within the county
☐ To serve beneficiaries temporarily residing outside the county
☐ To serve special populations (i.e. children/youth or older adult)
☐ To reduce travel time for healthcare professional staff
☐ To reduce travel time for beneficiaries
• Telehealth services are available with English and Spanish at two sites, Willows and Orland.
• Of the 1338 total telehealth services provided over a 12 month period, approximately nine telehealth sessions were conducted in Spanish and Hmong. The MHP’s capacity to serve non-English speaking beneficiaries is limited via telehealth.
• Recently, the MHP has increased telehealth services to 29 hours/week.
• Kings View is the MHP’s telehealth vendor.
- 34 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Summary of Technology and Data Analytical Staffing
MHP self-reported IT staff changes by full-time equivalents (FTE) since the previous CalEQRO review are shown in Table 10.
Table 10: Technology Staff
Fiscal Year
IT FTEs (Include
Employees and Contractors)
# of New FTEs
# Employees / Contractors Retired,
Transferred, Terminated
Current # Unfilled
Positions
2019-20 2 0 0 0
2018-19 2 0 0 0
2017-18 2 1 1 1
MHP self-reported data analytical staff changes by FTEs since the previous CalEQRO review are shown in Table 11.
Table 11: Data Analytical Staff
Fiscal Year
IT FTEs (Include
Employees and Contractors)
# of New FTEs
# Employees / Contractors Retired,
Transferred, Terminated
Current # Unfilled
Positions
2019-20 8 0 0 0
2018-19 7 1 0 0
2017-18 3 0 0 0
The following should be noted with regard to the above information:
• The two technology FTEs are County IT staff and support county systems like email, they do not support the EHR.
• The eight data analytical FTEs are Quality Improvement (QI) staff who have multiple responsibilities, but all do some reporting for the MHP. The MHP estimated its data analytical staffing to be one FTE.
- 35 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
• Kings View provides data analytics support to the MHP such as producing the monthly penetration rates report.
• IDEA Consulting has also done work for the MHP such as geo-maps.
Current Operations
• The MHP uses CCBH as its EHR with Promotion 229.
• Kings View hosts CCBH as an Application Service Provider (ASP).
• Glenn County’s Parkside facility in Orland was Medi-Cal certified in April 2019. Billing for services from October 2018 to April 2019 were submitted in June 2019.
• Two QI staff are designated CCBH super users and serve as Kings View’s liaison.
• Youth for Change is the MHP’s contract provider.
• The MHP has secured funding for the Cerner Millennium upgrade.
Table 12 lists the primary systems and applications the MHP uses to conduct business and manage operations. These systems support data collection and storage; provide EHR functionality; produce Short-Doyle Medi-Cal (SDMC) and other third-party claims; track revenue; perform managed care activities; and provide information for analyses and reporting.
Table 12: Primary EHR Systems/Applications
System/Application Function Vendor/Supplier Years Used
Operated By
CCBH EHR Cerner 7 Kings View
The MHP’s Priorities for the Coming Year
• Network Adequacy geo-mapping.
• Client and Service Information (CSI) system for timeliness implementation and reporting.
• Create CANS-50 reports.
• Create data dashboards.
- 36 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Major Changes since Prior Year
• CCBH was updated from Promotion 226 to 229.
• Completed CANS-50 and Family Advocacy and Support Tool (FAST) reporting.
• Developed Network Adequacy reporting.
• Built a new medical necessity form in the EHR that indicates which level of care (specialty mental health or mild to moderate) a beneficiary should be referred to.
• Developed a new Brief Assessment form in the EHR.
• Updated an Access Contact Log to support new CSI data requirements and to track more meaningful data.
Other Areas for Improvement
• The MHP reported 16 percent of active beneficiaries have co-occurring mental health and substance use diagnoses.
• The MHP initiated discussions with the Glenn Medical Center and North Valley Indian Health Center to set up a Health Information Exchange pilot; however, both providers have expressed no interest in the project.
Plans for Information Systems Change
• The MHP is in discussion with Kings View about upgrading to Cerner Millennium.
Current EHR Status
Table 13 summarizes the ratings given to the MHP for EHR functionality.
Table 13: EHR Functionality
Rating
Function System/Application Present Partially Present
Not Present
Not Rated
Alerts Cerner/CCBH X
Assessments Cerner/CCBH X
Care Coordination X
Document Imaging/ Storage
Cerner/CCBH X
- 37 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table 13: EHR Functionality
Rating
Function System/Application Present Partially Present
Not Present
Not Rated
Electronic Signature—MHP Beneficiary
Cerner/CCBH X
Laboratory results (eLab) X
Level of Care/Level of Service
X
Outcomes Cerner/CCBH X
Prescriptions (eRx) Cerner/CCBH X
Progress Notes Cerner/CCBH X
Referral Management X
Treatment Plans Cerner/CCBH X
Summary Totals for EHR Functionality:
FY 2019-20 Summary Totals for EHR Functionality:
8 1 3 0
FY 2018-19 Summary Totals for EHR Functionality*:
7 0 5 0
FY 2017-18 Summary Totals for EHR Functionality:
6 2 1 0
*Two new EHR functionalities were added to the list beginning in FY 2017-18. Progress and issues associated with implementing an EHR over the past year are summarized below:
• The MHP has made small incremental progress towards implementing a fully functional EHR system over the past three years.
Personal Health Record (PHR)
Do beneficiaries have online access to their health records through a PHR feature provided within the EHR, a beneficiary portal, or third-party PHR?
☐ Yes ☐ In Test Phase ☒ No
- 38 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
If no, provide the expected implementation timeline.
☐ Within 6 months ☐ Within the next year
☐ Within the next two years ☒ Longer than 2 years
Medi-Cal Claims Processing MHP performs end-to-end (837/835) claim transaction reconciliations: If yes, product or application:
Dimension Reports
Method used to submit Medicare Part B claims:
☐ Paper ☒ Electronic ☐ Clearinghouse
Table 14 summarizes the MHP’s SDMC claims.
Service
Month
Number
Submitted
Dollars
Billed
Number
Denied
Dollars
Denied
Percent
Denied
Dollars
Adjudicated
Dollars
Approved
TOTAL 14,034 $4,083,334 134 $47,856 1.17% $4,035,478 $2,988,654
JAN18 1,195 $289,289 4 $969 0.33% $288,320 $229,627
FEB18 1,069 $285,161 4 $938 0.33% $284,223 $213,366
MAR18 1,275 $312,261 7 $2,131 0.68% $310,130 $240,749
APR18 1,150 $287,934 1 $382 0.13% $287,552 $226,742
MAY18 1,249 $382,141 3 $572 0.15% $381,569 $289,562
JUN18 1,144 $391,867 6 $1,610 0.41% $390,257 $274,710
JUL18 1,086 $360,394 43 $15,786 4.38% $344,608 $245,762
AUG18 1,242 $383,157 31 $15,506 4.05% $367,651 $259,023
SEP18 1,957 $608,513 18 $5,593 0.92% $602,920 $443,006
OCT18 1,032 $302,536 14 $3,420 1.13% $299,116 $220,066
NOV18 845 $249,872 2 $537 0.21% $249,335 $180,900
DEC18 790 $230,208 1 $413 0.18% $229,795 $165,141
Table 14. Summary of CY 2018 Short Doyle/Medi-Cal Claims
Glenn MHP
Includes services provided during CY 2018 with the most recent DHCS claim processing date of June 7, 2019.
Only reports Short-Doyle/Medi-Cal claim transactions, does not include Inpatient Consolidated IPC hospital claims.
Statewide denial rate for CY 2018 was 3.25 percent.
☒ Yes ☐ No
- 39 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table 15 summarizes the top three reasons for claim denial.
Denial Reason DescriptionNumber
Denied
Dollars
Denied
Percent
of Total
Denied
Invalid procedure code and modifier combination OR single service
exceeds maximum minutes per day.6 $18,150 38%
Medicare or Other Health Coverage must be billed before submission
of claim.45 $12,107 25%
Service line is a duplicate and repeat service procedure modifer is not
present.55 $8,925 19%
TOTAL 134 $47,856 N/A
The total denied claims information does not represent a sum of the top three reasons. It is a sum of all denials.
Table 15. Summary of CY 2018 Top Three Reasons for Claim Denial
Glenn MHP
- 40 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
CONSUMER AND FAMILY MEMBER FOCUS
GROUP(S)
CalEQRO conducted one 90-minute focus group with consumers (MHP beneficiaries) and/or their family members during the site review of the MHP. As part of the pre-site planning process, CalEQRO originally requested two focus groups with 10 to 12 participants each, the details of which can be found in each section below. Through collaborative planning, the MHP and CalEQRO mutually decided that one focus group was sufficient.
The consumer and family member (CFM) focus group is an important component of the CalEQRO site review process. Feedback from those who are receiving services provides important information regarding quality, access, timeliness, and outcomes. The focus group questions emphasize the availability of timely access to care, recovery, peer support, cultural competence, improved outcomes, and CFM involvement. CalEQRO provides gift cards to thank the CFMs for their participation.
CFM Focus Group One
CalEQRO requested a culturally diverse group of adult beneficiaries who are mostly new beneficiaries who have initiated/utilized services within the past 12 months. CalEQRO also requested a culturally diverse group of parents/caregivers of child/youth beneficiaries who are mostly new beneficiaries who have initiated/utilized services within the past 12 months. As mentioned above, the review moved forward with one group. Group attendees included both adult male and female clients and parent/caregivers, who were mostly white. The focus group was held at Glenn County Behavioral Health, 1187 E. South St., Orland, CA 95963.
Number of participants: 14
The ten participants who entered services within the past year described their experiences as the following:
• Participants were referred for services through a variety of means including the wellness center, probation, walk-in and self-referral.
• Most participants accessed services in less than two weeks.
• The majority of participants indicated high satisfaction with services.
Participants’ general comments regarding service delivery included the following:
• Overall, participants received weekly or biweekly therapy. Most receive psychiatry services on a monthly basis.
• All participants stated that the frequency of service was sufficient to improve functioning.
- 41 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
• Support groups are available and regularly attended by group members.
• Most participants have a Wellness Recovery Action Plan (WRAP).
• All participants felt that extra services were available if need be and that they could be seen within a day or two if they were in crisis or had an urgent need.
Participants’ recommendations for improving care included the following:
• Increase transportation and drivers.
• Separate transportation for Harmony House and appointments. Being late or missing appointments can sometimes be an issue and the current two drivers drive all over the county and get very busy.
• Include a music program at the Transition Age Youth (TAY) drop-in center.
• Provide same day after-hours urgent care.
• Implement automatic refill system for medications.
• Update the office area so that it feels warm and welcoming, not clinical.
• Provide calming, peaceful music or television for children who are waiting for appointments.
• Include more services for men at Harmony House, especially support groups for homeless men, men in domestic violence situations, and men who are single parents.
• Offer closed groups, i.e. WRAP groups, at the Harmony House campus that will not be interrupted by people who are there for drop-in services only.
Interpreter used for focus group one: No
- 42 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
PERFORMANCE AND QUALITY MANAGEMENT KEY
COMPONENTS
CalEQRO emphasizes the MHP’s use of data to promote quality and improve performance. Components widely recognized as critical to successful performance management include Access to Care, Timeliness of Services, Quality of Care, Beneficiary Progress/Outcomes, and Structure and Operations. The following tables in this section summarize CalEQRO’s findings in each of these areas.
Access to Care
Table 16 lists the components that CalEQRO considers representative of a broad service delivery system that provides access to beneficiaries and family members. An examination of capacity, penetration rates, cultural competency, integration, and collaboration of services with other providers forms the foundation of access to and delivery of quality services.
Table 16: Access to Care Components
Component Maximum Possible
MHP Score
1A Service Access and Availability 14 14
The MHP provides beneficiaries a written Guide to Services. New clinicians attend orientation. The MHP provides beneficiaries a van schedule for the four drivers who provide transportation. The MHP has an IT team which monitors website usage to ensure links work and are current. The MHP’s System-Wide Mental Health. Assessment Response Treatment (SMART) team provides presentations on school violence prevention. This team is a collaboration with law enforcement.
1B Capacity Management 10 10
Last year there were six vacancies in both adult and children’s’ services. The MHP has now been fully staffed for a year; however, recently two therapists left (July 2019). The MHP is now in the process of advertising. To assist with recruitment, the MHP raised clinician salaries and now offers a 10 percent differential for licensure.
1C Integration and Collaboration 24 24
The MHP collaborates with several community-based organizations and agencies including managed care organizations, local churches and Indian Health Service. The MHP also holds outreach game nights locally in Willows and the following unincorporated areas – Orland, Hamilton City, Oak Creek, and Artois.
- 43 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Timeliness of Services
As shown in Table 17, CalEQRO identifies the following components as necessary for timely access to comprehensive specialty mental health services.
Table 17: Timeliness of Services Components
Component Maximum Possible
MHP Score
2A First Offered Appointment 16 16
Last year, overall, 88.7 percent of offered appointments met the 10-day standard (as compared to the year before, 80 percent).
2B Assessment Follow-up and Routine Appointments 8 4
The MHP reported that it tracks and trends assessment follow-up and routine appointments; however, data was not included in the self-assessment.
2C First Offered Psychiatry Appointment 12 10
From FY 2017 to FY 2018, the MHP made improvements to timeliness of first offered psychiatry appointment, 28.8 percent to 60.4 percent; however, the State standard was not met.
The MHP explained that Colusa had a new county counsel who changed how services were authorized for FC. County Counsel revoked CWS deputies’ ability to sign consent. A new county counsel will take over soon and as a result, the MHP will revisit this.
The Plan must offer a psychiatric appointment within 15 business days. The MHP needs to comply with the state timeliness metric as per IN 18-011.
2D Timely Appointments for Urgent Conditions 18 12
Data was not provided on either types of urgent appointments including looking at the
average and median times, percentages of times the standard is met, and the range.
2E Timely Access to Follow-up Appointments after Hospitalization
10 10
MHP beneficiaries experienced an increase in hospitalizations (59 to 75) after the local Carr Fire and Camp Fire. Many residents, including MHP staff, suffered significant trauma and loss. Of 75 hospitalization discharge appointments, 93.5 percent met the 7-day standard.
- 44 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table 17: Timeliness of Services Components
Component Maximum Possible
MHP Score
2F Tracks and Trends Data on Rehospitalizations 6 6
The hospital readmission rate (within 30 days) was reported as 5.3 percent.
2G Tracks and Trends No-Shows 10 10
The MHP offers a single point of contact for scheduling for first assessment appointments and therapy appointments.
Individual health records contain no-show information. Cancellation type (by staff or beneficiary) is also tracked.
Average no-shows for psychiatrists are 14.3 percent. Average no-shows for clinicians are 11.1 percent. The MHP has a standard of 10 percent for both psychiatrists and clinicians.
Quality of Care
In Table 18, CalEQRO identifies the components of an organization that is dedicated to the overall quality of care. These components ensure that the quality improvement efforts are aligned with the system’s objectives and contributes to meaningful changes in the system to improve beneficiary care characteristics.
Table 18: Quality of Care Components
Component Maximum Possible
MHP Score
3A Beneficiary Needs are Matched to the Continuum of Care
12 12
The MHP provides a monthly support group for staff on secondary traumatization. Weekly support for clinical debriefing of difficult cases is also available. This practice improves staff retention and quality of care. The MHP uses an outside therapist for staff support groups.
The MHP held a treatment planning meeting in June 2019 for staff. Training covered how to collaborate with beneficiaries beyond documentation. While this was an in-person training, there was no evaluation. However, during chart review, 15 percent of charts had missing/late treatment plan signatures. So far for this year, chart review showed 10 percent missing/late notes.
- 45 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
The MHP has 15-20 conservatees in psychiatric facilities They were able to step 5-6 beneficiaries down to less intensive services.
3B Quality Improvement Plan 10 10
The MHP provided a QI Work Plan and an annual evaluation. The Work Plan has measurable goals and provided analysis of disparity in services. The MHP receives a Kings View report on penetration rates quarterly.
3C Quality Management Structure 14 14
The MHP’s QIC is headed by a compliance and quality improvement manager and is attended by all QI staff, billing, EHR, analysts and contracts divisions. Peer mentors and coaches also attend the QIC; moreover, they hold focus groups monthly called the “Consumer Voice” meeting. Information is brought back to the QIC. The QIC meets quarterly, along with the SIC.
3D QM Reports Act as a Change Agent in the System 10 10
The MHP’s utilizes data extraction to evaluate timeliness on a quarterly basis. Data guides scheduling and the number of assessment slots offered. Chart review is used to build training opportunities and sometimes make changes to the EHR.
The MHP reported that for billing Medi-Cal dollars and services provided, they have a disallowance rate of 2.9 percent.
3E Medication Management 12 9
The MHP’s psychiatrist provides medication information to parents and caretakers, who also sign a consent and an acknowledgement. For foster youth not residing with a parent, the psychiatrist ensures that a caretaker is present. The degree that collaboration with primary care providers occurs is unclear. The MHP does not track and trend prescribing practices at the aggregate level.
- 46 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Beneficiary Progress/Outcomes
In Table 19, CalEQRO identifies the components of an organization that is dedicated to beneficiary progress and outcomes as a result of the treatment. These components also include beneficiary perception or satisfaction with treatment and any resulting improvement in beneficiary conditions, as well as capture the MHP’s efforts in supporting its beneficiaries through wellness and recovery.
Table 19: Beneficiary Progress/Outcomes Components
Component Maximum Possible
MHP Score
4A Beneficiary Progress 16 16
The MHP utilizes the CANS-50 and Pediatric Symptom Checklist (PSC-35) for children/youth. The MHP also uses an outside consultant to assist with the collection and analysis.
4B Beneficiary Perceptions 10 10
The MHP administers the DHCS-administered Consumer Perception Survey (CPS). The MHP also has regular opportunities for beneficiaries to provide feedback, at the wellness center, QIC focus groups, and those related to the MHSA stakeholder process.
4C Supporting Beneficiaries through Wellness and Recovery
4 4
The MHP’s wellness center, Harmony House, is a peer-run and peer-driven drop-in community wellness center. It is located at 343 Yolo Street in Orland. Drop-in hours are Monday 12:00 p.m. to 4:30 p.m. and Tuesday through Friday 10:00 a.m. to 4:30 p.m. Transportation is provided if needed by scheduling a day in advance. All support groups and events are open to the public, with some offered in Spanish.
The TAY Center is a youth peer-run, youth friendly wellness center environment offering peer support, expressive arts, mentoring and counseling for youth ages 14--24. The center hours are Monday through Thursday, 12:00 p.m. to 6:00 p.m. and Friday 12:00 p.m. to 5:00 p.m. The center is located at 612 4th Street in Orland.
- 47 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Structure and Operations
In Table 20, CalEQRO identifies the structural and operational components of an organization that is facilitates access, timeliness, quality, and beneficiary outcomes.
Table 20: Structure and Operations Components
Component Quality Rating
5A Capability and Capacity of the MHP 30 28
The MHP’s Parkside location for adult services was certified in April 2019. The MHP received approval retroactive to September 2018.
The MHP is working with managed care providers to provide the required training and education for contracted providers. In that regard, the MHP changed its medical necessity form to closely match managed care forms to facilitate exchange of referrals.
The MHP’s SUD staff attended a Tribal Opioid Coalition. The MHP is in the process of creating a policy and procedure for Narcan.
The MHP held a Mindfulness training in January 2019. All adult staff are certified and using the modality with beneficiaries.
5B Network Adequacy 18 18
The MHP provides 29 hours per week of telehealth. Five hours were added per month starting July 2019. The MHP is recruiting a locum provider one day per week.
5C Subcontracts/Contract Providers 16 N/A
MHP does not have contract providers (less than 3 percent)
5D Stakeholder Engagement 12 12
Staff and focus group participants reported positive communication with MHP management and leadership.
5E Peer Employment 8 6
The MHP has four part-time peer coaches and two peer mentors employed at Harmony House and the TAY center. An established career ladder is in place and employees are aware of how to pursue employment opportunities. Employees start as peer mentors (TAY) or coaches (Harmony House) and can then become case managers. Those with further educational goals can pursue other employment
- 48 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
opportunities. Peer employees are regularly evaluated for job performance on a yearly basis. The MHP does not have supervisory or management positions designated for beneficiaries.
5F Peer-Run Programs 10 9
The MHP has two peer-run peer-driven programs located in Orland. Harmony House is located at 343 Yolo Street and the TAY center at 612 4th street (more information about the centers is provided above in section 4C). Beneficiaries are informed about these programs through informational brochures, flyers, calendars provided by the MHP, case managers, therapists and the Glenn county website. More wellness center locations would potentially help those beneficiaries who are in more isolated areas of the county.
5G Cultural Competency 12 12
The Ethnic Services Committee (ESC) monitors assessment slots for Spanish-speaking beneficiaries and a Spanish-speaking service provider is available. This also applies to telehealth. The ESC meets with monthly and then with clinicians weekly to see what that week’s need is. The ESC was formed to test potential bilingual staff; moreover, this year the ESC became a consultation group for the MHP, acting as a resource for interpreter services. They also review Spanish-language forms. The MHP is hiring a Clinician III to manage the ESC and their work. The MHP has 19 direct service staff, eight direct service speak Spanish.
The MHP uses the Kings View Penetration Report to produce housing fund requests along with transportation logs to identify underserved populations.
The Cultural Competence Committee (CCC) meets monthly, keeps minutes and has CFM representation.
- 49 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
SUMMARY OF FINDINGS
This section summarizes the CalEQRO findings from the FY 2019-20 review of Glenn MHP related to access, timeliness, and quality of care.
MHP Environment – Changes, Strengths, and Opportunities
PIP Status
Clinical PIP Status: Inactive, developed in a prior year (not rated)
Non-clinical PIP Status: Active and ongoing
Access to Care
Changes within the Past Year:
• The Parkside facility in Orland was Medi-Cal certified in April 2019 and it houses the adult unit that offers outpatient services.
• Treatment court started in June 2018 as a pilot and it is now fully operational with a coordinator and program manager.
• In the past year, the MHP added five hours of telehealth services per month to bring the total to 29 hours per week.
• A support group for Lesbian Gay Bisexual Transgender and Queer/Questioning beneficiaries (LGBTQ) was recently started.
• The MHP established an MOU with NVIH for a referral process between the systems and for beneficiary services on a single case-by-case basis.
Strengths:
• The MHP has a van that goes between Willows and Orland and it is supported by four drivers. This van provides transportation to beneficiaries. Case managers also drive beneficiaries to appointments using county vehicles when necessary.
Opportunities for Improvement:
• Stakeholders indicate a need for more drivers and transportation options.
• Stakeholders indicate the need for after-hours urgent care.
• Stakeholders identified an underserved group - men who are homeless who would benefit from increased support services.
- 50 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Timeliness of Services
Changes within the Past Year:
• A process was established in March 2019 for reception to schedule all initial face-to-face appointments. This change ensures that all service entry data is tracked.
Strengths:
• Calls are made to beneficiaries to remind them about scheduled appointments.
• Some groups who need more timely access to treatment (e.g. beneficiaries released from jail) have priority to telehealth services.
Opportunities for Improvement:
• The MHP did not provide information on timeliness of assessment follow-up and routine appointments.
• 60.4 percent of the MHP's first offered psychiatry appointments meet the 15-day standard.
• The MHP did not provide data related to the tracking of timely appointments for urgent conditions.
Quality of Care
Changes within the Past Year:
• In January 2019, adult services clinical staff were trained and certified on mindfulness, an evidence-based practice.
• In May 2019, providers received coding training on service delivery types ICC and IHBS.
• In June 2019, clinical staff were trained on how to build treatment plans.
Strengths:
• Using an outside consultant to run support groups for staff, the MHP provides ongoing support for secondary traumatization and clinical debriefing of difficult cases to prevent burnout and improve quality of care.
Opportunities for Improvement:
• The MHP does not track and trend prescribing practices at the aggregate level.
- 51 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Beneficiary Outcomes
Changes within the Past Year:
• Clinical staff were trained and certified on CANS-50.
Strengths:
• None noted.
Opportunities for Improvement:
• The MHP continues to use the Risk and Resiliency Factors Inventory. While the MHP is electing to not make any changes at this time, and instead is waiting for more information from DHCS, the MHP would benefit from researching evidence-based practice outcomes tools for adults.
• While the MHP provides relevant trainings to its staff, evaluation of the effectiveness of the training is lacking.
Foster Care
Changes within the Past Year:
• The new county counsel changed how services were authorized for FC.
• County Counsel revoked the child welfare deputy’s ability to sign consent; however, a new County Counsel will be in this role soon. The MHP will be working to restore the right to consent with the new County Counsel.
Strengths:
• The MHP is in the process of standardizing FC referrals and screenings for all beneficiaries up to age 21 for ICC and IHBS services.
Opportunities for Improvement:
• Changes made to consent process for FC children delay assessment and treatment.
Information Systems
Changes within the Past Year:
• The MHP updated its EHR promotion and added forms to support medical necessity determination and beneficiary brief assessment.
• With support from Kings View, the MHP was able to meet Network Adequacy and CANS-50 reporting requirements.
- 52 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Strengths:
• The MHP has steadily increased its IT budget since 2016-17 to support EHR operations and DHCS requirements such as the implementation of CANS-50, PSC-35, Network Adequacy reporting and CSI timeliness reporting. The MHP dedicated 2.78 percent of its total budget in FY 2016-17 for IT. Subsequent year-over-year increases have brought the IT budget to 8.16 percent of the total budget for FY 2019-20.
Opportunities for Improvement:
• The MHP is aware co-occurring diagnoses are under-reported. Clinical staff received training in January 2019 on how to document co-occurrence. The MHP also corrected where co-occurrence data should be extracted from the EHR.
Structure and Operations
Changes within the Past Year:
• The MHP had discussions with GMC and NVIHC regarding a HIE pilot but both providers expressed no interest in the project.
• The MHP filled six clinical staff vacancies in both adult and children services. They were fully staffed for one year, but in the week prior to the review two staff left.
• This last year, the MHP raised clinician salaries and added a 10 percent differential when interns become licensed.
Strengths:
• The MHP is continuing efforts within the county to co-locate Children’s Mental Health and CWS, as well as other agency partners (i.e., eligibility and probation).
Opportunities for Improvement:
• The MHP does not have supervisory or management positions designated for beneficiaries.
• Wellness center locations do not include the more rural areas of the county.
- 53 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
FY 2019-20 Recommendations
PIP Status
1. Title 42, CFR, §438.330 requires two PIPs; the MHP is urged to meet this requirement going forward.
2. The MHP added interventions to its clinical PIP; however, interventions were not beneficiary focused. For multi-year PIPs, a new intervention needs to be included. (This recommendation is a carry-over from FY 2018-19.)
3. The MHP did consult on occasion with CalEQRO; however, the MHP would have benefited from ongoing consultation with CalEQRO for PIP improvement as well as new PIP development. (This recommendation is a carry-over from FY 2018-19.)
Access to Care
4. Evaluate current transportation resources to determine any gaps. Implement a plan to improve identified gaps.
5. Evaluate current after-hours care options to determine areas for improvement and/or expansion. Implement a plan to remedy shortfalls.
6. Perform or access a needs assessment for the local homeless population. Provide outreach and supportive services as identified through the needs assessment.
Timeliness of Services
7. Track and trend timeliness of assessment follow-up and routine appointments.
8. The MHP must offer a psychiatric appointment within 15 business days. The MHP should comply with the state timeliness metric as per IN 18-011.
9. Track and trend timeliness of urgent appointments in hours, 48 for those who do not need preauthorization and 96 hours for those requiring preauthorization.
Quality of Care
10. Track and trend prescribing practices and medication on an aggregate level.
Beneficiary Outcomes
11. Research evidence-based practice outcomes tools to consider implementing for adult beneficiaries. (This recommendation is a carry-over from FY 2018-19.)
12. Implement an evaluation for each staff training to determine the effectiveness and usefulness of training topics (i.e., pre/post training surveys).
- 54 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Foster Care
13. Collaborate and explore consent options with county counsel to develop a consent process that eliminates the delay in accessing treatment.
Information Systems
14. Conduct periodic chart reviews on dually diagnosed beneficiaries against data reports to assess if the co-occurring under-reporting problem has been rectified.
Structure and Operations
15. Expand management opportunities for peer employees and incorporate management/supervisory position within career ladder for beneficiaries.
16. Investigate the potential benefit and need for wellness centers in more rural areas of the county. Where appropriate develop a plan and timeline for wellness center site expansion.
- 55 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
ATTACHMENTS
Attachment A: On-site Review Agenda Attachment B: On-site Review Participants Attachment C: Approved Claims Source Data Attachment D: List of Commonly Used Acronyms in EQRO Reports Attachment F: PIP Validation Tools
- 56 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Attachment A—On-site Review Agenda
The following sessions were held during the MHP on-site review, either individually or in combination with other sessions.
Table A1—EQRO Review Sessions - Glenn MHP
Opening Session – Changes in the past year; current initiatives; and status of previous year’s recommendations
Use of Data to Support Program Operations
Cultural Competence, Disparities and Performance Measures
Timeliness Performance Measures/Timeliness Self-Assessment
Quality Management, Quality Improvement and System-wide Outcomes
Beneficiary Satisfaction and Other Surveys
Performance Improvement Projects
Clinical Line Staff Group Interview
Consumer and Family Member Focus Group(s)
Peer Employee/Parent Partner Group Interview
Validation of Findings for Pathways to Mental Health Services (Katie A./CCR)
Information Systems Billing and Fiscal Interview
Information Systems Capabilities Assessment (ISCA)
Wellness Center Site Visit
Final Questions and Answers - Exit Interview
- 57 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Attachment B—Review Participants
CalEQRO Reviewers
Cyndi Lancaster – Lead Quality Reviewer Angela Kozak-Embrey – Quality Reviewer Caroline Yip – Information Systems Reviewer Gloria Marrin – Consumer/Family Member Consultant Additional CalEQRO staff members were involved in the review process, assessments, and recommendations. They provided significant contributions to the overall review by participating in both the pre-site and the post-site meetings and in preparing the recommendations within this report.
Sites of MHP Review
MHP Sites
Glenn County Behavioral Health 1187 E. South St. Orland, CA 95963
- 58 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table B1—Participants Representing the MHP
Last Name First Name Position Agency
Alvarez Nancy Staff Services Specialist GCBH
Anderson Shelby TAY Peer Mentor GCBH
Arnold Amber Compliance and QI Coordinator GCBH
Baillergeon Roxann Program Manager GCBH
Bobadilla Dago Staff Services Specialist GCBH
Cartes Priscilla Case Manager II GCBH
Christianson Travis Senior MH Counselor HHSA
Crowell Patti Senior Mental Health Counselor GCBH
Cull Lisa Senior Program Coordinator GCBH
Davis Mandi Senior MH Counselor GCBH
Diamond Lisa Harmony House Coach GCBH
Doyle Kristin Senior Program Coordinator GCBH
Enriquez Brenda Administrative Services Analyst GCBH
Flewelling Tara Deputy Probation Off. Probation
Gomez Maria Program Coordinator GCBH
Hallett Joe Compliance and QI Manager GCBH
Hanni Tami Administrative Analyst/HIPAA Officer GCBH
Jordan Michael HH PSE Coach GCBH
Kahl Julie Case Manager II GCBH
Lindsey Amy Behavioral Health Director GCBH
Noel Patrick Senior Program Coordinator GCBH
Priest David HH Case Manager GCBH
Prose Ellen Program Manager GCBH
Randolph Melanie HH Coach GCBH
Ross Cindy Senior Program Coordinator GCBH
- 59 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table B1—Participants Representing the MHP
Last Name First Name Position Agency
Rust Stephanie Program Manager II GCBH
Salinas Francisco Tay Intern GCBH
Smith Pamela Senior MH Counselor GCBH
Solis Linyu Senior Mental Health Counselor GCBH
Tarango Leo Peer Mentor GCBH
Thalken Steve Program Manger GCBH
Trevineo-
Tzintzun Ulises Case Manager II
GCBH
Watkins Irvin Case Manager II GCBH
Waygoner Thomas Senior MH Counselor GCBH
Wong Christina Senior Program Coordinator GCBH
- 60 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Attachment C—Approved Claims Source Data
Approved Claims Summaries are provided separately to the MHP in a HIPAA-compliant manner. Values are suppressed to protect confidentiality of the individuals summarized in the data sets where beneficiary count is less than or equal to 11 (*). Additionally, suppression may be required to prevent calculation of initially suppressed data, corresponding penetration rate percentages (n/a); and cells containing zero, missing data or dollar amounts (-).
Table C1 shows the penetration rate and ACB for just the CY 2016 ACA Penetration Rate and ACB. Starting with CY 2016 performance measures, CalEQRO has incorporated the ACA Expansion data in the total Medi-Cal enrollees and beneficiaries served.
Table C2 shows the distribution of the MHP beneficiaries served by ACB range for three cost categories: under $20,000; $20,000 to $30,000, and above $30,000.
Entity
Average
Monthly ACA
Enrollees
Beneficiaries
Served
Penetration
Rate
Total
Approved
Claims
ACB
Statewide 3,807,829 152,568 4.01% $832,986,475 $5,460
Small-Rural 31,091 2,287 7.36% $6,904,894 $3,019
MHP 2,728 211 7.73% $528,459 $2,505
Table C1. CY 2018 Medi-Cal Expansion (ACA) Penetration Rate and ACB
Glenn MHP
Entity
Average
Monthly ACA
Enrollees
Beneficiaries
Served
Penetration
Rate
Total
Approved
Claims
ACB
Statewide 3,807,829 152,568 4.01% $832,986,475 $5,460
Small-Rural 31,091 2,287 7.36% $6,904,894 $3,019
MHP 2,728 211 7.73% $528,459 $2,505
Table C1. CY 2018 Medi-Cal Expansion (ACA) Penetration Rate and ACB
Glenn MHP
- 61 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Attachment D—List of Commonly Used Acronyms
Table D1—List of Commonly Used Acronyms
ACA Affordable Care Act
ACL All County Letter
ACT Assertive Community Treatment
ART Aggression Replacement Therapy
CAHPS Consumer Assessment of Healthcare Providers and Systems
CalEQRO California External Quality Review Organization
CARE California Access to Recovery Effort
CBT Cognitive Behavioral Therapy
CDSS California Department of Social Services
CFM Consumer and Family Member
CFR Code of Federal Regulations
CFT Child Family Team
CMS Centers for Medicare and Medicaid Services
CPM Core Practice Model
CPS Child Protective Service
CPS (alt) Consumer Perception Survey (alt)
CSU Crisis Stabilization Unit
CWS Child Welfare Services
CY Calendar Year
DBT Dialectical Behavioral Therapy
DHCS Department of Health Care Services
DPI Department of Program Integrity
DSRIP Delivery System Reform Incentive Payment
EBP Evidence-based Program or Practice
EHR Electronic Health Record
EMR Electronic Medical Record
EPSDT Early and Periodic Screening, Diagnosis, and Treatment
EQR External Quality Review
EQRO External Quality Review Organization
FY Fiscal Year
HCB High-Cost Beneficiary
HIE Health Information Exchange
HIPAA Health Insurance Portability and Accountability Act
ehr Health Information System
HITECH Health Information Technology for Economic and Clinical Health Act
HPSA Health Professional Shortage Area
HRSA Health Resources and Services Administration
IA Inter-Agency Agreement
ICC Intensive Care Coordination
ISCA Information Systems Capabilities Assessment
- 62 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table D1—List of Commonly Used Acronyms
IHBS Intensive Home Based Services
IT Information Technology
LEA Local Education Agency
LGBTQ Lesbian, Gay, Bisexual, Transgender or Questioning
LOS Length of Stay
LSU Litigation Support Unit
M2M Mild-to-Moderate
MDT Multi-Disciplinary Team
MHBG Mental Health Block Grant
MHFA Mental Health First Aid
MHP Mental Health Plan
MHSA Mental Health Services Act
MHSD Mental Health Services Division (of DHCS)
MHSIP Mental Health Statistics Improvement Project
MHST Mental Health Screening Tool
MHWA Mental Health Wellness Act (SB 82)
MOU Memorandum of Understanding
MRT Moral Reconation Therapy
NP Nurse Practitioner
PA Physician Assistant
PATH Projects for Assistance in Transition from Homelessness
PHI Protected Health Information
PIHP Prepaid Inpatient Health Plan
PIP Performance Improvement Project
PM Performance Measure
QI Quality Improvement
QIC Quality Improvement Committee
RN Registered Nurse
ROI Release of Information
SAR Service Authorization Request
SB Senate Bill
SBIRT Screening, Brief Intervention, and Referral to Treatment
SDMC Short-Doyle Medi-Cal
SELPA Special Education Local Planning Area
SED Seriously Emotionally Disturbed
SMHS Specialty Mental Health Services
SMI Seriously Mentally Ill
SOP Safety Organized Practice
SUD Substance Use Disorders
TAY Transition Age Youth
TBS Therapeutic Behavioral Services
TFC Therapeutic Foster Care
TSA Timeliness Self-Assessment
- 63 -
Glenn County MHP CalEQRO Report Fiscal Year 2019-20
Table D1—List of Commonly Used Acronyms
WET Workforce Education and Training
WRAP Wellness Recovery Action Plan
YSS Youth Satisfaction Survey
YSS-F Youth Satisfaction Survey-Family Version
- 64 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
Attachment E—PIP Validation Tools
PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY 2018-19 CLINICAL PIP
GENERAL INFORMATION
MHP: Glenn
PIP Title: Intensive Home-Based Services (IHBS)
Start Date:07/01/2017
Completion Date: ongoing
Projected Study Period: 24 Months
Completed: Yes ☐ No ☒
Date(s) of On-Site Review:
08/06/2019
Name of Reviewer: Cyndi Lancaster
Status of PIP (Only Active and ongoing, and completed PIPs are rated):
Rated
☐ Active and ongoing (baseline established and interventions started)
☐ Completed since the prior External Quality Review (EQR)
Not rated. Comments provided in the PIP Validation Tool for technical
assistance purposes only.
☐ Concept only, not yet active (interventions not started)
☒ Inactive, developed in a prior year
☐ Submission determined not to be a PIP
☐ No Clinical PIP was submitted
Brief Description of PIP (including goal and what PIP is attempting to accomplish)
The goal of this PIP, as stated last year, was to increase the number of IHBS provided to beneficiaries with open CWS cases, increase beneficiary resiliency as evidenced by scoring on the Risk and Resiliency Factors Inventory, and, thereby, promote an environment that supports permanency and/or stable living situation. The MHP revised its question to clarify that it aimed to increase reunification through increased IBHS, and to rate the success of the process via the CANS-50. The MHP, having implemented the
- 65 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
CANS-50 in compliance with the State, named the outcome tool as an intervention. It also added staff training; however, both of its new interventions are not true clinical interventions with a direct impact on beneficiaries. The CANS-50 is a measurement tool, and staff training on codes and modalities is a process to improve quality and integrity of data entry; further, it was not clear why the MHP feels that children need more IHBS. The indicators presented need further explanation as to how they measure performance and are relevant to the study question.
ACTIVITY 1: ASSESS THE STUDY METHODOLOGY
STEP 1: Review the Selected Study Topic(s)
Component/Standard Score Comments
1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
The PIP includes a multifunctional team; however, no beneficiaries were listed as part of the team.
1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
Data showed that IHBS services were lower than expected, while CWS reported higher rate of multiple placement and that mental health services were not sufficiently intensive.
Select the category for each PIP:
Clinical:
☐ Prevention of an acute or chronic condition ☐ High volume
services
☒ Care for an acute or chronic condition ☐ High risk
conditions
Non-clinical:
☐ Process of accessing or delivering care
- 66 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?
Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
The MHP is using the CANS-50 as a measurement tool. Accordingly, that the MHP should add a goal percentage or target number to the study question. They did not amend the question with a goal.
1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?
Demographics:
☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language
☐ Other
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
All open CWS cases.
Totals 2 Met 2 Partially Met 0 Not Met 0 UTD
STEP 2: Review the Study Question(s)
2.1 Was the study question(s) stated clearly in writing?
Does the question have a measurable impact for the defined study population?
Include study question as stated in narrative:
“Will increasing the delivery of IHBS services to children, youth, and caregivers who have an open CWS case improve the family’s resiliency and increase the number of children who are reunified with their family, as indicated by an improved score in the Life Domain Functioning section of the California CANS-50?”
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
The MHP is using the CANS-50 as a measurement tool. The MHP did not add a goal percentage or target number to the study question.
Totals 1 Met 1 Partially Met 0 Not Met 0 UTD
- 67 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
STEP 3: Review the Identified Study Population
3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?
Demographics:
☒ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language
☐ Other
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
The PIP includes all children and youth with an open CWS case who meet Medical Necessity for specialty mental health services and who live in Glenn County. The MHP expects to include approximately 30 individuals each year.
3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?
Methods of identifying participants:
☒ Utilization data ☐ Referral ☐ Self-identification
☐ Other: <Text if checked>
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
Totals 2 Met 0 Partially Met 0 Not Met 0 UTD
- 68 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
STEP 4: Review Selected Study Indicators
4.1 Did the study use objective, clearly defined, measurable indicators?
List indicators:
1. Percent of IHBS beneficiaries who improved their overall Risk and Resiliency Factors (RRF) score
2. Percent of IHBS beneficiaries who received 10 or more hours of IHBS services per year
3. Percent of CWS youth who are open to Glenn County HHS and receiving IHBS services
4. Percent of IHBS beneficiaries with a CWS length of stay of less than 2 years
Added this year:
5. Percent of CWS beneficiaries open to GCBH who received a CANS-50 assessment
6. Percent of IHBS beneficiaries who had an improved score in the Life Domain Functioning section of the CANS-50
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
The indicators presented need further explanation as to how they measure performance and their relevance to the study question.
In Step 4 of the PIP submission tool, there needs to be discussion in the narrative on why the MHP thinks that children need more IHBS.
- 69 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.
☐ Health Status ☒ Functional Status
☐ Member Satisfaction ☐ Provider Satisfaction
Are long-term outcomes clearly stated? ☐ Yes ☒ No
Are long-term outcomes implied? ☒ Yes ☐ No
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
It is the MHP’s goal to ensure children and caregivers receive the services they need, to ensure positive outcomes including improving access to IHBS services, improved resiliency factors, reduced out of home placement, and reduced changes in living situation.
Totals 2 Met 0 Partially Met 0 Not Met 0 UTD
STEP 5: Review Sampling Methods
5.1 Did the sampling technique consider and specify the:
a) True (or estimated) frequency of occurrence of the event?
b) Confidence interval to be used?
c) Margin of error that will be acceptable?
☐ Met
☐ Partially Met
☐ Not Met
☒ Not
Applicable
☐ Unable to
Determine
- 70 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
5.2 Were valid sampling techniques that protected against bias employed?
Specify the type of sampling or census used:
☐ Met
☐ Partially Met
☐ Not Met
☒ Not
Applicable
☐ Unable to
Determine
5.3 Did the sample contain a sufficient number of enrollees?
______N of enrollees in sampling frame
______N of sample
______N of participants (i.e. – return rate)
☐ Met
☐ Partially Met
☐ Not Met
☒ Not
Applicable
☐ Unable to
Determine
Totals 0 Met 0 Partially Met 0 Not Met 3 NA 0 UTD
- 71 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
STEP 6: Review Data Collection Procedures
6.1 Did the study design clearly specify the data to be collected?
When a beneficiary is opened to IHBS services, the Eligibility Assessment Form is completed, as well as the Referral Form. Any services the beneficiary receives is then tracked in the EHR and online data system, Cerner.
Data was collected on the services received by beneficiaries with an open CWS case and the number who were referred and met the criteria for specialty mental health services. For those who were open to mental health, services were collected in the EHR.
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
6.2 Did the study design clearly specify the sources of data?
Sources of data:
☐ Member ☐ Claims ☐ Provider
☒ Other: Eligibility Assessment Form and Referral
Form
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
- 72 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?
Instruments used:
☐ Survey ☐ Medical record abstraction tool
☒ Outcomes tool ☐ Level of Care tools
☒ Other: Assessment and referral forms
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
6.5 Did the study design prospectively specify a data analysis plan?
Did the plan include contingencies for untoward results?
The MHP analyzed data on CWS youth who received services from Glenn County HHS and the number of IHBS services that were received, and if they improved their scores collected on the RRF.
Data was routinely discussed to determine if beneficiaries are receiving the right amount of IHBS services. Mental health staff record the services in the EHR. I.D.E.A. Consulting analyzed the data from the EHR for the performance indicators
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
No contingencies were given for untoward results.
- 73 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
6.6 Were qualified staff and personnel used to collect the data?
Project leader:
Name:
Title:
Role:
Other team members:
Names:
☐ Met
☐ Partially Met
☒ Not Met
☐ Unable to
Determine
Names/titles of personnel used to collect the data were not given.
Totals 2 Met 3 Partially Met 1 Not Met 0 UTD
- 74 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
STEP 7: Assess Improvement Strategies
7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?
Describe Interventions:
1. Hire qualified staff to fill clinical and case management positions to have enough staff to provide these services; utilize additional staff and conduct ongoing recruitment to have enough staff to provide the intensive, wrap around level of services.
2. Implement weekly staff meetings and offer training to increase knowledge and skills in delivering mental health services, quality of service contacts, IHBS services, and improve the frequency of face-to-face services; Train the entire children’s unit on the Core Practice Model, and provide updates in the new edition of the model coming out
3. Obtain a clear vision and communication from the Behavioral Health Director that all treatment staff will attend each Family Team Meeting.
4. Provide monthly data and feedback on the number of IHBS services delivered to CWS families at SIC/QIC and bi- weekly feedback of the data during staff meetings to identify barriers and celebrate success.
5. Provide staff training on collecting and scoring the RRF and monitor RRF completion monthly.
☐ Met
☐ Partially Met
☒ Not Met
☐ Unable to
Determine
There needs to be at least one intervention that touches the beneficiary and speaks to the clinical outcome for the beneficiary.
The MHP added:
• Implemented and certified staff to utilize the CANS-50 to replace the RRF.
• Trained Mental Health staff on appropriate code usage and modalities of service, including IHBS.
However, these are not interventions. The CANS-50
is a tool for measuring outcomes, and staff training is
for quality control of data entry.
- 75 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
Totals 0 Met 1 Partially Met 0Not Met 0 UTD
STEP 8: Review Data Analysis and Interpretation of Study Results
8.1 Was an analysis of the findings performed according to the data analysis plan?
☐ Met
☒ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
MHP staff record results in the EHR and I.D.E.A. Consulting analyzes the data.
8.2 Were the PIP results and findings presented accurately and clearly?
Are tables and figures labeled?
☒ Yes ☐ No
Are they labeled clearly and accurately?
☒ Yes ☐ No
☒ Met
☐ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
- 76 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?
Indicate the time periods of measurements: ___________________
Indicate the statistical analysis used: _________________________
Indicate the statistical significance level or confidence level if available/known: ____percent ______Unable to determine
☐ Met
☐ Partially Met
☒ Not Met
☐ Not
Applicable
☐ Unable to
Determine
No statistical analysis has been conducted on this data yet.
8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?
Limitations described:
Conclusions regarding the success of the interpretation:
Recommendations for follow-up:
☐ Met
☐ Partially Met
☒ Not Met
☐ Not
Applicable
☐ Unable to
Determine
No statistical analysis has been conducted on this data yet.
Totals 0 Met 1 Partially Met 3 Not Met 0 NA 0 UTD
- 77 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
STEP 9: Assess Whether Improvement is “Real” Improvement
9.1 Was the same methodology as the baseline measurement used when measurement was repeated?
Ask: At what interval(s) was the data measurement
repeated?
Were the same sources of data used?
Did they use the same method of data collection?
Were the same participants examined?
Did they utilize the same measurement tools?
☐ Met
☐ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
9.2 Was there any documented, quantitative improvement in processes or outcomes of care?
Was there: ☒ Improvement ☐ Deterioration
Statistical significance: ☐ Yes ☒ No
Clinical significance: ☒ Yes ☐ No
☒ Met
☐ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
To summarize, the baseline for this indicator was May 2017- April 2018, and there were 210 requests identified as five-day indicators of which 89 met the timeliness standard (42.4 percent). This improved to 146 out of 256 (57 percent) upon close of the PIP, representing an improvement of 9.4 percent.
9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?
Degree to which the intervention was the reason for change:
☐ No relevance ☐ Small ☒ Fair ☐ High
☐ Met
☒ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
While there was improvement, it is unclear which intervention over the last two years had the greatest impact. Remeasurement after each intervention would provide more information.
- 78 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
9.4 Is there any statistical evidence that any observed performance improvement is true improvement?
☐ Weak ☐ Moderate ☐ Strong
☐ Met
☐ Partially Met
☒ Not Met
☐ Not
Applicable
☐ Unable to
Determine
9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?
☐ Met
☐ Partially Met
☒ Not Met
☐ Not
Applicable
☒ Unable to
Determine
Totals 0 Met 0 Partially Met 0 Not Met 0 NA 5 UTD
ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)
Component/Standard Score Comments
Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?
☐ Yes
☒ No
No statistical analysis has been conducted on this data yet.
- 79 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS
Conclusions:
No statistical analysis has been conducted on this data yet.
Recommendations:
Add interventions that are more clearly clinical outcomes of beneficiaries.
State a measurable goal in percent or numbers in the study question.
Analyze the results at least quarterly.
Explain why more children need services and how many more, as per Step 4.2 of the PIP validation.
Indicate the clinical significance to which the indicator measure or relate.
Check one: ☐ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results
☐ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible
☒ Confidence in PIP results cannot be determined at this time
- 80 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY 2018-19 NON-CLINICAL PIP
GENERAL INFORMATION
MHP: Glenn
PIP Title: Timely Access
Start Date: 09/26/16
Completion Date: 09/01/2019
Projected Study Period: 24
Completed: Yes ☒ No ☐
Date(s) of On-Site Review: 08/06/2019
Name of Reviewer: Cyndi Lancaster
Status of PIP (Only Active and ongoing, and completed PIPs are rated):
Rated
☐ Active and ongoing (baseline established and interventions started)
☒ Completed since the prior External Quality Review (EQR)
Not rated. Comments provided in the PIP Validation Tool for technical
assistance purposes only.
☐ Concept only, not yet active (interventions not started)
☐ Inactive, developed in a prior year
☐ Submission determined not to be a PIP
☐ No Non-clinical PIP was submitted
Brief Description of PIP (including goal and what PIP is attempting to accomplish):
The goal of this non-clinical PIP was to implement a triage assessment process to help high-need beneficiaries access services as quickly as possible, as well as ensure all beneficiaries have access to services in a timely manner. Annual External Quality Reviews have shown that in recent years Glenn County did not score well on timeliness to mental health and/or psychiatric services. The MHP identified this as an important area for improvement to ensure proper and consistent access to care for their beneficiaries.
- 81 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
In the initial year of the PIP, the MHP used 7, 14, and 21 days for prioritizing referrals for urgent, moderate, and routine beneficiaries, and have since updated the standards to include the Final Rule timeliness regulations of 15 business to psychiatric and 10 business days to mental health services.
The MHP initially developed interventions to directly impact the access processes and improve timeliness to services for beneficiaries. For this year, the MHP added the expansion of available appointments through hiring a dedicated staff person for assessments and adding assessment slots to supervisor schedules. The MHP also implemented a scheduling process which allowed the front office to schedule follow-ups efficiently.
For 5-day urgent requests for service, at baseline, 42.4 percent met the standard. At the time of remeasurement, 57 percent met the standard, representing an improvement of 9.4 percentage points.
ACTIVITY 1: ASSESS THE STUDY METHODOLOGY
STEP 1: Review the Selected Study Topic(s)
Component/Standard Score Comments
1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
The MHP assembled a multifunctional team; however, no beneficiary representation was noted.
- 82 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
Annual External Quality Reviews have shown that in recent years Glenn County did not score well on timeliness to outpatient mental health and/or psychiatric services. They analyzed data for timeliness to mental health services and psychiatric services for January – May 2016 and verified that there was an issue with access for beneficiaries requesting initial services.
Although the initial PIP and problem were identified through data, further evaluation as to the related causes and/or barriers was not completed.
Select the category for each PIP:
Non-clinical:
☐ Prevention of an acute or chronic condition ☐ High volume services
☐ Care for an acute or chronic condition ☐ High risk conditions
☒ Process of accessing or delivering care
1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?
Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
The MHP developed interventions to directly impact the access process.
- 83 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?
Demographics:
☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language
☒ Other All consumers of all ages.
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
The PIP includes all individuals, of all ages, who request mental health, including psychiatric services, from Glenn County Behavioral Health.
Totals 2 Met 2 Partially Met 0 Not Met 0 UTD
STEP 2: Review the Study Question(s)
2.1 Was the study question(s) stated clearly in writing?
Does the question have a measurable impact for the defined study population?
Include study question as stated in narrative:
“Will implementing a triage assessment process help high-need beneficiaries with an urgent need access services within five days; ensure beneficiaries with a routine need for services access services within 10 days; and beneficiaries requesting psychiatric services receive services within 15 days for at least 75 percent of all beneficiaries?”
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
Although the question is measurable, its multipart structure may be too complicated to answer definitively.
Simple, straight forward questions which identify the problem, intervention and outcome measure might prove more answerable.
“Will (intervention X) improve (stated problem Y) as measured by (outcome measure Z)?”
Totals 0 Met 1 Partially Met 0 Not Met 0 UTD
STEP 3: Review the Identified Study Population
3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?
Demographics:
☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language
☒ Other All beneficiaries of all ages
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
All individuals, of all ages, who request mental health including psychiatric services from Glenn County Behavioral Health.
- 84 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?
Methods of identifying participants:
☒ Utilization data ☐ Referral ☐ Self-identification
☒ Other: Internal Access Log
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
All individuals, of all ages, who request mental health including psychiatric services from Glenn County Behavioral Health.
Totals 2 Met 0 Partially Met 0 Not Met 0 UTD
STEP 4: Review Selected Study Indicators
4.1 Did the study use objective, clearly defined, measurable indicators?
List indicators:
1. Percent of the urgent mental health service requests with an assessment scheduled within 5 calendar days
2. Percent of the routine mental health services requests with an assessment scheduled within 10 calendar days
3. Percent of the psychiatric services requests with an assessment scheduled within 15 calendar days
4. Percent of beneficiaries who received an assessment who received a second mental health service within 14 calendar days after their Assessment
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
- 85 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.
☒ Health Status ☐ Functional Status
☐ Member Satisfaction ☐ Provider Satisfaction
Are long-term outcomes clearly stated? ☒ Yes ☐ No
Are long-term outcomes implied? ☒ Yes ☐ No
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
Totals 2 Met 0 Partially Met 0 Not Met 0 UTD
STEP 5: Review Sampling Methods
5.1 Did the sampling technique consider and specify the:
a) True (or estimated) frequency of occurrence of the event?
b) Confidence interval to be used?
c) Margin of error that will be acceptable?
☐ Met
☐ Partially Met
☐ Not Met
☒ Not
Applicable
☐ Unable to
Determine
- 86 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
5.2 Were valid sampling techniques that protected against bias employed?
Specify the type of sampling or census used:
☐ Met
☐ Partially Met
☐ Not Met
☒ Not
Applicable
☐ Unable to
Determine
5.3 Did the sample contain a sufficient number of enrollees?
______N of enrollees in sampling frame
______N of sample
______N of participants (i.e. – return rate)
☐ Met
☐ Partially Met
☐ Not Met
☒ Not
Applicable
☐ Unable to
Determine
Totals 0 Met 0 Partially Met 0 Not Met 3 NA 0 UTD
STEP 6: Review Data Collection Procedures
6.1 Did the study design clearly specify the data to be collected?
Utilized internal Access Log of service requests for mental health services and psychiatry services.
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
6.2 Did the study design clearly specify the sources of data?
Sources of data:
☒ Met
☐ Partially Met
☐ Not Met
- 87 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
☐ Member ☐ Claims ☐ Provider
☒ Other: Other: Internal Access Log and EHR
☐ Unable to
Determine
6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
All requests for services are in Internal Access Log and second appointments are recorded in the EHR.
6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?
Instruments used:
☐ Survey ☐ Medical record abstraction tool
☐ Outcomes tool ☐ Level of Care tools
☒ Other: Internal Access Log
☒ Met
☐ Partially Met
☐ Not Met
☐ Unable to
Determine
Front office staff logged the service requests on the Access Log. Clinical staff performed the assessments and administrative staff updated the appointments on the Access Log.
6.5 Did the study design prospectively specify a data analysis plan?
Did the plan include contingencies for untoward results?
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
Data was routinely discussed to determine the timeliness of services and ways to expedite the process. I.D.E.A. Consulting analyzed the data from the Access Log.
Who collected the data? How often was it collected? Who analyzed the data? How often was it analyzed? What tools were used to analyze it? Why was that person qualified to analyze the data? What did the MHP expect the data to show?
- 88 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
6.6 Were qualified staff and personnel used to collect the data?
Project leader:
Name:
Title:
Role:
Other team members:
Names:
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
Specific names and titles were not given in the submission tool. I.D.E.A. consulting and front office staff are mentioned.
Totals 4 Met 2 Partially Met 0 Not Met 0 UTD
STEP 7: Assess Improvement Strategies
7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?
Describe Interventions:
1. Create different levels of need on the Referral Form. Schedule and assess urgent acuity beneficiaries within five calendar days and routine beneficiaries within 10 calendar days.
2. Implement a wait list so when a beneficiary cancels an assessment appointment, the spot can be filled.
3. If a beneficiary does not show up within 15 minutes of their appointment time, staff call beneficiaries on the wait list to fill the spot. Staff also call and remind beneficiaries of their appointment the day prior to minimize “no-show” appointments.
☐ Met
☒ Partially Met
☐ Not Met
☐ Unable to
Determine
The MHP added the expansion of available appointments through hiring and adding assessment available to supervisor schedules. The MHP also implemented a scheduling process which allowed the front office to schedule follow-ups efficiently.
Interventions launched in 2017 are not considered active. Also, a review of data collection methods and improved tracking is not an intervention but rather an approach to data integrity. A barrier analysis on the causes of the problem would have allowed a more fine-tuned approach to intervention selection.
- 89 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
4. Expedite the assessment process for high-need medication beneficiaries with a shortened Treatment Plan completed by one of the referring staff or Managers.
Newly added:
5. Add a bi-weekly psychiatric evaluation time slot to be scheduled for persons with an urgent medication need, and fill with standard evaluations if not filled one week prior (July 2017)
6. Hire a PSE clinician to focus on providing additional assessment appointments. (August 2018) (active intervention).
7. Review Data collection methods at QIC and implement tracking of business days rather than calendar days, when to start the date of request, and looking at first offered appointment in addition to first accepted. (October 2018)
8. Create a scheduling process for all clinicians so that front office can schedule all follow-up appointments after assessment and help to get the beneficiary into treatment more efficiently and consistently. (November 2018)
9. Have clinical supervisors who have an expectation to see beneficiaries also provide assessment appointments to increase the number of appointments overall. (May 2019) (active intervention)
- 90 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
Totals 0 Met 1 Partially Met 0 Not Met 0 UTD
STEP 8: Review Data Analysis and Interpretation of Study Results
8.1 Was an analysis of the findings performed according to the data analysis plan?
☒ Met
☐ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
The MHP analyzed data to show the number and percent of beneficiaries who received an assessment within the 5-day goal for urgent needs, 10-day goal for routine needs of mental health services, and 15-day goal for psychiatric services. Upon implementation of the interventions, the MHP analyzed the data again. They added a new indicator to measure the time from receiving an assessment to receiving a second mental health service. May 2017 – April 2018 was the baseline with remeasurement July 2018 through May 2019.
8.2 Were the PIP results and findings presented accurately and clearly?
Are tables and figures labeled?
☒ Yes ☐ No
Are they labeled clearly and accurately?
☐ Yes ☒ No
☐ Met
☒ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
They did not use sampling as all new beneficiaries are included in this PIP. The MHP did not use statistical significance for this PIP. Once they fully implemented the various levels of need for requests, they analyzed the data separately for each level. The PIP committee routinely reviewed the analyses; however, the remeasurement findings contained calculation errors. The MHP provided a corrected table.
- 91 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?
Indicate the time periods of measurements: ___________________
Indicate the statistical analysis used: ___________n/a______________
Indicate the statistical significance level or confidence level if available/known: ____percent ______Unable to determine
☒ Met
☐ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?
Limitations described:
Conclusions regarding the success of the interpretation:
Recommendations for follow-up:
☒ Met
☐ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
Totals 3 Met 1 Partially Met 0 Not Met 0 NA 0 UTD
- 92 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
STEP 9: Assess Whether Improvement is “Real” Improvement
9.1 Was the same methodology as the baseline measurement used when measurement was repeated?
Ask:
At what interval(s) was the data measurement repeated?
Were the same sources of data used?
Did they use the same method of data collection?
Were the same participants examined?
Did they utilize the same measurement tools?
☒ Met
☐ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
The MHP analyzed the baseline data for services delivered in FY 2016-17. The MHP then implemented the interventions, and analyzed the data again for FY 2017-18, and reviewed in our EQRO that year. This PIP was continued for FY 2018-19, and data was analyzed quarterly at the Quality Improvement Committee meetings, and then overall upon close of the Fiscal Year. Since beneficiaries need a second CANS-50 score to analyze improvement for this indicator, we added measurement results at the end of the study period.
Baseline measurements were:
January – May 2016; May 2017 – April 2018
Remeasurement occurred:
July 2018 – May 2019
9.2 Was there any documented, quantitative improvement in processes or outcomes of care?
Was there: ☒ Improvement ☐ Deterioration
Statistical significance: ☐ Yes ☒ No
Clinical significance: ☒ Yes ☐ No
☒ Met
☐ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
The baseline was for May 2017-April 2018, and there were 210 requests for services identified as 5-day urgent indicators. 89 requests met the timeliness standard (42.4 percent). This improved to 146 out of 256 (57 percent) upon close of the PIP, representing an improvement of 9.4 percent.
- 93 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?
Degree to which the intervention was the reason for change:
☐ No relevance ☐ Small ☒ Fair ☐ High
☐ Met
☒ Partially Met
☐ Not Met
☐ Not
Applicable
☐ Unable to
Determine
While there was improvement, it is unclear which intervention over the last two years had the greatest impact. Remeasurement after each intervention would provide more information.
9.4 Is there any statistical evidence that any observed performance improvement is true improvement?
☐ Weak ☒ Moderate ☐ Strong
☐ Met
☐ Partially Met
☒ Not Met
☐ Not
Applicable
☐ Unable to
Determine
9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?
☐ Met
☐ Partially Met
☒ Not Met
☐ Not
Applicable
☐ Unable to
Determine
Totals 2 Met 1 Partially Met 2 Not Met 0 NA 0 UTD
- 94 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)
Component/Standard Score Comments
Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?
☐ Yes
☒ No
ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS
Conclusions:
PIP Validation #s
Met 15
Partially Met 8
Not Met 2
UTD 0
# Not applicable 3
Score 76.00%
#applicable 25
Suggestions to improve the PIP: The PIP could have benefited from beneficiary inclusion on the PIP team. Also, re-measurement should occur quarterly at minimum. Although the initial PIP and problem were identified through data, further evaluation as to the related causes and/or barriers was not completed. Although the study question was measurable, its multipart structure was complicated. Simple, straight-forward questions which identify the problem, intervention and outcome measure might prove more answerable.
- 95 -
Glenn County MHP CalEQRO Report Fiscal Year 2018-19
ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS
Recommendations:
Conclude the PIP and begin a new Non-clinical PIP.
Relevant details of these issues and recommendations are included within the comments found in the PIP validation tool.
The TA provided to the MHP by CalEQRO consisted of recommending that the MHP measure results quarterly. Discussion onsite involved the MHP’s plan to end the PIP and begin a new Non-clinical PIP on the topic of improving beneficiary access to Harmony House, the MHP’s wellness center. The MHP was also encouraged to consult with EQRO early and often during the development of new PIPs. Recommendations for new PIPs include using a simple, straight forward question which identifies the problem, intervention and outcome measure might prove more answerable; moreover, the new PIP should include more in-depth data evaluation, a barrier analysis, to ensure that interventions are related to problem causes.
Check one: ☐ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results
☒ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible
☐ Confidence in PIP results cannot be determined at this time