measuring outcomes: a methodological challenge m9205 october 24, 2000

Measuring Outcomes: a methodological challenge

M9205 October 24, 2000

Columbia University School of Nursing M6920, Fall, 2000

Dummy variable

variable in a regression equation with a finite number of values so that different categories of a nominal variable can be identified.

also called an indicator variable Examples

1=treatment0=no treatment

1=female2=male

1=female2=male

1=residence in West 0=residence is Central-1=residence is East


Identification of the time frame

Immediately following the intervention

3-6 months post intervention 5 years Lifetime


Selection of objective measures

Use of services Return to work Life expectancy


Selection of subjective measures

Provider assessment Quality of life or other patient

reports Observer reports


Development of data

Access to multiple sources Developing a cohort Retrospective cohort identification


HEDIS as an example of the issues

A set of 75 performance measures for comparison of HMOs

Designed for purchasers of care plans


Measures should be

relevant scientifically sound feasible


Sample HEDIS Measures

Effectiveness of care• beta blocker

treatments after heart attack

• childhood immunization status

Access to care• availability of

primary care providers

• annual dental visit Cost of Care

• rate trends


Measures, cont.

Informed care choices• language/

translation services

Descriptive information• provider

compensation• pediatric mental

health network• family planing


A nursing perspective on the challenges*

increasing nursing involvement in quality initiatives

identifying strategies that are effective

changing the clinical framework demonstrating nursing’s

contributions *Jones KR et al 1997 Policy Issues Associated with analyzing outcomes of care Image 29:3 (261-7)


ANA Report Card Goals

Test relationships between nurse staffing and specific outcome indicators

Assess the feasibility of capturing information necessary to develop measures with acceptable reliability and validity


ANA Methodology

quantify nurse staffing at a sample of hospitals

quantify patient incidents and length of stay at same hospitals

measure relationship between these two sets of variables


ANA Data challenges

For two states, had to allocate total numbers of hours worked to cost centers

California proportion of hours per cost center used as base of NY and Mass allocations


ANA Data challenges, cont.

Nursing intensity weights used as developed in NY

dimensions include • assessment, • planing, • physical needs, • medical needs, • socioemotional support, • teaching


ANA Quality indicators for acute care settings

Outcomes• Patient satisfaction

with nursing care, pain management, patient education

• Patient injury rate• Nosocomial infection

rate

Process• maintenance of skin

integrity• nursing staff

satisfaction Structure

• mix of RN, LPN, unlicensed staff

• total care hours per patient


Blegan, Goode, Reed Nurse Staffing

Objective: describe at the level of the nursing care unit the relationships among total hours of care, RN skill mix and adverse outcomes

Methods: corellational and multivariate analyses controlling for patient acuity


Measures selected included rates of

medication errors (from incident reports)

patient falls (from incident reports) skin breakdown patient and family complaints infections deaths


Nurse staffing variables

hours of care per patient day from all nursing personnel

hours of care provided by RNs


Key analysis

“total hours of patient care. . .was associated with higher rates of decubiti, complaints and death. . . .Given the high correlation between acuity and total nursing care hours, the interpretation of these coefficients must be done with care” (p. 49)


Key analysis

“Most of the previous research in this area was multiinstitutional and generalizeable but suffered from an accompanying lack of detail. The results of this project are more detailed and specific but less generalizable.” (p. 49)


Chen Top 100 Hospitals

Issue: do Medicare patients with acute MI admitted to a “top 100” hospital receive better care or have better outcomes than patients treated in other hospitals?


Quality of care measures

use of aspirin during hospitalization

reperfusion therapy in-hospital use of beta blockers


Findings

top 100 hospitals more likely to have higher AMI volume, be private for-profit, have on-site facilities for procedures

major differences were in length of stay and cost per AMI admission ($1,014 to $1,855 lower per admission; highly correlated with LOS)


Comment

goal of ‘top 100’ was to identify superior financial management, operations and clinical practices.

it may, instead, identify lower LOS and cost for same outcomes


Frank et al, Treating Depression

Why care? 1/2 of private insurance for MH is on depression

Defining value and benefits Critical of either traditional cost

effectiveness analysis or supply-demand analysis


Frank: Definitions

system effectiveness: sum of all effects produced by health care in a system, including those persons treated by various methods and those not treated at all

system cost: sum of all direct treatment costs

Systems cost effectiveness is the ration of system effects to system costs


Data identification

identifying persons and episodes: claims data

treatment effects: clinical research on efficacy and effectiveness


Findings

cost for highest expected outcomes was $1,059/case

20% of all spending was in the range expected to equal no treatment.


Kelleher

Concurrent, prospective study Consecutive admissions Resident assessment of variables Outcome of interest: length of stay Scales created for the study


Kelleher

Regression analysis to identify explanatory power

Severity has been common indicator and does help

adding difficulty increases explanatory power

peak values and fluctuation strongly predictive


Borzekowski

Outcome of interest: response to anti-violence PSA’s

Interest, understanding, credibility, effect

Sample size? location? Low but suggestive significance


Sochalski et al

Research review Proposal of an agenda Focus on nursing staff patterns

measuring outcomes: a methodological challenge m9205 october 24, 2000

Documents

nursing personnelhours

nursing care unit

total hours of care

outcomes of care image

patient day

patient incidents

nursing involvement

patient acuity measures