1

Using Medicare Files as Sampling Using Medicare Files as Sampling framesframes

for a Survey of Physiciansfor a Survey of Physicians

Using Medicare Files as Sampling Using Medicare Files as Sampling framesframes

for a Survey of Physiciansfor a Survey of Physicians

Vasudha NarayananPresented at

Third International Conference on Establishment SurveysJune 20, 2007

2

AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements

Co-Authors

Pamela Giambo & Sherm Edwards from Westat

Project Officers

Gladys Valentin & David Clark from Centers for Medicare & Medicaid ServicesThis material was prepared by Westat Inc., under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy.

3

Issues with Establishment Survey Issues with Establishment Survey Sampling FramesSampling Frames

Issues with Establishment Survey Issues with Establishment Survey Sampling FramesSampling Frames

• Cox (1997); Konschnik (1988)

• Complex ownership and management structures

• Frames clogged with inactive and duplicate records

• Coverage error

• Unreliable contact information

4

About MCPSSAbout MCPSSAbout MCPSSAbout MCPSS

• Measures satisfaction of physicians and other medical providers with the services offered by Medicare Contractors

• Purpose of MCPSS

• Satisfy Medicare Modernization Act (2003) requirements

• Establish a uniform measure of provider satisfaction with Contractors

• Provide feedback to Contractors for QI

• Annual Survey

5

About MCPSSAbout MCPSSAbout MCPSSAbout MCPSS

• Samples customers of 51 Medicare Contractors

• 19 “carriers” handle most physician claims

• Within Contractor, sample stratified by provider type and state

• Screening for a “knowledgeable” respondent

• Survey can be completed over the web, telephone or paper

• Response rate: 65%

6

Who is the Customer/Analysis UnitWho is the Customer/Analysis UnitWho is the Customer/Analysis UnitWho is the Customer/Analysis Unit

• Individual(s) who interact with Contractor

• Typically not the physician

• Could have multiple customers within a practice

• Could represent a single physician, a practice, multiple solo practices, or multiple groups

• Customer may interact with more than one contractor

7

Sample Design UtopiaSample Design UtopiaSample Design UtopiaSample Design Utopia

Analysis UnitAnalysis UnitAnalysis UnitAnalysis Unit

Responding UnitResponding UnitResponding UnitResponding Unit

Sampling UnitSampling UnitSampling UnitSampling Unit

8

Variables Required for MCPSS Variables Required for MCPSS SamplingSampling

Variables Required for MCPSS Variables Required for MCPSS SamplingSampling

• Contractor ID

• Tax ID

• Number of claims in reference year

• Provider type

• State

• Contact information

9

Files Available for MCPSS Files Available for MCPSS Physician FramePhysician Frame

Files Available for MCPSS Files Available for MCPSS Physician FramePhysician Frame

• CMS files• Claims History

• Unique Physician Identification Number (UPIN)

• Provider Enrollment, Chain, and Ownership System (PECOS)

• Non-CMS frames• AMA

• Commercial Vendors

10

Sources of Contact InformationSources of Contact InformationSources of Contact InformationSources of Contact Information

• 2006 sample [2%]

• PECOS [41%]• [Did not use UPIN]

• Commercial vendor [39%]

• Contractors [as a source of last resort] [16%]

• Other files [1%][Information missing for 1% of the sample]

11

Paradise LostParadise LostParadise LostParadise Lost• A disconnect between the sampling unit and

responding unit

• Sampling unit: a Tax ID with at least one claim in the last 12 months

• Responding Unit (s): a physician representative who has experience with a Medicare Contractor

• As mentioned earlier• One respondent for multiple sampled units• Multiple respondents in one sampled unit

12

Screening Sampled ProvidersScreening Sampled ProvidersScreening Sampled ProvidersScreening Sampled Providers

• To locate the most knowledgeable respondent(s)• Hence determine mapping between sampled

unit and responding unit(s)

• To determine if the responding unit is a third-party billing service

13

2007 Sample Performance2007 Sample Performance2007 Sample Performance2007 Sample PerformancePhysician Sample Rest of Sample

Sample Size 7,895 28,464

1. Completes 51.7% 48.0%

2. Partial completes 2.6% 2.1%

3. Ineligible 6.1% 3.5%

4. Ineligible-Chains 1.4% 23.3%

5. Unknown Eligibility 12.8% 6.6%

6. Refusals 4.4% 2.1%

7. Other Non-response 21.1% 2.1%

RR 56.5% 67.3%

Eligibility rate 91.4% 71.3%

Response rate Calculation (AAPOR-RR3)

1 / ( (1 + 2 + 6 + 7) + (5*e) )

Where e = (1 + 2 + 6 + 7) / (1 + 2 + 6 + 7 +3 + 4

14

Paradise Regained?Paradise Regained?Paradise Regained?Paradise Regained?

• Utopia is inherently unrealistic

• The frame allows for a pretty good mapping of sampling and analysis units

• Nothing can be done about the ineligibles

• Problem of unknown eligibility• Poor contact information

15

Contacting the AuthorsContacting the AuthorsContacting the AuthorsContacting the Authors

Vasudha Narayanan (vasudhanarayanan@westat.com)

Pamela Giambo (pamelagiambo@westat.com)

Sherm Edwards (shermedwards@westat.com)

16

Thank youThank youThank youThank you