Secondary Data Analysis:

Opportunities and Pitfalls

Who am I and why am I here?

Laurel A. Copeland, BS MPH PhD – VA Health Services Research &

Development Investigator; UTHSCSA Department of Psychiatry Assistant Professor

copelandl@uthscsa.edu http://czresearch.com/dropbox

What Can Secondary Data Analysis Do for You?

• Provide preliminary data for grant proposals• Facilitate publication while you apply for

grants• Explore new areas

My CV Shope JT, Copeland LA, Maharg R, Dielman TE, Butchart AT (1993). Health Education Quarterly 20: 373-390. Shope JT, Copeland LA, Dielman TE (1994). Alcoholism: Clin Exper Res 18: 726-733. Shope JT, Copeland LA, Marcoux BC, Kamp ME (1996). J Drug Educ, 26: 323-337. Shope JT, Copeland LA, Maharg R, Dielman TE (1996). Alcoholism: Clin Exper Res 20: 791-798. Copeland LA, Shope JT, Waller PF (1996). J School Health 66: 254-260. Barry KL, Fleming MF, Manwell L, Copeland LA (1997). J Fam Prac 45: 151-158. Zimmerman MA, Copeland LA, Shope JT, Dielman, TE (1997). J Youth Adolescence 26: 117-141. Barry KL, Fleming MF, Manwell L, Copeland LA, Appel, Scott (1998). Fam Med 30:366-371. Blow FC, Barry KL, BootsMiller BJ, Copeland LA, McCormick R, Visnic S (1998). J Psychiatric Research 32: 311-319. Valenstein M, Barry KL, Blow FC, Copeland LA, Ullman E (1998). Psychiatric Services 49: 1043-1048. Shope JT, Copeland LA, Kamp ME, Lang SW (1998). J Drug Educ 28: 185-197. Kales HC, Blow FC, Copeland LA, Bingham RC, Kammerer EE, Mellow AF (1999). Am J Psychiatry 156: 550-556. Blow FC, Barry KL, Copeland LA, McCormick R, Lehmann L, Ullman E (1999). Psychiatric Services 50: 390-394. Maio RF, Shope JT, Blow FC, Copeland LA, Gregor MA, Brockmann LM, Weber JE, Metrou ME (2000). Annals Emerg Med 35:252-7. Kales HC, Blow FC, Bingham RC, Copeland LA, Mellow AM (2000). Psychiatric Services 51: 795-800. Roberts JS, Blow FC, Copeland LA, Barry KL, Van Stone W (2000). J Ger Psychiatry Neurology 13: 78-86. Blow FC, Walton MA, Barry KL, Coyne JC, Mudd SA, Copeland LA (2000). JAGS 48: 769-774. Kales HC, Blow FC, Bingham RC, Roberts JS, Copeland LA, Mellow AM (2000). Am J Geriatric Psychiatry 8:301-309. Blow FC, Ullman E, Barry KL, Bingham CR, Copeland LA, McCormick R, Van Stone W (2000). Am J Orthopsychiatry 70: 389-400. Valenstein M, Copeland LA, Owen R, Blow FC, Visnic S (2001). J Clin Psychiatry 62: 545-551. Valenstein M, Copeland LA, Owen R, Blow FC, Visnic S (2001). Psychiatric Services 52: 1242-4. Duffy SA, Terrell, Jeff, Valenstein M, Ronis, David, Copeland LA, Connors, Mary (2002). Gen Hosp Psychiatry 24: 140-7. Valenstein M, Copeland LA, Blow FC, McCarthy JF, Zeber JE, Gillon L, Bingham CR, Stavenger T (2002). Med Care 40: 630-9. Hegedus, Andrea M, Copeland LA, Barry KL, Blow FC. (2002). American Journal of Orthopsychiatry 72: 331-340. Copeland LA, Blow FC, Barry KL (2003). Health Education & Behavior 30:305-21. Copeland LA, Zeber JE, Valenstein M, Blow FC (2003). Am J Psychiatry 160:1817-1822. Valenstein M, Blow FC, Copeland LA, McCarthy JF, Zeber JE, Gillon L, Bingham CR, Stavenger T (2004). Schizophr Bull 30: 255-64. Janz NK, Wren PA, Copeland LA, Lowery JC, Goldfarb SL, Wilkins EG. (2004). J Clin Oncology 22:3091-3098. Lambert MT, Terrell JE, Copeland LA, Ronis DL, Duffy SA. (2005). Nicotine & Tobacco Research 7:233-241. Copeland LA, Fletcher CE, Patterson J(2005). Military Medicine 170:602-606. Copeland LA, Elshaikh MA, Jackson J, Penner LA, Underwood III W (2005). Cancer 104:1372-1380. O'Neill JL, Flanagan PS, Zaleon CR, Copeland LA (2005). Pharmacotherapy 25:1560-1565. Pugh MJV, Copeland LA, Zeber JE, Cramer JA, Amuan MT, Cavazos JE, Kazis LE (2005). Epilepsia 46:1820-7. Copeland LA, Zeber JE, Rosenheck RA, Miller AL (2006). Med Care 44:110-116. Lambert MT, Copeland LA, Sampson N, Duffy SA (2006). Progress Neuropsychopharm Biol Psychiatry 30:919-23. Hopp FP, Woodbridge P, Subramanian U, Copeland LA, Smith D, Lowery JC (2006). Telemedicine and eHealth 12:297-307. Silveira MJ, Copeland LA, Feudtner C (2006). Am J Public Health 96:1243-1248. Zeber JE, Copeland LA, Grazier KL (2006). Military Medicine 171:619-26. Wilkins EG, Lowery JC, Copeland LA, Goldfarb SL, Wren PA, Janz NA (2006). Medical Decision Making 26:589-598. Copeland LA, Mortensen EM, Zeber JE, Pugh MJ, Restrepo MI, Dalack GW (2007). Progress Neuro-Psychopharm Biol Psychiatry 31:720-6 Alexander JA, Copeland LA, Metzger ME (2007). Clinical Toxicology 45(5):440-50. Kilbourne AM, Copeland LA, Zeber JE, Bauer MS, Lasky EC, Good CB (2007). Psychopharmacology Bulletin 40:104-15 Fletcher CE, Baker SJ, Copeland LA, Reeves PJ, Lowery JC (2007). Journal of Nursing Scholarship 39:358-62 Duffy SA, Copeland LA, Hopp FP, Zalenski RJ (2007). Journal of Palliative Medicine 10:1137-45 Mortensen EM, Restrepo MI, Copeland LA, Pugh JA, Anzueto AR, Cornell JE, Pugh MJ (2007). Pharmacotherapy 27: 1619-26 Kilbourne AM, Post EP, Bauer MS, Zeber JE, Copeland LA, Good CB, Pincus HA (2007). J Affective Disorders 102:145-51. Zeber JE, Copeland LA, Amuan ME, Cramer JA, Pugh MJ (2007). Epilepsy & Behavior 10:539-46. Mortensen EM, Pugh MJ, Copeland LA, Restrepo MI, Cornell JE, Anzueto A, Pugh JA (2008). Eur Respir J 31(3):611-7. Copeland LA, Zeber JE, Salloum IM, Pincus HA, Fine MJ, Kilbourne AM (2008). J Nerv Ment Disease 196:16-21. Leibowitz RQ, Jeffreys MD, Copeland LA, Noel PH (2008). Gen Hosp Psychiatry 30:100-3. Pugh MJ, Van Cott AC, Cramer JA, Knoefel JE, Amuan ME, Tabares J, Ramsay RE, Berlowitz DR; TIGER Team (2008). Neurology 70:2171-8. Zeber JE, Copeland LA, Good CB, Fine MJ, Bauer MS, Kilbourne AM (2008). J Affect Disord 107:53-62. Copeland LA, Zeber JE, Pugh MJ, Mortensen EM, Restrepo MI, Lawrence VA (2008). Annals Surg 248:31-8. Pugh MJ, Zeber JE, Copeland LA, Tabares JV, Cramer JA (2008). Psychiatric Services 59:925-8. Zeber JE, Copeland LA, Hosek BJ, Karnad AB, Lawrence VA, Sanchez-Reilly SE (2008). Crit Rev Oncology/Hematology 67:237-42. Mortensen EM, Restrepo MI, Copeland LA, Pugh JA, Anzueto AR (in press). Am J Med Sci Zeber JE, Copeland LA, McCarthy JF, Bauer MS, Kilbourne AM (in press). Am J Public Health Copeland LA, Miller AL, Welsh DE, McCarthy JF, Zeber JE, Kilbourne AM (in press). Am J Public Health

My CV without 2° Data Shope JT, Copeland LA, Maharg R, Dielman TE, Butchart AT (1993). Health Education Quarterly 20: 373-390. Shope JT, Copeland LA, Dielman TE (1994). Alcoholism: Clin Exper Res 18: 726-733. Shope JT, Copeland LA, Marcoux BC, Kamp ME (1996). J Drug Educ, 26: 323-337. Shope JT, Copeland LA, Maharg R, Dielman TE (1996). Alcoholism: Clin Exper Res 20: 791-798. Copeland LA, Shope JT, Waller PF (1996). J School Health 66: 254-260. Barry KL, Fleming MF, Manwell L, Copeland LA (1997). J Fam Prac 45: 151-158. Zimmerman MA, Copeland LA, Shope JT, Dielman, TE (1997). J Youth Adolescence 26: 117-141. Barry KL, Fleming MF, Manwell L, Copeland LA, Appel, Scott (1998). Fam Med 30:366-371. Blow FC, Barry KL, BootsMiller BJ, Copeland LA, McCormick R, Visnic S (1998). J Psychiatric Research 32: 311-319. Valenstein M, Barry KL, Blow FC, Copeland LA, Ullman E (1998). Psychiatric Services 49: 1043-1048. Shope JT, Copeland LA, Kamp ME, Lang SW (1998). J Drug Educ 28: 185-197. Kales HC, Blow FC, Copeland LA, Bingham RC, Kammerer EE, Mellow AF (1999). Am J Psychiatry 156: 550-556. Blow FC, Barry KL, Copeland LA, McCormick R, Lehmann L, Ullman E (1999). Psychiatric Services 50: 390-394. Maio RF, Shope JT, Blow FC, Copeland LA, Gregor MA, Brockmann LM, Weber JE, Metrou ME (2000). Annals Emerg Med 35:252-7. Kales HC, Blow FC, Bingham RC, Copeland LA, Mellow AM (2000). Psychiatric Services 51: 795-800. Roberts JS, Blow FC, Copeland LA, Barry KL, Van Stone W (2000). J Ger Psychiatry Neurology 13: 78-86. Blow FC, Walton MA, Barry KL, Coyne JC, Mudd SA, Copeland LA (2000). JAGS 48: 769-774. Kales HC, Blow FC, Bingham RC, Roberts JS, Copeland LA, Mellow AM (2000). Am J Geriatric Psychiatry 8:301-309. Blow FC, Ullman E, Barry KL, Bingham CR, Copeland LA, McCormick R, Van Stone W (2000). Am J Orthopsychiatry 70: 389-400. Valenstein M, Copeland LA, Owen R, Blow FC, Visnic S (2001). J Clin Psychiatry 62: 545-551. Valenstein M, Copeland LA, Owen R, Blow FC, Visnic S (2001). Psychiatric Services 52: 1242-4. Duffy SA, Terrell, Jeff, Valenstein M, Ronis, David, Copeland LA, Connors, Mary (2002). Gen Hosp Psychiatry 24: 140-7. Valenstein M, Copeland LA, Blow FC, McCarthy JF, Zeber JE, Gillon L, Bingham CR, Stavenger T (2002). Med Care 40: 630-9. Hegedus, Andrea M, Copeland LA, Barry KL, Blow FC. (2002). American Journal of Orthopsychiatry 72: 331-340. Copeland LA, Blow FC, Barry KL (2003). Health Education & Behavior 30:305-21. Copeland LA, Zeber JE, Valenstein M, Blow FC (2003). Am J Psychiatry 160:1817-1822. Valenstein M, Blow FC, Copeland LA, McCarthy JF, Zeber JE, Gillon L, Bingham CR, Stavenger T (2004). Schizophr Bull 30: 255-64. Janz NK, Wren PA, Copeland LA, Lowery JC, Goldfarb SL, Wilkins EG. (2004). J Clin Oncology 22:3091-3098. Lambert MT, Terrell JE, Copeland LA, Ronis DL, Duffy SA. (2005). Nicotine & Tobacco Research 7:233-241. Copeland LA, Fletcher CE, Patterson J(2005). Military Medicine 170:602-606. Copeland LA, Elshaikh MA, Jackson J, Penner LA, Underwood III W (2005). Cancer 104:1372-1380. O'Neill JL, Flanagan PS, Zaleon CR, Copeland LA (2005). Pharmacotherapy 25:1560-1565. Pugh MJV, Copeland LA, Zeber JE, Cramer JA, Amuan MT, Cavazos JE, Kazis LE (2005). Epilepsia 46:1820-7. Copeland LA, Zeber JE, Rosenheck RA, Miller AL (2006). Med Care 44:110-116. Lambert MT, Copeland LA, Sampson N, Duffy SA (2006). Progress Neuropsychopharm Biol Psychiatry 30:919-23. Hopp FP, Woodbridge P, Subramanian U, Copeland LA, Smith D, Lowery JC (2006). Telemedicine and eHealth 12:297-307. Silveira MJ, Copeland LA, Feudtner C (2006). Am J Public Health 96:1243-1248. Zeber JE, Copeland LA, Grazier KL (2006). Military Medicine 171:619-26. Wilkins EG, Lowery JC, Copeland LA, Goldfarb SL, Wren PA, Janz NA (2006). Medical Decision Making 26:589-598. Copeland LA, Mortensen EM, Zeber JE, Pugh MJ, Restrepo MI, Dalack GW (2007). Progress Neuro-Psychopharm Biol Psychiatry 31:720-6 Alexander JA, Copeland LA, Metzger ME (2007). Clinical Toxicology 45(5):440-50. Kilbourne AM, Copeland LA, Zeber JE, Bauer MS, Lasky EC, Good CB (2007). Psychopharmacology Bulletin 40:104-15 Fletcher CE, Baker SJ, Copeland LA, Reeves PJ, Lowery JC (2007). Journal of Nursing Scholarship 39:358-62 Duffy SA, Copeland LA, Hopp FP, Zalenski RJ (2007). Journal of Palliative Medicine 10:1137-45 Mortensen EM, Restrepo MI, Copeland LA, Pugh JA, Anzueto AR, Cornell JE, Pugh MJ (2007). Pharmacotherapy 27: 1619-26 Kilbourne AM, Post EP, Bauer MS, Zeber JE, Copeland LA, Good CB, Pincus HA (2007). J Affective Disorders 102:145-51. Zeber JE, Copeland LA, Amuan ME, Cramer JA, Pugh MJ (2007). Epilepsy & Behavior 10:539-46. Mortensen EM, Pugh MJ, Copeland LA, Restrepo MI, Cornell JE, Anzueto A, Pugh JA (2008). Eur Respir J 31(3):611-7. Copeland LA, Zeber JE, Salloum IM, Pincus HA, Fine MJ, Kilbourne AM (2008). J Nerv Ment Disease 196:16-21. Leibowitz RQ, Jeffreys MD, Copeland LA, Noel PH (2008). Gen Hosp Psychiatry 30:100-3. Pugh MJ, Van Cott AC, Cramer JA, Knoefel JE, Amuan ME, Tabares J, Ramsay RE, Berlowitz DR; TIGER Team (2008). Neurology 70:2171-8. Zeber JE, Copeland LA, Good CB, Fine MJ, Bauer MS, Kilbourne AM (2008). J Affect Disord 107:53-62. Copeland LA, Zeber JE, Pugh MJ, Mortensen EM, Restrepo MI, Lawrence VA (2008). Annals Surg 248:31-8. Pugh MJ, Zeber JE, Copeland LA, Tabares JV, Cramer JA (2008). Psychiatric Services 59:925-8. Zeber JE, Copeland LA, Hosek BJ, Karnad AB, Lawrence VA, Sanchez-Reilly SE (2008). Crit Rev Oncology/Hematology 67:237-42. Mortensen EM, Restrepo MI, Copeland LA, Pugh JA, Anzueto AR (in press). Am J Med Sci Zeber JE, Copeland LA, McCarthy JF, Bauer MS, Kilbourne AM (in press). Am J Public Health Copeland LA, Miller AL, Welsh DE, McCarthy JF, Zeber JE, Kilbourne AM (in press). Am J Public Health

Is it scientifically valid to use data for purposes for which it was not originally collected?

Yes, because…. No, because… Maybe…please use caution

Today and next week

Uses of secondary data Common biases encountered with secondary

data Methods of adjustment Sources of secondary data Possible effects of HIPAA on this type of

research

Different Uses Health Care Delivery

Quality assessment Geographic variation; under-/over-utilization Adverse events Outcomes of a particular treatment Cost

Natural history incidence, prevalence prognosis Association not causation

Other Regulatory

Some Examples

To show you the range of usesTo show you the work is publishable

Regulatory enforcement

TENET HEALTHCARE - 2002 Medicare Report Jan-Oct 2004 included these findings: – At Alvarado Hospital, prosecutors accused Tenet of running covert

kickback arrangements – Document requests were received from federal prosecutors in L.A. – False claims were uncovered and the company agreed to pay

$22.5M to settle allegations – Redding Medical Center was determined to be performing

unnecessary cardiac surgery – Tenet sold some holdings to Hospital Partners of America to

satisfy part of the 2003 settlement with federal officials – Whistleblowers were to receive over $8M

Whose data set is it anyway? Sharing raw data

from randomized trials

Andrew J VickersTrials. 2006 May 16;7:15

http://www.trialsjournal.com/content/pdf/1745-6215-7-15.pdf

Risk of death in elderly users of conventional vs. atypical antipsychotic medications

Wang PS, Schneeweiss S, Avorn J, Fischer MA, Mogun H, Solomon DH,

Brookhart MA. N Engl J Med. 2005 Dec 1;353(22):2335-41

http://content.nejm.org/cgi/reprint/353/22/2335.pdf

Cancer incidence in Kentucky, Pennsylvania, and West

Virginia: disparities in Appalachia

Lengerich EJ, Tucker TC, Powell RK, Colsher P, Lehman E, Ward AJ,

Siedlecki JC, Wyatt SW (2005). J Rural Health 21(1):39-47.

http://www.blackwell-synergy.com/toc/jrh/21/1

Does Medicare coverage of colonoscopy reduce racial/ethnic

disparities in cancer screening among the elderly?

Shih YC, Zhao L, Elting LS (2006). Health Aff (Millwood). Jul-Aug;

25(4):1153-62. http://content.healthaffairs.org/cgi/content/full/25/4/1153

Can you think of examples of secondary data analysis that you have read in the medical literature?

Any that have changed your practice or your research direction?

Methods of Learning

Lecture In class epidemiologic exercise Journal article evaluation – readings are

online (http://czresearch.com/dropbox/) Optional brief proposal for your own

secondary data analysis

Exercise #1 Policy Decision on

FundingOf Regional Trauma

Center (handout)

Secondary Data Exercise #1

A Regional Trauma Center (which encourages the surrounding hospitals to refer patients with serious injuries to it for expert care) is seeking additional funds from next year’s health budget for more equipment and staff. A local politician (who would rather spend such money on a new hospital named after his father) criticizes this request for additional resources by claming that Regional Trauma Centers do not, in fact, save lives, and he submits the following data to back up his claim.

You are asked for your opinion on these data. How should you respond?

Severity of trauma surrounding hospitals regional trauma center

# of cases # of deaths # of cases # of deaths

Total 7266 458 3354 212

Case-fatality rate 458/7266 = 6.3% 212/3354 = 6.3%

Secondary Data Exercise #1A Regional Trauma Center (which encourages the surrounding hospitals to refer patients

with serious injuries to it for expert care) is seeking additional funds from next year’s health budget for more equipment and staff. A local politician (who would rather spend such money on a new hospital named after his father) criticizes this request for additional resources by claming that Regional Trauma Centers do not, in fact, save lives, and he submits the following data to back up his claim.

By examining the data stratified by severity, the value of the regional trauma center becomes clear.

Severity of trauma surrounding hospitals regional trauma center

# of cases # of deaths # of cases # of deaths

Mild 3734 37 687 3

Moderate 1887 94 1238 37

Severe 1645 327 1429 172

Total 7266 458 3354 212

Case-fatality rate 458/7266 = 6.3% 212/3354 = 6.3%

Bias

Systematic error in measurement or a systematic difference (other than the one of interest) between groups Selection

For cohorts, assembly, migration, contamination, and referral bias

Measurement Confounding

Restriction (may lose generalizability) Matching Stratification Standardization Multivariate Adjustment Assuming the worst (sensitivity analyses) Discussing possible effects on your results

Bias: Anticipate and Control

Example: Hospital Mortality Report Cards

Originally unadjusted Hospitals without trauma centers, doing

primarily elective surgery, etc., looked really good

Made hospitals who took care of the sickest of the sick look bad

Quality Assessment

Data Quality: Garbage in, garbage out

Risk Adjustment: To remove the confounding effect of different institutions providing care to patients with dissimilar severity of illness and case complexity

Interpret data carefully

Data Quality

• If no reliable (accurate and adequate) data are available, questions about risk adjustment are moot

• Inconsistent practices in assigning standard ICD9 codes & names to diseases exist—but this has improved over time (Do you know who assigns the codes?) http://www.eicd.com/

• Lack of specificity of ICD9 system in some diagnostic areas, especially with regard to severity

Administrative Data Limitations

Need patient-specific identifiers to link episodes Need multi-year data when outcomes are infrequent Limited generalizability if restricted by type of

institution or hospital, by type of payor, or by location/region

May lack important clinical variables known to be related to outcomes (especially clinical tests or qualitative evaluations of severity)

Risk Adjustment

Controls for those patient characteristics that are related to the outcomes of interest

Removes the confounding effect of different institutions providing care to patients with dissimilar severity of illness and case complexity

Addresses regional variations Inadequate case-mix adjustment can lead to

misclassification of outlier status

Risk Adjustment

Primary data collection vs. administrative data

Disease-specific vs. generic Commercial vs. developed for your study Predictors vary by outcomes being predicted

Essential Elements of Risk Adjustment

Outcome-specific Contains specification of the principal

diagnosis Contains demographics as proxies for

preexisting physiological reserve Measure count of comorbidities and all the

most important comorbidities to assume their own empirically derived coefficients

Classification of Disease States

ICD-9: too many specific codes (n~10,000) Clinical Classifications for Health Policy

Research (CCHPR): good for chronic illness and longitudinal care [http://www.ahrq.gov/data/hcup/his.htm]

Primary diagnosis: good for studies that focus on a single episode of care

Risk Adjustment: Charlson

Advantages: Commonly used case-mix classification system in

the health care industry System with which most clinicians and reviewers

are familiar

Risk Adjustment: Charlson

Disadvantages Principal diagnosis not differentiated Original work did not specify ICD-9 codes that

went into the disease categories Developed on inpatients predicting mortality; may

not be well suited to outpatients at low risk of death

Not good for longitudinal care / chronic illness

Demographic Factors to Consider in Risk Adjustment

Age (e.g., age-adjusted Charlson) Proxies of Social Support

Marital status Race Gender SES (occupation, employment status,

education) Proxies of Socioeconomic Status

Health insurance status Home address zip code average income

Race and Gender

Don’t adjust for automatically Ideally adjust for variation in the patients’

physiological reserve and disease burden but not for variation in care rendered to patients

Propensity Scores Useful when dataset is small, to conserve power Need a good proxy to develop a propensity score Ask: propensity for what? Include as many predictors as possible to get

predicted probability of group membership (Rosenbaum & Rubin)

Published schema may include predictors you want to study separately

Best for non-randomized studies of treatment effect where you want to adjust for the factors that may have influenced the treatment choices

Your Assignment

Read; think If you want extra feedback, e-mail me a 1-

page proposal (abstract) for a database analysis related to your area of interest any time in next few yearscopelandl@uthscsa.edu