Click to edit Master title style

Classic Papers in Medical Informatics

Michael Hogarth, MD, FACPProfessor, Internal Medicine

Professor, Pathology and Laboratory MedicineChair, Health Informatics Graduate Program

UC Davis

Objective

• Identify “classic” papers in medical informatics• Considerations– What is meant by ‘classic’?• What metric/measurement

– Author, citation, topic, historical

• Is it the most impactful?• Is it an innovation?

– What is “the discipline”?• Medical informatics, clinical informatics, biomedical

informatics, health informatics, ?

Prior Art

• Characterizing the “impact” of papers and topics is not new

2011

2013

1965

A classic method: “Impact Factor”

• Journal Impact Factor (JIF) – obtained from the Journal Citation Reports (Thomson Reuters)– “a measure of journal prestige and impact”– Uses the Science Citation Index (SCI), a citation database

originally developed by the Institute for Scientific Information in 1964

– Calculated by dividing the number of citations in the present year by all the articles published in the two prior years

#citations of journal articles this year

#of papers in the journal in the prior 2 years

A newer method - Eigenfactor

• Eigenfactor ranks the influence of journals using network theory and an approach similar to Google’s PageRank algorithm

• Journals are considered influential if they are cited by other influential journals

(Journal A cites Journal B 4 times)

Are “bibliometrics” a correct metric?

JAMIA Most Cited

• Earliest article – 1994

• None after

• Only lists JAMIA articles

• Citations counts can change over time – – ?shouldn’t a classic

article remain classic forever?

Most Cited – Journal of Biomedical Informatics

• Only papers since 2010• Only articles from JBI• Citation counts change

with time

http://www.journals.elsevier.com/journal-of-biomedical-informatics/most-cited-articles/

Expert searcher vs. Content expert

• Informatics experts asked by AHRQ to create a Health IT “bibliography” of ‘seminal articles’ in 11 categories

• Using the same categories, an expert librarian searcher compiled a list of ‘best’ health informatics articles using information seeking and retrieval tools

• Lists overlapped by only 27%

2009

So what did I do?

I tossed all that aside...

My Approach – educator and discipline “expert”

• Authors and Papers– Authors - individuals who were/are influential in the discipline

• Even if they didn’t publish many papers (ie, Morris Collen)• Individuals often noted to be important by myself and my peers

– Papers – Important innovations and/or new knowledge in the discipline

• Classic = “Timeless”– I reached as far back as the beginning of the discipline– Principles in the paper are ‘timeless’ – relevant today as they were

when published

• Discipline– “medical” (clinical) informatics

Some Biases

• I decided to select a “Top 20”– Why not a top 10? – just couldn’t do it...

• Historical articles are being evaluated by me without a personal context (I was not in the discipline before 1995)

So what are you getting?

• In my opinion, these are papers that every student of medical (clinical) informatics should have read!

• I can’t be wrong – it is my opinion!

1) “Reasoning Foundations of Medical Diagnosis”(Ledley and Lusted, 1959)

• First paper analyzing the reasoning processes inherent in medical diagnosis

• Introduced the use of Bayesian probability and value theory (game theory) as elements in making a medical diagnosis and choosing a therapy

• The paper clearly outlines the derivation of a form of Bayes that can be used to determine the probability that a patient (P) had a disease (f) given a set of symptoms (G)

• Paper also outlines how to use game theory to choose the treatment of greatest value to the patient

Dr. Robert Ledley -- a pioneer

• Physicist and DDS• Pioneered the use of digital

computers in biology and medicine

• First exposed to computer programming through his wife (SAEC)

• Led the development of the Automatic Computerized Transverse Axial (ACTA) scanner...the CT scan!

• National Medal of Technology (1997)

2) “Computers in Medical Data Processing”(Ledley and Lusted, 1960)

• Outlines the utility of using ‘electronic computers’ in medical diagnosis

• The first time the notion of a ‘computer network’ is mentioned in the context of clinical computing• “a great significance of a national health computer

network cannot be overestimated”

• First time the notion of using clinical ‘data’ to optimize individual care is mentioned in our literature– “patient’s like mine” or evidence-based

practice

3) “Digital Computer as Aid to Differential Diagnosis”(Lipkin, et al. 1961)

• First example of decision support in healthcare• Used structured data (information codes) • Used weights for relevant findings (ie,

‘pathognomonic = 10; positive sickle-cell prep is a 10 for sickle cell dz)

• DSS accomplished by ‘matching’ the input findings to those extracted (and coded) from a textbook (Wintrobe)

• A ranked the diagnosis list by ‘matched items’• Computer provided correct diagnosis in 57% of the

cases, but the correct diagnosis was at least in the differential list for all cases

‘information codes’ used to represent data

4) “Medical Records that Guide and Teach” (Weed; 1968)

• Paper describing “problem oriented medical records”– As a core component of using the

chart to evaluate the quality of care being rendered

• Introduced the notion of a “flowsheet” with key data over time for a particular problem

• One of my favorite quotes in the discipline:

5) “Design and Implementation of a Clinical Data Management System”(Greenes, Pappalardo, Marble, and Barnett; 1969)

• Paper outlines the nature of clinical data and the best way to ‘store’ it in a computer– “criteria for the design of a clinical data

management system include flexibility in its interface with its environment, the capability of handling variable length text string data, and of organizing it in tree-structured files”

– “with the exception of laboratory data, much of the clinical information in the medical record is generally recorded in narrative or free text form”

– “the expense and inefficiency of writing, debugging, and modifying such programs have been serious obstacles..”

• The first paper describes the “MGH Utility Multi-Programming System” (MUMPS)

The importance of MUMPS

• An interpreted language (real-time interpreter)– Recognition that clinical data management

environments do not require ‘pure central processing’

– Interpreters make it easier to have cross platform software systems

• Combining logic/computation and data storage in the same language– Provides significant speed and robustness

• Arrays as data storage – fast, simple• Multi-user by design

– multiple users executing different parts of the program at the same time using the same interpreter...

– Uses a ‘re-entrant’ interpreter

• Estimated that >70% of all health records in the US today are stored in a MUMPS-based system (Epic, VISTA, Meditech)

6) “Computer-based consultations in clinical therapeutics”(Shortliffe, 1975)

• The first example of an “expert system” based on artificial intelligence principles used in healthcare– Decision rules stored in a knowledge base– Rule system included a rule-acquisition

capability where new information could be incorporated by modifying rules in the decision rules database

• Demonstrated performance superior to infectious disease experts– MYCIN chose correctly 65% of the time– Faculty specialists: 42.5%-62.5%

• Introduced the notion of “explanations” for expert systems– critical for a system to explain its

decisions/recommendations to retain user (clinician) trust

7) “An Application-Independent Subsystem for Free-Text Analysis”

(Fenichel and Barnett; 1975)

• The first paper to describe “free text analysis” (natural language processing) on clinical records

• Implemented in MUMPS• No evaluation, though

8) “Quality Assurance through Automated Monitoring and Concurrent Feedback Using a Computer-Based Medical Information System”

(Barnett, et al; 1978)

• The first report of computerized clinical records being used for quality improvement

• An early example of “alerts and reminders” as an effective decision support tool

• The Harvard Community Health Plan implemented COSTAR in 1970– HCHP was a ‘pre-paid’ health plan (HMO)– COSTAR was the first clinical information

system to be broadly implemented

9) “Origins of Medical Informatics”(Morris Collen; 1986)

• A historical perspective of the discipline by one of its “founding fathers”– 1950-1970’s– Describes the genesis of the discipline,

including how it became known as “informatics” in the US

– Origins of information science• Hollerith, Billings and the 1890 census• Punch-card electro-mechanical machine

used to process 62milliion records in 3 years

• Hollerith in 1896 created Tabulating Machine Company, renamed IBM Corp in 1924

10) “Surveillance Using a Hospital Information System”(Classen, et al.;1991)

• A paper summarizing the use of the “Health Evaluation through Logical Processing” (HELP) for surveillance– HELP included an integrated clinical record (with data from

multiple ancillary systems)– Also has a decision support module for optimizing antibiotic

prophylaxis preoperatively

• One of the first reports of a computer based decision support system improving outcomes– Increased the appropriate use of peri-operative antibiotic

prophylaxis from 40% to 58%– Post-surgical wound infection rate decreased from 1.8% to 0.9%

• One of the first CPOE systems to be implemented routinely in a large institution– 1975 all medication ordering was computerized at LDS hospital– Implemented ‘drug alerts’ (90% resulted in changes)– Observed seizure rate for a particular antibiotic was 0.3%

compared to a national average of 2% at other medical centers

• Early use of ‘predictive analytics’– Using data from 150,000 admissions, a logistic regression model

was developed to predict hospital-acquired infection– Model correctly predicted in 65% of patients in a 1,000 patient

cohort

11) “As We May Think: The Concept Space and Medical Hypertext”(Cimino, Elkin, Barnett; 1992)

• Described the notion of “hypertext” navigation of clinical data (pre-Web)– The WWW was first described by Tim

Berner’s Lee in a proposal he wrote for his boss at the CERN lab in 1988, Mosaic was first introduced in 1993.

• Highlighted common problems with hypertext navigation– We see these often in the web – cognitive

overhead required to review multiple web page retrieval results to find if one (or more) has what you want/need

• Introduced the notion of the ‘concept space’ (semantic space)– An early description of what might be

considered a ‘semantic web’

12) “Physician Inpatient Order Writing on Microcomputer Workstations”

(Tierney, McDonald, et al.; 1993)

• One of the first examples of a randomized controlled trial (RCT) of an informatics intervention

• First large-scale study in medical informatics – 5,219 patients– 68 IM teams of students, residents, faculty

• First demonstration of the utilization/productivity improvement from physician order entry– The value of providing information “at the point of

ordering” to influence the utilization of resources

• Key Findings– Intervention reduced ‘charges’ by 12.7% -- ie, CPOE

can lead to more appropriate ordering– Length of stay reduced 0.89 days for intervention

teams– Intervention Interns spent 33min per 10-hours

*longer* to enter orders than the control group did to write them

13) “Doctors, Patients, and Computers: Will Information Technology Dehumanize Health-Care Delivery?

(Shortliffe, 1993)

• Explores the “human dimension” in using computers in healthcare– The resistance of physicians to using

computers for clinical (although they use computers for many other things)

– The perceptions of patients when they encounter a ‘computer-using’ physician

– Whether computers will ‘dehumanize’ healthcare

– The changing nature of society when it comes to acceptance of technology

– Predicts that computers will ‘recede into the environment’ as ubiquitous computing emerges

14) “A General Natural-Language Text Processor for Radiology”(Friedman et al; 1994))

• Describes the first ‘successful’ NLP system used in a clinical environment

• 230 radiology reports spanning 4 diseases (neoplasm, CHF, bacterial pneumonia, COPD)

• System has a recall and precision of 70% and 87% respectively

15) “Effect of Computerized Physician Order Entry and Team Intervention on Prevention of Serious Medication Errors”

(Bates, Leape, et al.; 1998)

• The most cited article in the discipline

• Compared two interventions for preventing serious medication errors – one intervention was CPOE

• Showed CPOE reduces non-intercepted serious medication errors by 55%

• Demonstrated the patient-safety benefit from CPOE

16) “Types of Unintended Consequences Related to Computerized Provider Order Entry”

(Campbell, Sittig, Ash, et al; 2006)

• One of the first studies focused on unintended problems/errors caused by health information technology

• Focused on CPOE• Cases from 5 large academic

medical centers• Unintended consequences

– More/new work – 19.8%– Workflow issues – 17.6 %– System demands – 14.8%

• Overhead in maintaining the system, keeping up with regulatory compliance, etc..

17) “Clinical Classification and Terminology: Some History and Current Observations”

(Chute; 2000)

• One of my favorite papers• A comprehensive ‘story’ of

terminological systems in healthcare

• A treatise on the rationale for the use of concept representation and terminologies in healthcare

• An excellent definition of “clinical terminology”:

18) “Electronic Health Records in Ambulatory Care – A National Survey of Physicians”

(DesRoches, et al; 2008)

• A survey of physician use of EHRs that revealed only 14% used any kind of EHR

• Was an important moment for the country and its policy makers with regards to EHR adoption

• Showed that a large fraction of practicing physicians are in small groups (1-3 physicians) and adoption in that segment was extremely low

19) “Decrease in Hospital-wide Mortality Rate After Implementation of a Commercially Sold Computerized Physician Order Entry System”

(Longhurst, et al; 2010)

• Demonstrated that CPOE reduced hospital-wide mortality– Estimated decrease in

adjusted mortality by 20%

– 36 fewer deaths over an 18-month period

20) “Grid Binary LOgistic REgression (GLORE): Building shared models without sharing data

(Wu, Ohno-Machado, et al; 2012)

• A paper describing a method for performing decomposable regression models against distributed remote clinical data repositories

• Fulfills the original vision promoted by Ledley and Lusted 55 years ago– a ‘national computer network’ where

clinical data could be used to optimize the decisions being made by a physician in the context of a specific patient

Honorable Mention:“What it will take to achieve the as-yet-unfulfilled promises of

health information technology”(Kellerman and Jones; Health Affairs, 2012)

the largest benefit of health IT will be overall system productivity improvement....

Questions?