The Clinical Utility of META:An Analysis for Hypertension

James R. CampbellUniversity of Nebraska, Omaha, NE

Gene A. Kallenberg Robert C. SherrickGeorge Washington University, Washington, DC

ABSTRACTTo evaluate the clinical compkteness of the

NationalLibrary ofMedicine Metathesaurus(META), wecoded the conceptual informationfound in 2000problemorented (SOAP) notes for hypertension from oneCOSTAR site. To minimize the effects of practceidiosyncracy, we analyzed an additonal 500 notesfroma second, geographicaly remote site. Conceptsoccuming at either site numbered 1337. We classifiedconcepts occumng at both sites as core concepts andthese numbered 121. We attempted to find a matchingconcept of the proper semantic type in META for eachof the items. AU matching was done by program with amanual review by a physician. The overaU success ratefor matching was:

AU CoreConcepts Concepts

SUBJECTIVE 68% 76%OBJECTIVE 20% 38%ASSESSMENT 75% 72%PLAN 64% 80%We observed the greatest frequency of unmatchedconcepts in physical examination, medications,symptoms, personal behavior, non-medical therapies andcounseling.

We conclude that the current release ofMETAis not sufficiently rch to describe the process of care inthe ambulatory management ofhypertension. However,the construction and breadth of the current schemeholdspromise for medical knowledge representaton andtranslation.

INTRODUCTIONThe National Library of Medicine(NLM) has

undertaken a monumental project in order to developa unifonn medical language system (UMLS) as a"gateway" to the medical literature and medicalinformation in generaI4l] This consists of themetathesaurus (META) which is an expandedconceptual and semantic representation scheme, asemantic network which links the concepts functionallyand hierarchically, and an information sources mapwhich identifies computer resources available to themedical professional.

Although designed as a resource for access toinfonnation, a stated purpose of the development is toprovide knowledge access for clinical care. Since thequestions posed by the clinician should be outlined inthe medical record, it is reasonable to claim thatUMLS must be sufficient to index the content of thatdocument in order to plunb the questions that residetherein. The additional benefits of an acceptedstandard for medical data recording in thecomputerized medical record cannot be understated.

Despite these aspirations, and recentsuggestions that the NLM is moving to make METAmore clinically robust,[21 little has been writtendescribing the ability of the META resource tocompletely represent the content of the medical record.We undertook to examine whether META possessedsufficient conceptual detail to codify the process ofmedical care in the most common adult ambulatoryproblem -- hypertension.

METHIODSMedical records were reviewed and collated

from two clinical sites. The Internal Medicine clinic atthe University of Nebraska is a resident training clinicserving an ambulatory population with an average ageof 56 and a predominance of Medicaid and Medicarethird party coverage. Most patients at this site arereceiving ongoing care for multiple medical problems.COSTAR medical records have been servingambulatory care delivery at this site since 1983. TheGeorge Washington University Health Plan,Department of Health Care Sciences provides care forthe University owned Health Plan, a 50,000 memberhealth maintenance organization. The patientpopulation at this site has an average age of 45, elevenyears younger than the Nebraska patients. Thepractice serves as a training site for both medicalstudents and primary care residents. COSTAR is usedas the medical record for one of the practice teamsconsisting of 4 physicians, 2 physician assistants and aresident.

0195-4210/92/$5.00O©1993 AMIA, Inc. 397

As a part of a controlled trial of computerizedrecords undertaken at the Nebraska site, all problemnotes for a one year period were encoded fromtransribed source documents. This coding was doneconceptually with a manual review of all records bythe primary author and a spially trained coder.Synonyms of concepts that we identified, as wel aslexical variants, were ignored for this analysis. AUcoding maintained a strict problem oriented "SOAP'hierarchy so that the coded element was cross-refemeed by problem and the note "segment" (forexample - SUBJECTIVE).

For this project, 2000 records coded for theproblem of hypertension were taken from theNebraska site. These were selected only forcompleteness, and were included in the evaluation ifthey contained at least 3 coded elements. Acompilation of the primary concepts, expanded toinclude drug trade names, was then matched byprogram against concepts in the metathesaurusprovided by NLM.

Each main concept in META can havemultiple senantic types. The senantic types definedby NLM denote allowable "contexts" for each concept.From our initial results, the santic types of matcheswere correlated with the SOAP segment and comparedagainst the definitions of semantic types provided byNLM. In some cases, confusing areas were discussedwith NLM staff and a final "mapping" of an idealizedSOAP note into the NLM semantic types was done.This is summarized in Table 1.

This mapping was then used to define a validmatch from the hypertensive notes into the NLMmetathesaurus. If a concept coded from thehypertensive note was found to have a matchingconcept in META, and that META entry was of anallowable santic type, then a "match" was declared.If the concept could not be located, or if the semantictype of the META concept was not in the mapping,then the concept was declared to have no match inMETA (even though an attractive concept might existas an alternative semantic type).

We defined the union of all the semantic typeslisted in Table 1 as META "clinical" types. AMUMPS alphabetic cross-reference of all clinicalMETA codes was built for review of the hypertensionnotes. A manually reviewed, machine assistedmatching of all clinical concepts from the original2000 notes was then done by the primary author.

In order to minimize the chances of missingimportant concepts due to regional variation, 500hypertension notes were then identified from theWashington, DC site. These notes were textual,problem oriented docunents detailing encounters onlyfor hypertension. These were manually reviewed bythe authors for concepts consistent with the granularityof the Nebraska coding scheme. Tallies of codefrequency were not kept because of differences in thevolumes of notes reviewed between the sites, and inpatient mix. Concepts identified by the manual reviewat the Washington site were added to the master list ofconcepts and a final matching againstMETA conceptswas done using the procedure outlined above.

Trabie 1

398

Concepts appearing at either site will be calledhypertension concepts. Concepts appearing in clinicnotes at both sites were declared to be core concepts.We judged that core concepts were morerepresentative of management variables forhypertension as an ambulatory problem, althoughconceptual outliers in patients with many clinicalproblems were inevitable. The frequency ofappearance of hypertension concepts in META wascalculated and patterns of "misses" and "hits" werereviewed by the authors. Similar tallies were done forcore concepts.

RESULTSFrom the Nebraska sample of 2000 coded

notes, 1186 clinical concepts were identified. Theseare broken down by SOAP segment in Table 2,showing a slight predominance of concepts in"Subjective" and "Plan" segments. This may beweighted somewhat by virtue of a decision to includedrug trade names as concepts where they occurred inthe medical record. In parentheses following eachtabulation are listed the frequency with whichmatching concepts were found in META.

Column 2 of Table 2 summarizs the clinicalconcepts found on review of notes at the Washington,DC site. The smaller number of concepts relates tothe smaller number of notes available, differences inrecording habits and the heavy orientation of this sitetowards primary care. The third column summarizestotal numbers of unique concepts and the cumulativefrequency of matching into the Metathesaurus. Thefinal column tallies number of concepts and matchingfrequency for core concepts found on review at bothsites.

Following machine and manual matching intoMETA, the codes from all sites without matches weresummarized by the authors into generic classes ofinformation. Table 3 is a summary of those areaswithout match. For each segment of the problem note,any conceptual group judged by the authors to have 5or more failed matches is listed along with the totalnunber of concepts outstanding for that class. Forexample, 42 patient symptoms could not be found inMETA.

Finally, Table 4 summarizes the 121 coreconcepts found independently at both sites. For eachsegment of the SOAP note, those terms matching andfailing to match META are listed for review.

DISCUSSIONThe development and evolution of the NLM

metathesaurus is a matter of strong interest to thoseworking in medical informatics. Conventional codingschemes such as ICD-9-CM or CPI-V have thestrengths of standardization and acceptance but arenot clinically comprehensive. Schemes that weredeveloped for billing patient care generally reflect agranularity and organization that supports the billingfunction but rarely are useful to describe the processof medical care delivery, much less serve as a medicalrecord.

Although META and the related resourcespublished by NLM are designed for access to themedical literature, pressure for clinical utility andrelevance of this representational scheme is building.Problems of transportability of medical data, non-uniformity of representation and confusion regardingmedical meaning would be greatly ameliorated ifMETA developed into a coding resource whichdevelopers could employ with confidence.

lanle z

399

This paper represents an initial attempt toexamine the conceptual content of META for acommon ambulatory medical problem. Ignoringcompletely lexical issues involved in interpretingnatural language, we only attempted to determinewhether basic conceptual representation exsts inMETA for this arena of ambulatory medical process.Clearly, our comments cannot reflect on the ability ofMETA to serve as a resource for text parsing or othertextual analysis since we did not analyze for frequencyof synonyms or lexical variants in this project.

Nonetheless, we found similar patterns ofdiscrepancy when matching META against coreconcepts and the larger set of hypertension concepts.This suggests that we can make useful observationsregarding the dinical utility of the current release ofMETA. The area of greatest clinical concen from ouranalysis is clearly the objective portion of the note -specifically the physical examination. Better than 30%of all concepts not existing in META were physicalfindings or portions of the physical ex tion. Eventhough physical examination concepts were only 20%of the total hypeensive concepts reviewed, thisportion of META is woefully incomplete.

The second most frequent pattern of matchingfailure was attributed to medicinals - both trade andgeneric drugs. Although META does have many drugnames and most medication classes, the sheer volumeof clinical pharmaceuticals used in hypertension andrelated diseases points to a second area which requiresadditional work if META is to serve as a dinicalcoding scheme.

Table 3 outlines several other areas that couldbenefit from concept additions. Personal symptoms,concepts attributed to nonnal behavior and function,some diagnoses, patient education and counselling, andnon-medical therapies are the learest examples.

From the foregoing results, it should be dearthat META in its current release is not adequate toconceptually represent the process of ambulatoryhypertensive management. Despite this conclusion,META has many stts including a rich andgrowing scheme of symptoms, diagnoses andtreatments.

The organization of META, and therelationship of the semantic types to the process ofclinical care can be confusing at first. The mapping ofMETA types into a SOAP note, summarized in Figure2, is an example of that complexity. Clearly the areaof greatest variety, and therefore confusion, exists inthe "subjective" area where the types of informationentering the record may be very diverse. However,the authors found that this was a minor point ofconfusion. We believe that this approach shouldultimately prove useful for textual analysis where theorganization of the semantic net will aid the dissectionof content of structured medical documents.

Finally, this paper makes no attempt to dealwith the issues inherent in the structure and utility ofthe semantic network. Clearly, the accuracy anduniformity of the network relationships will provide alarge benefit to the clinical developer who is trying todevelop knowledge based programs which effectivelydeal with incomplete clinical data and uncertainty.

lTaDLe 3

400

........ ....................... ._.......

.. ............ ....... ..........

...... coww-.....I......... ........_...................

PM...-., ....................... ..........__ .....................

.......

.............-......................... .........._........... ._........_........................_.

consw..p.xo co..

Olik to-.'- -.R .M ...*... ......O.....t pance-tono.. .. .........................

...

:.1..!.F.:..:.'6.!. .:... t .........

... .................tum ifiiaIefilu,Ow...I]1,11vOYS p -d

........ .. ........................... ................. ........................

.............................................S-Vncirom...............................

......... .......... ........ ................................. I'L ............InMW0 ....gqW.`:'M ...... .pro ....... ... :::.a... ...

... .................... .............

..L......... ............. .. ....V

14) 115,

.......

..................... .......... .....

i(CWM917=...Mimpressp. :p4pie ...........L......

..... -- ............ _..........................L.............. ............. ......

...................... .. ......_...............

...... ....... __

'es """"'W,.`:X.W r" 'iti "S1...novenow, "crossing::::::044es",:.:::::ext".'

ra .j ......... ..

.. ..... ... ....... .. ..

... ... ...

.. .. ......

p la.. ..

jL :::::Mpo .ttwc-a........

I.O.:::... .. .. .. .... .. .. .. .. .. .. ..

-Y0 clenpyePO W" ig

..

tilIh" .

-exercise .M.................... bei6.d i9.pain

An examination of the paucity of the relationshipsbetween META physical examination concepts pointsto an area of our concern relative to the clinical utilityof the semantic network.

With all of the preceding comments in mind,we observe that the breadth and depth of META in itscurrent release is still a wonderful enrichment overexisting coding schemes. Since it tries to handle thefull scope of medical care, the current release isalready better than many coding resources. If theNLM continues to enhance the clinical areas we havediscussed, we are convinced this will have substantialutility for knowledge engineers and medical data basedevelopers. We applaud this initiative by NLM andcan only hope as clinician infonnation specialists thattheir emphasis will continue to develop META in waysmore helpful for clinical medicine.

[1] Humphreys BL, Lindberg DA. Building the UnifiedMedical Language System. Proceedings of theThirteenth Annual SCAMC. IEEE Computer SocietyPress; 1989; 475480.[2] Humphreys BL, Lindberg DA, Hole WT. Assessingand Enhancing the Value of the UMLS KnowledgeSources. Proceedings of the Fifteenth Annual SCAMC.IEEE Computer Society Press, 1991; 78-82.

401