Db

I

f

CG

�

�

�

�

�

�

�

�

FM

MCA

DSA

AT

CF

UC

0©d

Journal Club Roundtable www.AJOG.org

iscussion: ‘Predicting neonatal acidemiay computer analysis’ by Costa et al

n the roundtable that follows, clinicians discuss a study published in this issue of the Journal in light of its methodology, relevance to practice, and implications

or future research. Article discussed:

osta A, Ayres-de-Campos D, Costa F, et al. Prediction of neonatal acidemia by computer analysis of fetal heart rate and ST event signals. Am J Obstet

ynecol 2009;201:464.e1-6.

IE(mStvoocS(dhttno

BPSgampdpfio9dact

ttii

fa1btcawacs

SPssDaps(laPtTfevcdtentmasptata

DISCUSSION QUESTIONS

What is the specific question being asked?

What do the inclusion and exclusioncriteria tell us about the study?

What was the primary outcome?

Why were tracings from the last hour oflabor used?

What difference does lack of traditionalfetal heart rate interpretation make?

Would you categorize this tool as adiagnostic test or a screening test?

Which statistical assessments weremost important?

What evidence supports higher accuracyof combined computer analysis?

rom the Yale University School ofedicine, New Haven, CT:

oderatorhristian M. Pettker, MDssistant professor

iscussantstephen F. Thung, MDssistant professor

ntonette Dulay, MDhird-year fellow

hristina Han, MDirst-year fellow

nzila Ali, MDhief resident

002-9378/$36.002009 Published by Mosby, Inc.

oi: 10.1016/j.ajog.2009.08.014

See related article, page 464

For a summary and analysis of this

Gdiscussion, see page 543

NTRODUCTIONlectronic fetal heart rate monitoringEFM) has been used since the 1950s toonitor the fetus’s status during labor.1

ince its inception, EFM has been con-roversial, a point exemplified by its uni-ersal adoption despite failure to dem-nstrate improvements in cesarean ratesr long-term fetal/neonatal injury.2 Re-ent advances, most notably the STAN31 ECG fetal heart rate (FHR) monitorNeoventa Medical AB, Mölndal, Swe-en), show promise in monitoringigher-risk patients. Today, we are goingo look at a study that examines a newechnology and its ability to predict neo-atal acidemia with computer analysisfFHR and ST event signals.

Christian M. Pettker, MDand George A. Macones, MD, MSCE,

Associate Editor

ACKGROUNDettker: An FDA-approved device,TAN examines specific electrocardio-ram (ECG) changes, such as T-wavemplitude and ST-interval duration,uch like an adult ECG. Results, at this

oint, demonstrate that STAN may re-uce the incidence of fetal scalp pH sam-ling (relative risk [RR], 0.65; 95% con-dence interval [CI], 0.59 – 0.72) andperative vaginal delivery (RR, 0.88;5% CI, 0.80 – 0.97).3 Work is beingone to advance computerized FHRnalysis so that it can also perceivehanges in FHR variability and decelera-ions.

Most practitioners are familiar withhe controversies surrounding tradi-ional EFM; this test has a very poor pos-tive predictive value (PPV). The Amer-can College of Obstetricians and

ynecologists has concluded that EFM’s r

NOVEMBER 2009 Am

alse-positive rate in the prediction ofdverse outcomes is high, with only 1-2/000 abnormal tracings leading to cere-ral palsy.4 A new study by Costa et al aimso estimate the prognostic capabilities of aomputerized ante- and intrapartum FHRnalysis program. The overall objective ofork like this is to improve EFM predict-

bility, to reduce false positives, and to de-rease the need for operative vaginal or ce-arean intervention.

TUDY DESIGNettker: I think it is always helpful to

tart with study design. What type oftudy is this?ulay: This is a prospective study of di-

gnostic test accuracy that evaluates theredictive capacity of a computerizedystem, the Omniview-SisPorto 3.5Speculum, Lisbon, Portugal), for ana-yzing FHR patterns in screening for fetalcidemia.ettker: What is the objective? What is

he specific question being asked?hung: EFM is very subjective and

raught with many challenges; the great-st are the high false-positive rate andarying interpretations between clini-ians. Dr Costa and her group wanted toetermine if computerized analysis ofhe FHR tracing, combined with ST-vent analysis, can accurately predicteonatal acidemia at birth. To do this,

hey described the primary test perfor-ance parameters of computerized

nalysis with ST-event signals, includingensitivity, specificity, PPV, and negativeredictive value (NPV) and comparedhis information to computerized FHRnalysis alone. This expands on researchhat has already been done using STANlone, and it lays a foundation for future

andomized trials.

erican Journal of Obstetrics & Gynecology e1

PpdsesvPiaaHb3pamarfmitotvtshhawdafiwPwohscpsAPrwmcfliept

tdsOtcettamwwtaTanwiwNetT2hatpPttStbnbttawPwTwptdawdiUw

aamTiiapmtPcDmfrSpchiubtwtsttadilptcPliAytgcosfnakculam

Journal Club Roundtable www.AJOG.org

e

ettker: It is good to put the study intoerspective like that. The researcherson’t attempt to make definitive conclu-ions on this type of EFM analysis; it isxploratory, and they report that thiseems to be a good test worth further in-estigation.ettker: What types of patients were

ncluded? What does that teach usbout the study and the system theyre using?an: Included patients were in active la-

or with singleton gestations older than6 weeks. The fetus had to be in cephalicresentation without evidence of fetalnomalies. Patients had to have rupturedembranes and been dilated enough forfetal scalp electrode. In addition, they

equired a generally accepted indicationor internal FHR monitoring, such as co-

orbid conditions, difficulties in trac-ng—such as might occur with obese pa-ients—and obstetrical concerns, likeligohydramnios. Generally, study par-icipants were at an elevated risk of ad-erse pregnancy outcomes. Certainly,his population is appropriate in an earlytudy such as this one because theseigh-risk pregnancies are likely to have aigher proportion of adverse outcomes,nd for an early investigation, you do notant to subject patients to undue proce-ures. However, while these patients aret higher risk, the study is not necessarilyocused on patients with abnormal trac-ngs. Generalizing this data to low-riskomen might not be appropriate.ettker: So, we are looking at patientsho are at risk for adverse fetalutcomes; not necessarily those whoave abnormal fetal tracings. It is alsoometimes helpful to look at theharacteristics of the excludedatients. What does this part of thetudy design tell us?li: Quite a few patients were excluded.atients were excluded if the tracing du-ation was less than 60 minutes, if thereas signal loss in the last hour that lastedore than 15% of the time, if there were

omplications with the potential to in-uence fetal oxygenation between trac-

ng end and delivery (such as difficultxtractions, cord prolapse, maternal hy-otension, shoulder dystocia, long delay

o vacuum or cesarean delivery), and if e

2 American Journal of Obstetrics & Gynecology N

here were anesthesia complicationsuring surgery. Inadequate cord bloodamples or values were also not included.verall, the exclusion criteria were ex-

ensive, in order to make for a ratherlean study population where the deliv-ry blood gas could be correlated to theracing. The study excluded many pa-ients who were at high risk of umbilicalrtery acidemia at birth, which is the pri-ary outcome of the study. Therefore, itill be important to keep this in mindhen interpreting the data and to realize

hat the findings may not be generaliz-ble or applicable to routine practice.hung: One thing you have to thinkbout when you introduce a new tech-ology is how often it will work. A testith high accuracy that fails to provide

nformation in the majority of patientsith indications for the Test Is Reallyot So Useful In Clinical Practice. These

xcluded patients were patients in whomhe new test probably wouldn’t work.hus, the test may not be able to work in3% of our patients, and perhaps theseave a particularly high risk for neonatalcidemia. It would have been interestingo see the data on cord pHs for excludedatients.ettker: Remember that in this study,

hey are applying the analysis of the fetalracings retrospectively. The Omniview-isPorto 3.5 is not being evaluated in realime, and its function in practice is noteing evaluated. For instance, we haveo idea how differences of interpretationetween the Omniview-SisPorto 3.5 andhe provider would be resolved in prac-ice. The authors are just correlating thenalysis done by their unique systemith neonatal outcomes.ettker: This brings up the next point:hat was the primary outcome?hung: Neonatal umbilical artery pHas the primary outcome; specifically, aH �7.05. This cutoff is somewhat arbi-rary though, and the best cutoff can beebated. Why not a cutoff of 7.20? Afterll, many clinicians use this cutoff whene perform umbilical scalp sampling toetermine whether immediate delivery

s required. Why not a cutoff of �7.00?ltimately, I think most clinicians, asell as their patients, are not as inter-

sted in the umbilical artery pH as they a

OVEMBER 2009

re in the long-term health of the baby,nd the latter would make an ideal pri-ary outcome if it were feasible to use.hat being said, the umbilical artery pH

s easy to measure, immediately availablen most cases, somewhat associated withdverse outcomes when coupled withersistently low Apgar scores, and com-on enough for a small study such as

his one.ettker: How does this type ofomputerized FHR analysis work?ulay: Standard fetal scalp electrodeonitoring uses R-wave intervals in the

etal ECG complex to calculate heartate. STAN combines this capability withT-waveform analysis, based on therinciple that identifiable ST-segmenthanges occur when a fetus experiencesypoxia; it uses cardiotocography to

dentify a high-risk group. This method,sed for fetal monitoring during child-irth, requires use of a fetal scalp elec-rode. The Omniview-SisPorto 3.5,hich also involves internal fetal moni-

oring, provides alerts during periods ofpecific ST-segment changes but also de-ects and sends alerts for FHR changeshat are considered to be strongly associ-ted with fetal hypoxemia and/or aci-emia. Its aim is to reduce human error

n tracing interpretation and possible de-ays in taking action. The system has theotential to act as an adjunct to the de-ection of fetal hypoxia, particularly inases that do not display ST alerts.ettker: Why were tracings during the

ast hour of labor used? Does this havemplications?li: This was a logistical necessity. Anal-sis of tracings was only carried out inhe last 60 minutes because the investi-ators believed that this period was mostlosely related to immediate newbornutcome. That assumption is not neces-arily accurate. While it may be true foretuses with respiratory acidosis, it is notecessarily true for those with metaboliccidosis. Neonatal outcome is well-nown to be influenced by chronic pro-esses that exist before the last 60 min-tes of labor— or prior to the onset of

abor. Again, this may make the data lesspplicable to routine practice. It alsoight not be attributable to events that

re occurring earlier in labor.

PidHtdwttsmttagpmFccuhbttPdtsrHsepnwtppttoThiidwcC(cpo

SPaaliOwpTaspntivftastwpAorctgshcanmawsgTmiptPpfldhcPk3

Pocubtcakif

RPrA4wuolNhwbbps9wTatpPAFwcamaccmpmPeotcf

www.AJOG.org Journal Club Roundtable

ettker: Traditional FHRnterpretation was not used. Whatoes this tell us about the study?an: The use of computerized interpre-

ation of FHR tracings, as opposed to tra-itional visual interpretation, may be aeakness of the paper. It doesn’t allow us

o compare the Omniview-SisPorto 3.5o the standard of care, flawed as thetandard of care might be. As the authors

entioned, computerized interpreta-ion of FHR tracings during the intrapar-um period has several limitations, suchs frequent signal loss and artifact,reater signal instability impeding com-uter interpretation, and high computeremory requirements. Computerized

HR tracing analysis has not been ac-epted into routine obstetric care, and itannot yet be used as a substitute for thebiquitous visual interpretation byealthcare professionals. It might haveeen interesting to compare it to tradi-ional analysis, though this may be a fu-ure study for this group.ettker: So, would we consider this aiagnostic test or a screening test? Ihink this issue is important when wetart talking about statistics andesults interpretation.an: Computerized FHR tracing analy-

is is a screening test that identifies mark-rs for fetal hypoxemia and acidemia. Inractice, it is often thought of as a diag-ostic test that results in interventionith delivery. The follow-up diagnostic

est that could be applicable is fetal scalpH sampling, which provides a directortal to the fetal acid-base status. Or,he case may be that the diagnostic test ishe later test that identifies cerebral palsyr some other birth-related injury.hung: It is going to be a challenge toave someone actually reading the trac-

ngs simultaneously with the computer-zed analysis. You are often going to getifferent answers, and who is to knowhether the human or the computer is

orrect? The new National Institute ofhild Health and Human Development

NICHD) categories may help achieveonsistency between human and com-uterized assessment, but I am not aware

f work being done on this.5 t

TATISTICAL ANALYSES

ettker: The study uses the typicalssessments for test validity: sensitivitynd specificity, PPV and NPV, and

ikelihood ratios. Which is moremportant for interpretation of themniview-SisPorto 3.5’s utility, andhich is more important in clinicalractice?hung: As you hint in the question, thenswer is that it depends. From the per-pective of a public health expert or ahysician, who is deciding whether orot to use the screening test in practice,

he sensitivity and specificity are moremportant parameters, because they pro-ide information on false-positive andalse-negative rates. Sensitivity is similaro detection rate; a sensitive test is goodt identifying those with the disease. Apecific test shows an ability to identifyhose without the disease. Ideally, youant to maximize both, but often, to im-rove 1, you have to sacrifice another.li: The problem with the current statef EFM is the false-positive rate, which iseflected in the specificity. So, to show alinically relevant test, one that is betterhan human EFM interpretation, you areoing to have to show improvements inpecificity. Furthermore, I think thatere, specificity is more important be-ause the positive test result is making usct on something, as it would in a diag-ostic test. We are acting with a treat-ent—an operative delivery. We don’t

ct on a screening test, so I am not suree can treat this like a screening test. I

ense that if we are evaluating its utility ineneral, we should focus on specificity.hung: However, PPV and NPV are farore important when a physician is car-

ng for an individual pregnancy, as thesearameters determine the probabilityhat the testing results are indeed correct.redictive values tell the provider therobability that a specific result will re-ect the absence or presence of the con-ition we are testing for. This may beelpful for the general reader; and in thisase, we probably want a test with a goodPV. The managing physician wants tonow that when the Omniview-SisPorto.5 sounds an alert, there is likely to be a

rue concerning fetal blood pH. j

NOVEMBER 2009 Am

ettker: What about likelihood ratios? Iften look at these when thinking aboutlinical scenarios. Unlike predictive val-es, likelihood ratios are not influencedy the prevalence of the disease we arerying to predict. So, we can more judi-iously apply the numbers to high-risknd low-risk patients, though we musteep in mind that their pretest probabil-ties may be different. I think that is help-ul, given the reported findings.

ESULTS

ettker: Let’s just quickly review theesults. Who would like to summarize?li: Initially, 193 patients were enrolled;5 were excluded, and thus, 148 tracingsere evaluated. There were 7 cases withmbilical artery pH �7.05 and no casesf neonatal hypoxic-ischemic encepha-

opathy. Sensitivity, specificity, PPV,PV, and positive and negative likeli-ood ratios—plus a 95% CI for each—ere calculated for “red alerts” elicitedy combined FHR and ST analysis andy FHR analysis alone. Combined com-uterized FHR and ST analysis had aensitivity of 100% and specificity of4%. The sensitivity decreased to 57%ith computerized FHR analysis alone.he NPV with combined analysis waslso 100% with a PPV of 47%, comparedo 98% and 50%, respectively, with com-uterized FHR analysis alone.ettker: What do we make of that?li: Overall, the combined computerHR and ST analysis test performed well,ith good sensitivity and specificity. The

ombined system picked up all cases ofcidemia. This wasn’t evaluated, but itay be better than what we have avail-

ble; though, we still have a low PPV withombined analysis. On the other hand,omputerized FHR analysis aloneissed about half of the cases with low

H, reflected in the sensitivity of 57%,aking it not such an ideal test.

ettker: If we take the isolated comput-rized FHR analysis as a surrogate forur present method of interpreting FHRracings, you might be able to say that theombined FHR plus ST analysis per-orms better. But, that might be a leap in

udgment.

erican Journal of Obstetrics & Gynecology e3

DaotcSmlPrrhtpatdiPwaNHSw1odrsssepPhDtTblAbrDc�iT

pcPrashDhrptintac1

CPlDmaismustFttplrtctmPcaaaeAc

aatrphwatsaptPtHasmgdtcNecautab

R1e2cfi23ft4cpn5JHpd

Journal Club Roundtable www.AJOG.org

e

ulay: And remember, unlike sensitivitynd specificity, predictive values dependn prevalence, and here we have selectedhe prevalence of “disease” to be high be-ause of inclusion and exclusion criteria.o, in a lower-risk population, the PPVsay be worse, because the prevalence of

ow cord gases will be lower.ettker: Getting back to the likelihoodatios, in this case, the positive likelihoodatio for predicting acidemia was veryigh (17.6, 95%; CI, 9.0 –34.5). A posi-ive likelihood ratio of this magnitude,articularly in a high-risk patient, prob-bly helps improve our ability to judgehe probability of the outcome, aci-emia. This result actually is quite prom-

sing.ettker: What do you think of theide CIs for the sensitivity and PPVnd the smaller CIs for specificity andPV?an: The sensitivity of the Omniview-

isPorto 3.5, with its combined analyses,as reported as 1, with a 95% CI of 0.56 –.0, and the PPV was 0.47, with a 95% CIf 0.22– 0.72. The wide CIs for these in-icate the imprecision of the study andesult from the small sample size andmall number of acidemic neonates. Amall sample size makes it difficult to as-ess just how informative a test might be,specially when applied across variousatient subgroups.ettker: There were no cases ofypoxic-ischemic encephalopathy.oes someone want to comment on

hat?hung: Neonatal encephalopathy maye caused by hypoxic-ischemic encepha-

opathy, which is also referred to as HIE.lthough no definition for HIE haseen agreed upon, we believe it to occurarely; in about 1.6/10,000 live births.ue to the rareness of this potential out-

ome, unlike an umbilical cord pH7.05, it is unlikely that a case would be

dentified in a study as small as this one.

he study would need to be greatly ex- i

4 American Journal of Obstetrics & Gynecology N

anded to use HIE as a primary out-ome.ettker: The authors mention in theesults that they showed the test to beccurate. What is meant whenomeone uses the term accuracy andow does one quantify accuracy?ulay: Accuracy, in general, refers toow true the test is at showing a correctesult. An accurate test has a high pro-ortion of true positives and true nega-ives. You can quantify accuracy by add-ng the true positives and the trueegatives and dividing this sum by the

otal number of cases. For this test,mong the patients they studied, the ac-uracy in predicting a low cord pH is40/148, or 94.5%.

ONCLUSIONSettker: What are some of the

imitations of this study?ulay: This is a small study, and it isainly exploratory in nature. The results

re very promising, but they are not go-ng to change clinical practice, yet. In-tead, this observational study should

ark the beginning of attempts to eval-ate the Omniview-SisPorto 3.5 pro-pectively, in a clinical trial, using a con-rol group and a large sample size.urthermore, many of the excluded pa-ients are the ones with troublesomeracings or are at even higher risk for hy-oxemia. The generalizability is thus

imited as a result. Finally, there are otherelevant outcomes that may be impor-ant to look at. As we have said before, itould be argued that a pH cutoff of 7.2 orhe incidence of cerebral palsy could be

ore clinically-relevant outcomes.ettker: The authors state thatombined computer analysis of FHRnd ST events provides higherccuracy in predicting neonatalcidemia. Do we have sufficientvidence for that?li: In this type of setting, combined

omputer analysis of FHR and ST events

s more accurate than computer FHR l

OVEMBER 2009

nalysis alone. The evidence for its over-ll accuracy is a little more limited,hough. This analysis was performed ret-ospectively and not in a typical clinicalractice setting, so we just don’t knowow accurate it is in the real world. Thereas no direct comparison of ST analysis

nd/or computerized EFM interpreta-ion with visual analysis by health profes-ionals. The data also have large CIs withsmall sample size. Therefore, we shouldrobably temper this conclusion fromhe data presented.ettker: What studies might be helpful

o build on this information?an: An ideal follow-up study would beprospective randomized controlled

tudy with a large sample size to identifyore acidemic fetuses. The control

roup should be evaluated using the tra-itional visual interpretation of FHRracings with NICHD terminology andategorization. Including the newICHD categories would be very inter-

sting. Furthermore, other neonatal out-omes, such as need for resuscitation,dmission to the neonatal intensive carenit (NICU), days in the NICU, indica-

ions for operative or cesarean delivery,nd long-term outcomes should proba-ly be included. f

EFERENCES. Hon EH, Hess OW. Instrumentation of fetallectrocardiography. Science 1957;125:553-4.. Alfirevic Z, Devane D, Gyte GM. Continuousardiotocography (CTG) as a form of electronicetal monitoring (EFM) for fetal assessment dur-ng labor. Cochrane Database Syst Rev006;3:CD006066.. Neilson JP. Fetal electrocardiogram (ECG)

or fetal monitoring during labor. Cochrane Da-abase Syst Rev 2006;3:CD000116.. American College of Obstetricians and Gyne-ologists. ACOG Practice Bulletin No 70: Intra-artum fetal heart rate monitoring. Obstet Gy-ecol 2005;106:1453-60.. Macones GA, Hankins GD, Spong CY, Hauth, Moore T. The 2008 National Institute of Childealth and Human Development workshop re-ort on electronic fetal monitoring: update onefinitions, interpretation, and research guide-

ines. Obstet Gynecol 2008;112:661-6.