ROC Based Evaluation and Comparison ofClassifiers for IVF Implantation Prediction

Aslı Uyar, Ayşe BenerBoğaziçi University, Department of Computer

Engineering, Istanbul

Nadir Çıray, Mustafa BahçeciIVF Unit of German Hospital, Istanbul

PhD Thesis 2nd Progress Presentation 15th June 2009

IN-VITRO FERTILIZATION (IVF) IVF is a common infertility treatment method during which

female germ cells (oocytes) are inseminated by sperm under laboratory conditions.

Fertilized oocytes are cultured between 2-6 days in special medical equipments and embryonic growth is observed and recorded by embryologists.

Finally, selected embryo(s) are transferred into the woman's womb.

PROBLEM STATEMENT

In-vitro fertilization (IVF) A complex and costly process requiring automated decision support

Many critical decisions affecting the success of treatment Analysis of factors affecting treatment outcome Number of embryos to be transferred Embryo selection Decision of transfer day

Initial consideration Implantation prediction of individual embryos Yielding reliable elective single embryo transfer (eSET) avoiding

multiple pregnancies

MOTIVATION AND OBJECTIVES MOTIVATION:

Lack of reliable eSET criteria and public IVF datasets Limited number of machine learning based

implantation prediction studies Conflicting prediction results No consensus on:

input feature sets training and testing strategies performance measures

OBJECTIVES Dataset construction Benchmarking machine learning based predictor

models Enhancement of prediction results by methodological

improvements or novel techniques Statistical validation and generalization of proposed

methods

METHODOLOGY

DATASET Database of German Hospital, Bahceci IVF Center Cycles performed from January 2007 through August

2008 Two classes of total 2453 embryos Each embryo was represented as an individual record

with 18 input features related to clinical patient and embryo variables

1853 embryos with proven negative implantation (cycles with negative outcome)

270 embryos with proven positive implantation outcome (cycles in which number of visualized pregnancy sacs were equal to number of transferred embryos)

Imbalanced class distribution with 89% negative and 11% positive cases

Considering both FP and FN rates

DATASET FEATURESDataset Features Data TypePatient CharacteristicsWoman age NumericalPrimary or secondary infertility CategoricalClinical Diagnosis and TreatmentInfertility factor CategoricalTreatment protocol CategoricalDuration of stimulation NumericalFollicular stimulating hormone dosage NumericalPeak E2 level NumericalEndometrium thickness NumericalSperm quality CategoricalEmbryo Related DataTransfer day CategoricalNumber of cells NumericalNucleus characteristic CategoricalFragmentation rate CategoricalEquality of blastomeres CategoricalAppearance of cytoplasm CategoricalThickness zona pellucida CategoricalEarly cleavage time NumericalEarly cleavage morphology Categorical

PERFORMANCE MEASURES AND ROC ANALYSIS

A single performance measure representing discriminative power of binary classification Default threshold t_0 = 0.5 Ideal case: (0,1) left corner of ROC curve Optimum threshold t_opt: threshold value that maps to nearest point to (0,1)

Accuracy = TP + TN / (TP + FN + FP + TN)

Sensitivity = TP / (TP + FN) ~ TPR

Specificity = TN / (TN + FP)

(1 – Specificity) ~ FPR

BENCHMARKING CLASSIFIERS

Classifier

NB RBF MLP SVM kNN DT

AUC 0.739 ± 0.036

0.712 ± 0.036

0.675 ± 0.039

0.657 ± 0.020

0.612 ± 0.030

0.550 ± 0.056

Only NB and RBF provided acceptable discrimination with 0.7 ≤ AUC < 0.8. (Hosmer and Lemeshow, 2000)

Thanks for listening!