ppt
TRANSCRIPT
Prediction of Cerebral Aneurysm Rupture
Q. Peter Lau, Wynne Hsu, Mong Li LeeNational University of Singapore
{plau, whsu, leeml}@comp.nus.edu.sgYing Mao, Liang Chen
Huashan Hospital, [email protected], [email protected]
Cerebral Aneurysms
• Weak or thin spots on blood vessels in the brain that balloon out
• Common in adult populations, about 5%
• Majority are small and do not rupture
Cerebral Aneurysms
• Picture taken from: J. L. Brisman, J. K. Song, and D. W. Newell. Cerebral Aneurysms. The New England Journal of Medicine,355(9):928–939, 2006.
Cerebral Aneurysms
• Treatment involves invasive neurosurgery
• Surgical treatment often affects the patient’s quality of life
• However a rupture can cause death
Domain Experts (Neurosurgeons)
• Currently expert opinion on the factors that cause rupture is controversial
• Experts make decisions based on local experience
• There have been some study on individual risk factors using statistical techniques
Motivation
• At Huashan Hospital advances in imaging technology has led to many patients with aneurysms discovered
• The current doctrine is surgical treatment since it is the lesser of the two evils
Motivation
• Too many require surgery:– a neurosurgeon may operate on 4 patients a
day each requiring 4 hours
• Need another way of predicting ruptures while expert opinion is still divided
Data-mining Approach
• Patient data consists of patient records and human annoted diagnosis from images
• Issues:– Automatic class labeling– Unequal Misclassification Cost– Class Imbalance
Automatic Class Labeling
• Binary class:– LR : Aneurysm will rupture within n months
– LNR : Aneurysm will not rupture within n months
• n months correspond to consultation interval
• n may be varied
Unequal Misclassification Cost
• Case 1: Will not rupture classified as will rupture.– Patient undergoes needless surgery but likely
to be alive
• Case 2: Will rupture classified as will not– Rupture occurs and is likely to be fatal
• Cost of case 2 is higher than case 1
Class Imbalance
• Most of the available data is from will rupture cases
• There is a tendency to focus more on the serious cases of the disease
• Thus “will not rupture” is the rare class (15% of dataset)
Data-mining Approach
• Much work done in investigating individually:– Classification– Rare class or rare case– Feature selection– Unequal Misclassification costs– Ensemble techniques
• Effects in combination not well established
Methodology
• Run a combination of algorithms for different tasks in order to find “best” combination
• Many combinations when considering ensembles – pruning required
Methodology (Filters)
Classification Feature Selection Class Imbalance
Methodology (Ensembles)
Single Ensemble Multiple Ensemble
Methodology (Evaluation)
• Need to compare combinations
• Use evaluation metric to reflect unequal misclassification costs
• Score algorithm combinations accordingly
Methodology (Evaluation)
• Precision & Recall of “will not rupture”– Precision(LNR) = T(LNR) / [T(LNR) + F(LNR)]
• The will not ruptures should not include will ruptures
– Recall (LNR) = T(LNR) / [T(LNR) + F(LR)]• We want to detect reasonable amount of will not ruptures
• Weighted F-measure (1 + b) Precision(LNR) Recall (LNR)
Fb = -------------------------------------------
b Precision(LNR) + Recall (LNR)
Methodology (Evaluation)
• Use b = 0.5 to weight precision more favourably
• Intuition: We want to predict will not rupture cases more accurately than we want to detect them
Methodology (Two stage)
• Stage 1: Run filter combinations with 10-fold CV
• Stage 2: Take the top few combinations and use them in single ensemble and multiple ensemble variants
• Output: combination with best F score
Methodology (Inputs)
• Classification:– J48 (C4.5) decision tree– Rule-based variant– SMO variant of SVM– Averaged One-dependence Estimators
(AODE)– k-NN classifier using Heterogeneous Value
Difference Metric
Methodology (Inputs)
• Feature selection/transformation:– Fast Correlation Based Filter using
Symmetrical Uncertainty– Ranking using Symmetrical Uncertainty– Principle Component Analysis
Methodology (Inputs)
• Some Class Imbalance Algos (out of 17):– Tomek Links for under-sampling– Cluster-Based Sampling– Synthetic Minority Over-sampling TEchnique
(SMOTE)– Cluster-Based SMOTE– Sampling with Tomek Links
• Sample rare class to 1:1 and 1:2 ratios
Methodology (Inputs)
• Single Ensemble– Bagging– Boosting
• Multiple Ensemble– Stacking with various meta-classification
algorithms– Voting (un-weighted sum)
Experiments
• Used algorithms in WEKA augmented with those absent
• Run the methodology on 12 other UCI datasets
• 10-fold CV for each combination is done in parallel
Experiments
• Score gain over best plain classification
• Methodology handles rare class problem with varying success
• No dataset is worse off after stage one
0
5
10
15
20
25
30
0 2 4 6 8 10 12 14
maj/min class ratio
Sco
re G
ain
ecoli glass flags
aneurysm sponge zoo
vehicle hepatitis autos
credit-german haberman wine
primary-tumor
Experiments
• Most of the gain is in stage 1
• For our aneurysm dataset, every bit of gain is useful
0 5 10 15 20 25 30
ecoli
glass
flags
aneurysm
sponge
zoo
vehicle
hepatitis
autos
credit-german
haberman
w ine
primary-tumor
Score Gain
Stage 2 Gain Stage 1 Gain
Observations
• If a filter performs better than another, it is not always true that any combination with it is better than the rest
• Taking the top combinations from stage 1 for ensemble methods is not always optimal
• The number of base algorithms to combine in multiple ensembles do not directly relate to a better score
Prediction Tool
• A prediction tool implemented uses best algorithm combination from methodology
• This normally involves an ensemble– The model is not human understandable– Attempt to allow novice user to explore
prediction reasoning
Prediction Tool• On-the-fly prediction
Prediction Tool• Nearest Neighbours visualization
Prediction Tool (Limitations)
• More robust techniques will be required to provide human understandable reasoning
• However this is difficult, algorithms that create such models perform poorly (in terms of accuracy) on our dataset
Conclusion
• An application of a large variety of data-mining techniques to predict aneurysm rupture
• A systematic methodology to achieve the best combination of algorithms for prediction was presented
Conclusion
• Prediction tool is implemented and deployed at hospital, currently the 10-fold CV accuracy is 92%
• More advanced techniques needed for exploring prediction reasoning to improve user confidence
• Alternatively, test more patients in a trial-run to “show” accuracy
The End.
• Q&A
Q. Peter Lau, Wynne Hsu, Mong Li LeeNational University of Singapore
{plau, whsu, leeml}@comp.nus.edu.sgYing Mao, Liang Chen
Huashan Hospital, [email protected], [email protected]