applications of deep learning in radiology - espr · applications of deep learning in radiology and...

Applications of DeepLearning in Radiology and

Pediatric Radiology

Dr. İlker Özgür KoskaAfyon SB University Pediatric Radiology

İzmir Dokuz Eylül Universtiy Biomedical Technologies

Prof Dr. Hüdaver Alper

Outline

Building blocks of deep learning

*Artificial neural networks (ANN)

*Working principle of ANN

*Fully Connected Deep Neural Networks

*Convolutional Deep Neural Networks

*Residual Deep Neural Networks

*Autoencoders

Application examples of deep learning in radiology

Artificial neural networks

Malign

0.1 0.3 0.410.2 0.7 0.150.3 0.2 0.320.4 0.3 0.80.2 0.5 0.12

M1 M2 M3

İ1İ2İ3İ4İ5

0.14 0.250.3 0.440.18 0.7

O1 O2

M1M2M3

Deep learning model

How it worksRandom initial weights

Inputs and weights are multiplied and addedthen feed the result to activation functions(feed forward)

Computing the error between the target valueand computed value

Computing how it can minimise the error bydifferentiation (gradient descent)

Computing new values of weights which willdecrease the error(back propagation)

Repeating the process till satisfiying thepredetermined stop criteria (epochs)

Gradient descent

Fully connected (dense) network

Short diameter

Spiculation ratio

Histogram mean

GLCM heterogeneity

GLRLM short run emphasis

Input layer

Hidden layer

Output layer

0 Benign

1 Malign

Convolution

1 0 1

0 1 0

1 0 1

Mask

*

What does convolution do?

-1 -1 -1

-1 5 -1

-1 -1 -1

=*

Pooling Activation functions

Convolutional neural networks

Liver

Gallbladder

Spleen..Kidney

Residual learning

U-Net

Convolutional AutoencoderAutoencoder

Sparse CT Reconstruction

Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)

Hu Chen, Yi Zhang*, Member, IEEE, Mannudeep K. Kalra, Feng Lin, Yang Chen, Peixo Liao, Jiliu

Zhou, Senior Member, IEEE, and Ge Wang, Fellow, IEEE

QUANTITATIVE RESULTS (MEAN±SDs)

PNSR RMSE SSIM

LDCT 39.4314±1.5206 0.0109±0.0021 0.9122±0.0280

TV-POCS 41.7496±1.1522 0.0083±0.0012 0.9535±0.0143

K-SVD 42.7203±1.4260 0.0074±0.0014 0.9531±0.0167

BM3D 42.7661±1.0471 0.0073±0.0009 0.9563±0.0125

CNN10 43.6561±1.1323 0.0066±0.0009 0.9664±0.0100

KAIST-Net 43.9668±1.2169 0.0064±0.0009 0.9688±0.0110

RED-CNN 44.4187±1.2118 0.0060±0.0009 0.9705±0.0087

(a) NDCT, (b) LDCT, (c) TV-POCS, (d) K-SVD, (e) BM3D, (f) CNN10, (g)

KAIST-Net, and (h) RED-CNN

Sparse MR Reconstructionk-Space Deep Learning for Accelerated MRIYoseob Han, and Jong Chul Ye, Senior Member, IEEE

arXiv:1805.03779v1 [cs.CV] 10 May 2018

a) Image domain learning, (b) cascaded network,c) AUTOMAP d) k-space learning..

Artefact ReductionConvolutional Neural Network based Metal ArtifactReduction in X-ray Computed TomographyYanbo Zhang, Senior Member, IEEE, and Hengyong Yu*, Senior Member, IEEE

arXiv:1709.01581v2 [physics.med-ph] 20 Apr 2018

RMSE OF EACH IMAGE IN THE NUMERICAL SIMULATION STUDY. (UNIT: HU).Original BHC LI NMAR1 NMAR2 CNN CNN-MAR

Case 1 155.0 86.3 46.2 121.2 35.4 33.1 29.1Case 2 71.5 44.4 54.5 50.4 41.4 31.5 22.8Case 3 320.3 183.5 107.3 234.9 82.3 83.4 58.4

SSIM OF EACH IMAGE IN THE NUMERICAL SIMULATION STUDY.

Original BHC LI NMAR1 NMAR2 CNN CNN-MARCase 1 0.565 0.576 0.576 0.887 0.935 0.940 0.943Case 2 0.883 0.854 0.931 0.955 0.950 0.965 0.977Case 3 0.522 0.536 0.886 0.833 0.942 0.932 0.967

Image detection/classificationCheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning

Pranav Rajpurkar* 1

Jeremy Irvin* 1

Kaylie Zhu1 Brandon Yang

1 Hershel Mehta

1 Tony Duan

1 Daisy Ding

1 Aarti Bagul

1 Robyn L. Ball

2

arXiv:1711.05225v3 [cs.CV] 25 Dec 2017

Image detection/classification/regression

Original image with superimposed saliency map for sample hand radiographic

images in three male patients age 4 years (a), 15 years (b), and 17 years

Radiology: Volume 287: Number 1—April 2018 n radiology.rsna.org

Performance of a Deep-Learning Neural Network Model in Assessing Skeletal Maturity on Pediatric Hand Radiographs1

Summary Statistics of Paired Interobserver Difference between Bone Age Estimate of Each Reviewer and Mean of the Other Three Human Reviewers’ Estimates, Compared

with That of Model

Variable Clinical Report Reviewer 1 Reviewer 2 Reviewer 3 Mean

Mean

Reviewer 0.08 -0.07 -0.07 0.06 0.00

Model 0.02 -0.01 -0.01 0.02 0.00

P value (paired t test) .41 .34 .36 .57

RMS

Reviewer 0.87 0.73 0.73 0.95 0.82

Model 0.65 0.67 0.67 0.68 0.67

P value (F test, comparing ratio of variances) < .01 .26 .23 < .01

MAD

Reviewer 0.65 0.55 0.53 0.69 0.61

Model 0.51 0.53 0.53 0.53 0.52

P value (paired t test) <.01 .50 .99 <.01

Deep learning and its application to medical image segmentationHolger R. ROTH1, Chen SHEN1, Hirohisa ODA2, Masahiro ODA1, Yuichi

HAYASHI1, Kazunari MISAWA3, Kensaku MORIarXiv:1803.08691v1 [cs.CV] 23 Mar 2018

TABLE : Quantitative results of the 3D FCN network in testing (n=37).Dice (%) Avg. Std. Min. Max.Artery 83.5% 4.1% 73.7% 91.1%vein 80.5% 6.8% 49.0% 89.4%liver 97.1% 1.0% 93.5% 98.3%spleen 97.7% 0.8% 95.2% 98.9%stomach 96.1% 7.9% 49.4% 98.9%Gallbladder 85.1% 15.7% 28.6% 97.4%Pancreas 84.9% 9.1% 52.5% 95.1%Total Avg. 89.3% 6.5% 63.1% 95.6%

Segmentation

The architecture of our 3D U-Net like fully convolutional network. It applies an end-to-end architecture using same size convolutions (via zero padding) with kernel sizes of 3 3 3.

Radiogenomics

CT synthesis from MRIDeep Embedding Convolutional Neural Network for Synthesizing CT Image

from T1-Weighted MR Image Lei Xiang1, Qian Wang1,*, Xiyao Jin1, Dong Nie3, Yu Qiao2, Dinggang Shen3,4,*

https://arxiv.org/pdf/1709.02073.pdf

Many more

• Tumor histology• Tumor grade• Prediction to RT response potential• Metastasis potential• Prediction to theraphy response• Prediction of survival• Prediction of motion and motion suppression• CT synthesis from MRI (MRI only RT)• Intracranial hemoraghy and infarct detection• Pulmonary thromboemboli detection• ….

Limitations:

• Training is data hungry

• How to train, in the case of practically unlimitednumber of targets (whole set of diagnosis, variation and artefact conditions)

• Each new scenoria requires new training, scalability to new conditions

Future pespective

• Scalibility

• Integration in different clinial environments

• Machine vs man--------->Machine+man vs man

• Imageomics……> Central position in precisionmedicine (Dr. Bradley Erickson Mayo Clinic)

Thank you for your attention…

https://mse238blog.stanford.edu/2017/08/imunizr/ai-takes-on-radiology/

Classification pipeline: Segmentation

• The extracted 3D tumor volume is saved in NRRD format in slicer and this is fed to ouralgorithm.

Classification pipeline: Feaure exrraction

Feature space, decision surface

Overfitting

Regularization:

*L2/L1 regularization*Drop out*Data augmentation*Early stopping

What is artificial intelligence

AI: Performing tasks of a machine

which are attributed to human

Machine learning: Performing tasks of a

machine without explicitly programming;

instead learning from the data fed to it

Artificial neural network:Computational modelsinspired by human neural cell

Deep learning: Computational models built bystacking multipl artfical neural network layers

Machine learning

• Supervised learning…Artifical neural networks

…Support vector machines (SVM)

…Decision trees

• Unsupervised learning…Self organising maps (SOM)

…k means clustering and otherclustering methods

http://slideplayer.com/slide/4380892/