Upload File

interpretable discovery in large image data...

  • Home
  • Documents
  • Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff
19
Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff and Jake Lee Jet Propulsion Laboratory, California Institute of Technology December 7, 2017 NIPS Interpretable Machine Learning Symposium © 2017, California Institute of Technology. Government sponsorship acknowledged. This work was performed in part at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with NASA.

Upload: others

Post on 13-Jun-2020

7 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

InterpretableDiscoveryinLargeImageDataSets

KiriL.WagstaffandJakeLee

JetPropulsionLaboratory,CaliforniaInstituteofTechnology

December7,2017

NIPSInterpretableMachineLearningSymposium

©2017,California InstituteofTechnology.Governmentsponsorshipacknowledged.ThisworkwasperformedinpartattheJetPropulsionLaboratory,CaliforniaInstituteofTechnology,underacontractwithNASA.

Page 2: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

DiscoveryinLargeDataSets• Scientificdiscoveriesoftencomefromoutliers

WagstaffandLee- NIPS2017 2

ByFlickruserKlaus

Page 3: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

DiscoveryinLargeImage DataSets

• Challenges:Representation,Explanations

WagstaffandLee- NIPS2017 3

HiRISE– 1.4MimagesCredit:NASA/JPL-Caltech/Univ.ofArizona

HumanfacesCredit:Pixabay userGeralt

Surveillance PlanetaryScience

Page 4: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

NoveltyDetectionMethods• Clustering• IsolationForest[Liuetal.,2008]• Density-based(e.g.,LocalOutlierFactor[Breunig etal.,2000])• SVD• DEMUD:SVD-based+explanations• Explanation:SVDresidual;informationthemodelcouldnotexplain[Wagstaffetal.,2013]

WagstaffandLee- NIPS2017 4

Page 5: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

WhyDEMUD?

0 5 10 15 20 25 30 351

2

3

4

5

6

Number of selected examples

Num

ber o

f cla

sses

disc

over

ed

DEMUDCLOVERNNDMSEDERInterleaveStatic SVDRandom

Rareclassdiscovery– UCIglassdata

WagstaffandLee- NIPS2017 5

Explanations– UCIglassdata

• IncrementaldiscoveryusingSVDmodelofselections• Especiallygoodfordiscoveringrareclasses• Explanationsjustifyselections

Page 6: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

WhyDEMUD?• IncrementaldiscoveryusingSVDmodelofselections• Especiallygoodfordiscoveringrareclasses• Explanationsjustifyselections• Explanationshelpusersclassifyitems

0 5 10 15 200

10

20

30

40

50

60

70

80

90

100

Selection number

Cum

ulat

ive

accu

racy

DEMUD explanationsNo explanationsRandom

ChemCam expertclassificationperformance

WagstaffandLee- NIPS2017 6

200 300 400 500 600 7000

2

4

6

8

10

12

14

16

18

20 x 10−4

Wavelength (nm)

Inte

nsity

+Mn 259.34−Fe 273.91−Fe 274.62−Fe 274.67−Fe 274.88−Fe 274.93−Fe 275.53−Fe 275.58+Mn 293.28+Mn 293.86+Mn 293.91+Mn 294.88−Mg 516.73−Mg 517.19−Mg 518.34

Explanations:ChemCam spectra

Rhodochrosite:MnCO3

Page 7: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

DEMUDforImages• Representation• Rawpixels• SIFT[Lowe,2004],HOG[Dalal &Triggs,2005]• CNNfeatures[Razavian etal.,2014]

WagstaffandLee- NIPS2017 7

Page 8: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

DEMUD+CNNRepresentations

Classprobs

WagstaffandLee- NIPS2017 8

Images

[Krizhevsky etal.,2012]

Page 9: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

DEMUD+CNNRepresentations

Images

Classprobs

WagstaffandLee- NIPS2017 9

DEMUD

Features

[Krizhevsky etal.,2012]

Page 10: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

DEMUDExplanationswithCNNFeatures

WagstaffandLee- NIPS2017 10

DEMUDFeatures

Selection

Explanation

?

Invertresidualstogetvisualexplanations

Page 11: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

DEMUDExplanationswithCNNFeatures

WagstaffandLee- NIPS2017 11

DEMUDFeatures

Selection

Explanation

?

DeepGoggle:Generateinputthatyieldsfeaturevalues

(Mahendran&Vedaldi,2015)

CNNFeatureInversionMethods

Invertresidualstogetvisualexplanations

Page 12: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

DEMUDExplanationswithCNNFeatures

WagstaffandLee- NIPS2017 12

DEMUDFeatures

Selection

Explanation

?

DeepGoggle:Generateinputthatyieldsfeaturevalues

(Mahendran&Vedaldi,2015)

Up-Conv:Predictoriginalimage

withsecondNN(Dosovitskiy &Brox,2016)

CNNFeatureInversionMethods

Invertresidualstogetvisualexplanations

Page 13: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

Experiments– ImageNet

WagstaffandLee- NIPS2017 13

• 1000images• 10classes• Evenlydistributed

Page 14: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

Experiments– ImageNet

WagstaffandLee- NIPS2017 14

DEMUD-CNNSVD-CNNDEMUD-pixelSVD-pixelRandom

Page 15: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

WagstaffandLee- NIPS2017 15

Selection

Deep

Goggle

Up-Con

vExplanations– ImageNetBassoon Dial Foodpacket Dogsled Zucchini

Page 16: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

Experiments– MSLRoverimages

WagstaffandLee- NIPS2017 16

• 6737images• 26classes• Unevendistribution

Page 17: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

Experiments– MSLRoverimages

WagstaffandLee- NIPS2017 17

DEMUD-CNNSVD-CNNDEMUD-pixelSVD-pixelRandom

Page 18: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

WagstaffandLee- NIPS2017 18

Explanations– MSLRoverimagesSelection

Deep

Goggle

Up-Con

v

Chemin inlet REMSUVsensor MAHLIcal target Turret Ground

Page 19: Interpretable Discovery in Large Image Data Setss.interpretable.ml/nips_interpretable_ml_2017_kiri_wagstaff.pdf · Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff

Summary• DEMUD+CNNfeatures+CNNfeatureinversion• Fastdiscoveryofnovelimages

• Withvisualexplanations

• Whatwillyoufindinyourimagedataset?

WagstaffandLee- NIPS2017 19

Thankyou:NASAPlanetaryDataSystem(PDS)ImagingNode


Interpretable and Globally Optimal Prediction for Textual … · Interpretable and Globally Optimal Prediction for Textual Grounding using Image Concepts Raymond A. Yeh, Jinjun Xiongy,

AttentionMeSH: Simple, Effective and Interpretable

Image, Shape & Multimedia Resource Discovery...Image, Shape & Multimedia Resource Discovery Stefan Rüger Multimedia & Information Systems mmis.doc.ic.ac.uk Frederic Fol Leymarie Arts

Designing Interpretable Molecular Property Predictors

Explainable and Interpretable Machine Learning

Knowledge Discovery in Large Image Databases: Dealing with ...gweiss/selected... · Knowledge Discovery in Large Image Databases: Dealing with Uncertainties in Ground Truth Padhraic

Interpretable Emotion Classification Using Temporal

Image Discovery Week

Interpretable recommender system with heterogeneous

ROBUST AND INTERPRETABLE BLIND IMAGE FREE CONVOLUTIONAL … · past decade, convolutional neural networks (LeCun et al., 2015) have achieved state-of-the-art results in image denoising

An Interpretable Graph-based Image Classifierispac.diet.uniroma1.it/scardapane/pdf/2014 - An interpretable graph... · An Interpretable Graph-based Image Classifier Filippo M. Bianchi,

GIF: Generative Interpretable Faces

Recovering the Unseen: Benchmarking the Generalization of ...openaccess.thecvf.com/content_CVPRW_2019/papers... · image enhancement methods in a more intuitive and interpretable

Unsupervised Interpretable Pattern Discovery in Time ... · Unsupervised Interpretable Pattern Discovery in Time Series Using Autoencoders Kevin Bascol 1, R emi Emonet , Elisa Fromont

Image cataloging as a tool for marine biodiversity discovery

Interpretable Deep Learning in Drug Discovery · Interpretable Deep Learning in Drug Discovery KristinaPreuer 1,GünterKlambauer ,FriedrichRippmann2,SeppHochreiter1, andThomasUnterthiner1

Reproducible and Interpretable Spiculation Quantification

This Looks Like That: Deep Learning for Interpretable Image Recognition · 2020-02-13 · This Looks Like That: Deep Learning for Interpretable Image Recognition Chaofan Chen Duke

Interpretable Convolutional Neural Networksopenaccess.thecvf.com/content...Convolutional_Neural_CVPR_2018_… · Interpretable Convolutional Neural Networks Quanshi Zhang, Ying Nian

Unsupervised Discovery of Interpretable Directions in the GAN ...discovery of semantically meaningful directions in the GAN latent space. The approach is model-agnostic and does not

Image Processing Strategies Image Biomarkers for Drug Discovery Steve Pieper, PhD Isomics, Inc.

Near Duplicate Image Discovery on One Billion Images - POSTECH …mlg.postech.ac.kr/~seungjin/publications/wacv15_KimSH.pdf · 2015-01-02 · Near Duplicate Image Discovery on One

Algorithm Unrolling: Interpretable, Efficient Deep …1 Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing Vishal Monga, Senior Member, IEEE,

Feature-based Aerial Image Discovery System with Multi ...repo.lib.hosei.ac.jp/bitstream/10114/13387/1/gradCIS_14t2006.pdf · Feature-based Aerial Image Discovery System with Multi-Scale

Image Discovery Week: A holistic approach to marketing image resources

Interactive image discovery

Towards Interpretable Face Recognition

Interpretable Neural Predictions with Differentiable

HISTOGRAM POOLING OPERATORS AN INTERPRETABLE …

Interpretable Clustering: An Optimization Approach

Probabilistic modeling approach for interpretable

MDNet: A Semantically and Visually Interpretable Medical ...openaccess.thecvf.com/content_cvpr_2017/papers/Zhang...MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis

Data-driven discovery of coordinates and governing equationsOct 18, 2019  · neural networks to discover interpretable dynamical models. Critically, the proposed method provides a

Interpretable Intuitive Physics Model - openaccess.thecvf.comopenaccess.thecvf.com/content_ECCV_2018/papers/Tian_Ye_Interpretable... · Interpretable Intuitive Physics Model 3 unseen

This Looks Like That: Deep Learning for Interpretable ...papers.nips.cc/paper/9095-this-looks...interpretable-image-recognitio… · MIT Lincoln Laboratoryy [email protected] Cynthia