deep learning: a brief overview on the possibility for ... · deep learning for studies of galaxy...

Deep Learning: a brief overview on the possibilityfor astrophysics

Francois-Xavier Dupe

(LIS/Aix-Marseille Universite, France)

Journee SKA @ LAM

Introduction

What is Machine Learning?

The aim of Machine Learning is to build a mathematical functionwhich solve a human task.

Today tasks include

classification/regression;

representation (or feature) learning;

transfert learning;

reinforcement learning;

. . .

F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 2 / 25

Introduction

Some history about Machine Learning

1956: the Dartmouth workshop

Proposal

We propose that a 2 month, 10 man study of artificial intelligence becarried out during the summer of 1956 at Dartmouth College inHanover, New Hampshire. The study is to proceed on the basis of theconjecture that every aspect of learning or any other feature ofintelligence can in principle be so precisely described that a machinecan be made to simulate it. An attempt will be made to find how tomake machines use language, form abstractions and concepts, solvekinds of problems now reserved for humans, and improve themselves.We think that a significant advance can be made in one or more ofthese problems if a carefully selected group of scientists work on ittogether for a summer.


Introduction

Some history about Machine Learning (2)


Introduction

A brief timeline

from Andrew L. Beam


Introduction

Today’s talk

1 Deep learning

2 Interactions with astrophysics

3 What next?


Deep learning

Today’s talk

1 Deep learning


3 What next?


Deep learning

What is it?

Deep learning ⇒ hierarchical learning with high order features.


Deep learning

The location of deep learning

From https://www.machinecurve.com/index.php/2017/09/30/

the-differences-between-artificial-intelligence-machine-learning-more/


https://www.machinecurve.com/index.php/2017/09/30/the-differences-between-artificial-intelligence-machine-learning-more/

https://www.machinecurve.com/index.php/2017/09/30/the-differences-between-artificial-intelligence-machine-learning-more/

Deep learning

The zoo

From http://www.asimovinstitute.org/neural-network-zoo/


http://www.asimovinstitute.org/neural-network-zoo/

Deep learning

AlexNet

ImageNet Classification with Deep Convolutional Neural Networks byA. Krizhevsky et al (NIPS 2012)


Deep learning

AlexNet (results)


Deep learning

Long Short-Time Memory (LSTM)

Example of a recurrent neural network.

From http://colah.github.io/posts/2015-08-Understanding-LSTMs/


http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Deep learning

Long Short-Time Memory (LSTM): details

Input gate: encode theinput data.

Output gate: create thenext output (from theinput).

Forget gate: removeinformation from inputdata.

Block input: output datafrom another block.


Deep learning

Generative Adversarial Networks (GAN)

Idea: building a generator that can fool a discriminator

Generator: a NN thatproduce new data fromnoise.

Discriminator: aclassifier which distinguishfake data from true.

A set of real samples.

Generative adversarial nets by I. Goodfellow et al (NIPS 2014)


Deep learning

Generative Adversarial Networks (GAN): some results

From https://adeshpande3.github.io/Deep-Learning-Research-Review-Week-1-Generative-Adversarial-Nets


https://adeshpande3.github.io/Deep-Learning-Research-Review-Week-1-Generative-Adversarial-Nets

Interactions with astrophysics

Today’s talk

1 Deep learning


3 What next?



Some examples

Star-galaxy Classification Using Deep Convolutional Neural Networksby Edward J. Kim Robert J. Brunner (MNRAS 2016)

Idea: Use ConvNet on the reduced calibrated pixel values.



Some examples (2)

Fast Cosmic Web Simulations with Generative Adversarial Networksby A.C. Rodrıguez et al (arXiv:1801.09070)

Idea: use deep learning to avoid the computational cost of N-bodysimulation.



Some other examples from arXiv

Deep learning for studies of galaxy morphology (arXiv:1701.05917)

Detecting Solar-like Oscillations in Red Giants with Deep Learning(arXiv:1804.07495)

Lunar Crater Identification via Deep Learning (arXiv:1803.02192)

Deep learning from 21-cm images of the Cosmic Dawn (arXiv:1805.02699)

Fast Point Spread Function Modeling with Deep Learning (arXiv:1801.07615)


What next?

Today’s talk

1 Deep learning


3 What next?


What next?

Explainable Machine Learning

Deep learning a black-box?

Source: Department of Defense, Advanced Research Projects Agency.


What next?

Evaluation

One of the open question in Machine Learning: how to fullyevaluate a method?

No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.

By OswaldLR - From: A Corny Concerto (2).png, Public Domain


What next?

Evaluation


No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.

Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.

By OswaldLR - From: A Corny Concerto (2).png, Public Domain


What next?

Evaluation


No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.

By OswaldLR - From: A Corny Concerto (2).png, Public DomainF.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 23 / 25

Conclusion

Take away message

Evaluation is primordial! Must be coherent with the task.

Learning representation is a way to avoid some bias.

Deep learning asks for big datasets, but scale very well.

There is no free lunch!


Conclusion

Thank you for your attention.

Any questions?


deep learning: a brief overview on the possibility for ... · deep learning for studies of galaxy...

Documents