generative adversarial networksgenerative adversarial networks presented by yi zhang. deep...

Generative Adversarial Networks Presented by Yi Zhang

Upload: others

Post on 25-May-2020

31 views

Category:

Documents

1 download

Report

Download

Embed Size (px):

TRANSCRIPT

Generative Adversarial NetworksPresented by Yi Zhang

Deep Generative Models

N(O, I) Variational Auto-EncodersGANs

Unreasonable Effectiveness of GANs

Page 4: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

GANs

Discriminator tries to distinguish genuine data from fake ones

Generator turns noise into imitations of data, trying to fool the Discriminator

Idea 1: deep nets are good at recognizing images, then let it judge of the outputs of a generative model

Idea 2: If a good discriminator net has been trained, use it to provide “gradient feedback” that improves the generative model.

Idea 3: Turn the training of the generative model into a game of many moves or alternations.

● f(x) = log(x) for ordinary GANs

Page 5: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

GANs

Discriminator

Generator

Training a GAN playing a two-player game:

Page 6: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Some open questions

1. Does an equilibrium exist?Pure equilibrium may not exist in general.i.e, the rock/paper/scissor game

2. Suppose the equilibrium exists. Does the generator win at the equilibrium?

It seems to be the case in practice, with various training techniques.

3. Suppose the generator wins. What does this say about whether or not generator’s distribution is close to the real distribution?

Most GANs’ research focus on this area. i.e design objective functions.

Page 7: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Objective functions of GANsOriginal GANs f(x) = log(x)

The optimal Discriminator is

Then the objective function becomes the Jenson-Shannon Divergence!

Page 8: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Objective functions of GANsOriginal loss function f(x) = log(x)

Consider a generalized family of JSD

(Pi = 0.5 recovers the ordinary JSD)

Page 9: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Objective functions of GANsWasserstein GANs f(x) = x

Wasserstein Distance:

the amount of probability mass that must be transported from P to Q

Page 10: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Objective functions of GANsWasserstein GANs f(x) = x

Page 11: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Objective functions of GANsWasserstein GANs f(x) = x

Wasserstein Distance:

the amount of probability mass must be transported from P to Q

Page 12: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

However…JSD and Wasserstein Distance don’t generalize !!!

Page 13: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

However…JSD and Wasserstein Distance don’t generalize !!!

The training objective should not be interpreted as minimizing these two distances between distributions !

Page 14: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Neural-Network Divergence

where NN is a class of (small) neural networks.

If the class NN is small, then we have generalization guarantee:

Page 15: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

But here is the bad news….

is usually small compared to the of modes of real-life data distribution,i.e. # of faces

The generator can fool the discriminator by memorizing a small number of images.

Page 16: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

But here is the bad news….is usually small compared to the of modes of real-life data distribution,i.e. # of faces

The generator can fool the discriminator by memorizing a small number of images.

evidence of lack of diversity

Page 17: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Equilibrium in GANs

GAN is a two-player game with the following payoff function

where u, v are the parameters of the generator G and the discriminator D.

Equilibrium may not exist, for pure strategies!!!(rock/paper/scissors)

Page 18: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Equilibrium in GANsEquilibrium may not exist for pure strategies!!!

(rock/paper/scissors)

Page 19: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Equilibrium in GANsEquilibrium may not exist for pure strategies!!!

(rock/paper/scissors)

[von Neumann’s Min-Max Theorem]

There exists an equilibrium, if players are allowed to play mixed-strategies.

i.e. infinite mixture of generators infinite mixture of discriminators

Page 20: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Equilibrium in GANsThen the equilibrium exists. But does the (infinite mixture of) generator(s) win the game at it?

Yes!!!!Infinite mixture of generators

Each generator is responsible for generating a single image

and in real life…..eps-approximate if we use a mixture of size O(n log(n) / eps^2)

Page 21: Generative Adversarial NetworksGenerative Adversarial Networks Presented by Yi Zhang. Deep Generative Models N(O, I) Variational Auto-Encoders GANs. Unreasonable Effectiveness of GANs

Conclusion● Distinguish between JSD/Wasserstein Distance and Neural-Net Divergence

● Neural-Net Divergence ensures generalization, but doesn’t encourage

diversity

● Pure equilibrium may not exist; we need to allow mixed strategies

● Then the equilibrium exists, and the generators win the game

● In practice, we can approximate the equilibrium using a finite mixture

Machine Learning (CSE 446): Generative Adversarial Networks … · Machine Learning (CSE 446): Generative Adversarial Networks (GANs) Sham M Kakade c 2019 University of Washington

GENERATIVE ADVERSARIAL NETWORKSraul/RS/GANs.pdf · Generative Adversarial Network GANs adopt the following alternative approach: Sample from a simple distribution, e.g., random noise,

Generative Adversarial Networks - TJ Machine LearningGenerative Adversarial Networks Justin Zhang July 2017 1 Introduction Generative Adversarial Networks (GANs) are a seminal type

Generative Adversarial Networks (GANs) Past, Present…web.eecs.utk.edu/~zzhang61/docs/reports/2017.03 - GANs_past_pres… · Generative Adversarial Networks (GANs) Past, Present,

Lipschitz constrained GANs via boundedness and continuityKeywords Generative adversarial networks Lipschitz constraint Boundedness Continuity 1 Introduction Generative adversarial

Generative Adversarial Networks (GAN) · Unrolled generative adversarial networks.arXivpreprint arXiv:1611.02163. Evaluation of GANs is quite challenging In explicit density models,

Generative Adversarial Networks (GANs) Adversarial Networks (GANs) Hossein Azizpour Most of the slides are courtesy of Dr. Ian Goodfellow (Research Scientist at OpenAI) and from his

Generative Adversarial Networks (GANs)courses.missouristate.edu/anthonyclark/790DL/lectures/16-gans.pdf · Generative Adversarial Networks, Goodfellowet al. (2014) GANs Real Data

Semantic Segmentation using Adversarial NetworksTheir generative adversarial networks (GANs) take samples z from a ﬁxed ... An adversarial network is used to deﬁne a loss function

An Introduction to Generative Adversarial Networks and ... · Generative Adversarial Networks Issues with GANs Deblurring with GANs Conclusion Main Message Brief introduction into

Generative Adversarial Networksbhiksha/courses/deeplearning/Spring.2018/... · 2018. 3. 29. · Generative Adversarial Networks Generative adversarial networks (GANs) are relatively

Progressive Augmentation of GANs · [email protected] Abstract Training of Generative Adversarial Networks (GANs) is notoriously fragile, re-quiring to maintain a careful balance

Deep Semantic Hashing with Generative Adversarial … generative adversarial networks (GANs), which leverages largely unlabeled and limited labeled training data ... state-of-the-art

Least Squares Generative Adversarial Networks...Generative Adversarial Networks (GANs) were pro-posed by Goodfellow et al. [6], who explained the the-ory of GANs learning based on

Generative Adversarial Networks (GANs) in Networking: A

Generative Adversarial Networks (GANs) - Ian Goodfellow · 2020. 9. 24. · Generative Adversarial Networks (GANs) Ian Goodfellow, OpenAI Research Scientist Presentation at Berkeley

Structured Generative Adversarial Networks · Among various DGMs, variational autoencoders (VAEs) and generative adversarial networks (GANs) can be trained unsupervisedly to map a

Generative Adversarial Networks (GANs) Synthetic report · 2020. 12. 8. · Generative adversarial networks (GANs) are a generative model with implicit density estimation, part of

Generative Adversarial Networks (GANs) - Ian Goodfellotions. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to gen-erate highly compelling images of

A Survey of Image Synthesis and Editing with Generative ...kun/papers/gan_survey_final.pdf · 2 Generative Adversarial Networks Generative adversarial networks (GANs) were proposed

An Introduction to Generative Adversarial Networks · 2021. 8. 1. · Unsupervised: Circular GANs DiscoGAN: “Learning to Discover Cross-Domain Relations with Generative Adversarial

MetricGAN: Generative Adversarial Networks based Black-box ...proceedings.mlr.press/v97/fu19b/fu19b.pdf · Generative adversarial networks (GANs) (Goodfellow et al., 2014) has shown

Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAI

Feedback Adversarial Learning: Spatial Feedback for Improving ... · by Generative Adversarial Networks (GANs) Is the discriminator’s feedback useful for improving generated samples?

Generative Adversarial Networks (GANs) · where D is our discriminator, G is our generator, and and are their corresponding parameters. [ 2 ] E. A. Goodfellow, Ian, Generative Adversarial

GENERATIVE ADVERSARIAL NETWORKS FOR SINGLE …...Generative Adversarial Networks (GANs - Goodfellow et al., 2014) were recently proposed. This paper is focused on providing a reliable

GAN D V U ENERATIVE ADVERSARIAL NETWORKSbzhou.ie.cuhk.edu.hk/publication/iclr19_gandissection.pdf · Generative Adversarial Networks (GANs) have recently achieved impressive results

Generative Adversarial Networks (GANs) · 2020. 9. 6. · Generative Adversarial Networks (GANs) Akshan Sameullah April 2020 1 Introduction Imagine two people are locked in a world-class

Photo Editing With Generative Adversarial Networks (GANs)on-demand.gputechconf.com/gtc/2017/presentation/s... · GTC, May 2017. Greg Heinrich. Photo Editing With Generative Adversarial

Generative Adversarial Networks (GANs) - Emanuele Sansone · List of recent papers on Generative Adversarial Networks: PAPERs about GANs 20142016 Generative Adversarial Networks ICLR

GANs in Action: Deep learning with generative adversarial ......welcome Welcome to MEAP for GANs in Action: Deep Learning with Generative Adversarial Networks. GANs are an exciting

“Deep Fakes” using Generative Adversarial Networks (GAN)noiselab.ucsd.edu/ECE228_2018/Reports/Report16.pdfstyles. Generative adversarial networks (GANs) provide us an available

Generative Adversarial Networks (GANs) Advanced …...Generative Adversarial Networks (GANs) CS109B, PROTOPAPAS, GLICKMAN Outline Concept and Math Applications Common Problems Wasserstein

Generative Adversarial Nets(GANs) - UMDdrp.math.umd.edu/Project-Slides/CarySpring2019.pdf · 2019. 8. 22. · •A generative adversarial net is a type of neural net, used in deep

A Beginner's Guide to Generative Adversarial Networks (GANs) · GANs, Autoencoders and VAEs It may be useful to compare generative adversarial networks to other neural networks, such