learning neurosymbolic generative models via program...
TRANSCRIPT
![Page 1: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/1.jpg)
Learning Neurosymbolic Generative Models via
Program Synthesis
Halley Young, Osbert Bastani, Mayur Naik
University of Pennsylvania
![Page 2: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/2.jpg)
Generative Models and Global Structure
Full Image ⅓ of Image Baseline Completion
![Page 3: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/3.jpg)
Our Approach: Global Structure as Programs
![Page 4: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/4.jpg)
Application to Image Completion
Full Image Third Image Baseline Completion (No representation of global
structure)
Our Completion
![Page 5: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/5.jpg)
Learning via Program Synthesis - Phase 1
Program Synthesis
![Page 6: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/6.jpg)
Learning via Program Synthesis - Phase 1
Program Synthesis
![Page 7: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/7.jpg)
Learning via Program Synthesis - Phase 1
Program Synthesis
Structure Rendering
![Page 8: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/8.jpg)
Learning via Program Synthesis - Phase 2
![Page 9: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/9.jpg)
Comparison to Baselines
Our approach (Synthesis-Guided Generative Model, SGM) significantly outperforms the baseline in 5 out of 6 experiments in image completion (as well as in all image generation experiments).
(Scores for GLCIC/CycleGAN represent Fréchet Inception Distance, scores for VED represent negative log likelihood).
![Page 10: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/10.jpg)
Examples
Experimental Results
![Page 11: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/11.jpg)
Examples
Experimental Results
![Page 12: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/12.jpg)
Examples
Experimental Results
![Page 13: Learning Neurosymbolic Generative Models via Program Synthesis12-16-00)-12-17-15-5121-learning_neuros.pdf · Learning Neurosymbolic Generative Models via Program Synthesis Halley](https://reader033.vdocuments.site/reader033/viewer/2022041922/5e6c66ccd085395807224dd1/html5/thumbnails/13.jpg)
Future Work
•More expressive programs
•Better ways of incorporating program structure
•Domains beyond images