cloud programming: from doom and gloom to boom and bloom peter alvaro, neil conway faculty recs:...
TRANSCRIPT
![Page 1: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/1.jpg)
Cloud Programming:From Doom and Gloom to
BOOM and Bloom
Peter Alvaro, Neil Conway
Faculty Recs: Joseph M. Hellerstein, Rastislav BodikCollaborators: Tyson Condie, Bill Marczak, Rusty Sears
UC Berkeley
![Page 2: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/2.jpg)
Writing reliable, scalable distributed software remains extremely difficult
![Page 3: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/3.jpg)
Three Hardware Trends
1. Cloud Computing2. Powerful, heterogeneous mobile clients3. Many-Core
![Page 4: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/4.jpg)
Implications
1. Nearly every non-trivial program will be physically distributed
2. Increasingly heterogeneous clients, unpredictable cloud environments
3. Distributed programming will no longer be confined to highly-trained experts
![Page 5: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/5.jpg)
The Anatomy of a Distributed Program
• In a typical distributed program, we see:– Communication, messaging, serialization– Event handling– Concurrency, coordination– Explicit fault tolerance, ad-hoc error handling
• What are we looking for?– Correctness (safety, liveness, …)– Conformance to specification– High-level performance properties; behavior under
network edge-cases
![Page 6: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/6.jpg)
Data-Centric Programming
• Goal: Fundamentally raise the level of abstraction for distributed programming
• MapReduce: data-centric batch programming– Programmers apply transformations to data sets
• Can we apply a data-centric approach to distributed programming in general?
![Page 7: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/7.jpg)
Bloom and BOOM
1. Bloom: A high-level, data-centric language designed for distributed computing
2. BOOM: Berkeley Orders of Magnitude– OOM bigger systems in OOM less code– Use Bloom to build real distributed systems
![Page 8: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/8.jpg)
Agenda: Foundation
• Begin with a precise formal semantics– Datalog w/ negation, state update, and non-
determinism• Include primitives for distributed computation• Enable formal methods for distributed
programming– Model checking, theorem proving, …
![Page 9: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/9.jpg)
Agenda: Engineering
1. Efficient, low-latency dataflow engine (C4)2. Network-oriented continuous program
optimization– Automatically co-locate code and data– Adapt to current network and client conditions– Optimize for both power and performance– Leverage formal semantics: how does
distribution change program behavior?
![Page 10: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/10.jpg)
Agenda: Language Design
• How to expose these concepts to developers?– What are the right developer abstractions for
common distributed programs?• Bloom language design goals:
1. Familiar syntax (list comprehensions, callbacks)2. Integration with imperative languages3. Modularity, encapsulation, and composition
![Page 11: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/11.jpg)
Agenda: Validation
• How do we know that we’re solving real problems?– Build real systems
• Initial work: BOOM Analytics– Hadoop + HDFS in distributed logic
• Goal: Use Bloom to build a complete cloud computing stack– Google in 10KLOC?
![Page 12: Cloud Programming: From Doom and Gloom to BOOM and Bloom Peter Alvaro, Neil Conway Faculty Recs: Joseph M. Hellerstein, Rastislav Bodik Collaborators:](https://reader036.vdocuments.site/reader036/viewer/2022082505/56649eef5503460f94bff3b3/html5/thumbnails/12.jpg)
Thank you!