networking, cloud computing, and course...
TRANSCRIPT
1/8/2019
1
NETWORKING, CLOUD COMPUTING, AND COURSE OVERVIEW
George PorterJan 7, 2019
ATTRIBUTION• These slides are released under an Attribution-NonCommercial-ShareAlike 3.0
Unported (CC BY-NC-SA 3.0) Creative Commons license
• These slides incorporate material from:
• Michael Freedman and Kyle Jamieson, Princeton University
1
2
1/8/2019
2
Outline
1. Networking and cloud computing
2. Course overview
3. Open Q&A
4. Network terminology
COURSE OBJECTIVES
• Add networking support to software
• Between two computers
• Between computer and datacenter (“The Cloud”)
• Develop software that is:
• Scalable (handles 100s of M to 1+ billion users)
• Fault-tolerant (survives failures)
• Performant (runs fast)
• Evolvable (how to support different versions?)
• Secure
3
4
1/8/2019
3
OUR LIVES ARE (LARGELY) ONLINE!
NETWORKED SERVICES DRIVEN BY DATA
Data + = Product
Recommendations
Data + =Custom
Stations
Data + = Personalized
Search
5
6
1/8/2019
4
DATA-DRIVEN, PER-USER CUSTOMIZATION + ML
Data + = Product
Recommendations
App 1
App 3
App ...
App 2
App ...App ... App ...
App ...App ...App ...
App ...
App ...
App ...
DATACENTERS:THE HOME OF ALL THIS COMPUTING AND STORAGE
Google Facebook
Microsoft
7
8
1/8/2019
5
9
Google 2012
10
Microsoft
9
10
1/8/2019
6
11
12
11
12
1/8/2019
7
13
MASSIVE NETWORKED INFRASTRUCTURE
• To build:
• Google spends about $3B per year
• Microsoft spent $15B in total
• To operate:
• 1-2% of global energy consumption1
• 91 billion kWh (34 500-MW coal-fired power plants)2
• By 20202: (less than a year from now!)
• 140 billion kWh (50 power plants)
• $13 billion in electricity bills
• 100 metric tons of carbon pollution per year
1. LBNL, 2013
2. NRDC report
13
14
1/8/2019
8
THE NETWORK HAS SEEN RAPID GROWTH
THE NETWORK HAS SEEN RAPID GROWTH
1989 1993 1997 2001 2005 2009 2013
Web
Created
1989 1993 1997 2001 2005 2009 2013
15
16
1/8/2019
9
THE NETWORK HAS SEEN RAPID GROWTH
1989 1993 1997 2001 2005 2009 2013
Web
Created
1989 1993 1997 2001 2005 2009 2013
Google’s 1st cluster
(15 years)
THE NETWORK HAS SEEN RAPID GROWTH
1989 1993 1997 2001 2005 2009 2013
Web
Created
1989 1993 1997 2001 2005 2009 2013
Google’s 1st cluster
(15 years)
(10 years)
17
18
1/8/2019
10
HOW TO BUILD SUCH LARGE SYSTEMS?
• Systems…
• Built on top of abstractions…
• Built on software…
• Built on hardware…
THE IMPORTANCE OF SCALE
• Network abstractions are designed to scale
• Techniques we learn are directly applicable to global-scale services like Google, Facebook, …
• Your projects will be tested in small scale
• Yet could scale immensely with minimal to no modifications
19
20
1/8/2019
11
CSE 123 VS. 124
124: Networked services
• How to program networked software
• Programming abstractions:
• Socket programming
• RPC
• Message passing
• Protocol design and implementation
• Consensus and consistency
• Security, TLS, …
CSE 123: Computer Networks
• How the Internet works
• How to route packets through multiple networks
• How to deliver a reliable, in-order byte stream over unreliable packet delivery
• Resource sharing
• Network-level errors
WHY FOCUS ON CORRECTNESS?
21
22
1/8/2019
12
SELF-DRIVING CARS
SMART CITIES AND SMART GRIDS
http://electrical-engineering-portal.com/an-overview-of-smart-power-grid
23
24
1/8/2019
13
THE CHALLENGE OF NETWORKING
• Undergraduate program includes:
• Algorithms
• Programming languages
• Architecture
• Data structures
• Etc...
• How does the network change each of these areas?
Outline
1. Networking and cloud computing
2. Course overview
3. Open Q&A
4. Network terminology
25
26
1/8/2019
14
RESOURCES
• Course web page
• Linked off of www.cs.ucsd.edu/~gmporter
• Syllabus, schedule, and blog/updates
• Books
• TA discussion sections
• Class meetings
• Google Q&A
• Each other!
CLASS MEETINGS (M/W/F)
• Overview of material, work through examples/demos
• To help you do what you need to do for your projects/homeworks
• Be involved--don’t expect 45 minute speeches!
• Engagement is good:
• Being unengaged saps energy from your peers and me
• 10am isn’t too bad…
27
28
1/8/2019
15
BOOKS
• All free, available online via course webpage
PROGRAMMING LANGUAGE: C/C++
• Pluralsight online training (link on course page)
• Usually quite expensive, but UCSD is covering the cost
• Need: Functions, basic data structures, containers (Map, Vector/List, String)
• Maybe: Classes, unit testing
• Don’t need: Inheritance, metaprogramming, templates, advanced features, …
• Correctness, but also code quality
29
30
1/8/2019
16
124 TEACHING ASSISTANTS
• Aashi Jain
• Kunal Kashilikar
• Siddarth Ravichandran
• Discussion: Fri 12-1pm
• Small(er) group meeting to work through examples, ask questions, seek out help on the projects/homeworks, etc.
• Midterm will be given during a discussion mid-way into the term
WEEKLY LECTURES
• Electronic device policy: None allowed in first few rows (ok in back)
No devices in first few rows
Devices permitted
(But no TV, movies, video, or games!)
Picture courtesy http://tinyurl.com/znkuezc
31
32
1/8/2019
17
124 ASSESSMENT
• Assignments (60%)
• Three major programming projects
• Smaller written homeworks
• Mid-term (15%)
• Final (25%)
PROJECTS
• Build your own web server
• Build a cloud-based file storage system patterned off of how Dropbox works
• Geodistributed across the globe
• Have it keep working even when portions crash/fail
33
34
1/8/2019
18
AMAZON CLOUD SERVICES
• Deploy your code on Amazon AWS to datacenters on five continents
• Mumbai, India; Dublin Ireland; Sao Paulo Brazil; Seoul, Korea, San Diego, Calif.
COLLABORATION POLICY
• Groups of 1 or 2
• GradeScope
• For homeworks and projects
• Automatic grading of the project (a bit experimental)
• Can use GitHub:
• For all assignments, if you use GitHub you must use private GitHub repos that we will provide to you
• Do not post code online, on the web, in a public repo, on discussion forums, etc.
35
36
1/8/2019
19
Stack overflow, Google, etc…
• Be aware of Googling for answers
• Isn’t that what “real” programmers do?
• Nope!
• Short story about using online resources at a real company
• Examples of OK resources
• C++ API docs, Library documentation
PIAZZA
Piazza’s business model is based on selling your data to 3rd parties
37
38
1/8/2019
20
PIAZZA GOOGLE Q&A
• We’ll use Google’s message board instead
• Seems to have similar features such as upvoting a question to indicate you have it too, marking as answer as the “correct” answer
• Can adjust email frequency
• No emails, email for every message, periodic digests (e.g. every 12 hours), subscribe to just one thread, …
• No data sold to third parties
Outline
1. Networking and cloud computing
2. Course overview
3. Open Q&A
4. Network terminology
39
40
1/8/2019
21
Outline
1. Networking and cloud computing
2. Course overview
3. Open Q&A
4. Network terminology
NETWORK TERMINOLOGY
• Some network terms
• Cloud
• Hosts
• Switches
• internetwork
• Router/gateway
• Host-to-host connectivity
• Address
• Routing
• Unicast/broadcast/multicast
(a) A switched network
(b) Interconnection of networks
41
42
1/8/2019
22
NETWORK TERMINOLOGY
• Some common terminology
• Scale
• Link
• Nodes
• Point-to-point
• Multiple access
• Switched Network
• Packet, message
• Store-and-forward
(a) Point-to-point
(b) Multiple access
RESOURCE SHARING
• Resource: links and nodes
• How to share a link?
• Multiplexing
• De-multiplexing
• Queuing
Multiplexing multiple logical flows
over a single physical link
43
44
1/8/2019
23
QUESTIONS? COMMENTS?
Consult the course web page for this week’s readings
45