rethinking data transport for a truly mobile world · i can’t suspend/resume i highly variable...

Rethinking Data Transport for aTruly Mobile World

Keith Winstein, Anirudh Sivaraman, Hari Balakrishnan

[email protected]. Computer Science and Artificial Intelligence Lab

January 2013http://wireless.csail.mit.edu

Mosh: Graceful mobility Sprout: Flow control for cellular networks

The Era of Truly Mobile Computing

I The current (and future) Internet is mobile first

I Mobile & wireless networks are very different from thetraditional Internet

I Current protocols and applications perform poorly:Should we fix the end points or fix the network?

Keith Winstein, Anirudh Sivaraman, Hari Balakrishnan Wireless@MIT M.I.T. Computer Science and Artificial Intelligence Lab

Rethinking Data Transport for a Truly Mobile World


Truly Mobile Computing

0

400

800

1200

1600

2000

2007 2008 2009 2010 2011 2012 2013 2014 2015

Inte

rnet Us

ers

(Milli

ons

) Dinosaurs

Source: Morgan Stanley Research

I 5.5B mobile phones (> 1B with “broadband” access)I Exceeds people with access to shoes, electricity, toothbrushesI In Q2, 2012:

I Population growth: 0.25 millionI Smartphone growth: 1.7 million




Many Problems

I Can’t roam across IP addresses (WiFi ↔ cellular)

I Can’t suspend/resume

I Highly variable rates

I Highly variable and often high delays




Highly Variable Rates

0

500

1000

1500

2000

0:00 0:30 1:00 1:30

thro

ugh

pu

t(k

bp

s)

Verizon LTE uplink throughput




Awful Delays: The Too-Reliable Network

6.02 Fall 2012 Lecture 21, Slide #16 http://nms.csail.mit.edu/papers/index.php?detail=208

Verizon LTE TCP RTT (Cambridge, MA)

6.02 Fall 2012 Lecture 21, Slide #17 17

Round-trip delay

Del

ay

(millise

con

ds)

AT&T Wireless on iPhone 3G

mu: 1697.2 ms stddev: 2346.5 ms min:155.6 ms max:12126.6 ms

Time (s)

RTT (m

s)

On the Acela Amtrak train (BOS à NY)




Outline

New transport protocols to adapt applications to thevagaries of modern wireless and mobile networks

Mosh: Graceful mobility

Sprout: Flow control for cellular networks




Motivation for Mosh: Frustration with SSH

I Cannot roamI . . . across Wi-Fi networksI . . . from Wi-Fi to cellular or vice versa

I Can’t sleep and wake up (usually): home ↔ work fails

I . . . TCP disconnects if unacked data for “a while”I . . . . . . or if peer away for “too long”

I TCP (and SSH) responds poorly to packet loss

I All UI requires round-trip to server




Octet Stream: Wrong Abstraction

I Client wants latest screen

I After interruption, don’t want to replay megabytes of “oldstuff”

I But SSH runs over TCP, which sends everything in order

I TCP fills buffers, so Control-C takes forever




What We Built

1. Transport protocol for low-latency object synchronization

I with fast, secure roamingI and its own flow controlI through suspend/resumeI over lossy network paths

2. Supports rolling latency compensation on client side

3. Mobile shell application to replace SSH:runs over object sync protocol with predictive local echo

Mosh project: http://mosh.mit.eduK. Winstein, H. Balakrishnan, USENIX Annual Conf., June 2012—, article in ;login:, August 2012




State Synchronization Protocol (SSP)

I Runs over UDP

I Instead of sending octet streams, synchronize objectsI Object must support:

I diff: make vector from state A→ BI patch: apply vector to A to make B

I Object implementation, not protocol, defines synchronizationsemantics.

I Flow control: two object operations per RTT (one in flight,another in process)




Secure Quick Roaming

I Protected by Offset Codebook (OCB) mode of AES-128(Krovetz 2011; Rogoway’s scheme)

I Integrity and confidentiality with one key

I Each packet is an idempotent operation with incrementingsequence number

I Roaming is easy:I Source address of latest authentic packet from client⇒ server’s new target

I Client does not even know it has roamedI Server can’t move




Loss Resilience: Prophylactic Retransmissions

SSP has options in choosing which diff to send:

1. Last ack was for state #3. Then state changes to #4.

2. Host sends diff from 3→ 4.

3. Object changes to state #5.

4. If no timeout yet, make next diff as 4→ 5.

5. Also make diff from 3→ 5: the prophylactic retransmission.

6. If P·Retransmission is shorter or not much longer, sendinstead.




Rolling Predictions

I Client runs predictive model of server UI.

I Make predictions in epochs.

I If any from epoch n is confirmed, show whole epoch.

I If prediction from epoch n is wrong, hide that epoch.

I If user does something difficult to handle, become tentative:increment epoch.




Demo




Deployment

I Mac OS X: in MacPorts,Homebrew, FreeBSD ports

I In most Linux distros (Nov. 2012cover of Linux Magazine)

I Debian, Ubuntu, Fedora, Gentoo,Arch, Slackware

I Red Hat, CentOS, Oracle Linux

I iOS (iSSH), Android (expt)

I Works on Windows Cygwin, Solaris

I 300,000+ page views, ≈100,000known downloads, 1,600+ VCSfollowers

I Top repo of the month on GitHub




Performance

Keystroke response times CDF over Sprint EV-DO (3G) network

0

10

20

30

40

50

60

70

80

90

100

0 0.2 0.4 0.6 0.8 1

Per

cen

tage

Keystroke response time (seconds)

Mosh

median: 5 ms

mean: 173 ms

median: 503 ms

mean: 515 ms

SSH




Reviews

I USENIX accepted the paper, grudgingly. . .I “Not a research paper”I “ISO 2022 locking escape sequences oh flying spaghetti

monster please kill me now”

I The Twitterati are far kinder . . .I “After 48 hours . . . I can say it rocks!” — paulrougetI “Mosh is the only thing that makes internet on Greyhound

useful whatsoever.” — standaloneSAI “I’m on a train, tethering. SSH keeps failing hard. Installed

Mosh. Instant win.” — maciejmaleckiI “MOSH: mobile shell that’s suitable for real-world connectivity;

example, accessing my Euro boxes from India” — danparsonsI “Mosh is world-changing.” — mftbI “Mosh gives me faith in humanity.” — runaI “Go get mosh: NOW” — echorand




SSP for all Mobile Apps?

Many Web and native apps have trouble with roaming andintermittent connectivity.

Screenshot from July 5!

I Gmail app, Skype, Google Chat, quora.com and other websites, Google Voice, Twitter, . . .

I Problems may be expressed as state sync problems andhandled well using SSP




Mosh: Graceful mobility

Sprout: Flow control for cellular networks




Mobile Wireless Networks are Variable

0

500

1000

1500

2000

0:00 0:30 1:00 1:30

thro

ugh

pu

t(k

bp

s)

Verizon LTE uplink throughput




Videoconferencing over LTE

I We measured cellular networks while driving:

I Verizon LTEI Verizon 3G (1xEV-DO)I AT&T LTEI T-Mobile 3G (UMTS)

I Then ran apps across emulated network:

I Skype (Windows 7)I Google Hangout (Chrome on Windows 7)I Apple Facetime (OS X)




Why is Wireless Videoconferencing So Bad?

I Applications/transport protocols react to congestion signalsI Today, these are

I Packet lossI Increase in round-trip time

I This feedback comes too late to help

I The killer: self-inflicted queueing delay

I Any overshoot means a queue filling up with packets

I No network model, no prediction ⇒ Poor adaptation




Characterizing Performance

0

500

1000

1500

2000

2500

3000

3500

300500100020003000500010000

Th

rou

gh

pu

t(k

bp

s)

Self-inflicted delay (ms)

Skype

Better




Sprout’s Goal

I As much throughput as possible, with

I bounded risk of delay > D (e.g., D = 100 ms).

Alfalfa project (http://alfalfa.mit.edu)“How to Beat Skype, Hangout, and Facetime”K. Winstein, A. Sivaraman, H. Balakrishnan, NSDI 2013




Bounded Risk of Delay

I Infer rate from interarrival distribution

I Predict future link rate and convey prediction to sender

I Don’t wait for congestion

I Control: Send as fast as possible, but require:

I 95% probability all packets will arrive within 100 ms




Infer Rate from Interarrival Distribution

0.0001

0.001

0.01

0.1

1

10

100

(< 0.5)

39041 10 100 1000

Per

cen

tin

tera

rriv

als

interarrival time (ms)

t−3.27

A flicker noise processKeith Winstein, Anirudh Sivaraman, Hari Balakrishnan Wireless@MIT M.I.T. Computer Science and Artificial Intelligence Lab



Predict Future Link Rate

I Count packets in every 20 ms tick

I Use Bayesian updating to infer (uncertain) link speed

I Make a cautious forecast




Network Model

Sender Queue Receiver

Poisson process

Rate λ controls

λ

σ

Brownian motion

λz

If in an outage,

drains queue

Poisson process

of σ√t varies λ λz is escape rate.




Bayesian Update

I Discrete set of possible rates, λ (e.g., 0 to 1000 packets/s)

I Initially, each λ is equi-probable

I Each tick (τ seconds), if we receive k bytes, run update step:

Pnew(λ = x)←Pold(λ = x) · (xτ)k

k! e−(xτ)

Z,

where Z ensures that the probabilities sum to 1




The Cautious Forecast

I Receiver has “cloud” of current link speeds

I For eight ticks in the future:

I Predict future link rate by simulating Brownian motion of ratesI Find 5th percentile of cumulative packets

I Send forecast to sender on ACKs

I Most of the math is pre-computed




Limitations

I Stochastic model has not been tuned

I Designed for cellular link with per-user queue

I If other users can cause you big delay, can’t solve end-to-end




Verizon LTE uplink

0

500

1000

1500

2000

2500

3000

3500

300500100020003000500010000

Th

rou

gh

pu

t(k

bp

s)


Skype

Better




Verizon LTE uplink

0

500

1000

1500

2000

2500

3000

3500

300500100020003000500010000

Th

rou

gh

pu

t(k

bp

s)


Hangout

Skype Facetime

Better




Verizon LTE uplink

0

500

1000

1500

2000

2500

3000

3500

300500100020003000500010000

Th

rou

gh

pu

t(k

bp

s)


LEDBAT

Hangout

Skype Facetime

TCP Vegas

Compound TCP

Linux TCP

Better




Verizon LTE uplink

0

500

1000

1500

2000

2500

3000

3500

300500100020003000500010000

Th

rou

gh

pu

t(k

bp

s)


LEDBAT

Hangout

Skype Facetime

TCP Vegas

Sprout

Compound TCP

Linux TCP

Better




Verizon LTE downlink

0

1000

2000

3000

4000

5000

1002003005001000200030005000

Th

rou

gh

pu

t(k

bp

s)


LEDBAT

HangoutSkype

Facetime

TCP Vegas

Sprout

Compound TCP

Linux TCP




AT&T LTE uplink

0

100

200

300

400

500

600

700

800

20030050010002000300050001000020000

Th

rou

gh

pu

t(k

bp

s)


LEDBAT

Hangout

SkypeFacetime

TCP Vegas

Sprout

Compound TCPLinux TCP




AT&T LTE downlink

0

500

1000

1500

2000

30501002003005001000200030005000

Th

rou

gh

pu

t(k

bp

s)


LEDBAT

Hangout

Skype

FacetimeTCP Vegas

Sprout

Compound TCP

Linux TCP




Overall Results from Four Networks

Sprout vs. Avg. speedup Delay reduction

Skype 2.2× 7.9×Hangout 4.4× 7.2×Facetime 1.9× 8.7×Compound 1.3× 4.8×TCP Vegas 1.1× 2.1×LEDBAT Same 2.8×Linux TCP (CUBIC) 1.1× 79×




Competes with AQM even though end-to-end

0

20

40

60

80

100

2003005001000300050001000030000

Uti

lizat

ion

(per

cen

t)


Sprout

Sprout-EWMA

Linux TCP

Linux TCP-over-CoDel




On-going & Future Work

I Sprout relay for easy deployment

I Alfalfa: videoconferencing utility using SproutI Public contest for best predictor

I Incorporate winning ideas into SproutI Open courseware → Open research




Sprout Summary: High Throughput, Tolerable Delay

I Infer link speed from interarrival distribution

I Predict future link speed

I Control risk of large delay

I Yields 2–4× throughput of Skype, Facetime, Hangout

I Achieves 7–9× reduction in self-inflicted delay

I Matches active queue management without router changes




Conclusion

I The current (and future) Internet is mobile first

I Mobile & wireless networks are very different from thetraditional Internet

I Current protocols and applications perform poorlyI Mosh and Sprout show how to handle

I Changing addresses, suspend/resume connectivity, high lossesI High rate variabilityI Delay variability

I Contact: [email protected] & http://wireless.csail.mit.eduAlso http://mosh.mit.edu & http://alfalfa.mit.edu



rethinking data transport for a truly mobile world · i can’t suspend/resume i highly variable...

Documents