high availability server apps - ziade.org · pdf filehigh availability server apps tarek ziade...

High AvailabilityServer apps

Tarek Ziade & Benoît Chesneau

1Monday, October 3, 2011

• The C10K problem

• “It's time for web servers to handle ten thousand clients simultaneously, don't you think? After all, the web is a big place now. “

• http://www.kegel.com/c10k.html

2Monday, October 3, 2011

More & more hits3Monday, October 3, 2011

More & more media4Monday, October 3, 2011

• 1993 - 1998 - CGI (and java applets...)

• Apache & Prefork - simple

• Multiprocessing - threads vs events

A bit of history5Monday, October 3, 2011

6Monday, October 3, 2011

• 1993 - 1998 - CGI (and java applets...)

• Apache & Prefork - simple

• Multiprocessing - threads vs events

A bit of history7Monday, October 3, 2011

Clients

Socket Port 80

Worker

Application Minitel8Monday, October 3, 2011

Waiting...9Monday, October 3, 2011

Moar10Monday, October 3, 2011

Fork11Monday, October 3, 2011

Fork12Monday, October 3, 2011

Fork13Monday, October 3, 2011

Fork14Monday, October 3, 2011

Multiprocessing15Monday, October 3, 2011

multiprocessing16Monday, October 3, 2011

• Threads or Events ?

• Apache, Nginx, Node.js, .... (Erlang)

17Monday, October 3, 2011

• MPM Prefork

• MPM Worker (hybride prefork + threads par requêtes)

$ ab -n 1000000 -c 250 http://127.0.0.1/$ ps ax|grep apache2|wc -l152

$ ps ax|grep apache2|wc -l5

18Monday, October 3, 2011

• 1 master process, several workers

• Prefork

• Accepts several connections per worker

• Events

19Monday, October 3, 2011

• 1 single thread

• 1 event loop (READ/WRITE)

• Context switching

Event Loop20Monday, October 3, 2011

• Nginx VS Apache

• http://blog.webfaction.com/a-little-holiday-present

21Monday, October 3, 2011

Nginx versus Apache (with the worker-MPM) for serving a small static file:

22Monday, October 3, 2011

Memory usage

23Monday, October 3, 2011

Choicehttp://www.ostinelli.net/a-comparison-between-misultin-mochiweb-cowboy-nodejs-and-tornadoweb/

24Monday, October 3, 2011

• WSGI (PEP 3333)

• Prefork - shared socket

• async & sync workers (gevent, eventlet, tornado, ...)

• Easy to use in Django, Paster, ...

25Monday, October 3, 2011

Fork26Monday, October 3, 2011

Fork + async27Monday, October 3, 2011

• HTTP Stream

• Restartless upgrades ala Nginx

• Simple CLI:

$ gunicorn -w 3 test:app

• greins (https://github.com/meebo/greins)

28Monday, October 3, 2011

29Monday, October 3, 2011

30Monday, October 3, 2011

asynchronous or

synchronous

31Monday, October 3, 2011

• bandwidth-bound & CPU-bound apps

• Fast operations required

• Needs a way to hold incoming connections (cf nginx + gunicorn sync)

synchronous32Monday, October 3, 2011

• websocket, longpolling

• apps with long, blocking calls

• streaming

• slow clients

• slowloris & other DoS

asynchronous33Monday, October 3, 2011

gevent & eventlet

• Network libraries

• synchronous API

• monkey patch

• co-routines

• Code is always running async

gevent & eventlet34Monday, October 3, 2011

twisted

• asynchronous library

• Events & callbacks

• Twisted server == single event loop

twisted35Monday, October 3, 2011

36Monday, October 3, 2011

• message passing

• greenlets

• py.py --withmod-_stackless

http://www.grant-olson.net/python/intro-to-stackless-python

stackless37Monday, October 3, 2011

• ulimit is your friend

• somaxcon

• ...

Tuning38Monday, October 3, 2011

39Monday, October 3, 2011

40Monday, October 3, 2011

Firefox Sync

41Monday, October 3, 2011

42Monday, October 3, 2011

• 10 frontends web

• +123 mysql (percona)

• 6 Open Ldap servers (2 masters, 4 slaves)

• ~1.5 M active users

• ~4 M registered users

• ~800 RPS for the storage

• CPU: 20% !!! ETooMuch

43Monday, October 3, 2011

• silverlining (http://cloudsilverlining.org/) by @ianb

• kraftwerk (http://kraftwerk-wsgi.org)

Deploy44Monday, October 3, 2011

• libcloud (http://libcloud.apache.org/): deploy on the “clouds”

• juju (https://juju.ubuntu.com/) : ubuntu cloud (openstack + ensemble or ec2)

Deploy45Monday, October 3, 2011

• Apache Bench (ab)

• httperf

• Apache Jmeter

• Grinder

• Funkload

Bench46Monday, October 3, 2011

Funkload47Monday, October 3, 2011

Funkload48Monday, October 3, 2011

Thanks !

• @benoitc

• @tarek_ziade

Moar ???49Monday, October 3, 2011