distribute the workload, php barcelona 2011
TRANSCRIPT
![Page 1: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/1.jpg)
Distribute the workload
Helgi Þormar ÞorbjörnssonPHP Barcelona, 29th of October 2011
Saturday, 29 October 11
![Page 2: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/2.jpg)
Who am I?
Saturday, 29 October 11
![Page 3: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/3.jpg)
Co-founded Orchestra.io
Developer at PEAR
From Iceland
@h on Twitter
Helgi
Saturday, 29 October 11
![Page 4: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/4.jpg)
Why Distribute?
Budget
Efficiency
Perception
Saturday, 29 October 11
![Page 5: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/5.jpg)
Efficiency
10 small servers > 1 big
Saturday, 29 October 11
![Page 6: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/6.jpg)
Budget
Spend wisely
Commodity servers
Cloud Computing (EC2)
Saturday, 29 October 11
![Page 7: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/7.jpg)
Perception
Defer intensive processes
Give instant feedback
Users keep on browsing
Saturday, 29 October 11
![Page 8: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/8.jpg)
Saturday, 29 October 11
![Page 9: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/9.jpg)
Ant Colonies
Saturday, 29 October 11
![Page 10: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/10.jpg)
Teamwork
When faced with a problem they will solve the problem as one.
Saturday, 29 October 11
![Page 11: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/11.jpg)
Saturday, 29 October 11
![Page 12: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/12.jpg)
Saturday, 29 October 11
![Page 13: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/13.jpg)
Architect for Distribution
Saturday, 29 October 11
![Page 14: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/14.jpg)
Characteristics
Decoupling
Elasticity
High Availability
Concurrency
Saturday, 29 October 11
![Page 15: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/15.jpg)
Decoupling
Saturday, 29 October 11
![Page 16: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/16.jpg)
Application
DB API
Cache FE
Saturday, 29 October 11
![Page 17: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/17.jpg)
Application
DB API
Cache FE
Cache
API
API
Saturday, 29 October 11
![Page 18: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/18.jpg)
Elasticity
Saturday, 29 October 11
![Page 19: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/19.jpg)
Cloud Computing
Saturday, 29 October 11
![Page 20: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/20.jpg)
Load Balancing
Saturday, 29 October 11
![Page 21: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/21.jpg)
HA Proxy
Nginx
My Favourite
Saturday, 29 October 11
![Page 22: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/22.jpg)
Monitoring
Saturday, 29 October 11
![Page 23: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/23.jpg)
When do I need more servers?
Saturday, 29 October 11
![Page 24: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/24.jpg)
Needs to be around from the start!
Saturday, 29 October 11
![Page 25: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/25.jpg)
Keep records
Saturday, 29 October 11
![Page 26: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/26.jpg)
Spot trends
Saturday, 29 October 11
![Page 27: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/27.jpg)
Different types
Hardware Performance
Software Performance
Availability
Resourcing
Saturday, 29 October 11
![Page 28: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/28.jpg)
ApplicationsNew Relic
CloudKick
ScoutApp
Nagios
Cacti
Circonus
Saturday, 29 October 11
![Page 29: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/29.jpg)
Automation
Saturday, 29 October 11
![Page 30: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/30.jpg)
Plug into your monitoring
Saturday, 29 October 11
![Page 31: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/31.jpg)
Bringing together Monitoring and Elastic behaviour into one
beautiful whole!
Saturday, 29 October 11
![Page 32: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/32.jpg)
Add some intelligence to add / remove servers as needed based
on current information.
Saturday, 29 October 11
![Page 33: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/33.jpg)
Just make sure it doesn’t turn into...
Saturday, 29 October 11
![Page 34: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/34.jpg)
Skynet!!Saturday, 29 October 11
![Page 35: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/35.jpg)
High Availability
Saturday, 29 October 11
![Page 36: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/36.jpg)
Get a highly available and resilient setup by following a few
of those recommendations
Saturday, 29 October 11
![Page 37: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/37.jpg)
Remember, even Google has outages
Saturday, 29 October 11
![Page 38: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/38.jpg)
What to avoid
Saturday, 29 October 11
![Page 39: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/39.jpg)
Local Sessions
Saturday, 29 October 11
![Page 40: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/40.jpg)
Store sessions in DB / Memcache
Solution
Saturday, 29 October 11
![Page 41: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/41.jpg)
Local Memory
Saturday, 29 October 11
![Page 42: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/42.jpg)
Networked Memcache
Solution
Saturday, 29 October 11
![Page 43: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/43.jpg)
Local Files
Saturday, 29 October 11
![Page 44: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/44.jpg)
Local Uploads
Saturday, 29 October 11
![Page 45: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/45.jpg)
Writing to /tmp
Saturday, 29 October 11
![Page 46: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/46.jpg)
Store on S3 or a networked FS
Solution
Saturday, 29 October 11
![Page 47: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/47.jpg)
Serve up static files from CDNs
Solution
Saturday, 29 October 11
![Page 48: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/48.jpg)
Servers can vanish at any given time
Saturday, 29 October 11
![Page 49: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/49.jpg)
Internal APIs
Saturday, 29 October 11
![Page 50: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/50.jpg)
Application
S3GFS FS
Internal Storage API
Saturday, 29 October 11
![Page 51: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/51.jpg)
Application
MySQLMongo Cache
Internal DB API
Saturday, 29 October 11
![Page 52: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/52.jpg)
SOA
Saturday, 29 October 11
![Page 53: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/53.jpg)
Service Oriented Architecture
Saturday, 29 October 11
![Page 54: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/54.jpg)
Sort of :-)
Saturday, 29 October 11
![Page 55: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/55.jpg)
Eventually Consistent
Saturday, 29 October 11
![Page 56: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/56.jpg)
CAP Therom
Saturday, 29 October 11
![Page 57: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/57.jpg)
Consistency
Availability
Partition Tolerance
Saturday, 29 October 11
![Page 58: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/58.jpg)
Consistency
All nodes see the same data at the same time
Saturday, 29 October 11
![Page 59: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/59.jpg)
Availability
Node failures do not prevent survivors from continuing to
operate
Saturday, 29 October 11
![Page 60: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/60.jpg)
Partition Tolerance
The system continues to operate despite arbitrary message loss
Saturday, 29 October 11
![Page 61: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/61.jpg)
Consistency
Availability
Partition Tolerance
Saturday, 29 October 11
![Page 62: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/62.jpg)
Queue Systems
Saturday, 29 October 11
![Page 63: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/63.jpg)
Good forImage Processing
Distributed Logs
Data Mining
Mass Emails
Intensive transformation
Search
Saturday, 29 October 11
![Page 64: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/64.jpg)
Common Tools
Gearman
Hadoop
ZeroMQ
RabbitMQ
And many others!
Saturday, 29 October 11
![Page 65: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/65.jpg)
New York Times
4TB of TIFF files
Needed to get 11 million PDF versions
Used Hadoop and EC2
100 machines took 24 hours
Saturday, 29 October 11
![Page 66: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/66.jpg)
Map/Reduce
Saturday, 29 October 11
![Page 67: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/67.jpg)
Map
Master gets a problem to solve
Breaks into multiple sub-problems
Distributed to multiple workers
A worker can take the same steps
Answer passed back to Master
Saturday, 29 October 11
![Page 68: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/68.jpg)
Reduce
Takes in answers from the map workers
Combines together to get an answer
There can be multiple reducers
Saturday, 29 October 11
![Page 69: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/69.jpg)
process petabytes of data in few hours on commodity server farm
Saturday, 29 October 11
![Page 70: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/70.jpg)
CouchDB
Saturday, 29 October 11
![Page 71: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/71.jpg)
CouchDB
Highly Concurrent
Schema free, document based
RESTful API
Map/Reduce Views
Easy Replication
Saturday, 29 October 11
![Page 72: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/72.jpg)
Gearman
Saturday, 29 October 11
![Page 73: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/73.jpg)
Your Client Code
Gearman Client API(C, PHP, Perl, MySQL UDF, ...)
Gearman Job Servergearmand
Gearman Worker API(C, PHP, Perl, Python, ...)
Your Worker Code
Your App Gearman
Saturday, 29 October 11
![Page 74: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/74.jpg)
pear.php.net/net_gearman
Saturday, 29 October 11
![Page 75: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/75.jpg)
A Story!
Saturday, 29 October 11
![Page 76: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/76.jpg)
Financial Software
Saturday, 29 October 11
![Page 77: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/77.jpg)
3000+ Clients
Saturday, 29 October 11
![Page 78: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/78.jpg)
Each one has 5 external data sources
Saturday, 29 October 11
![Page 79: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/79.jpg)
Each data source is a web service
Saturday, 29 October 11
![Page 80: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/80.jpg)
Ran every 6 hours every day
Saturday, 29 October 11
![Page 81: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/81.jpg)
Cron
Gearman
Job 11
2
3
4
5
Web Services
1
43
2
5
Processing
Saturday, 29 October 11
![Page 82: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/82.jpg)
But! That wasn’t enough
Saturday, 29 October 11
![Page 83: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/83.jpg)
Job kicked off on login
Saturday, 29 October 11
![Page 84: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/84.jpg)
Supervisord
Saturday, 29 October 11
![Page 86: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/86.jpg)
Another Story!
Saturday, 29 October 11
![Page 87: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/87.jpg)
CloudSplit
Saturday, 29 October 11
![Page 88: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/88.jpg)
Near Real Time Cloud Analytics
Saturday, 29 October 11
![Page 89: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/89.jpg)
Clients install logging agent locally
Saturday, 29 October 11
![Page 90: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/90.jpg)
syslogd
Saturday, 29 October 11
![Page 91: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/91.jpg)
Public API
Saturday, 29 October 11
![Page 92: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/92.jpg)
Multiple Persistent Gearman Servers
Saturday, 29 October 11
![Page 93: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/93.jpg)
Internal DB API
Saturday, 29 October 11
![Page 94: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/94.jpg)
Agent syslogd
API
Gearman
Gearman
CouchDB
Worker
Worker
Worker
Internal API
Load Balanced
Load Balanced
PersistentSaturday, 29 October 11
![Page 95: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/95.jpg)
CouchDB Setup
Saturday, 29 October 11
![Page 96: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/96.jpg)
Write vs Read
Saturday, 29 October 11
![Page 97: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/97.jpg)
Writes
Multi Master setup
Replicated
Deals with writes only
Saturday, 29 October 11
![Page 98: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/98.jpg)
Reads
Multi Master setup
Replicated from write cluster
Slaves handle website requests
Saturday, 29 October 11
![Page 99: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/99.jpg)
Heavy Map/Reduce usage for data
Saturday, 29 October 11
![Page 100: Distribute the workload, PHP Barcelona 2011](https://reader031.vdocuments.site/reader031/viewer/2022022412/58f31ebf1a28abdb1d8b4577/html5/thumbnails/100.jpg)
Questions?
Joind.in: http://joind.in/4326
Saturday, 29 October 11