1. 1. Copyright NTT Communications Corporation. All rights reserved. Storm Couchbase Server NTT 2015611
2. 2. Copyright NTT Communications Corporation. All rights reserved. 2 n Couchbase ServerCouchbase Japan n n Couchbase ServerStorm n Couchbase Server
3. 3. Copyright NTT Communications Corporation. All rights reserved. 3 { tetsuyam, NTT, , }, twitter@tetsuyam_twt, [, , ]
4. 4. Copyright NTT Communications Corporation. All rights reserved. Hadoop, Spark, BigQuery, Redshift Storm, Norikra, Dataow
5. 5. Copyright NTT Communications Corporation. All rights reserved. Nathan Marz 3 Batch Layer Speed Layer Serving Layer Lambda Architecture Raw Data Raw Data Raw Data Batch Layer Speed Layer Serving Layer Lambda Architecture
6. 6. Copyright NTT Communications Corporation. All rights reserved. Nathan Marz 3 Batch Layer Speed Layer Serving Layer Lambda Architecture Raw Data Raw Data Raw Data Batch Layer Speed Layer Serving Layer Lambda Architecture
7. 7. Copyright NTT Communications Corporation. All rights reserved. PUSH 7 n NTTPUSH n 1010PUSH n TBSTBS 10 TBS 100DL PUSH
8. 8. Copyright NTT Communications Corporation. All rights reserved. 8 n 10 n n
9. 9. Copyright NTT Communications Corporation. All rights reserved. on Cloud TBS Storm, CouchbaseServer NTTcloud RabbitMQ Storm PUSH n
10. 10. Copyright NTT Communications Corporation. All rights reserved. userIDactionIDtimestampcontentsID XXXX131397239563377YYYY1 XXXX251397240192325YYYY2 1433151060:cid|gender|state 1433151060:cid|gender 1433151060:cidTimestamp: PUSH
11. 11. Copyright NTT Communications Corporation. All rights reserved. Apache Storm 11 n n BackTypeTwitterOSS n SpoutBolt n https://storm.apache.org/ Spout Tuple bolt Tuple Tuple
12. 12. Copyright NTT Communications Corporation. All rights reserved. Apache Storm 12 n SPOF n Nimbus ZooKeeper Supervisor Master Node Cluster Coordinator Worker processes ZooKeeper ZooKeeper Supervisor Supervisor Worker Process Worker Process Worker Process Nimbus
13. 13. Copyright NTT Communications Corporation. All rights reserved. 13 Push ZK ZK ZK Nim SV SV SV SV ZoneA ZoneB
14. 14. Copyright NTT Communications Corporation. All rights reserved. 14 n n IO Wait
15. 15. Copyright NTT Communications Corporation. All rights reserved. 15
16. 16. Copyright NTT Communications Corporation. All rights reserved. 16 Y^Y^Y^Y^Y^Y^Y
17. 17. Copyright NTT Communications Corporation. All rights reserved. 17 NODE1 NODE2 NODE3 1 2 3 1 23 Client Library Cluster Map APP
18. 18. Copyright NTT Communications Corporation. All rights reserved. 18 NODE1 NODE2 NODE3 1 2 3 1 23 Client Library Cluster Map APP
19. 19. Copyright NTT Communications Corporation. All rights reserved. 19 NODE1 NODE2 NODE3 1 2 3 1 23 Client Library Cluster Map APP 4 Write Error
20. 20. Copyright NTT Communications Corporation. All rights reserved. 20 NODE1 NODE2 NODE3 1 2 3 1 23 Client Library Cluster Map APP 4 Write Error
21. 21. Copyright NTT Communications Corporation. All rights reserved. 21 NODE1 NODE2 NODE3 1 2 3 1 23 Client Library Cluster Map APP 4 Write Error 30
22. 22. Copyright NTT Communications Corporation. All rights reserved. 22 NODE1 NODE2 NODE3 1 2 3 1 23 Client Library Cluster Map APP 4 Write Error 30 30 30
23. 23. Copyright NTT Communications Corporation. All rights reserved. 23 NODE1 NODE2 NODE3 1 2 3 1 23 Client Library Cluster Map APP 4 Write Error 30 XDCR
24. 24. Copyright NTT Communications Corporation. All rights reserved. 24 Client Library Cluster Map APP ClusterA ClusterB XDCR NODE1 NODE NODE NODE1 NODE NODE
25. 25. Copyright NTT Communications Corporation. All rights reserved. 25 Client Library Cluster Map APP ClusterA ClusterB XDCR NODE1 NODE NODE NODE1 NODE NODE
26. 26. Copyright NTT Communications Corporation. All rights reserved. 26 Client Library Cluster Map APP ClusterA ClusterB XDCR NODE1 NODE NODE NODE1 NODE NODE Write Error
27. 27. Copyright NTT Communications Corporation. All rights reserved. 27 Client Library Cluster Map APP ClusterA ClusterB XDCR NODE1 NODE NODE NODE1 NODE NODE
28. 28. Copyright NTT Communications Corporation. All rights reserved. 28 n n 30 n XDCR
29. 29. Copyright NTT Communications Corporation. All rights reserved. IO Wait 29 RAM
30. 30. Copyright NTT Communications Corporation. All rights reserved. IO Wait 30 RAM
31. 31. Copyright NTT Communications Corporation. All rights reserved. IO Wait 31 RAM
32. 32. Copyright NTT Communications Corporation. All rights reserved. IO Wait 32 RAM Disk write queue IO WaitCPU
33. 33. Copyright NTT Communications Corporation. All rights reserved. IO Wait 33 RAM Disk write queue IO WaitCPU SV Y^Y^Y^Y^Y
34. 34. Copyright NTT Communications Corporation. All rights reserved. IO Wait 34 Disk write queue IO WaitCPU SV Y^Y^Y^Y^Y Couchbase ServerRAM CouchbaseRAM (Couchbase http://www.couchbase.com/jp/couchbase-server/features
35. 35. Copyright NTT Communications Corporation. All rights reserved. IO Wait 35 RAM
36. 36. Copyright NTT Communications Corporation. All rights reserved. IO Wait 36 RAM tmpfs
37. 37. Copyright NTT Communications Corporation. All rights reserved. IO Wait 37 tmpfsUnixOStmpfs HDD (wikipedia
38. 38. Copyright NTT Communications Corporation. All rights reserved. IO Wait 38 tmpfsUnixOStmpfs HDD (wikipedia
39. 39. Copyright NTT Communications Corporation. All rights reserved. IO Wait 39 tmpfsUnixOStmpfs HDD (wikipedia
40. 40. Copyright NTT Communications Corporation. All rights reserved. IO Wait 40 Couchbase Server Couchbase 2 3 (Couchbase Server http://www.couchbase.com/jp/couchbase- server/architecture
41. 41. Copyright NTT Communications Corporation. All rights reserved. IO Wait 41 n Couchbase Server DB n n n tmpfs, SSD
42. 42. Copyright NTT Communications Corporation. All rights reserved. 42 n n Couchbase Server 1 n Couchbase ServerDB
43. 43. Copyright NTT Communications Corporation. All rights reserved. Couchbase Server 43 n DL n SV n n KVS JSON
44. 44. Copyright NTT Communications Corporation. All rights reserved. Couchbase ServerL 44 n Redis TTL n memcached Couchbase Server Memcached
45. 45. Copyright NTT Communications Corporation. All rights reserved. 45