![Page 1: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/1.jpg)
Make it fast for everyonePerformance and Middleware design
S. Suhothayan
Technical Lead, WSO2 Inc.
![Page 2: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/2.jpg)
Optimising Applications for
Performance
![Page 3: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/3.jpg)
Latency vs Throughput
Latency measures the end-to-end time processing time. In a messaging environment, teams determine latency by measuring the time between sending a request and receiving the response. Latency is measured from the client machine and includes the network overhead as well.
Throughput measures the amount of messages that a server processes during a specific time interval (e.g. per second). Throughput is calculated by measuring the time taken to processes a set of messages.
![Page 4: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/4.jpg)
Latency vs Throughput
http://wso2.com/library/articles/2012/03/importance-performance-wso2-esb-handles-nonobvious/
![Page 5: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/5.jpg)
WSO2 CEP
![Page 6: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/6.jpg)
Data structures
o Linked List vs Array List vs Map
CEP window processor :
o Average temperature of last 10 min. o Inmemory
o Average temperature of last 24 hour.o Disc Based implementation
![Page 7: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/7.jpg)
Sactificing Accuracy for Performance (Bloom filters)
http://lkozma.net/blog/sketching-data-structures/
Initial configuration : 0000000000Insert("Sally") : 0100000001 # h1("Sally") = 2, h2("Sally") = 10Insert("Jane") : 1110000001 # h1("Jane") = 1, h2("Jane") = 3Insert("Mary") : 1110100001 # h1("Mary") = 5, h2("Mary") = 2 [collision]
Query("Sally") # bits 2 and 10 are set, # return HITQuery("John") # h1("John") = 10 set, but h2("John") = 4 not set # return MISSQuery("Bob") # h1("Bob") = 5 set, h2("Bob") = 1 set # return HIT (false positive)
![Page 8: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/8.jpg)
Syncing Nodes (Merkle tree)
Working with untrusted peer to peer networks (Torrents, Cassandra ring)
hash 0 = hash( hash 0-0 + hash 0-1 )
http://en.wikipedia.org/wiki/Merkle_tree
![Page 9: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/9.jpg)
Improve performance by Multi threading
Use correct number of threads too much or too little is not good.
Scaling Vertical : Improve Throughput reduce LatencyHorizontal : Improve both Throughput & Latency
Commonly used data structure : ArrayBlockingQueue
![Page 10: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/10.jpg)
Removing contingency (disruptor)
https://lmax-exchange.github.io/disruptor/
![Page 11: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/11.jpg)
Removing contingency (disruptor)
o Minimise thread contentiono Maximise the use of single threaded algorithms and data
structures.
https://lmax-exchange.github.io/disruptor/
![Page 12: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/12.jpg)
Optimising Performance over Network
![Page 13: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/13.jpg)
Blocking vs Non Blocking
o Handling of thousands of connections simultaneously (significant number of connections may be in idle state as well)
o Handling high latency connectionso Request/response handling needs to be
decoupledo Minimize latency, maximize throughput and
avoiding unnecessary CPU cycles
![Page 14: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/14.jpg)
WSO2 ESB
Routing: Header based, content based, rule-based and priority-based routing
Mediation: EIPs (including scatter/gather, message filters, recipient list, dead-letter channels, guaranteed delivery and message enrichment), database integration, event publishing, logging & auditing, validation
Transformation: XSLT 1.0/2.0, XPath, XQuery, Smooks
![Page 15: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/15.jpg)
Reactor Pattern
Uses Readiness selection: It is the ability to choose a socket that will not block when read or written. This is a very efficient way of handling thousands of concurrent clients and scales well.
http://wso2.com/library/articles/2013/12/demystifying-wso2-esb-pass-through-transport-part-i/
![Page 16: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/16.jpg)
Message Flow
![Page 17: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/17.jpg)
NHTTP
![Page 18: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/18.jpg)
Message Relay
Why we need to read the whole message if the necessary info is only at the message headers?
Read only the headers and stream the message as bytes !
![Page 19: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/19.jpg)
Passthrough Transport
Going beyond design patterns & conventions
![Page 20: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/20.jpg)
Performance Comparison
![Page 21: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/21.jpg)
XML vs JSON & SOAP vs REST
XML vs JSONo Serialization and Deserialization Performanceo Data Storageo Data Transfero Interoperability SOAP vs RESTo REST has minimal overhead on top of HTTPo REST easy to scale don't have any server side sessionso REST Provides easier interface for cliento SOAP is better when it comes to security o SOAP has better tools for schema, code generation and
validation
![Page 22: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/22.jpg)
Binary Transports (Thrift, Protocol buffers)
http://ganges.usc.edu/pgroupW/images/a/a9/Serializarion_Framework.pdf
![Page 23: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/23.jpg)
Binary Transports (Thrift, Protocol buffers)
http://ganges.usc.edu/pgroupW/images/a/a9/Serializarion_Framework.pdf
![Page 24: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/24.jpg)
Monitor all times (Java Flight Recorder)
![Page 25: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/25.jpg)
Monitor all times (Java Flight Recorder)
![Page 26: Make it fast for everyone - performance and middleware design](https://reader031.vdocuments.site/reader031/viewer/2022032114/55ab797e1a28abb6418b4619/html5/thumbnails/26.jpg)
Monitor all times (Java Flight Recorder)