understanding business apis through statistics
TRANSCRIPT
![Page 1: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/1.jpg)
Understanding Business APIs through Statistics
Amila De SilvaDinusha Senanayaka
![Page 2: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/2.jpg)
About WSO2
• Providing the only complete open source componentized cloud platform
– Dedicated to removing all the stumbling blocks to enterprise agility– Enabling you to focus on business logic and business value
• Recognized by leading analyst firms as visionaries and leaders– Gartner cites WSO2 as visionaries in all 3 categories of
application infrastructure– Forrester places WSO2 in top 2 for API Management
• Global corporation with offices in USA, UK & Sri Lanka– 200+ employees and growing
• Business model of selling comprehensive support & maintenance for our products
![Page 3: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/3.jpg)
150+ globally positioned support customers
![Page 4: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/4.jpg)
Overview
● Role of Statistics in API Management○ Statistics to understand user communities○ As tool for making strategic decisions.
● Different Options for analysing data○ Phases in Data analysis○ Offline vs Real-time analysis
● BAM as an offline data analytic engine○ How large event streams gets summarized
● CEP for real time processing○ Architecture of CEP○ Stages of an execution plan
● GA as a feedback channel for app developers● Options for visualising data.
○ Generating reports out of BAM summarized data.
![Page 5: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/5.jpg)
APIs - the way to go
● APIs are no longer a luxury - they are becoming a necessity.
● API - A common interface for disparate platforms
● More open you are more users you will attract.
● Build an ecosystem around your APIs.
![Page 6: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/6.jpg)
Knowing your users
![Page 7: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/7.jpg)
Why do you need statistics?
● There are many connected components● One wrong move can make disastrous
consequences.● Feedback channels are important -
○ See how users access them and learn - It explains what they say.
● Discover hidden potentials within your communities○ Going for a new billing model - See how users
access them.● Can you check them all manually?
![Page 8: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/8.jpg)
What we provide
● Two branches in analysis○ Offline Data analysis○ Realtime Analaysis
● Offline Data Analysis○ Gives you a general idea of your APIs○ Who your users are.○ Leads you to make long term decisions.
● Realtime analysis○ Identifies co-occurrence of several useful business
flows.○ Find and act on it.
![Page 9: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/9.jpg)
How we analyse it
● Done in two main phases○ Data Gathering○ Analysis.
● Gather data at product level.○ multiple event streams for request, response and faults○ each invocation as an event
● Delegating analysis to a different product○ Simply any tool capable of processing event streams can
analyse data.○ Currently BAM and CEP are used.
● Extensions to achieve product specific analysis○ In BAM this is done through a hive script○ In CEP this can be done by a transport adaptor + execution
plan.
![Page 10: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/10.jpg)
Data Gathering
● Event = Invocation● In the gateway we collect data related to
each invocation● Three different event streams
○ Request Stream○ Response Stream○ Fault stream
![Page 11: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/11.jpg)
Structure of a StreamRequest Stream
'payloadData':[ {'name':'consumerKey','type':'STRING'}, {'name':'context','type':'STRING'}, {'name':'api_version','type':'STRING'}, {'name':'api','type':'STRING'}, {'name':'resource','type':'STRING'}, {'name':'method','type':'STRING'}, {'name':'version','type':'STRING'}, {'name':'request','type':'INT'}, {'name':'requestTime','type':'LONG'}, {'name':'userId','type':'STRING'}, {'name':'tenantDomain','type':'STRING'}, {'name':'hostName','type':'STRING'}, {'name':'apiPublisher','type':'STRING'},
{'name':'applicationName','type':'STRING'}, {'name':'applicationId','type':'STRING'} ]
Response Stream
'payloadData':[ {'name':'consumerKey','type':'STRING'}, {'name':'context','type':'STRING'}, {'name':'api_version','type':'STRING'}, {'name':'api','type':'STRING'}, {'name':'resource','type':'STRING'}, {'name':'method','type':'STRING'}, {'name':'version','type':'STRING'}, {'name':'response','type':'INT'}, {'name':'responseTime','type':'LONG'}, {'name':'serviceTime','type':'LONG'}, {'name':'userId','type':'STRING'}, {'name':'tenantDomain','type':'STRING'}, {'name':'hostName','type':'STRING'}, {'name':'apiPublisher','type':'STRING'}, {'name':'applicationName','type':'STRING'}, {'name':'applicationId','type':'STRING'} ]
Fault Stream
'payloadData':[ … {'name':errorCode','type': 'STRING '}, {'name':‘errorMessage','type': 'STRING '}, .. ]
![Page 12: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/12.jpg)
Analysing Data with BAM...
![Page 13: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/13.jpg)
Analysing Data with BAM
● Used for offline Analysis.● Aggregates, stores, then analyses data.● Capable of receiving events from multiple data
agents.○ In APIM 1.5.0 we use a load balancing data publisher.
● All the incoming events are first written to a Cassandra DB
● Data analysed using a hive script running on top of a hadoop cluster.
● Summarised data written back to a Relational DB.
![Page 14: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/14.jpg)
Analysing Data with BAM...
Summarized tables
![Page 15: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/15.jpg)
Analysing Data with BAM...
Table Descriptions
![Page 16: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/16.jpg)
Analysing Data with BAM...
Visualizing summarized data
![Page 17: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/17.jpg)
Analysing with BAM...
Summarized data usages for billing
● Decide on a billing model● Integrate with existing billing engines
![Page 18: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/18.jpg)
Real-time Analysis with CEP
● APIM can publish events to multiple data agents.
● Same streams are used to publish events to CEP
● Whenever it receives a new event CEP executes a query and evaluates the event.
● Siddhi queries define the evaluation criteria.
![Page 19: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/19.jpg)
CEP Architecture
![Page 20: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/20.jpg)
CEP Architecture...
● Transport Adaptors to read/write event sources.○ We can create alternate event streams out of the
original streams.● Event builders convert events to a standard
format.● Queries defined as query plans.● Event processor executes a query plan.
○ It might generate additional events.● Event formatters convert back the output.
![Page 21: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/21.jpg)
Example
● Task : Identify client applications sending too many repeated requests○ Too Many : 5 requests ○ repeated : within 1 minute
● How to identify a client○ Consumer key
And the query is:from AMRequest#window.time(1 min)insert into outStreamcount(consumerKey) as myCount,api_version,api,consumerKeygroup by consumerKey,api_version,apihaving (myCount > 5);
![Page 22: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/22.jpg)
Flow in CEP...
● Transport adaptor reads requestStreamDefn● Creates a second input stream AMRequest
○ This only contains subset of data.● Event builder transform the input stream● Query is defined in a query plan.
○ Even processor executes each of the registered query plans.
● Upon meeting the condition an event is generated.● Event formatter converts the event.● Mail is sent through the transport adaptor.
![Page 23: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/23.jpg)
Monitoring Statistics with Google Analytics
• API Manager can be easily configured with Google Analytics
<GoogleAnalyticsTracking><Enabled>false</Enabled>
<TrackingID>UA-XXXXXXXX-X</TrackingID></GoogleAnalyticsTracking>
![Page 24: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/24.jpg)
Monitoring Statistics with Google Analytics
![Page 25: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/25.jpg)
Visualising
● Needed for interpreting statistics.● Gadgets provided in BAM● Plugin any external tool for generating reports● Combining summary tables to create new
views.
![Page 26: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/26.jpg)
Summary
• Statistics as a channel to verify business objectives.
• WSO2 API Manager provides different mechanisms to analyze data.
• WSO2 BAM can be integrated with API Manager process offline data.• Gather -> Cassandra• Analyze -> Relational DB
• WSO2 CEP can be integrated with API Manager and real time data analyzing.
• Google Analytics integration with API Manager.
![Page 27: Understanding Business APIs through statistics](https://reader034.vdocuments.site/reader034/viewer/2022042717/55d4fb7ebb61eb95528b45ba/html5/thumbnails/27.jpg)
Engage with WSO2
• Helping you get the most out of your deployments• From project evaluation and inception to development
and going into production, WSO2 is your partner in ensuring 100% project success