![Page 1: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/1.jpg)
Data Analytics using Mobile, 3Scale and radanalytics.ioPutting it all together in A Bike Sharing StoryJuana NakfourSenior Technical Account Manager
Vinay BhaleraoSenior Solution Architect
Michael McCunePrincipal Software Engineer
May 10th, 2018
![Page 2: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/2.jpg)
Who are we ?
Juana NakfourSenior Technical Account Manager
Software EngineerRed Hat Inc.
Michael McCunePrincipal Software Engineer
Red Hat Inc.
Data processing on OpenShift and OpenStack
I have a curiously strong love of the Grateful Dead
Vinay BhaleraoSenior Solution Architect
Red Hat Inc.
I love APIs and Nutella! Always Inventing
![Page 3: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/3.jpg)
AgendaStory of Renting Bikes
Putting it All Together- Architecture
Our API gateway using 3Scale
Data Analysis and Machine Learning using radanalytics.io
Actually Renting Bikes in a Demo
![Page 4: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/4.jpg)
Thoughts
![Page 5: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/5.jpg)
Story of Renting Bikes - Last Mile
“Dockless”
Where are the bikes?
Where should we stock bikes?
How are users using our mobile app?
How do we get analytics data?
How do we integrate backend network components?
What are the bike renting location trends?
How do we collect and organize our data?
Which platform component can provide data?
Analytics on 3rd party app providers using our API?
![Page 6: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/6.jpg)
Story of Renting Bikes - Last Mile
![Page 7: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/7.jpg)
Putting it All Together Architecture
radanalytics.io
REST API
REST API
![Page 8: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/8.jpg)
3Scale
![Page 9: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/9.jpg)
Red Hat 3scale API Management Overview
Admin Portal
● Dashboard● Developer /
Application / Key Management
● CMS● Analytics● Billing
API Consumers
(App Developers)Authorize & Report Traffic
Developer Apps
Developer Portal
● API Provider Branded ● API Description● Signup● ActiveDocs (OAS)
API BackendAPI Gateway
API Manager
API Provider
(Line Of Business / Product Manager,
Developers, Writers, Ops)
API Request Authorized API Request
![Page 10: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/10.jpg)
SECURITY & ACCESS CONTROL
How do you manage who gets access to your API? Can you establish different levels of access for different types of users? Can you control how different applications interact with your API?Access control features are essential to making sure you determine exactly who uses your API, how it is used and how much they can use it. We make it easy to centrally set up and manage policy and application plans for all your APIs on one platform
It goes without saying that if you’re planning to open an API, security needs to be carefully considered from the start. Whether your API is public, private or internal, with 3scale you can choose the authentication type most appropriate to your needs. We offer a range of authentication patterns and credentials to choose from, including unique API keys, and OAuth tokens.
Easy the consumption of APIs without losing control
![Page 11: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/11.jpg)
YOUR API SECURITY
Multiple authentication mechanisms
Authenticate traffic
Restrict by policy
Drop unwelcome calls
Protect backend services
Generate overage alerts
Impose rate limits
– API Key – App ID / App Key
Authenticate and restrict access to your APIs. Protect backend services.
– OpenID Connect
![Page 12: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/12.jpg)
REPORTS & ANALYTICSTrack and monitor usage. Get reports by API, app, method and metric.
Gain and share API program insights.
Monitor and set alerts on traffic flow. Provide partners and developers with reports on their traffic with a user dashboard designed for them. Analyze your API traffic through detailed traffic analytics by account, application or service and share performance insights across the organization with crisp clear reporting.
High-level data at your fingertips
The Dashboard part of the Admin Portal gives you quick, centrally located visibility into any traffic and customer engagement opportunities or issues with your APIs. It is available now on all 3scale API Management plans from free through enterprise.
![Page 13: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/13.jpg)
The API Gateway is responsible for enforcing the API policies that are defined in the API Manager Admin Portal.
The API Gateway consults with the API Manager on incoming calls, and enforces the policies, either returning an error or proxying the API call to the customer’s API backend.
API GATEWAYGateway Layer Policy Enforcer
![Page 14: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/14.jpg)
Our API gateway using 3-Scale- Demo
![Page 15: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/15.jpg)
radanalytics.io
![Page 16: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/16.jpg)
What is radanalytics.io?
An open source community working to empower intelligent application lifecycles on OpenShift
A collection of projects to enable analytics and machine learning frameworks on OpenShift
radanalytics.io
![Page 17: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/17.jpg)
Basic architecture radanalytics.io
![Page 18: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/18.jpg)
Apache Spark radanalytics.io
![Page 19: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/19.jpg)
Project Oshinko
Deploy and manage Apache Spark clusters on OpenShift through browser and command line tooling
Utilize source-to-image based repositories to automatically deploy ephemeral Apache Spark clusters
alongside your applications
radanalytics.io
![Page 20: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/20.jpg)
Oshinko WebUI radanalytics.io
![Page 21: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/21.jpg)
Oshinko WebUI radanalytics.io
![Page 22: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/22.jpg)
Oshinko Command Line radanalytics.io
![Page 23: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/23.jpg)
Oshinko Source-to-Image radanalytics.io
![Page 24: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/24.jpg)
Source-to-Image language support radanalytics.io
![Page 25: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/25.jpg)
Learn more!
radanalytics.io
![Page 26: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/26.jpg)
Data Analysis and Machine Learning using radanalytics.io - Demo
Radanalytics
REST API
REST API
radanalytics.io
![Page 27: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/27.jpg)
Data Analysis and Machine Learning using radanalytics.io - Demo
oc new-app --template=oshinko-pyspark-build-dc -p APPLICATION_NAME=python-analytics -p GIT_URI=https://github.com/nakfour/mobile-analytics.git -p SPARK_OPTIONS='--packages org.mongodb.spark:mongo-spark-connector_2.11:2.2.0' OSHINKO_CLUSTER_NAME=test
Deployment Config
Creating Pod
radanalytics.io
![Page 28: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/28.jpg)
Data Analysis and Machine Learning using radanalytics.io - Demo
Creating Spark Session
spark = SparkSession.builder.appName("mobileanalytics").config("spark.mongodb.input.uri", "mongodb://admin:<pass>@mongodb/sampledb.bikerental").config("spark.mongodb.output.uri",
"mongodb://admin:<pass>@mongodb/sampledb.bikerental").getOrCreate()
radanalytics.io
![Page 29: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/29.jpg)
Data Analysis and Machine Learning using radanalytics.io - Demo
Creating DataFrames
bikerentaldf = spark.read.format("com.mongodb.spark.sql.DefaultSource").load()
radanalytics.io
![Page 30: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/30.jpg)
Data Analysis and Machine Learning using radanalytics.io - Demo
Creating Linear Regression Model assembler=VectorAssembler(inputCols=['startstationid', 'daypartInt', 'startstationlat', 'startstationlon'], outputCol="features")
radanalytics.io
output = assembler.transform(bikrentaldf)lr = LinearRegression(maxIter=10, regParam=0.3, elasticNetParam=0.8,featuresCol=assembler.getOutputCol(),labelCol="rentalcount")
pipelineLG= Pipeline(stages=[assembler,lr])
modelLR = pipelineLG.fit(bikerentaldf)
https://spark.apache.org/docs/2.1.0/ml-classification-regression.html#linear-regression
![Page 31: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/31.jpg)
Data Analysis and Machine Learning using radanalytics.io - Demo
Applying model for Predictions
predictions = modelLR.transform(testbikerentaldf)
radanalytics.io
![Page 32: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/32.jpg)
Demo
![Page 33: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/33.jpg)
Actually Renting Bikes in a Demo
![Page 34: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/34.jpg)
Lessons Learned
![Page 35: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/35.jpg)
References
● radanalytics.io● 3scale.net● openshift.com
● https://rampages.us/pedal2play/2016/10/13/post-4/● https://timesofsandiego.com/business/2018/03/03/dockless-b
icycles-pop-up-downtown-offering-transportation-flexibility/● https://blog.producthunt.com/we-tried-every-shared-bike-and-
scooter-in-san-francisco-bb766abd0a96● https://mashable.com/2017/01/18/bike-sharing-pile-up-china/#
whSoCvTV1PqA● https://www.theguardian.com/cities/2017/mar/22/bike-wars-d
ockless-china-millions-bicycles-hangzhou
Images
● https://www.citibikenyc.com/system-data
Data Source
![Page 36: and radanalytics.io Data Analytics using Mobile, 3Scale · 2018-05-10 · Track and monitor usage. Get reports by API, app, method and metric. ... Deploy and manage Apache Spark clusters](https://reader033.vdocuments.site/reader033/viewer/2022042910/5f411c5bca8d4c305805fd6f/html5/thumbnails/36.jpg)
THANK YOUplus.google.com/+RedHat
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHat