real time analytics with spring
DESCRIPTION
Speakers: Luke Taylor and David Turanski Today's solutions must provide the ability to interpret related events and understand trends that are happening right now. This session will cover some of the out of the box capabilities of Spring XD to tap into big data streams and generate metrics such as simple counters, aggregate counters, moving averages, rates of change, and histograms. Hands-on demos will show you how Spring XD uses Redis and GemFire's Continuous Query and Function Execution to incorporate real-time analytics into event-driven applications.TRANSCRIPT
Real Time Analytics WithSpring
David TuranskiLuke Taylor
Real Time AnalyticsWhat's happening right now?
Event drivenNotify humans or systems in real timeSimple counters and metricsAggregate CountersGauges, Moving AverageComplex Event Processing
Spring XD OverviewOne stop shop for:
Data IngestionRealtime AnalyticsWorkflow OrchestrationData ExportBuilt on existing Spring Assets
Spring Integration, Batch, Data
XD='extreme data'
Spring XDOpen source Apache licenseOut of the box executable servers
not a bunch of jars
Pluggable Module systemConfiguration via High level DSL, CLI, andREST services
Spring XD DSLBased on Unix pipes and filters syntax
http | filehttp --port=8080 | file --rootDir=/data/
'http' and 'file' are Module names
Spring XD Streams
Anatomy of a Stream
XD Single Node
Spring XD TapsA Tap is a stream whose source is a point in anexisting streamTaps are primarily used for real time analysisExample:twittersearch --query=Bieber | file
tap:twittersearch > field-value-counter --
fieldName=entities.hashtags.text
Spring XD MetricsSpring XD provides persistent and inmemoryMetrics RepositoriesMetrics are modules
commonly act as sinks for taps
XD includes the following MetricsCounterField Value CounterAggregate CounterGaugeRich Gauge
See fordetails.
Spring XD Analytics Documentation
GemFire FunctionExecution
Efficient way to process large data volumes"where the data lives"Java code deployed to GemFire server nodesThe Function runs on local cache node andtransports results over the network to theprocess that executed the function (Execution)Function Execution aggregates results using aResultCollector
GemFire Continuous QueryGood for asynchronous notification of dataupdatesClient subscribes to a data query and binds it toa QueryListener callbackGemFire invokes the QueryListener wheneverthe query's result set changes state
The DemoPerform real time analytics on the Twitter streamCreate a Twitter ingest stream with XDCreate taps on the Twitter stream to feed XDMetricsCreate a tap to pull selected data into GemFirefor deeper analysis
Show how to build and deploy a custom processor
Execute a GemFire Function to perform acomplex analysis of hash tagsDashboard web app to view analytics
Questions?
© 2013 SpringOne 2GX Do not distribute without permission
Learn More. StayConnected
Project SitesSpring XD Spring Data GemFire
projects.spring.io/springxd
projects.spring.io/springdatagemfireRelated sessionsThu 12:45 Real Time Event Processing andDecision Making Lyndon AdamsXD and Pivotal HD Demo Demo lounge