getting started with azure event hubs and stream analytics services
TRANSCRIPT
Getting started with Azure Event Hubs and Stream
Analytic ServicesVladimir Bychkov
Who We Are
CUSTOM SOFTWARE DEVELOPERS
200 employees (180 engineers) Certified Amazon - AWS Technology PartnerScientists, mathematicians, & engineers Certified Azure/Microsoft Partner of the Year 2014Open Architecture Washington, DC Headquarters
Business Intelligence & Big Data
Cloud Solutions, IoT & DevOps
Identity & Access Management
System Integration & CRM Applications
Content Management & Enterprise Portals
Custom Mobile Solutions
Join our Team www.eastbanctech.com/careers/vacancies
Processing streaming data – use cases
• Automotive - fleet tracking, operations telemetry • Medical - vital signs, fitness and health monitoring • Meteorology - weather and environmental data • Software - applications performance and instrumentation
data • Home automation – energy efficiency, home security• Agriculture/farming – crops and cattle monitoring
Connected cow
End-to-end stream processing architecture
Producing sensor data with Raspberry Pi
Azure Service Bus
Source: https://channel9.msdn.com/Events/TechEd/Europe/2014/CDP-B307
Azure Event Hub
Cloud Services
Storage & Analytics
Custom Code & 3rd Party ServicesWeb/Mobile User InterfacesIntegration Services
Event Hub
- Hyper Scale -- Fully Managed -- Interoperable -
- Secure -- Cost Effective -
Azure Event Hub (cont.)
DEMO
• Create Azure Event Hub• Read events using console windows app• Send events using Python script on Raspberry Pi
Azure Stream Analytics
• Managed service in Azure Cloud• Build for hyper-scale• Cost effective• Real-time data processing• Developer productivity
Azure Stream Analytics: Basic Job Topology
Inputs
• Data Stream• Reference data• Supported data formats:
• JSON• CSV• Apache Avro (binary JSON)
Outputs
• Azure Tables or Blob• SQL Database• Event Hub• Service Bus Queue/Topic• PowerBI dataset• DocumentDB
Query - SQL like language with built in temporal semantics
Azure Stream Analytics: Grouping data - Windowing
Tumbling window
Aggregate per time interval
Hopping window
Schedule overlapping windows
Sliding window
Window constantly re-evaluated
DEMO
• Create Azure Stream Analytics job• Add input/output to job topology• Write pass-through query to archive events to Azure
Table• Write aggregate query to save temp averages only to
Azure Table• Web-dashboard demo
ANATOMY OF A H4CK3R ATTACK
DEMO
• Review CurcuitBreaker app• Review CurcuitBreaker ASA query• Run real-time Dashboard• Hack into Steel Mountain!
Q&A
THANK YOU
Vladimir BychkovTechnical Team [email protected]