fineo technical overview - nextsql for iot
TRANSCRIPT
FineoTurn-key Big Datafor Enterprise IoT
Fineo - Data Management Platform for IoT
NoSQL time-series database meets analytics warehouse
with native Business Intelligence integration and one-click ETL
Insights
10x lower cost100x more data
Scale
Grow easilyIterate faster
Faster adoptionLower complexity
Integration
“Simple” Big Data Deployment
Web scale architecture is hard. Supporting data scientists is even harder. Security/compliance is terrible. ETL is the worst. Integration is horrendous.
Web
ser
ver t
ier
UserC
achi
ng
ETL
AnalystData ScientistSecurity/AuditingStreaming
Data Warehouse
Hadoop
NoSQL SparkHive
Buffer
Realtime
● Growing time-series (event) data, but RDBMS does not scale○ Need low-latency access to recent data
● Data warehousing overhead (real-time database to analytics store)○ Increasingly complex with new/evolving data sources○ Many places to find “answers”
● Big Data implementation○ Hard to find people to manage & implement○ Fragmented, weak security story
● NoSQL integration○ NoSQL != no schema, but missing tools to leverage flexibility○ Does not play nice with rest of the enterprise
Enterprise Data Challenges
Solution
Painless Petabytes
BI IntegrationOne-ClickETL Tools
Instant Data Science
Built in Security & Compliance
A data platform that actively helps you
So you can focus on turning data into information
Using Fineo
● Simple integration○ Dashboard for viz. management○ Intuitive REST + SDKs○ Handle any shape of data○ SQL + BI integration (JDBC/ODBC)
● Seamless Scalability○ Transparently handle traffic spikes○ Dynamic query optimization
● Built-in Security○ SSL○ Strong Authentication + Authorization○ Audit in-platform with SQL
How does it work?
Fineo Internals
● Service Layer○ SSL + Authentication + Authorization○ Native native BI integration
● Next-Generation Compute Layer○ Infinitely scalable writes to cloud-scale buffer○ Staged pipelines for scale, flexibility and safety○ Dynamic query planning, optimization & execution
● Smart Storage Layer○ Flexible schema evolution○ Row-based database for real-time data access○ Columnar storage for analytics/data science
Built for the cloud, by Big Data experts
Fineo Internals
Web Servers
Execution Servers
Execution Tree
NoSQL
Next-Gen Compute
Smart Storage
SecureService
S3
ArchiveErrors Query
Stage Backup
Batch Process
Schema Store
Stream Processing
Cloud-Scale Buffer
“What I would want to build, given the time - it’s the ‘right thing’”
- Sean-Michael Lewis, CTO Cortex Intel
Architecture Advantages
● Seamless scalability● Native BI connectors make integration easy● Best resources selected for each query● Integrated schema for flexibility● Backups for every stage - at least 10 copies of data!
Fineo Internals - Stream Processing
Cloud BufferStream
Processors
Raw
Staged
DeviceEvents
Schema Application
Schema Store
Storage
NoSQL
S3
ArchiveErrors QuerySmart
Storage
Stream Architecture Advantages
● Highly modular● Shadow validation on real data● NoSQL store ready for immediate query● Multi-tenant aware, single tenant capable● No-Ops => low downtime● Leveraged for auditing
Flexible Schema Evolution
● Store data before creating schema○ Define the data model once you know what you need
● One-click mapping of fields to a single type○ New fields do not impact existing queries or require source systems to change
● Flexible event mapping○ Users only need to specify:
■ Target table■ Timestamp (human or machine-readable)
Temperature Temp tmp
temp
SELECT temp FROM devices
Flexible Schema Advantages
● Makes NoSQL viable for average user● Existing queries don’t change when adding schema● Resilient to ‘fat finger’ mistakes● Easy evolution of existing data● Only add schema when you need it● ETL is a click, not a job
Integrated Security
● Supports single tenant deployment● Natively multi-tenant
○ Segregated columnar storage○ Push-down filters per-tenant○ Tenant validation at all entry points
● All usage tracked in platform○ User events generate data points○ Use SQL to audit your usage
● API Keys, User/Device ACLs○ Strong authentication & authorization
What is Fineo?
● Simple, easy to use APIs● Scalable read/write● Integrated schema management● Multi-tenant, time-series data organization● Transparent, varied types of data stores● Logical recombination of data + schema● Distributed query optimization & execution
Why Fineo?
Rapid deployment, integration and evolution
Big Data expertise at startup prices
Integrated security & audit tools
10x faster on 100x more data
Want to hear more?Let’s chat!