c* summit eu 2013: capitalizing on data in telecommunications: the cassandra way
DESCRIPTION
Speakers: Rokesh Jankie, CTO at QAFE Inc. & Hallo Khaznadar CAO at QAFE Inc. Video: http://www.youtube.com/watch?v=PNLF5vNv5Qg&list=PLqcm6qE9lgKLoYaakl3YwIWP4hmGsHm5e&index=3 Telecommunications mediation is the process of handling Call Data Records (CDR's) coming through specialized hardware switches that are connected to the infrastructure of the telecommunications company. The standard industry requirements of these processes are the archiving of original raw data for auditing purposes, decoding and persisting of the extracted Call Data Records for applying rating, billing, reporting and feeding processed data to other sub systems specific to each telecoms platform. The number of CDR's and their size has exploded since mobility got an enormous boost in recent years. A new and non-traditional approach is needed to achieve these requirements. By combining QAFE-based architecture platform with the hardware power of the cutting edge Oracle Exalogic and the scaleable, flexible and highly performing DataStax enterprise, we achieved astonishing results. The solution not only allowed us to achieve industry standard requirements but also opened up the door for new requirements, realizing crucial functionality for the industry. Specifically the ability to perform analytics, reporting and KPI's in a time period spanning several years. In the presentation, there will be focus on the combination of a private cloud solution (public clouds are not an option here due to legislation) and DataStax enterprise. A working benchmark can show performance numbers of the real time processing and the storage of big amounts of CDR's from several sources.TRANSCRIPT
Rokesh Jankie |CTO, QAFE Inc. , [email protected] Hallo Khaznadar |CAO, QAFE Inc. , [email protected]
Capitalizing on Data in Telecommunications: The Cassandra way
#CASSANDRAEU CASSANDRASUMMITEU@QAFE www.qafe.com
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
*Introduction
*Business Case
*Solution Direction
*The Cassandra Way
*Improvements and Future Directions
*ConclusionsThe name of the Telecommunications Company cannot be mentioned due to NDA.
Agenda
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
IntroductionQAFE & Qualogy
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
*Qualogy is Oracle Platinum Partner
*Qualogy was founded in 1998
*QAFE is a Qualogy subsidiary founded in 2007
*QAFE is a development Platform focussed on
- Integration
- Front-end technologies
- Oracle Forms Modernization
Qualogy and QAFE
Visit www.qualogy.com and www.qafe.com for more information
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Business CaseCustomer driven situation
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Telecommunications and core Business*Billing is core business
*Input for billing are “CDR”s
*Call Data Records (CDR) contain information on calls made (besides Voice, also for Data).
*The process of handling CDRs is called Mediation
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Mediation!
!
!
“Telecom Mediation is a system which acts as an interface or a middle layer between network elements and billing system/down stream applications like Fraud Management Systems, Revenue Assurance, Data Ware house etc.”
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Call Data Records*Processing and parsing
*Dependent on switches and configuration
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Solution DirectionSetting up a scalable architecture
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Complexity and setup with Cassandra
CS#
R7#
R8#
R9#
Rx#
MEDIATION# Target#Sys.#
BSS#
DWH#
ICT#
ERS#
SWITCHES(
CS#
R7#
R8#
R9#
Rx#
Decoding# Media3on#Rules# Adap3ng#
MEDIATION# Target#Sys.#
BSS#
DWH#
ICT#
ERS#
SWITCHES(
CS#
R7#
R8#
R9#
Rx#
Decoder# Filtering# Enrichment# Adapter#
Decoding# Media<on#Rules# Adap<ng#
MEDIATION# Target#Sys.#
BSS#
DWH#
ICT#
ERS#
SWITCHES(
CS#
R7#
R8#
R9#
Rx#
Decoder# Filtering# Enrichment# Adapter#
Decoding# Media<on#Rules# Adap<ng#
MEDIATION# Target#Sys.#
BSS#
DWH#
ICT#
ERS#
SWITCHES(
CS#
R7#
R8#
R9#
Rx#
Decoder#
Decoder#
Filtering# Enrichment# Adapter#
Decoding# Media<on#Rules# Adap<ng#
MEDIATION#
Filtering# Enrichment# Adapter#
Target#Sys.#
BSS#
DWH#
ICT#
ERS#
SWITCHES(
CS#
R7#
R8#
R9#
Rx#
Decoder#
Decoder#
Decoder#
Filtering# Enrichment# Adapter#
Decoding# Media<on#Rules# Adap<ng#
MEDIATION#
Filtering# Enrichment# Adapter#
Filtering# Enrichment# Adapter#
Target#Sys.#
BSS#
DWH#
ICT#
ERS#
SWITCHES(
CS#
R7#
R8#
R9#
Rx#
Decoder#
Decoder#
Decoder#
Filtering# Enrichment# Adapter#
Decoding# Media<on#Rules# Adap<ng#
MEDIATION#
Filtering# Enrichment# Adapter#
Filtering# Enrichment# Adapter#
Decoder# Filtering# Enrichment# Adapter#
Target#Sys.#
BSS#
DWH#
ICT#
ERS#
SWITCHES(
IMS$
CS$
R7$
R8$
R9$
R7$
Rx$
Rx$
Decoder$
Decoder$
Decoder$
Filtering$ Enrichment$ Adapter$
Decoding$ Media=on$Rules$ Adap=ng$
MEDIATION$
Filtering$ Enrichment$ Adapter$
Filtering$ Enrichment$ Adapter$
Decoder$ Filtering$ Enrichment$ Adapter$
Target$Sys.$
BSS$
DWH$
ICT$
ERS$
SWITCHES(
IMS$
EPC$
CS$
R7$
R8$
R9$
R7$
Rx$
Rx$
R7$
Rx$
X$ Rx$
Decoder$
Decoder$
Decoder$
Filtering$ Enrichment$ Adapter$
Decoding$ Media?on$Rules$ Adap?ng$
MEDIATION$
Filtering$ Enrichment$ Adapter$
Filtering$ Enrichment$ Adapter$
Decoder$ Filtering$ Enrichment$ Adapter$
Target$Sys.$
BSS$
DWH$
ICT$
ERS$
SWITCHES(
IMS$
EPC$
CS$
R7$
R8$
R9$
R7$
Rx$
Rx$
R7$
Rx$
X$ Rx$
Decoder$
Decoder$
Decoder$
Filtering$ Enrichment$ Adapter$
Decoding$ Media?on$Rules$ Adap?ng$
MEDIATION$
Filtering$ Enrichment$ Adapter$
Filtering$ Enrichment$ Adapter$
Decoder$ Filtering$ Enrichment$ Adapter$
Target$Sys.$
BSS$
DWH$
ICT$
ERS$
SWITCHES(
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Think about hardware
*Next Generation Data Center Appliance
*Scalable
*Hardware and Software Engineered to work together
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Solution Direction: Data modelSetting up a scalable architecture
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
DataModel AbstractCREATE TABLE cdr_<recordType>_<mainIdentifier>By<subIdentifier1>And<subIdentifier2>...And<subIdentifierN>_Index ( mainIdentifier <anyType>, (callingNumber text,) date text, subIdentifier1 <anyType>, (answerTime timestamp,) subIdentifier2 <anyType>, (numberingPlan text,) . . subIdentifierN <anyType>, (natureOfAddress text,) recordId uuid PRIMARY KEY ((mainIdentifier, date), subIdentifier1, subIdentifier2, subIdentifierN) };!!INSERT INTO cdr_<recordType>_<mainIdentifier>By<subIdentifier1>And<subIdentifier2>...And<subIdentifierN>_Index(mainIdentifier,date,subIdentifier1,...,subIdentifierN, recordId) VALUES('59995611994','2013-10-01','subIdentifier1Value' ...,'subIdentifierNValue','c4e1ee6f-e053-41f5-9890-6746e6d51095')
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
DataModel: IndexesCREATE TABLE cdr_MoCallRecord_CallingNumberByAnswerTime_Index ( callingNumber text, date text, answerTime timestamp, recordId uuid PRIMARY KEY ((mainIdentifier, date), subIdentifier1, subIdentifier2, subIdentifierN) }; !
!INSERT INTO cdr_MoCallRecord_CallingNumberByAnswerTime_Index(mainIdentifier,date,answerTime,recordId) VALUES('59995611994','2013-10-01','13:43:40','c4e1ee6f-e053-41f5-9890-6746e6d51095') !SELECT recordId FROM cdr_MoCallRecord_CallingNumberByAnswerTime_Index WHERE callingNumber='59995611994' AND date='2013-10-01' AND answerTime > '13:00:00' AND answerTime < '14:00:00' !SELECT * FROM cdr_MoCallRecord IN (<list of recordId's>)
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
DataModel Visual/Implementation
UUID$ Da_P2$Da_P1$Id_Pn$Id_P1$ Da_Pn$
Actual'Data'
UUID$ Da_P2$Da_P1$Id_Pn$Id_P1$ Da_Pn$
uuid_v1$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$
Actual'Data'
UUID$ Da_P2$Da_P1$Id_Pn$Id_P1$ Da_Pn$
uuid_v1$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$uuid_v2$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$
Actual'Data'
UUID$ Da_P2$Da_P1$Id_Pn$Id_P1$ Da_Pn$
uuid_v1$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$uuid_v2$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$
uuid_vn$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$
Actual'Data'
UUID$ Da_P2$Da_P1$Id_Pn$Id_P1$ Da_Pn$
uuid_v1$Id_P2$Id_P1$ Id_P3_v1$
uuid_v1$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$uuid_v2$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$
uuid_vn$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$
uuid_v2$Id_P3_v2$
uuid_vn$Id_P3_vn$
Index&A&
Actual&Data&
UUID$ Da_P2$Da_P1$Id_Pn$Id_P1$ Da_Pn$
uuid_v1$Id_P2$Id_P1$ Id_P3_v1$
uuid_v1$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$uuid_v2$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$
uuid_vn$ Da_P2_v$Da_P1_v$Id_Pn_v$Id_P1_v$ Da_Pn_v$
uuid_v2$Id_P3_v2$
uuid_vn$Id_P3_vn$
Index&A&
Actual&Data&
uuid_v1$Id_P4$Id_P3$ Id_P5_v1$uuid_v2$Id_P5_v2$
uuid_vn$Id_P5_vn$
Index&Z&
UUID$ callType$dura/on$StartTime$Date$
Actual'Data'
CallingNm$UUID$ callType$dura/on$StartTime$Date$
73s7jabe$ Interna/onal$00:03:10$13:07:34$2013@01@03$
Actual'Data'
98asd23h$ local$00:01:23$22:18:09$2013@01@03$
kjbnfd72$ 00:12:02$08:10:03$2013@03@07$
CallingNm$
065782391$065782391$
069035802$ local$
UUID$ callType$dura/on$StartTime$Date$
73s7jabe$ Interna/onal$00:03:10$13:07:34$2013@01@03$
Index&CallingNm_By_Date_And_StartTime&
Actual&Data&
98asd23h$ local$00:01:23$22:18:09$2013@01@03$
kjbnfd72$ 00:12:02$08:10:03$2013@03@07$
CallingNm$
065782391$065782391$
069035802$ local$
UUID$ callType$dura/on$StartTime$Date$
73s7jabe$2013>01>03$065782391$ 13:07:34$
73s7jabe$ Interna/onal$00:03:10$13:07:34$2013>01>03$
Index&CallingNm_By_Date_And_StartTime&
Actual&Data&
98asd23h$ local$00:01:23$22:18:09$2013>01>03$
kjbnfd72$ 00:12:02$08:10:03$2013>03>07$
CallingNm$
065782391$065782391$
069035802$
22:18:09$98asd23h$
local$
UUID$ callType$dura/on$StartTime$Date$
73s7jabe$2013>01>03$065782391$ 13:07:34$
73s7jabe$ Interna/onal$00:03:10$13:07:34$2013>01>03$
Index&CallingNm_By_Date_And_StartTime&
Actual&Data&
98asd23h$ local$00:01:23$22:18:09$2013>01>03$
kjbnfd72$ 00:12:02$08:10:03$2013>03>07$
CallingNm$
065782391$065782391$
069035802$
22:18:09$98asd23h$
kjbnfd72$2013>03>07$069035802$ 08:10:03$
local$
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Specifying Cassandra ClusterImplementation of scaling plan
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Considerations*Datasize per month
*Time window for scaling up
*Replication Factor
*Decoding Factor
*Node size
*Cassandra Cluster size
!
* Technical note: Cassandra is not certified for Oracle Linux (yet)
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Planning on Data growth*Number of Nodes = 4
*HDD per node = 8TB
*RAM = minimum of 32 GB
*CPUs = 4 cores (at least 16 GB ram per core, so increasing this one requires increasing in RAM).
*Every 6 months we’ll have to add 4 nodes!
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
The Cassandra wayWhy choose Cassandra
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Requirements *Large Scale Storage
*Predictable Scalability
*High Availability
*Robustness of Mission Critical Application
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
What we found in Cassandra*Meets all our requirements!
*No compromise on performance (during data write)
*No single point of failure
*Easy to adopt TimeSeries
*Proven Technology
*Trusted High End Specialists you can count on!
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Dramatic Performance *When Hardware and software were combined!
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Improvements and Future DirectionsThings that can make a difference
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Using the ExaBus Optimization*ExaBus built on top of Oracle Infiniband
*Extreme Performance and Resource Efficient
*Socket Direct Protocol rather than TCP-IP
*APIs are available
!
*QAFE created solution to make this work in SDP and non-SDP environments.
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Conclusions
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
Creating more Business Value from Data*Create more valuable customer relations
*Reduction of Churn
*Tuned offerings for customers
*Targeted marketing
*Advanced network analyzing
#CASSANDRAEU CASSANDRASUMMITEU@QAFE
*Business case is real!
*Complexity that results in architecture choices
As well as Hardware as Software
*Very Promising (technology and business-wise)
*Room for improvement and more efficiency
What we discussed today...
Rokesh Jankie |CTO, QAFE Inc. , [email protected] Hallo Khaznadar |CAO, QAFE Inc. , [email protected]
Capitalizing on Data in Telecommunications: The Cassandra way
#CASSANDRAEU CASSANDRASUMMITEU@QAFE www.qafe.com