qo introduction v2

39
The old computing is about what computers could do. The new computing is about what people can do… Ben Shneiderman

Upload: joef

Post on 16-Jul-2015

237 views

Category:

Documents


2 download

TRANSCRIPT

The old computing is about what computers could do. The new computing is about what people can do…

Ben Shneiderman

Average company data storage triples every 18 to 24 months

• AT&T has 11,000 PB (107 TB) of wireline, wireless, and Internet dataGoogle’s Big Table is 6 PBWal-Mart DB is 500 TB and handles 107 transactions / day (2004 data)

• Sprint has 2.85 trillion rows historical data• New technical information doubles every 2 years

LHC experiment will generate 350 TB of data each week (15 PB / year)

A few facts …… … From the enterprisesFrom the enterprises

A few facts …… … From our digital livesFrom our digital lives

7.2B Web searches / month (3.9B by Google)• 161,000 PB (108 TB) of information was created or

replicated worldwide in 2006 … IDC estimates 6x growth by 2010 to 988,000 PB

• Cisco predicts that IP traffic will quintuple 2006 – 2011 to 11,000 PB / month

• 3+ B calls per day - wireline, wireless, and VoIP - are growing at 50% CAGR

• More than 6B text messages are sent every day• User Generated Content devices (cameras, phones, PCs,

video equip.) are 4+ B and will increase 50% by 2010

Increase speed of analysis and learn to efficiently solve unplanned problemsCollect and analyze data coming from many different sources

• Archive and retrieve information in business real time• Implement money and energy efficient data centers

Empower increasing numbers of individuals / organizations with actionable information through diverse channels

ConsequencesData ownersData owners willwill havehave toto:

OLTP Databases grow 2x every 5 yearsOLTP Workloads increase 4x every 3 years

• OLAP Databases grow 3x every 3 years• OLAP Workloads increase 2x every 3 years

Database perspective

… … But But ……In year 2000 10% of all worldwide data were in

relational databases …… In year 2010 this value will be 5%

• Between 2006 and 2010 number of non-relational DBs installation will increase 6x

Source: M. Brodie VLDB Conference 2007

Today the “one-size-fits-all” notion that has been the mantra of relational DBs vendors for decades has lost its gripNew needs, new data, and new volumes demand for new approaches.

The turning point

Computer science in the 20th century was about perfect solutions in closed domains and applications.

Computer science in the 21st century will be about approximate solutions and frameworks that capture the relationships of

partial solutions and requirements.

Dieter Fensel

To cope with emerging needs two approaches have been employed with increasing success:

Parallel processing database appliances– New database architectures

• Both have been around for years, but were often limited in the past to niche applications or specific verticalsQueryObject System technology has proven to be an effective approach to supplement existing relational databases or as a cornerstone for new demanding applications

Technology panorama

What Market strives for:

Remedy unsatisfactory performances;Reduce time to implement information change requests;

• Make better use of available resources;• Archive and retrieve with ease historical

information;• Have business efficiently drive information delivery.

Data Consolidation in Time

MAINFRAMMAINFRAMESES

DIS TRIBUTDIS TRIBUTEDED

S YS TEMSS YS TEMS

P HYS ICALP HYS ICAL

DATAWAREHOUDATAWAREHOUS ES E

FEDERATEDFEDERATED

S YS TEMSS YS TEMS

Data MartsData Marts

Distributed LogicalDistributed Logical

DatawarehousesDatawarehouses

From DRP …From DRP … … … to Just in Timeto Just in Time

OperationalOperational InformationalInformational

Current IT Panorama

IT has followed in recent times same evolution route manufacturing took when developing standard components and parts.IT equivalents are open source interfaces, Linux, XML, standard connectors, etc.

• This evolution has permitted the creation of specialized federated systems and fostered the adoption of Logical Datawarehouses

What Vendors propose - 1

RelationalRelational DatabasesDatabases VendorsVendors

What Vendors propose - 2

BusinessBusiness AnalyticsAnalytics SolutionsSolutions

QueryObject recognized as a key technology in traditional relational database implementations …… its important role in providing performances and features is considered an asset also by specialized vendors.

QueryObject in the Market

QueryObject market positioning as a database QueryObject market positioning as a database independent technology is confirmedindependent technology is confirmed

S o f t w a r eL ic e n s e

/ A p p lia n c e H W Ve n d o r

A g r e e m e n t

D a t aS e r v ic e

M P PA r c h it e c t u r e[ ]R e la t io n a l

Oracle TeradataNetezza Dataupia

Datallegro+MSFT

C o lu m n a r VerticaSenSageParAccel

VerticaParAccel+EMC2

/ G R ID C lo u d AsterData [MR]Hadoop

Greenplum [MR] Google [MR]Amazon Elastic Cloud

[MR]

E s o t e r ic Panoratio

Vendors vs. Model

S o lutionS o lutionTe c hno logy

Te c hno logy

What is QueryObjectQueryObject is a data consolidation multi-platform technology that:

Generates exact copies of data from databases, applications and systems.– Produces a complete, precise and compressed Master Data Store that is:

• Read only• Cost effective• Binary portable• Secure for content and access• Easy to access and query.

– Provides very short query response times independently from quantity and complexity of data in Master Store.

– Makes available in a single coherent environment:• Aggregates and atomic data• All query services [primary and secondary keys, analytics, drill]

QueryObject competitive advantage: its adoption overcomes some of the limits of relational technologies while leaving to users the freedom to continue designing and thinking in a relational way.

QueryObject Architecture

C o n n e c t P r e p a r e D e s ig n C o m p ile S e c u r e D e p lo y A n a ly z e

P r o t o t y p e & D e v e lo p C o m m a n d &A u t o m a t e

X M L Q u e r y O b je c t M e t a d a t a

S e c u r e &D e p lo y

D a t a S o u r c e s

D B M S

W a r e h o us e

D a t a

F la tF ile s

C S V

D e liv e r y

ODBC

JDBC

EIIServer HSJDBC

-M

ulti

ch

an

ne

l

C o n s u m e r s

P r o c e s s es

C o n n e c te d

R e p o r t in g To o ls

D is c o n n e c t e d

U s e r s

A p p lic a t ion s

E n g in e

U n io n

, ,

Co

nn

ec

tA

cc

es

s

Vie

w

A g g r e g a t e

Th r e s h o ld

In d e x

A.P.I.

WS A.P.I.

SoapServer

D a t a C o p y

U p d a t e

M e r g e

A c c e s s

JDBC

ODBC

HTTP

XML

MDX

A complete, compressed, de-normalized and secure image of accessed data setA set of self-sustaining and highly compressed indexes and views built including analytical contents derived from atomic data

• A set of correlation keys between analytical contents and atomic data

• Can be used as:A database

– A file– A web service

A static, dynamic or drill report– An Excel file

How operates

Data & Information Consumers

DetailedDATA

set

Return to details

Aggregatedindexes

Operational query

Data Services

Analytical query

As an add-on for databases [relational and non] in order to enable high performance analytics;To implement a fast, cost effective data consolidation platform;

• To provide simple data mobility infrastructure;• To ensure information consistency and non-

repudiation.

How QueryObject is used

Direct Competition:Hyperrol

– Query optimization strategies adopted by database vendors– In-memory solutions– Database appliances

• Competitive Advantages:– Database independent– Simple implementation– Business driven against IT driven – Lower cost and commodity HW– High scalability

• Best Practices:– OEM agreement with Dataupia– Verizon ATLAS Project– All major Telco projects

Add-on for DBs

Direct Competition:Database vendors

– Data storage appliances• Competitive Advantages:

– Low TCO– Simple implementation– Platform / DB independent– Scalability– Fast access to data– Read only to users

• Best Practices:– ULISSE Project– …– …

Data Consolidation

Direct Competition:Database vendors via multiple client installations

– Business Intelligence suites vendors• Competitive Advantages:

– Only truly disconnected solution– Cost effective: no client licenses– Platform / DB independent– Open standard access to information– Read Only

• Best Practices:– Poste Italiane– …– …

Data Mobility

Direct Competition:Certified data exchange solutions

– Business Intelligence suites vendors• Competitive Advantages:

– Provides built in non repudiation solution without the need of additional infrastructures

– Cost effective: no client licenses– Platform / DB independent– Open standard access to information

• Best Practices:– …– …– …

Non-repudiation

Usability

Performances

Customer Perception

Strong perceived differentiators with competing technologies also drive purchasing decisions > They represent “Must Have” features;QueryObject sales pitch can be “fine tuned” with respect to the audience:– Ease of integration, scalability and flexibility for the

technical team;– Un-structured and flexible use for the final users;– Very high performances for both.

Chart Analysis

Although several aspects of QueryObject technology are perceived as differentiators with competition, historically the following ones are considered competitive advantages:

The capability to efficiently build and maintain persistent materialized views of data;

– The scalability with much lower than linear resource requirements and no impact on performances;

– The ability to run on multiple platforms and seamlessly move the data;

– Full ad-hoc and fast queries capabilities;– Data consistency and non repudiation.

Competitive Advantages

• Data Integration– CDI (Customer Data Integration) projects:

• Sales & Marketing Solutions• Sales force Automation Solutions• Customer profiling

– Pre/Post Aggregation Processing– High Volume Data Movement– Multiple Data Sources Access

• Business Intelligence– Extreme BI projects– Data Mart Solutions– Historical Data Management Solutions– Business Intelligence on the fly– Disconnected-wireless BI– Infrastructures and Front-End tools

Performances crisis lifeguard• Data Services

– Data Exchange Services– Data Services Provisioning Solutions

Applications …

• Telcos:Network Analysis; Traffic Analysis; Churn Analysis; Billing; Campaign Management; Interconnection Traffic; IP Traffic

• Banking & Finance:Credit Card Usage; Customer Behaviour; Sales Force Information Distribution

• Retail:Basket Analysis; Customer Behaviour; Sales; Inventory Management; Store Replenishment

• Healthcare:Performance Analysis; Financial KPIs; Drugs Expenditure; Prescriptions Analysis

• Manufacturing & Services:Product Lifecycle Management; Maintenance Data Analysis•Media and Web AdvertisingCampaign Management; Real Time Campaign Telemetry; Behavioural Targeting

QueryObject has been used with success in following markets …

… with following vertical solutions:

… helping our Customers to:

• Certify conformity of data to Company’s classification Standards• Guaranty validity, integrity and consistency of data• Build data structures secure, certifiable, licensable, portable, platform

independent, tamper-proof• Create Data Stores dynamically normalized that can be:

– Used by all organization’s systems and in the extended enterprise– Accessed only by authorized users and applications

• Easily allow fast access to Master Data via SQL with all query options (primary and secondary keys, analytics, drill) available

• Provide inter-systems data access and reporting services with high levels of Data Governance, Availability, Reliability

• Quickly react in business real time to unplanned requests• Linearize TCO growth with data volumes

Impact on ROI and TCO

• The infrastructural investment in QO is justified by direct savings and increased efficiency measured by improved SLAs, Quality of Service, and KPIs.

• Objective elements for ROI measure are:

• Reduced Costs• Increased number of users with

same resources

Reuse of infrastructural elements already available

• Productivity increase

In c r e a s e In c r e a s e / S LA / S LA

Q u a lit yQ u a lit y

R e d u c eR e d u c eC o s t sC o s t s In c r e a s e s p e e d t o m a r k e t In c r e a s e s p e e d t o m a r k e t

R e d u c e c u s t o m e r R e d u c e c u s t o m e r s u p p o r t r e q u ir e m e n t s s u p p o r t r e q u ir e m e n t s

In c r e a s e n u m b e r o f u s e r s In c r e a s e n u m b e r o f u s e r s s e r v e d w it h s e r v e d w it h

t h e e x is t in g r e s o u r c e s t h e e x is t in g r e s o u r c e s

D r iv e r D ir e c t D ir e c t

/ S LA Q u a lit y / S LA Q u a lit yIm p a c tIm p a c t

In d ir e c tIn d ir e c t / S LA Q u a lit y / S LA Q u a lit y

Im p a c t Im p a c t

D ir e c t C o s t D ir e c t C o s tR e d u c t io nR e d u c t io n

In d ir e c t C o s t In d ir e c t C o s tR e d u c t io nR e d u c t io n

A c q u ir e n e w u s e r s A c q u ir e n e w u s e r s

In c r e a s e S LA f o r In c r e a s e S LA f o r e x is t in g u s e r s e x is t in g u s e r s

D e v e lo p n e w p r o d u c t s D e v e lo p n e w p r o d u c t s a n d s e r v ic e s a n d s e r v ic e s

In c r e a s e In c r e a s e / Q u a lit y o f D a t a S e r v ic e / Q u a lit y o f D a t a S e r v ic e

In c r e a s e a p p lic a t io n & In c r e a s e a p p lic a t io n & Te c h n ic a l P e r f o r m a n c e Te c h n ic a l P e r f o r m a n c e

In c r e a s e In c r e a s e u s e r s s a t is f a c t io n u s e r s s a t is f a c t io n

In c r e a s e In c r e a s e lo y a lt y o f c u s t o m e r s lo y a lt y o f c u s t o m e r s

Im p r o v e p r o d u c t iv it y Im p r o v e p r o d u c t iv it y

D is p la c e c o s t s D is p la c e c o s t s

R e d u c e c a p it a l R e d u c e c a p it a lr e q u ir e m e n t sr e q u ir e m e n t s

Direct Impact on Quality/SLA

• QueryObject based solutions keep constant or improve SLA independently from number of users or input data volumes. As a consequence, for a given investment and SLA level it is possible to increase users and input data volumes.

• Our experience shows that whenever QueryObject has been used a marked improvement in user-ICT infrastructure interaction is observed. And improved confidence of information consumer has always driven new investments.

A c q u ir e n e w u s e r s A c q u ir e n e w u s e r s In c r e a s e S LA f o r In c r e a s e S LA f o r e x is t in g u s e r s e x is t in g u s e r s

D e v e lo p n e w p r o d u c t s D e v e lo p n e w p r o d u c t s a n d s e r v ic e s a n d s e r v ic e s

Indirect Impact on Quality/SLA

QueryObject based solutions:• Improve data quality;

• Bring to zero the informational misalignment between operational and informational data enabling a quasi real time reconciliation;

• Allow the definition of SLAs independently from data volumes and number of users thus increasing user satisfaction and loyalty.

In c r e a s e In c r e a s e /Q u a li t y o f D a t a S e r v ic e /Q u a li t y o f D a t a S e r v ic e

In c r e a s e a p p lic a t io n & In c r e a s e a p p lic a t io n & Te c h n ic a l P e r f o r m a n c e Te c h n ic a l P e r f o r m a n c e

In c r e a s e In c r e a s e u s e r s s a t is f a c t io n u s e r s s a t is f a c t io n

In c r e a s e In c r e a s e lo y a lt y o f c u s t o m e r s lo y a lt y o f c u s t o m e r s

Reduction of Direct Costs

• Reduction:– Average of development times 1 to 6– Costs and time for training (max 2 weeks)– Maintenance 1to10– Disk space from 50% to 80%– Query times and pre-post aggregation … orders of magnitude

• Savings:– Elaboration resources for same data: 80%– Elaboration resources for same accesses– Time for checking data quality– Time for reconciliation of operational data with analytical data

Im p r o v e p r o d u c t iv it y Im p r o v e p r o d u c t iv it y D is p la c e c o s t s D is p la c e c o s t s R e d u c e c a p it a l R e d u c e c a p it a lr e q u ir e m e n t sr e q u ir e m e n t s

• Reuse of existing investments: the improved efficiency provided by QO leads to better performances of existing IT infrastructure [Databases, ETL, Front End, Applications, hardware] thus protecting investments.

• Reduction: – Support calls from final users: more complete information in QO Datamarts;

– Information delivery time: first answer against new requests from users in few hours.

• Increase: – Volume of stored data with same resources: constant performances with increasing

volumes

– Number of users with same resources: constant performances with increased accesses

Number of users accessing information: large scale distribution and multi-channel access.

In c r e a s e s p e e d t o m a r k e t In c r e a s e s p e e d t o m a r k e t R e d u c e c u s t o m e r R e d u c e c u s t o m e r

s u p p o r t r e q u ir e m e n t s s u p p o r t r e q u ir e m e n t s In c r e a s e n u m b e r o f u s e r s s e r v e d w it h In c r e a s e n u m b e r o f u s e r s s e r v e d w it h

t h e e x is t in g r e s o u r c e s t h e e x is t in g r e s o u r c e s

Reduction of Indirect Costs

Selected customers

Case Studies

– Client:Wholesale Telco Operator

– Application: International Traffic Analysis

Highlights: • 30M rec/day • 3h/day elaboration

• Client: Wireline Telco Operator

• Application: Traffic performance

• Highlights: • 600+ M rec/day• 15h/day elaboration

• Client: Wireline Telco Operator• Application: Work Order Management• Highlights:

• 20M rec/day • 3h/day elaboration

Advantages of QO vs Competition

• Infrastructure downsize (2x7CPUs vs 8x8 CPUs)

• Increase of functionalities with same infrastructure (2x16 CPU);

• 18 months of traffic on-line

• Infrastructure downsize (1x16 CPUs vs 5x64 CPUs);

• Lighter data treatment process

Selected references

Case Studies

• Client: Utility

• Application: Billing and Invoicing

• Highlights:

• 900M rec/year • 1h/day elaboration

• Client: Utility

• Application: Migration

• Highlights:

• 20K rec/day • Two minutes elaboration

• Client: Very large Retail

• Application: Sales performances and margins

• Highlights:

• 5Mrec/week• 1h/week elaboration

Advantages of QO vs Competition

• Speed of implementation (5 days for first project);

• Querytimes 400 times smaller; • Wider time window for analysis

• Creation of unique query functionalities;

• Infrastructure downsize (1 CPU vs 2x4 CPUs)

• Speed of implementation;• Data certification

Company Data

• Owner and developer of QueryObject System technology.

• Active in Master Data Archiving & Deployment projects in complex environments and with large data bases.

• Offices in Italy, USA, Poland with a total of25 direct employees.

• Operates in partnership with System Integrators and OEM.

Revenues 2008 around €3M.

1998 CrossZ Solutions SpA is founded Collaboration with QueryObject Systems Corp. is

established: CrossZ is VAR for Italy

2000

CrossZ develops analytical solutions based on QueryObject technology mostly for Telcos

CrossZ develops with Italtel ULISSE: the largest traffic analysis system ever developed in Italy

2002 Acquisition of QueryObject System Corp. assets and

establishment of CrossZ Solutions USA Inc. in NY Re-engineering of QueryObject started

2004 QueryObject Information Compiler v1.00 released QueryObject System v3.30 released

2006 QueryObject Version 4 released Sales of new version of product started

2007 QueryObject Appliance and Telco solutions released iQO Solutions is established in NY

2008

QueryObject Version 4.11 released iQO Solutions in US wins strategic references in web

advertising market Sales of Data Services solutions started

Targeting Global Leadership

• Provide Fast and Personalized Access to Information from Very Large Data Sources

• Distribute Actionable Information across Extended Enterprise

• Designed Specifically for Simple and Fast Initial Implementations

• Able to Seamlessly Grow to Full Enterprise-class Requirements

• Over 40 Customers across Diverse Industries and Geography

• QueryObject is Chosen by Companies for Whom Business Intelligence is or is Becoming Mission-Critical

• Provide Unrivalled Performances with Commo-dity Infrastructures at Low Entry Cost

• Low Total Cost of Ownership as the Analytical Application Implementation Grows

Ta r g e t in g F a s t G r o w in g D e m a n d

F o c u s e d o n C u s t o m e r Va lu e F u e lle d b y D e m a n d in g C u s t o m e r s

C o m p le t e M a r k e t C o v e r a g e

Selected references