datavault automation
DESCRIPTION
Our presentation on the DV automation congres on theTRANSCRIPT
Conspect G
roup
Wisdom never fails
06 October 2011
Data Vault Automation
Conspect G
roup
Wisdom never fails
So Conspect?
� Dutch ICT & Consulting organization
� Headquarters @ Almere
� 120 Highly Skilled Professionals
� Conspect is a company that excels in Attitude and Behavior
� Within the ICT domain we provide professional services in Application Integration, Business Intelligence, Custom Development and CRM on Demand
� Conspect = Open Culture - Social Engaged – Sustainability & Corporate Social Responsibility
Conspect G
roup
Wisdom never fails
Why Conspect….
‘Think Big, Start small, Scale fast’
Conspect G
roup
Wisdom never fails
Conspect and BI, our history
2001
Beginning of
Conspect
First Data Vault
2007
2009
Developed the idea of automating
our Data Vaults
Brought BI Team
together with
Conspect
2011
2011
made our ideas real
Conspect G
roup
Wisdom never fails
Who is this guy?
Name: Justin Eelzak
Date of birth: January 13th 1981
Employer: Conspect
Occupation: Business Intelligence Consultant
Active in BI since: 2008
BI Specialty: Reporting and as of today speaking to large groups about Data Vault
Conspect G
roup
Wisdom never fails
Next Generation of Data Warehouses
Conspect G
roup
Wisdom never fails
Conspect G
roup
Wisdom never fails
Conspect G
roup
Wisdom never fails
Conspect G
roup
Wisdom never fails
Conspect G
roup
Wisdom never fails
Conspect G
roup
Wisdom never fails
An Actual Slide Slide
� “I was so hoping it wouldn’t be a guy reading the bullet points from his slides.
� He lasted ten slides before failing…”
Value in automation• Complex• Dynamic• Great deal of initial work
Difficulty of automation• Non standard• Complex
Conspect G
roup
Wisdom never fails
Value in automation
Complex DynamicGreat deal of initial
work
Difficulty of automation
Non standard
Very Complex
DWH to Staging Out
Conspect G
roup
Wisdom never fails
Value in automation
Reasonably simple
Static once in place
Not a lot of initial work
Difficulty of automation
Non standard
Metadata rules are complex
Source to Staging
Conspect G
roup
Wisdom never fails
Staging to Data Vault
Value in automation
Moderate maintenance
costs
Moderate initial
development time
Difficulty of automation
Highly standardized
Simple Metadata
rules
Conspect G
roup
Wisdom never fails
What did we end up doing?
� CDC (hardly a confusing acronym)
� Fully automated the Staging to Data Vault E(T)L
� Fully automated the generation of the Data Vault Database
� Left a human being in charge of the Data Vault design.
Conspect G
roup
Wisdom never fails
100% Reusable ETL
Conspect G
roup
Wisdom never fails
Data Vault
datawarehouse
Generated Data Vault
Conspect G
roup
Wisdom never fails
Custom Component
Conspect G
roup
Wisdom never fails
Results
Current record: 5 hours of ETL in 5 minutes. Build ETL get your
data vault for free
Conspect G
roup
Wisdom never fails
What will we add later?