data warehouse testing: the querysurge data wizards
DESCRIPTION
What Do Harry Potter And Data Warehouse Testing Have In Common? They both use wizardry to get the job done! Introducing the new QuerySurge Data Wizards. See how easy it is to validate your data warehouse data throughout the ETL process with our new Wizards! The Wizards are also perfect for database migrations and upgrades. Why spend time writing SQL queries when you can simply point to your data sources and tell the wizard what tests you'd like to run - fast and easy, no coding necessary! We will walk you through use cases for this new functionality to show you how it can simplify your work and help you achieve efficiency and quality in the data warehouse development cycle.TRANSCRIPT
built by
Bill HaydukCEO/President
RTTS
Harry Potter and Data Warehouse TestingIntroducing the QuerySurge Data Wizards
Martin Henry-LesterProduct Specialist
RTTS
Laura PoggiMarketing Manager
RTTS
Webinar
About FACTSFounded: 1996
Primary Focus:
consulting services, software
Locations: New York, Atlanta, Philly, Phoenix
Geographic region:North America
Customer profile:Fortune 500, > 450 clients
Software:QuerySurge
The Software Quality Experts
RTTS is the leading provider of software quality for critical business systems
built by
The average organization loses $8.2 million annually through poor Data Quality.
- Gartner
46% of companies cite Data Quality as a barrier for adopting Business Intelligence products.
- InformationWeek
Data Quality Best Practices boost revenue by 66%. - Research firm Sirius Decisions
4
About QuerySurge
built by
QuerySurge is the
premier test tool built
to automate Data Warehouse testing
and the ETL Testing Process
What is QuerySurge?
built by
What does
QuerySurge ™do?
built by
QuerySurge finds bad data
• Most firms test < 10% of their data
• Your CxOs rely on BI apps to make strategic decisions
• BI apps sit on top of DWHs that have at best, untested data & at worst, bad data
• QuerySurge tests up to 100% of your data quickly & finds bad data
QuerySurge™ Architecture
built by
Que
rySu
rge
Arch
itect
ure
Target
Sources
8
Architecture-Under-Test
Sources
SQL Q
uery
Da
ta se
t
SQL Query
Data set
Data Set #1
Data Set #2
© 2011 Real-Time Technology Solutions, Inc.
QuerySurge Architecture
User(s)
Web interface
CompareData Sets
QuerySurge™: How it Works
built by
QuerySurge Roles & Uses
Testers - functional testing - regression testing
ETL Developers - unit testing
Data Analysts- review, analyze data - verify mappings and
failures.
Operations teams - monitoring
built by
built by
Design Library Create Query Pairs (source & target SQLs)
10
QuerySurge™ Modules
Scheduling Build groups of Query Pairs Schedule Test Runs
Deep-Dive Reporting Examine and automatically
email test results
Run Dashboard View real-time execution Analyze real-time results
QuerySurge™ Modules
built by
built by
QuerySurge™ Gallery: QueryPair Creation
Source Query
Target Query
built by
QuerySurge™ Gallery: Run Dashboard
QueryPair details
Runtime activity
Result sets
built by
QuerySurge™ Gallery: Summary Report
Duration= 34 sec
720,000 data sets
14,000 data failures
built by
QuerySurge™ Gallery: Detail Report
Individual QueryPair passes
Individual QueryPair fails
Business Problem
built by
In a survey of potential customers we asked:
What are your top
DWH testing challenges?
They need to automate the testing effort
Their Top 6 Responses
They need to speed up the testing cycle
built by
They need to be able to schedule test runs
They need to be able to test across different platforms
They need to share test results with their group
They need to verify more data than they currently do (typically <10%)
automates the testing effort the kickoff, the tests, the comparison, emailing the results
speeds up testing up to 1,000 times faster than manual testing
schedules test runs run now, every Tuesday at 11pm or right after ETL process
tests across different platformsany JDBC-compliant database, DWH, D-Mart, flat file, XML
views & shares results through automated emailing of reports
the QuerySurge solution…
verifies more data verifies upwards of 100% of all data
built by
QuerySurge Value-Add
QuerySurge provides value by either:
in testing data coverage from < 10% to upwards of 100%
in testing time by as much as 1,000 x
combination of in test coverage while in testing time
19built by
Return on Investment (ROI)
redeployment of head count by 75% because of an increase in coverage
a savings over manual testing (minus queries, manual compares, other)
an increase in better data due to shorter / more thorough testing cycle, saving $millions by preventing key decisions made on bad data.
20built by
FYI - Training Courses
built by
CoursesFundamentals of QuerySurge 1 dayIntroduction to SQL for QuerySurge 1 day Advanced SQL Techniques for QuerySurge 1 dayData Warehouse Fundamentals 1 day
Delivery Optionsin the Cloud in our Classroom private on-site training
new
built by
QuerySurge™ new release!!!
What do and
QuerySurge™ have in common?
23
They both use to get the job done!!!
Introducing the
built by
24
Fast and Easy. No programming needed.
built by
• Perfect for data warehouse testing, data migrations & upgrades
• Great for verifying that all rows have come over without issue
• Superb at verifying table-to-table transfers
• Ability to graphically create SQL stubs for advanced scripting
Uses:
Verify that the amount of rows match the amount that were expected
built by
Roles:
ETL Developer, Operations, Testers
SQL Skill set / Value added:
beginner: no coding needed
experienced user: saves time
Uses: - Verify data warehouse loads
when no transformation- Verifies data migrations,
upgrades
built by
Roles: ETL Developers, Testers
SQL Skill set / Value added: beginner: no coding neededexperienced user: saves
time
Uses: - Utilize either wizard to create stub
statements for writing SQL query pairs
built by
SQL
SQL
SQL
SQLSQL
SQL
SQL SQL
SQL
SQL
Roles: ETL Developers, Advanced Testers
SQL Skill set / Value added: For experienced users: - No need to look up schema(s) or
table names- Saves tons of time
Visually select source and target database (or DWH)
Wizards’ ease-of-use Select Row Count Wizard or Table Compare Wizard
Select schema(s) from source and targetSelect matching tables in source and target
QuerySurge automatically creates test suite
Now you are ready to run!
Ensuring Data Warehouse Quality
QuerySurgebuilt by
Demo