understanding system performance

Understanding System Performance – When is it Time for a System Tune-up?

Jim BlairData Warehouse Consultant / Marketing Specialist – Teradata

Agenda

• Overall System Performance> Overview of DBQL and

Viewpoint> Baselines and Benchmarks

– Metrics and reports

> Real-time alerts> Growth Patterns

• Time for a tune up> Best Practices in

Benchmarks> Query Tuning> Load considerations> Compression> Other Considerations

High Level Performance

• Inefficient or bad queries?• Updates and data loading?

> Locking tables and rows for too long

• Too many concurrent tasks?> Big consumption

workloads> Jobs that should

run later

Heatmap from DBQL

Total CPU usage – Is Your System Running HOT?HOT

Heartbeat - June 2009

0.1

1

10

100

6/1 6/2 6/5 6/6 6/7 6/8 6/9 6/12 6/13 6/14 6/15 6/16 6/19 6/20 6/21 6/22 6/23 6/26 6/27 6/28 6/29 6/30

Hour and Day (Business Days only 6 am - 6 pm)

!

Sec

on

ds

(Ela

pse

d)

Heartbeat Graph

What is Teradata Viewpoint?

• System administration portal> Portlets for status displays> Targets business and

technical users> Rewind for system analysis> Manage multiple systems

• Appliance > Server + OS + software> Completely supported by

Teradata• Future platform for Teradata

management products> All management Teradata

Tools and Utilities• NOT an Enterprise Portal

> Does not compete with WebSphere, BEA, SAP, etc.

Viewpoint Portlets in Action

Filtered Queries• List View of all sessions

on a Teradata system• View sessions by different

categories> Active, parsing, delayed, etc.

• Set thresholds to spot problem queries

• Drill down into a session, and access explain plan and SQL text

• Allows for DBAs to easily manage and track sessions and queries filtered by state

Benchmarks and Baselines

• Create a ‘real life’ benchmark> Body of 4 hrs real workload> Do not use static tables> Predictable results > Include all types of queries

– capture 25-40 different queries and extrapolate in 4 hrs of work

• Run the benchmark with consistency> Same # of concurrent queries each time> Same queries each time> Access production data> Capture metrics on each query

• Establish a Baseline> Run benchmark in lowest period of system usage> Quiese the system if possible> Capture metrics on all queries

• When to run the Benchmark process > Quarterly> Before/After every

Upgrade– HW, SW, – major implementations

> On Demand

• Analyze results > What’s changed since last run?

Standard Benchmark

0

50

100

150

200

250

300

350

Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10

Min

ute

s

Run time in minutes

Baselines and Benchmarks

Major Upgrade

Heartbeat - June 2009

0.1

1

10

100

6/1 6/2 6/5 6/6 6/7 6/8 6/9 6/12 6/13 6/14 6/15 6/16 6/19 6/20 6/21 6/22 6/23 6/26 6/27 6/28 6/29 6/30

Hour and Day (Business Days only 6 am - 6 pm)

!

Sec

on

ds

(Ela

pse

d)

Real Time Alerts

• Canary Queries> One in each work load> Set threshold for alerts

• DBA Alerts> High level of spool> Skewing> Number of users

• All production Raw data

• Benchmark tables

• Don’t forget Overhead!> Spool, DBC, DBA workspace

Track Growth Patterns

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10

Teradata Production data

Free

Data

Overhead

0

2

4

6

8

10

12

14

16

18

20

Jun-09

Jul-09

Aug-09

Sep-09

Oct-09

Nov-09

Dec-09

Jan-10

Feb-10

Mar-10

Apr-10

May-10

Jun-10

Jul-10

Aug-10

Sep-10

Oct-10

Nov-10

Ter

abyt

es

Overhead

Data

Free

Linear (Data)

Linear (Free)

Linear (Overhead)

Data Growth

Trends Over Time

Performance Tuning

Time to Get Under the Hood

Your

Sr. DBA

When to Run a Benchmark

• General performance trending

• To compare when migrating to or from Teradata

• Regressions

• Before and after Patches and Major or Minor Hardware or Software upgrades

• Test before and after certain project implementations

• May want to test before and after TASM implementations or modifications

Makings of a Successful Benchmark

• Consistent data being accessed

• Consistent indexing, views, security, priorities

• Queries should represent a true mixed workload

• Maintains consistent concurrency levels

• Correct results captured every time

• Queries should be run in same order every time

• Same number of queries completed every time

• Meaningful reports

Building the Benchmark Process

• Process should be 100% automated

• Capture Explains before and after

• Data and queries represent a true workload

• Process should ensure that all indices, statistics, join indices, etc. are current as expected

• Use DBQL to capture queries to represent workload

Building the Benchmark Process

• Capture short, medium and long running queries

• Process should allow for consistent number of concurrent queries

• Design queries to return a count and not return huge sets of rows – Network could skew timings

• Process should report on results when finished

Things to Check For

• Make sure response times for each query and overall process are not abnormally high

• Checking overall or individual response times is NOT enough!

• Make sure results are also accurate

• Examine explain plans to see if dramatically changed> Note: You probably will not care about this if query

is running much better.

• One query can throw off the entire benchmark

Result Report Example

Investigate!• Different results normally indicate a problem, but it could

be that the benchmark spanned two dates

QUERY NUM AVG TIME MIN TIME MAX TIME TIMES RUN

12 1:41:09 1:25:17 1:48:49 426 1:24:57 1:05:16 1:37:53 2017 1:11:02 1:04:50 1:16:54 414 0:53:57 0:36:20 1:01:54 422 0:29:16 0:23:36 0:35:28 824 0:18:13 0:17:45 0:18:38 4

6 0:18:33 0:14:43 0:24:43 24

Response Time Report Example

Look for Consistency and Compare to Past

Final Thoughts on Benchmarks

• Take special note when changes are made to views, indices, TASM, etc.

• These changes may unexpectedly improve or even impair your benchmark

• Keep the benchmark current

Number of Incoming Queries

RejectFilter

Delay Throttle

Reject Throttle

ExceptionReject

OutsideOf SLA

QueriesAfter

Filters and Throttles

Exception Reclass

Workload Management

Query Tuning

• Can save a tremendous load on your system

• Need process to tune query, but then inform and train users as well

• Identify offending queries and report

• Viewpoint-Query Replay

• Document all findings, strategies, time saved, CPU and IO saved, etc.

Query Tuning

• Look for common mistakes such as missing aliases, product joins, poor primary index selection on “Create Tables”

• Make sure necessary statistics are collected

diagnostic helpstats on for session;

• Try tricks like GROUP BY instead of DISTINCT if it applies

Query Tuning

• Look for ways to impact multiple queries with one tuning effort or enhancement

Load Techniques

• Goal – Avoid contention between loads, users, and backups

• Establish DDL naming conventions and document

• Automate process to validate all DDL

• Get Developers to collaborate with DBAs

• Educate Developers on database technology and features (i.e. Locking, Backups, Utilities)

• Educate DBAs on ETL complexities

Load Techniques

• Start using TPT

• Establish Load standards or conventions and enforce them at the ETL level

• Examples > Trickle Batch> Mini Batch> TPUMP> Dynamic stage table creation> Dual Database design

Compression

• Facts> Compress 255 values + Nulls> Cannot compress PI columns> Can improve performance

• May even pay to compress small tables so they can remain memory resident

• Changes coming soon> Compression on variable

length data types> Algorithmic compression

Other Ideas – Don’t reinvent the wheel!

• Materialize view definitions through Join Indexes or automation

• Utilize new features like Multilevel Partitioning

• Archive data

• Horizontal partitioning

• Upgrade – Query Rewrite and Optimizer improvements

• Investigate Hot/Cold Data

Thank You!

Questions?