présentation ibm db2 blu - fabrizio danusso

© 2013 IBM Corporation

IBM DB2 with BLU Acceleration Bois-Colombes - User Group Infosphere

Fabrizio DANUSSO | Client Technical Professional

16 octobre 2013

© 2013 IBM Corporation 2

Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in

which IBM operates.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for

informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant.

While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without

warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this

presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or

representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use

of IBM software.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have

achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended

to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other

results.

Acknowledgements and Disclaimers


Croissance des besoins d’analyse des données

Creating the need for

organizations to

understand and anticipate

customer behavior and

needs based on customer

insights across all

channels

The shift of power to the consumer 2

Creating the need for all

parts of the organization

to optimize all of their

processes to create new

opportunities, to mitigate

risk, and to increase

efficiency

Accelerating pressure to do more with less 3

The emergence of big data

Creating new

opportunities to capture

meaningful information

from new varieties of

data and content coming

at organizations in huge

volumes and at

accelerated velocity

1


Accélérateur BLU: c’est quoi?

BLU combine des technologies de pointe

- 25+ brevets dans 10 laboratoires sur 7 continents

BLU augmente les fonctionnalités existantes de DB2

- Utilise des tables organisées en colonnes qui coexistent avec les tables classique organisées en lignes

- Utilise un moteur d’exécution colonne qui coexiste avec le moteur traditionnel

- C’est du DB2: TBS, BP, utilitaires, SQL, ne changent pas

Les tables organisées en colonnes utilisent

- Compression des données

- Processus vectoriels

- Optimisation de ma mémoire

- Et plus


Accélérateur BLU: à quoi ça sert?

BLU est conçu pour vos requêtes analytiques

- Maximise l’utilisation de votre mémoire, mais…

- Ne nécessite pas une solution coûteuse « in-memory only »

BLU offre des avantages majeurs

- Performances ultra rapide (10x – 25x)

- Réduit le besoin de stockage (5x – 20x)

- Administration simplifié, « load-and-go »


Super Fast, Super Easy – Create, Load, and Go!

Database Design and Tuning

1. Decide on partition strategies 2. Select Compression Strategy 3. Create Table 4. Load data 5. Create Auxiliary Performance Structures

• Materialized views • Create indexes

• B+ indexes • Bitmap indexes

6. Tune memory 7. Tune I/O 8. Add Optimizer hints 9. Statistics collection

DB2 with BLU Acceleration

1. Create Table 2. Load data

Repeat

VS


Accélérateur BLU: pourquoi est-il spéciale?

Ses caractéristiques principales:

1. Facile à mettre en œuvre et à administrer

2. Extrême compression des données

3. Stockage des données en mode colonne

4. Data Skipping

5. Parallélisme multi processeur

6. Accélération CPU SIMD (Single Instruction Multi Data)

7. Cache mémoire intelligent


Accélérateur BLU: Facile à mettre en œuvre et à administrer

Set one registry variable: DB2_WORKLOAD=ANALYTICS

- Then create the database to get optimal settings for an analytic environment optimized to your hardware

- It’s still just DB2: use traditional storage, utilities, SQL, application interfaces, and so on

Now just “load and go”

- Compression automatically done and statistics automatically collected

- No need for indexes, partitions, MQTs (materialized views), MDCs, hints, statistical views, etc.

- Underlying structures that support BLU are created and updated automatically

Ongoing maintenance is also minimal

- REORGs (for space reclaim) and workload management is all automatic


Accélérateur BLU: Extrême compression des données

Massive compression with approximate Huffman (frequency-based) encoding

- Most frequent values encoded with fewest bits

- Extremely effective since all data in the column are the same data type

- Map entire values directly to column dictionary codes by both table and page levels

Register-friendly encoding greatly improves efficiency

- Encoded values are packed into bits matching the CPU register length

- Requires fewer I/Os and CPU cycles as well as better memory utilization

Encoded values do not need to be decompressed during evaluation

- Predicates and joins work directly on (actionable) encoded values

(Predicate examples: =, <>,<, >, >=, <=, between, etc.)


Accélérateur BLU: Stockage des données en mode colonne

Efficient columnar storage automatically done during LOAD

Result: minimal I/O

Work performed directly on columns

Improved memory density


Accélérateur BLU: Data Skipping Automatic detection of large sections of data that do not qualify for a query and can be ignored

Order of magnitude savings in all of I/O, RAM, and CPU

No DBA action to define or use – truly invisible


Accélérateur BLU: Parallélisme multi processeur BLU Acceleration builds on traditional multi-core parallelism

Multi-core parallelism is a combination of Data and Functional parallelism

Data parallelism assigns dynamic partitions to retrieve data from disk

Functional parallelism divides queries into multiple parts by function


Accélérateur BLU: Accélération CPU SIMD “Single Instruction Multiple Data “ (SIMD) CPU design:

- Performs the same operation on multiple data elements simultaneously

Various CPUs have built-in instructions to perform SIMD

- Your hardware already has this capability - BLU Acceleration simply exploits it (128-bit register usage is maximized as well)

- BLU Acceleration has been tested to perform on:

- AIX on Power - best performance on Power 7

- Linux 64-bit on Intel / AMD - best performance on Intel Nehalem or better


Accélérateur BLU: Cache mémoire intelligent

New algorithms cache in RAM effectively

High percent of interesting data fits in memory

- We leave the interesting data in memory with the new algorithms

Data can be larger than RAM

- No need to ensure all data fits in memory

- Optimization for in memory and I/O efficiency

RAM

DISKS

Near optimal caching


Accélérateur BLU : En conclusion

Technologie intégrée au moteur DB2

Column-organized table approach

Orienté pour les workloads hybrides et analytiques

Transparent pour les applications

Implémentation et administration simplifiés

Multi-core and SIMD parallelism

Data skipping

Meilleur taux de compression pour les données

Super Fast, Super Easy— Create, Load and Go!

No Indexes, No Aggregates, No Tuning, No SQL changes, No schema changes, No REORG/RUNSTATS


Fabrizio Danusso Client Technical Professional [email protected]

mailto:[email protected]


© Copyright IBM Corporation 2013. All rights reserved.

• U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

• IBM, the IBM logo, ibm.com, Informix, are trademarks or registered trademarks of International Business Machines Corporation in the

United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information

with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this

information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM

trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

Other company, product, or service names may be trademarks or service marks of others.

http://www.ibm.com/legal/copytrade.shtml







présentation ibm db2 blu - fabrizio danusso

Technology