supercharged analytics - sas...o strong capability in predictive analytics and data exploration is...

Supercharged AnalyticsSupercharged AnalyticsCase Study: Canadian Imperial Bank of Commerce

For what matters.2010

For what matters. © CIBC 2010 All Rights Reserved SAS GLOBAL FORUM 2010 2

Today’s Discussion

o Overview – Risk Management Analytics

o The Opportunity

o Making It Happen

o The Road Ahead


About CIBC

• Canadian Imperial Bank of Commerce (CIBC) is a leadingNorth American Financial institution

• we offer a full range of products and services to almost 11 million individuals and small businesses, corporate andinstitutional clients

• At year-end (October 31, 2009):• Market capitalization was $23.8 billion• Tier 1 capital ratio was 12.1%• employed nearly 40,000 employees worldwide• had 1,050 branches in Canada and more than 3,700 ABMs

• Constituent of the Dow Jones Sustainability Index (DJSI)for seven consecutive years (one of 25 banks worldwide)

All amounts in C$


Overview

Risk Management Analytics


Overview - Risk Management Analytics

o Key accountabilities of Risk Management Analytics include:

o acquiring relevant, accurate, complete and timely risk datao managing default event history and write-offs / recoveries

o developing and monitoring performance of risk modelso modeling multiple business scenarioso establishing Basel II Framework parameterso measuring performance of portfolios of assets

o supporting business programs with risk data insights

o regulatory and management reporting

o monitoring compliance to enterprise risk policies and standards


Risk Management - Critical Success Factors

o Strong capability in predictive analytics and data explorationis critical

o Effective Risk Management depends upon an integrated approach:

o skilled data and quantitative analysts

o efficient processes

o access to relevant, accurate, complete and timely data

o the right tools for the job

o robust and scalable infrastructure

o Senior Management commitment to ensure business processescapture accurate and complete data on a timely basis


Optimizing The Business Process

DataAcquisition

DataPreparation

DataExploration

Analytics Reporting

o Success in managing risk depends upon predicting future scenarios well enough to:

o take actions to mitigate risko recognize opportunities in the market

o Today’s discussion focuses on key enablers in this process in order to speed the overall time to value


Scope of Risk Data

Database Name

Description # Records each month

Size (GB) each month

# Attributes each file

History

Equifax Credit Bureau

File #1 millions 31 650 24 monthly versions

Equifax Credit Bureau

File #2 millions 7 Variable from 81 to

2,629

24 monthly versions

Card Products (VISA)

File #1 millions 4 98 84 monthly versions

o We need to regularly analyze very large datasets

o This includes extensive history of customer behaviour of allcredit products (e.g., mortgages, card products, loans, etc.)

ILLUSTRATIVE

o As an example, we’ll look at the analysis for one of the predictive risk factors used in the Basel II calculation of regulatory capital


Background on Basel II Framework

o A global framework issued by Bank of International Settlements (BIS) and managed by national supervisors

o Developed over the period 1999 – 2005 with broadconsultation globally along with quantitative impact studies

o The Basel II Committee Goals were:o to enhance risk sensitivity of capital requirementso greater emphasis on banks own assessment of risko improve transparency for market discipline

o Basel II was implemented November 1, 2007 by CIBC and other major banks in Canada


Distribution of Credit Risk

Default Rating

Exp

osu

re (

$)

Bank A Bank B

Best Worst

Corporate Loan Portfolio

o assume the credit portfolio size is identical for both banks but with a different mix of credit risk

ILLUSTRATIVE


Basel II: Risk Sensitive, More Capital

o The strategic implication is that banks with riskier portfolioswill have higher minimum regulatory capital requirements

ILLUSTRATIVEAIRB Approach

Total Capital Capital

Exposures CAR 1 Basel II

($) Bank A

Bank B

Corporate Loan Portfolio


Basel II Glossary: Credit Risk Capital

o The Basel II Framework allows the use of bank-specific estimates of risk components in determining the capital component for a given exposure:

• Probability of default (PD)

• Exposure at default (EAD)

• Loss given default (LGD)

• Effective maturity

• Firm-size adjustment for Small Medium Enterprises(SME)


Basel II Glossary: Credit Risk Capital

o Expected Loss (EL) = PD * EAD * LGD

o Unexpected Loss (UL) is calculated using sophisticated Basel II formulae incorporating PD, EAD, LGD

Loss

Probability of Default

Unexpectedloss

Expected loss

99.9th percentileof loss

o minimum regulatory capital is a function of the calculationof unexpected loss (UL) and expected loss (EL)


Overview: Parameter Estimationo risk rating systems rank order the quality of individual credit risk

exposures and groupings of exposures

o there are three important dimensions:o the risk of the borrower defaulting (PD)o factors specific to individual transactions to estimate

the economic loss, given default (LGD)o the calculation of exposure amount at default (EAD)

o the estimates for PDs need to be long-run averages ofthe actual one-year default rates

o LGDs must be developed from internal data about historicallosses and recoveries

o parameters must be good predictors of future loss events

o banks are expected to reflect conservative estimates


Developing Retail PD Estimates

o Basel II requires banks to “pool” retail exposures with similar risk characteristics and estimate the Probability of Default (PD)

o each individual exposure within the pool then acquires the parameters of the pool to which it belongs

Pool1

Pool2

Pool3

Pool4

Pool5

Pooln

Borr

ower

Met

rics

Transaction Metrics

Historic Portfolio Performance Data

Historic Economic Data

Pool1

Pool2

Pool3

Pool4

Pool5

Pooln

Borr

ower

Met

rics

Transaction Metrics

PD

Analytic Engine:• determines pools• forecasts PD for each pool• revises pools to ensure

appropriate Capital• stress testing

o PD pools display sufficiently homogeneous behaviour over time


Reviewing the Historical Performance DataILLUSTRATIVE

Consumer Loans - PD for Pools A,B,C

Consumer Loans - PD for Pools D,E,F


Next Steps – Deriving the Retail PDs

Pool ID Mean PD Std Min Max Adjusted PD

PD Estimate

Average Balance

A 0000.00 0000.00 0000.00 0000.00 0000.00 0000.00 00.0

B 0000.00 0000.00 0000.00 0000.00 0000.00 0000.00 00.0

C 0000.00 0000.00 0000.00 0000.00 0000.00 0000.00 00.0

D 0000.00 0000.00 0000.00 0000.00 0000.00 0000.00 00.0

E 0000.00 0000.00 0000.00 0000.00 0000.00 0000.00 00.0

F 0000.00 0000.00 0000.00 0000.00 0000.00 0000.00 00.0

ILLUSTRATIVE

o we tested the accuracy of our predictions

o we obtained approval and implemented the PD model intoproduction for calculation of Risk Weighted Assets (RWAs)


Parameter Monitoring – PD Example

ILLUSTRATIVE

LOB Total # of Pools in

LOB

Pools above Upper

Thresholds

Pools within

Thresholds

Pools Below Lower

Thresholds

Comments

% of Pools Outside

Thresholds

Avg # of Pools Above Upper

Thresholds

Avg # of Pools within

Thresholds

Avg # of Pools Below Lower

Threshold

Pooling Model 1

97 33 25 39 Click for details

74.2% 31 26 40

Pooling Model 2


68.8% 11 17 21

Pooling Model 3


90.9 0 1 10

Pooling Model 4


70.6% 4 5 8

Pooling Model 5


75.0% 10 2 0

for all Poolingmodels

PD Estimates vs. Actuals Monthly SummaryReporting Date: 201001 Estimation Date: 200901

n

o we monitor and analyze the observed default rate over time


Key Challenges

o Timely turnaround of analytic results

o Management of risk datao movement of large volumes of datao redundancy (same version?) of data across projects

o Resource-intensive model development / testing process


The Opportunity


Critical Success Factors

o Success depends upon an integrated approach:

o skilled data and quantitative analysts

o consistent processes

o access to accurate, complete and timely data

o the right tools for the job

o robust and scalable infrastructure

o Senior Management commitment to ensure business processescapture accurate and complete risk data on a timely basis

today’sdiscussion


Current State Architecture

SAS Application Server running32-bit Win OS6 TB SAN storage

SAS 9.1.3 components, including SAS EG, SAS EM, SAS Credit Scoring, SAS BI Server

2 x Windows database serversrunning 32-bit Win OS14 TB SAN storageSQL databases

FOCUS:

Upgrade Tools

Upgrade Infrastructurecapability

Analytic DatabasesProduction Databases


Proposed Solution

o Accelerate the data preparation, data exploration, and analytics portions of the business process

o Replace existing database servers with a Data Warehouse Appliance from Netezza Corp.

o Migrate existing SQL databases to Netezza databases

o Upgrade SAS tools to the version 9.2 components

o Optimize the SAS – Netezza integration


What Is A Data Warehouse Appliance?

o Integrates database, server, and storage in one compact system

o Optimized for analytical processing and designed for flexible growth

o Architecture principles include:o processing close to the data source o balanced, massively parallel architectureo platform is engineered for advanced analyticso appliance simplicity vs. traditional separate componentso Extreme scalability of internal storage capacity

o Advertised performance gains of 10 x to 100x baseline

o Not a “niche product”


Enhanced Architecture

SAS Application Server running64-bit Win OS6 TB SAN storageSAS 9.2 components, including SAS EG, SAS EM, SAS Credit Scoring, SAS BI ServerSAS / ACCESS Interface for Netezza

1 x Netezza data warehouse appliance in PROD, with30 TB storage1 x Netezza DEV/UAT, with 6 TB storage

Data Warehouse ApplianceProduction Databases

Replacement of SQL servers with Netezza Data Warehouse Appliance

Upgraded SAS Applications to 9.2

SAS/ACCESS Interface to Netezza


Accelerating Data Exploration

Current Architecture

Data Warehouse / Database

Data Preparationand

Data Exploration

SAS tools


Accelerating Data Exploration

Current Architecture In-Database Architecture

Data Warehouse / Database Data Warehouse Appliance

Data Preparationand

Data Exploration

SAS tools SAS tools

Data Preparationand

Data Exploration


SAS / ACCESS Interface to Netezza

o A SAS software solution that provides direct connectivitybetween SAS and the Netezza data warehouse appliance

o Leverages utilities from both SAS and Netezza foroptimized loads and extracts of data

o Supports two means of integration:o LIBNAME engine – requires minimal knowledge of

the data and SQL to surface the data

o Pass-Through Facility – greater flexibility, butrequires users to specify properly structured SQLlanguage


Using SAS/ACCESS Interface for Netezza

SQL QUERY EXAMPLE:

Proc SQL;Create table Results as Select coalesce (table_A.balance,

table_B,balance, table_C.balance) as OS_balance

From Table_A, Table_B, Table_CWhere Table_A.customer =

Table_B.customerand Table_B.customer =

Table_C.customer:Quit;


Performance Examples for Data Exploration

1 running SAS 9.1.3 with ODBC connection to SQL server and SQL databases2 running SAS 9.1.3 with ODBC connection to Netezza model 10/100 with Netezza databases

Description of Sequential Query Benchmark1

(time in seconds)

DW Appliance2

(time in seconds)

Size of Dataset

Credit Cards database 3,096 30 243 GB

Credit Bureau database Cannot complete in “one pass”

396 ~ 200 GB

o The following are actual measures of performance observed duringthe on-site “Proof of Concept” using production data that was loaded onto the Netezza data warehouse appliance

ILLUSTRATIVE

o on average, given a mix of complex queries and large datasets,we observed an improvement of more than 20x the benchmark

o we do not yet have metrics using the SAS/ACCESS Interface for Netezza


What About Changes for Analytics?

Current Architecture

Data Warehouse / Database

SAS tools

Data Prepand Data

Exploration

AnalyticModels

ModelScoring


Creating and Deploying Analytic Models

o Starts with data preparation and data exploration

o Development of analytic models is an iterative process

o At each step, the modeler may use different tools andanalytics, depending on the situation

o Multiple models are developedo evaluate and compare the performance of each model

o Finalize the model

o Deploy the model into production


What About Changes for Analytics?

Current Architecture In-Database Architecture

Data Warehouse / Database Data Warehouse Appliance

SAS tools SAS tools

Data Prepand Data

Exploration

Data Prepand Data

Exploration

AnalyticModels

ModelScoring

AnalyticModels

ModelScoring


Making It Happen


Selecting the Infrastructure Solution

o Industry Review of Technology Trends

o Proof of Concept with Multiple Vendors of Data WarehouseAppliances, conducted at their sites

o Selected Vendor (Netezza Corp.) to work with

o On-Site Proof of Concept with Netezza Corp.


Objectives: On-Site Proof of Concept

BUSINESS OBJECTIVES:o assess real-world performanceo understand data migration issueso assess impact on current userso conduct due diligence of vendoro assess cost-benefit of investment

TECHNOLOGY OBJECTIVES:o understand integration into shared production environmento ensure compliance with security standardso create support model with technology operations and vendoro test operations and database administrative functionso assess new technology


Implementation Plan

The key elements of the implementation plan include:

o install and test the DEV/UAT appliance

o finalise the operations support model with technology groupsand the vendor

o install and test the PROD appliance

o migrate SQL databases to new Netezza databaseso selectively optimize database structures

o upgrade the SAS Application Server (32 bit -> 64 bit OS)

o upgrade analytic applications to the SAS 9.2 componentso deploy SAS/ACCESS Interface to Netezza

o optimize applications


The Road Ahead


What’s Next?

o SAS/ACCESS Interface for Netezzao additional PROCs available for in-database

processing

o SAS Scoring Accelerator 1.6 for Netezzao translates SAS Enterprise Miner models into

in-database scoring functions

o Continuing R&D between SAS and Netezza to optimizethe SAS-Netezza analytic platform


Summary

o The future for “supercharged” analytics includes in-database processing using scalable, high performance data warehouse appliances and “made-to-fit” analytic tools

o The stated strategic direction of SAS Institute is to continue to develop sophisticated analytic “routines” and integration with data warehouse appliances to leverage this capability

o Business processes may need to change to fully leverage this new capability

o For CIBC, there is a good “payback” for this investment


Thank You

contact: [email protected]

supercharged analytics - sas...o strong capability in predictive analytics and data exploration is...

Documents