tm exposure data quality and catastrophe modeling rick anderson february 28, 2002

TMTM

Exposure Data Quality and Exposure Data Quality and Catastrophe ModelingCatastrophe Modeling

Rick AndersonRick Anderson

February 28, 2002February 28, 2002

© 2002 Risk Management © 2002 Risk Management Solutions, Inc.Solutions, Inc.

Data Quality IssuesData Quality Issues

Is the insurance to value being accurately reflected? Is the insurance to value being accurately reflected? Does my data capture my actual exposure on a Does my data capture my actual exposure on a

regional and peril basis? regional and peril basis? Do I understand the default assumptions in my data? Do I understand the default assumptions in my data? Do I know that the information my brokers and Do I know that the information my brokers and

agents are providing me is correct? agents are providing me is correct? Am I capturing my aggregate information correctly?Am I capturing my aggregate information correctly?


Statement of the ProblemStatement of the Problem

What is the impact of poor data quality on:What is the impact of poor data quality on:– Exposure data valuesExposure data values– Uncertainty in modeled lossesUncertainty in modeled losses– Business decisions (external and internal)Business decisions (external and internal)

How do I quantify / score data qualityHow do I quantify / score data quality– On a location basisOn a location basis– On a policy basisOn a policy basis– On an aggregate portfolio basisOn an aggregate portfolio basis

How do I optimize data quality given my current business How do I optimize data quality given my current business constraints?constraints?

What improvements should I be making?What improvements should I be making?


Tackling the ProblemTackling the Problem

Close working relationship with business partnersClose working relationship with business partners

– AgentsAgents

– ReinsurersReinsurers

– ModelerModeler Development of a structured data quality assessmentDevelopment of a structured data quality assessment Ability to identify specific data quality issues and their Ability to identify specific data quality issues and their

impact on portfolio risk assessment at all levels.impact on portfolio risk assessment at all levels. Development of a consistent independent data Development of a consistent independent data

quality measurequality measure

– Data Quality Index (DQI)Data Quality Index (DQI)


Data Quality in the Context of Data Flow Data Quality in the Context of Data Flow Primary Insurance Company PerspectivePrimary Insurance Company Perspective

Pricing,Reinsurance,

Cap. Allocation,etc.

Exposure Database

ProductionStream

Cat ModelAnalysis

Data Acquisition(Source Data)

Data Resolution Analysis

Process/ Operational Accuracy Analysis

Data Acquisition Accuracy Analysis

What does it mean?What matters?

DA

TA

QU

AL

ITY

DA

TA

FL

OW


Components of Data QualityComponents of Data Quality

Accuracy componentAccuracy component Resolution componentResolution component


Data AccuracyData Accuracy


Examining the Components of Examining the Components of Exposure Data Quality: Exposure Data Quality: Data AccuracyData Accuracy

How accurately is my data being captured and How accurately is my data being captured and processed?processed?

Examination of processes through interviews and Examination of processes through interviews and exposure data queriesexposure data queries – Data acquisitionData acquisition– Data processingData processing– Operations / systemsOperations / systems

Market dependentMarket dependent

Logic tree assessment framework Logic tree assessment framework


Data Accuracy ComponentsData Accuracy Components

Data acquisition (source of data origination)Data acquisition (source of data origination)– Conditional on type of source and line of Conditional on type of source and line of

businessbusiness– Source reputation / biasSource reputation / bias– Source data vintage / validity / consistency / Source data vintage / validity / consistency /

interpretabilityinterpretability Data processingData processing

– Conditional on line of businessConditional on line of business– Bias / vintage / validity / consistency / Bias / vintage / validity / consistency /

interpretabilityinterpretability Operations / systemsOperations / systems

– Data accessibility / data integration / systems Data accessibility / data integration / systems process / operations value to costprocess / operations value to cost


Accuracy Component of Data Quality Accuracy Component of Data Quality Assessment FrameworkAssessment Framework

1. Data Acquisition

2. Data Processing

3. Operations

Accuracy Component Data Quality

Questionnaire 1

Questionnaire 3

Questionnaire 2

W3

W2

W1

Warning flags from queries of the exposure database

Peril and LOB specific On-Site Questions

Components of Data Flow


Example Logic Tree with Data Accuracy CriteriaExample Logic Tree with Data Accuracy Criteria

Data Data AcquisitionAcquisitionAccuracy Accuracy ScoreScore

Reputation/BiasReputation/Bias

Data Data

ReputationReputation

BiasBias

VintageVintage

ValidityValidity

ConsistencyConsistency

InterpretabilityInterpretability

Question 1Question 1Question2Question2

..

..

..

Question 11Question 11Question 12Question 12

..

..

..

Question 29Question 29Question 30Question 30

..

..

..

0.30.3

0.70.7

0.30.3

0.50.5

0.50.5

0.20.2

0.40.4

0.10.1

DirectDirectIndependent AgentIndependent AgentWholesale BrokerWholesale BrokerRetail BrokerRetail BrokerRisk Retention GroupRisk Retention Group

Integrated Data SubmissionIntegrated Data SubmissionCatastrophe Model EDMCatastrophe Model EDMDigital (Spreadsheet, Word Doc, etc.)Digital (Spreadsheet, Word Doc, etc.)Paper SubmissionPaper Submission


Development of Data Accuracy Criteria Relative Development of Data Accuracy Criteria Relative Importance WeightsImportance Weights

Assessed as relative impact on modeled losses and Assessed as relative impact on modeled losses and key data quality issueskey data quality issues

Based on:Based on:– Extensive interviews with Cat managers, Extensive interviews with Cat managers,

underwriters and systems personnelunderwriters and systems personnel– Results of relative parameter impact analyses on Results of relative parameter impact analyses on

AAL (data validity criteria)AAL (data validity criteria)– Availability of other information from which to draw Availability of other information from which to draw

assumptionsassumptions

Line of business, peril, and region dependentLine of business, peril, and region dependent


Data Accuracy Criteria Data Accuracy Criteria Development of QuestionnaireDevelopment of Questionnaire

Questionnaire is administered through interview processQuestionnaire is administered through interview process

Questions are multiple choiceQuestions are multiple choice– Yes / NoYes / No– Always / Most of the Time / Occasionally / NeverAlways / Most of the Time / Occasionally / Never

Number and content of questions designed to Number and content of questions designed to adequately assess how criteria are addressed at adequately assess how criteria are addressed at companycompany

Normalized relative importance weighting applied to Normalized relative importance weighting applied to questions within each criteriaquestions within each criteria


Warning Flags Summaries from DB QueriesWarning Flags Summaries from DB Queries

Used as supporting information in answering Used as supporting information in answering questionnairequestionnaire

Warning flagsWarning flags– Data consistencyData consistency

• Address entryAddress entry• ValuesValues• Construction and occupancy class/schemaConstruction and occupancy class/schema

– Data vintageData vintage– Data biasData bias

• Secondary characteristicsSecondary characteristics• Primary characteristicsPrimary characteristics


Warning Flags Summaries from DB Queries – Warning Flags Summaries from DB Queries – Sample ResultsSample Results

Data Vintage – Use of Policy Status FlagData Vintage – Use of Policy Status Flag

StatusStatus # of Policies# of Policies % of Total Policies% of Total Policies

BOOKBOOK 11 16.7%16.7%

No Status No Status 55 83.3%83.3%

Data Consistency – Value Entry CheckData Consistency – Value Entry Check

AddressAddress Total Total TotalTotal AverageAverage MatchMatch LocationsLocations ValueValue ValueValue

Street LevelStreet Level 5 5 $1,270,000 $1,270,000 $254,020$254,020

Zip LevelZip Level 1 1 $147,500 $147,500 $147,500$147,500


1. Data Acquisition Accuracy1. Data Acquisition Accuracy

Conditional on type of provider of data, data Conditional on type of provider of data, data format, submission process, and line of businessformat, submission process, and line of business

Data acquisition accuracy componentsData acquisition accuracy components

– ValidityValidity

– VintageVintage

– Data provider bias Data provider bias

– Data provider reputation Data provider reputation

– ConsistencyConsistency

– InterpretabilityInterpretability


Acquisition Criteria: Relative ImportanceAcquisition Criteria: Relative Importance

Data vintageData vintage Location validity checksLocation validity checks Default value treatmentDefault value treatment Data acquisition biasData acquisition bias Data validity checksData validity checks Use of data alteration flagsUse of data alteration flags Data aggregationData aggregation Location entry consistencyLocation entry consistency Reputation of data providerReputation of data provider Secondary construction characteristics Secondary construction characteristics

treatmenttreatment

HighHigh

LowLow


2. Data Processing2. Data Processing

Conditional on database format, platform, and Conditional on database format, platform, and line of businessline of business

Incorporates results from queries of exposure Incorporates results from queries of exposure databasedatabase

Data processing accuracy componentsData processing accuracy components– BiasBias– ValidityValidity– InterpretabilityInterpretability– VintageVintage– ConsistencyConsistency


3. Systems / Operations Accuracy3. Systems / Operations Accuracy

Processing / operations data quality componentsProcessing / operations data quality components

– Data accessibility and storageData accessibility and storage

– Data integration and linkingData integration and linking

– Technology systems process/flowTechnology systems process/flow

– Operational value-to-costOperational value-to-cost


Data Accuracy - SummaryData Accuracy - Summary

Assessment of how closely processes arrive at the Assessment of how closely processes arrive at the true and accepted valuetrue and accepted value

Structured and consistent approachStructured and consistent approach

Ability to assess the contribution of individual Ability to assess the contribution of individual components to overall data accuracy score components to overall data accuracy score

Periodic assessment is valuable for internal process Periodic assessment is valuable for internal process reviewreview

Integral component to overall data qualityIntegral component to overall data quality


Data ResolutionData Resolution


Examining the Components Of Examining the Components Of Exposure Data Quality: Exposure Data Quality: Data ResolutionData Resolution

What data am I capturing and at what level?What data am I capturing and at what level?

Direct query of exposure data parametersDirect query of exposure data parameters – GeocodingGeocoding– ConstructionConstruction – OccupancyOccupancy – Year builtYear built – Building heightBuilding height – Construction modifiersConstruction modifiers

Peril, region and market dependentPeril, region and market dependent


Data Resolution Analysis Tree - Data Resolution Analysis Tree - California Earthquake ResidentialCalifornia Earthquake Residential

Cladding

HAZARD VULNERABILITY

Coordinate

Zip Code

County

Location Resolution

Const. Scheme

Occupancy Class

Secondary Characteristics

Construction Class

Year Built Number of Stories

Frame Bolted Down

Soft Story

Unknown

URM Chimney

Cripple Walls

UnknownInventoryRMS

ISO Fire Known

Unknown

Known

UnknownMFW Frame

SFW Frame

SF House

MF Housing

LOCATION

ATC


Florida HU – Location Sampling (10 km. Grid)Florida HU – Location Sampling (10 km. Grid)


Florida HU – Location Sampling (1 km. Grid)Florida HU – Location Sampling (1 km. Grid)

COLLIER

MONROE DADE

BROWARD

PALM BEACH

MARTIN


Development of Category WeightsDevelopment of Category Weights

Weights for individual categories are determined Weights for individual categories are determined through numerical simulation (analysis) of the impact through numerical simulation (analysis) of the impact of a given category on losses for the geography, of a given category on losses for the geography, peril, and LOB under considerationperil, and LOB under consideration

Final weights are normalized across the applicable Final weights are normalized across the applicable categoriescategories

CategoryCategory HighHigh Med.Med. LowLow

GeocodingGeocoding ww1a1a w w1b1b w w1c1c

Cons. SchemeCons. Scheme ww2a2a w w2b2b w w2c2c

Year BuiltYear Built ww5a5a w w5b5b w w5c5c

22ndnd. Char.. Char. ww6a6a w w6b6b w w6c6c

Extensive testing, validation, and benchmarkingExtensive testing, validation, and benchmarkingperformedperformed


Florida HU – Scoring RegionsFlorida HU – Scoring Regions

Scoring Region by Hazard

Very HighHighMediumLow


California EQ – Scoring RegionsCalifornia EQ – Scoring Regions

I-5

Scoring Region by Hazard

HighMediumLow


Resolution Geocoding Scores by Hazard Level Resolution Geocoding Scores by Hazard Level California Earthquake Residential California Earthquake Residential

100

75

25

100

80

30

100

85

40

0

10

20

30

40

50

60

70

80

90

100

Sco

re b

y G

eoco

din

g L

evel

High Hazard Medium Hazard Low hazardHazard Level

coordinates

street

zip

city

county


Data Resolution Category Score Summary – Data Resolution Category Score Summary – California Earthquake Residential High Hazard RegionCalifornia Earthquake Residential High Hazard Region

Category Attribute

Name Attribute

Score Catergory

Weight Weighted

Score Unknown 0 0.75 0.00 Coordinate 100 0.75 75.00 Street Address 95 0.75 71.25 Postal Code 80 0.75 60.00 City 25 0.75 18.75 County 25 0.75 18.75 State 0 0.75 0.00

Location Resolution

Cresta 0 0.75 0.00 RMS 100 0.01 1.00 ATC 100 0.01 1.00 ISO 85 0.01 0.85

Construction Scheme

ISO FIRE 75 0.01 0.75 Known 100 0.05 5.00 Construction Class Unknown 75 0.05 3.75 Known 100 0.04 4.00 Occupancy Class Unknown 70 0.04 2.80 Known 100 0.05 5.00 Year Built Unknown 50 0.05 2.50 Known 100 0.10 10.00 Few Unknown 60 0.10 6.00

Secondary Characteristics

Unknown 25 0.10 2.50


Data Resolution Category Weights Data Resolution Category Weights

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

High

Med

Low

Sei

smic

Reg

ion

s

Score (%)

Geocoding

Construction Scheme

Construction Class

Occupancy Class

Year Built

Secondary Chars.

California Earthquake Residential


Data Resolution Category WeightsData Resolution Category Weights

Florida Hurricane Commercial

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Very High

High

Med

Low

Ha

zard

R

eg

ion

s

Score (%)

Geocoding

Construction SchemeConstructionClassOccupancy Class

Number of Stories

Secondary Chars.


Data Resolution - Aggregation MethodologyData Resolution - Aggregation MethodologyProgression of Data Resolution ScoringProgression of Data Resolution Scoring

Account 1Account 1 Account 2Account 2 Account 3Account 3

Commercial Commercial PortfolioPortfolio

LocationLocation

PortfoliPortfolioo

PP11

ScoreScore

AA11 AA33AA22

LL11

LL22

LLnn

LL11

LL22

LLnn

LL11

LL22

LLnn


Data Resolution - Data Resolution - Development of Relative “Importance” FactorsDevelopment of Relative “Importance” Factors

Relative importance is an approximation of the AAL.Relative importance is an approximation of the AAL. Ground-up AAL approximated at ZIP code level Ground-up AAL approximated at ZIP code level

based on insurance industry exposure.based on insurance industry exposure. Gross AAL approximated by average ratio of gross / Gross AAL approximated by average ratio of gross /

ground-up AAL per attachment point.ground-up AAL per attachment point.


Layer contribution to AALLayer contribution to AAL

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10% 20% 30% 40% 50%

Attachment Point (% of Value)

Per

cent

of

AA

L


Sample Resolution ScoresSample Resolution Scores

- 10 20 30 40 50 60 70 80 90 100

Company C

Company B

Company A

Score (%)

Geocoding

Construction Scheme

Construction Class

Occupancy Class

Number of Stories

Secondary Chars.


Improving Data Resolution – Improving Data Resolution – Leveraging Account Data Resolution ScoresLeveraging Account Data Resolution Scores

Identify accounts with score less than than target Identify accounts with score less than than target scorescore

Determine account potential for improvement as:Determine account potential for improvement as:

(Score Difference) * (Account Importance)(Score Difference) * (Account Importance)

Identify accounts with biggest improvement potential Identify accounts with biggest improvement potential and decide on strategy for data improvementand decide on strategy for data improvement


Improving Data Resolution – Targeting AccountsImproving Data Resolution – Targeting Accounts

Score Difference = Target Score - Account ScoreScore Difference = Target Score - Account Score Potential Improvement = (Score Difference) * (Importance)Potential Improvement = (Score Difference) * (Importance)

Target score 82.25

Account ScoreScore

Difference ImportancePotential

ImprovementSV Office Center 80.21 2.04 23,517,279 47,975,249So Cal Management 79.92 2.33 16,329,576 38,047,912Crown Ltd 80.54 1.71 9,816,495 16,786,206Putt Putt Motors 86.86 -4.61 1,460,773 -6,734,164Beach Apartments 87.78 -5.53 6,425,708 -35,534,165Palo Alto Condos 86.88 -4.63 13,022,979 -60,296,393

Average 82.25


Combining the ComponentsCombining the Components


Options for Combining Accuracy and Resolution Options for Combining Accuracy and Resolution ComponentsComponents

Keep separateKeep separate AdditiveAdditive MultiplicativeMultiplicative MinimumMinimum

tm exposure data quality and catastrophe modeling rick anderson february 28, 2002

Documents

data accuracyhow

data acquisition2

data quality issuesis

impact of poor data

specific data quality

score data qualityon

key data quality issuesbased

relative impact