modeling reserve variability november 13, 2007 mark r. shapland, fcas, asa, maaa louise a. francis,...
Post on 04-Jan-2016
222 Views
Preview:
TRANSCRIPT
Modeling Reserve Modeling Reserve VariabilityVariabilityNovember 13, 2007November 13, 2007
Mark R. Shapland, FCAS, ASA, Mark R. Shapland, FCAS, ASA, MAAAMAAA
Louise A. Francis, FCAS, MAAALouise A. Francis, FCAS, MAAA
Definitions of TermsDefinitions of Terms
Ranges vs. DistributionsRanges vs. Distributions
Methods vs. ModelsMethods vs. Models
Types of Methods/ModelsTypes of Methods/Models
Advantages of DistributionsAdvantages of Distributions
Diagnostic TestingDiagnostic Testing
OverviewOverview
ReserveReserve – an amount carried in the liability – an amount carried in the liability section of a risk-bearing entity’s balance sheet for section of a risk-bearing entity’s balance sheet for claims incurred prior to a given accounting date.claims incurred prior to a given accounting date.
LiabilityLiability – the actual amount that is owed and will – the actual amount that is owed and will ultimately be paid by a risk-bearing entity for ultimately be paid by a risk-bearing entity for claims incurred prior to a given accounting date.claims incurred prior to a given accounting date.
Loss LiabilityLoss Liability – the – the expected valueexpected value of all of all estimated future claim estimated future claim paymentspayments..
RiskRisk ( (from the “risk-bearers” point of viewfrom the “risk-bearers” point of view) – the ) – the uncertainty (deviations from expected) in both uncertainty (deviations from expected) in both timing and amount of the future claim timing and amount of the future claim paymentpayment stream.stream.
Definitions of TermsDefinitions of Terms
Process RiskProcess Risk – the randomness of future – the randomness of future outcomes given a outcomes given a knownknown distribution of distribution of possible outcomes.possible outcomes.
Parameter RiskParameter Risk – the potential error in the – the potential error in the estimated parameters used to describe the estimated parameters used to describe the distribution of possible outcomes, assuming distribution of possible outcomes, assuming the process generating the outcomes is the process generating the outcomes is knownknown..
Model RiskModel Risk – the chance that the model – the chance that the model (“process”) used to estimate the distribution (“process”) used to estimate the distribution of possible outcomes is incorrect or of possible outcomes is incorrect or incomplete.incomplete.
Definitions of TermsDefinitions of Terms
RiskRisk – unknown outcomes with – unknown outcomes with quantifiablequantifiable probabilities. probabilities.
UncertaintyUncertainty – unknown outcomes that – unknown outcomes that cannot be cannot be estimatedestimated or or quantifiedquantified..
All entrepreneurship involves both All entrepreneurship involves both riskrisk (which can be transferred) and (which can be transferred) and uncertaintyuncertainty (which cannot be (which cannot be transferred).transferred).
Definitions of TermsDefinitions of Terms
Source: Knight, Frank H. 1921. “Risk, Uncertainty, and Profit.” Houghton Mifflin.
Variance, standard deviation, kurtosis, Variance, standard deviation, kurtosis, average absolute deviation, Value at Risk, average absolute deviation, Value at Risk, Tail Value at Risk, Tail Value at Risk, etcetc. which are measures . which are measures of dispersion. of dispersion.
Other measures useful in determining Other measures useful in determining “reasonableness” could include: mean, “reasonableness” could include: mean, mode, median, pain function, mode, median, pain function, etcetc..
The choice for measure of risk will also be The choice for measure of risk will also be important when considering the important when considering the “reasonableness” and “materiality” of the “reasonableness” and “materiality” of the reserves in relation to the capital position. reserves in relation to the capital position.
Definitions of TermsDefinitions of Terms
Measures of Risk from Statistics:Measures of Risk from Statistics:
A “Range” is not the same as a “Distribution”A “Range” is not the same as a “Distribution”
A Range of Reasonable EstimatesA Range of Reasonable Estimates is a range is a range of estimates that could be produced by of estimates that could be produced by appropriate actuarial methods or alternative appropriate actuarial methods or alternative sets of assumptions that the actuary judges to sets of assumptions that the actuary judges to be reasonable.be reasonable.
A A DistributionDistribution is a statistical function that is a statistical function that attempts to quantify probabilities of all attempts to quantify probabilities of all possible outcomes.possible outcomes.
Ranges vs. DistributionsRanges vs. Distributions
Ranges vs. DistributionsRanges vs. Distributions
A Range, by itself, creates problems:A Range, by itself, creates problems:
A range can be misleading to the A range can be misleading to the layperson – it can give the impression layperson – it can give the impression that any number in that range is equally that any number in that range is equally likely. likely.
A range can give the impression that as A range can give the impression that as long as the carried reserve is “within long as the carried reserve is “within the range” anything is reasonable.the range” anything is reasonable.
Ranges vs. DistributionsRanges vs. Distributions
A Range, by itself, creates problems:A Range, by itself, creates problems:
There is currently no specific guidance There is currently no specific guidance on how to consistently determine a on how to consistently determine a range within the actuarial community range within the actuarial community (e.g., +/- X%, +/- $X, using various (e.g., +/- X%, +/- $X, using various estimates, etc.). estimates, etc.).
A range, in and of itself, needs some A range, in and of itself, needs some other context to help define it (e.g., other context to help define it (e.g., how to you calculate a risk margin?)how to you calculate a risk margin?)
Ranges vs. DistributionsRanges vs. Distributions
A Distribution provides:A Distribution provides:
Information about “all” possible Information about “all” possible outcomes. outcomes.
Context for defining a variety of other Context for defining a variety of other measures (e.g., risk margin, measures (e.g., risk margin, materiality, risk based capital, etc.)materiality, risk based capital, etc.)
Ranges vs. DistributionsRanges vs. Distributions
Should we use the same:Should we use the same:
Criteria for judging the quality of a Criteria for judging the quality of a range vs. a distribution? range vs. a distribution?
Basis for determining materiality? risk Basis for determining materiality? risk margins?margins?
Selection process for which numbers Selection process for which numbers are “reasonable” to chose from?are “reasonable” to chose from?
A A MethodMethod is an algorithm or recipe is an algorithm or recipe – a series of steps that are followed – a series of steps that are followed to give an estimate of future to give an estimate of future payments.payments.
The well known chain ladder (CL) The well known chain ladder (CL) and Bornhuetter-Ferguson (BF) and Bornhuetter-Ferguson (BF) methods are examples.methods are examples.
The search for the “best” pattern.The search for the “best” pattern.
Methods vs. ModelsMethods vs. Models
A A ModelModel specifies statistical specifies statistical assumptions about the loss process, assumptions about the loss process, usually leaving some parameters to be usually leaving some parameters to be estimated.estimated.
Then estimating the parameters gives Then estimating the parameters gives an estimate of the ultimate losses and an estimate of the ultimate losses and some statistical properties of that some statistical properties of that estimate.estimate.
The search for the “best” distribution.The search for the “best” distribution.
Methods vs. ModelsMethods vs. Models
Methods vs. ModelsMethods vs. Models
Many good probability models have been Many good probability models have been built using “Collective Risk Theory” built using “Collective Risk Theory”
Each of these models make assumptions Each of these models make assumptions about the processes that are driving claims about the processes that are driving claims and their settlement valuesand their settlement values
None of them can ever completely None of them can ever completely eliminate “model risk”eliminate “model risk”
““All models are wrong. Some models are All models are wrong. Some models are useful.”useful.”
Types of ModelsTypes of Models
Individual Claim ModelsTriangle Based Models vs.
Conditional Models Unconditional Modelsvs.
Single Triangle Models Multiple Triangle Modelsvs.
Parametric Models Non-Parametric Modelsvs.
Diagonal Term No Diagonal Termvs.
Fixed Parameters Variable Parametersvs.
Types of ModelsTypes of Models
Processes used to calculate liability Processes used to calculate liability ranges or distributions can be grouped ranges or distributions can be grouped into four general categories: into four general categories:
1) Multiple Projection Methods,1) Multiple Projection Methods,
2) Statistics from Link Ratio Models,2) Statistics from Link Ratio Models,
3) Incremental Models, and3) Incremental Models, and
4) Simulation Models4) Simulation Models
Advantages of Advantages of DistributionsDistributions
$11M $16M
We could define a “reasonable” We could define a “reasonable” range based on probabilities of the range based on probabilities of the distribution of possible outcomes.distribution of possible outcomes.
This can be translated into a range This can be translated into a range of liabilities that correspond to of liabilities that correspond to those probabilities.those probabilities.
Advantages of Advantages of DistributionsDistributions
Premise:Premise:
The “risk” in the data defines the range.The “risk” in the data defines the range.
Adds context to other statistical measures.Adds context to other statistical measures.
A “reserve margin” can be defined more A “reserve margin” can be defined more precisely.precisely.
Can be related to risk of insolvency and Can be related to risk of insolvency and materiality issues.materiality issues.
Others can define what is reasonable for Others can define what is reasonable for them.them.
Advantages of Advantages of DistributionsDistributions
A probability range has several advantages:A probability range has several advantages:
150150100100909011511510010097975050thth to 75 to 75thth PercentilePercentile
12012010010080801201201001008080Expected +/- 20%Expected +/- 20%
HighHighEVEVLowLowHighHighEVEVLowLowMethodMethod
More Volatile More Volatile LOBLOB
Relatively Stable Relatively Stable LOBLOB
Advantages of Advantages of DistributionsDistributions
Comparison of “Reasonable” Reserve RangesComparison of “Reasonable” Reserve Rangesby Methodby Method
Advantages of Advantages of DistributionsDistributions
Comparison of “Normal” vs. “Skewed” Liability Distributions
Advantages of Advantages of DistributionsDistributions
Comparison of Aggregate Liability Distributions
A
B
C
Added(100% Correlation)
Correlated(0% Correlation)
Advantages of Advantages of DistributionsDistributions
Comparison of Aggregate Liability Distributions
Value
Pro
bab
ility
Value
Pro
bab
ility
Before CorrelationBefore Correlation After CorrelationAfter Correlation
Capital = $1,000M Capital = $600M
Advantages of Advantages of DistributionsDistributions
Reasonable & Prudent Margin
Reasonable & Conservative Margin
1%1%16016015%15%12012040%40%808050%50%100100
Prob. Prob. Of Ins.Of Ins.
AmouAmountnt
Prob. Prob. Of Ins.Of Ins.
AmouAmountnt
Prob. Prob. Of Ins.Of Ins.
AmouAmountntProb. Prob.
AmouAmountnt
Situation CSituation CSituation BSituation BSituation ASituation ALoss ReservesLoss Reserves
Corresponding Surplus Depending on SituationCorresponding Surplus Depending on Situation
1%1%15015015%15%11011040%40%707075%75%110110
1%1%14014015%15%10010040%40%606090%90%120120
Advantages of Advantages of DistributionsDistributions
Comparison of “Reasonable” Reserve RangesComparison of “Reasonable” Reserve Rangeswith Probabilities of Insolvencywith Probabilities of Insolvency
“Low” Reserve Risk
10%10%12012040%40%808060%60%404090%90%140140
10%10%14014040%40%10010060%60%606075%75%120120
10%10%16016040%40%12012060%60%808050%50%100100
Prob. Prob. Of Ins.Of Ins.
AmouAmountnt
Prob. Prob. Of Ins.Of Ins.
AmouAmountnt
Prob. Prob. Of Ins.Of Ins.
AmouAmountntProb. Prob.
AmouAmountnt
Situation CSituation CSituation BSituation BSituation ASituation ALoss ReservesLoss Reserves
Corresponding Surplus Depending on SituationCorresponding Surplus Depending on Situation
Advantages of Advantages of DistributionsDistributions
Comparison of “Reasonable” Reserve RangesComparison of “Reasonable” Reserve Rangeswith Probabilities of Insolvencywith Probabilities of Insolvency
“Medium” Reserve Risk
20%20%606050%50%202080%80%-20-2090%90%200200
20%20%11011050%50%707080%80%303075%75%150150
20%20%16016050%50%12012080%80%808050%50%100100
Prob. Prob. Of Ins.Of Ins.
AmouAmountnt
Prob. Prob. Of Ins.Of Ins.
AmouAmountnt
Prob. Prob. Of Ins.Of Ins.
AmouAmountntProb. Prob.
AmouAmountnt
Situation CSituation CSituation BSituation BSituation ASituation ALoss ReservesLoss Reserves
Corresponding Surplus Depending on SituationCorresponding Surplus Depending on Situation
Advantages of Advantages of DistributionsDistributions
Comparison of “Reasonable” Reserve RangesComparison of “Reasonable” Reserve Rangeswith Probabilities of Insolvencywith Probabilities of Insolvency
“High” Reserve Risk
Principle of Greatest Common InterestPrinciple of Greatest Common Interest – the “largest amount” considered – the “largest amount” considered “reasonable” when a variety of constituents “reasonable” when a variety of constituents share a common goal or interest, such that share a common goal or interest, such that all common goals or interests are met; and all common goals or interests are met; and thethe
Principle of Least Common InterestPrinciple of Least Common Interest – – the “smallest amount” considered the “smallest amount” considered “reasonable” when a variety of constituents “reasonable” when a variety of constituents share a common goal or interest, such that share a common goal or interest, such that all common goals or interests are met.all common goals or interests are met.
Advantages of Advantages of DistributionsDistributions
Satisfying Different Constituents:Satisfying Different Constituents:
Advantages of Advantages of DistributionsDistributions
$11M $16M
Advantages of Advantages of DistributionsDistributions
$11M $16M
Advantages of Advantages of DistributionsDistributions
$11M $16M
Advantages of Advantages of DistributionsDistributions
$11M $16M
Advantages of Advantages of DistributionsDistributions
$11M $16M
Advantages of Advantages of DistributionsDistributions
Reasonable Range
Reasonable Distribution
Advantages of Advantages of DistributionsDistributions
A Distribution can be used for:A Distribution can be used for:Risk-Based / Economic CapitalRisk-Based / Economic Capital
Reserve RiskReserve RiskPricing RiskPricing RiskALM RiskALM Risk
Pricing / ROEPricing / ROEReinsurance AnalysisReinsurance Analysis
Quota ShareQuota ShareAggregate ExcessAggregate ExcessStop LossStop LossLoss Portfolio TransferLoss Portfolio Transfer
Dynamic Risk Modeling (DFA)Dynamic Risk Modeling (DFA)Strategic Planning / ERMStrategic Planning / ERM
Allocated Capital
Risk Transfer
The Need for TestingThe Need for Testing
Actuaries Have Built Many Actuaries Have Built Many Sophisticated Models Based on Sophisticated Models Based on Collective Risk Theory Collective Risk Theory
All Models Make Simplifying All Models Make Simplifying AssumptionsAssumptions
How do we Evaluate Them?How do we Evaluate Them?
How Do We “Evaluate”?How Do We “Evaluate”?
$11M $16M
(Point Estimates)(Point Estimates)
Liability Estimates
Pro
bab
ility
How Do We “Evaluate”?How Do We “Evaluate”?(Multiple Distributions)(Multiple Distributions)
Liability Estimates
Pro
bab
ility
How Do We “Evaluate”?How Do We “Evaluate”?(Competing Distributions)(Competing Distributions)
Ranges vs. DistributionsRanges vs. Distributions
Ranges vs. DistributionsRanges vs. Distributions
Reasonable Range
Reasonable Distribution
The Need for TestingThe Need for Testing
Actuaries Have Built Many Actuaries Have Built Many Sophisticated Models Based on Sophisticated Models Based on Collective Risk Theory Collective Risk Theory
All Models Make Simplifying All Models Make Simplifying AssumptionsAssumptions
How do we Evaluate Them?How do we Evaluate Them?
Fundamental QuestionsFundamental Questions
How Well Does the Model How Well Does the Model Measure and Reflect the Measure and Reflect the Uncertainty Inherent in the Uncertainty Inherent in the Data?Data?
Does the Model do a Good Job of Does the Model do a Good Job of Capturing and Replicating the Capturing and Replicating the Statistical Features Found in the Statistical Features Found in the Data? Data?
Modeling GoalsModeling Goals
Is the Goal to Minimize the Range Is the Goal to Minimize the Range (or Uncertainty) that Results from (or Uncertainty) that Results from the Model?the Model?
Goal of Modeling is Goal of Modeling is NOTNOT to to Minimize Process Uncertainty!Minimize Process Uncertainty!
Goal is to Find the Best Statistical Goal is to Find the Best Statistical Model, While Minimizing Model, While Minimizing Parameter and Model Uncertainty. Parameter and Model Uncertainty.
Liability Estimates
Pro
bab
ility
How Do We “Evaluate”?How Do We “Evaluate”?
Diagnostic TestsDiagnostic Tests
Model Selection CriteriaModel Selection Criteria
Model Reasonability ChecksModel Reasonability Checks
Goodness-of-Fit & Prediction Goodness-of-Fit & Prediction ErrorsErrors
Model Selection CriteriaModel Selection Criteria
Criterion 1Criterion 1: : Aims of the AnalysisAims of the Analysis
Will the Procedure Achieve the Will the Procedure Achieve the Aims of the Analysis?Aims of the Analysis?
Criterion 2Criterion 2: : Data AvailabilityData Availability
Access to the Required Data Access to the Required Data Elements?Elements?
Unit Record-Level Data or Unit Record-Level Data or Summarized “Triangle” Data?Summarized “Triangle” Data?
Model Selection CriteriaModel Selection Criteria
Criterion 3Criterion 3: : Non-Data Specific Non-Data Specific Modeling Technique EvaluationModeling Technique Evaluation
Has Procedure been Validated Has Procedure been Validated Against Historical Data? Against Historical Data?
Verified to Perform Well Against Verified to Perform Well Against Dataset with Similar Features?Dataset with Similar Features?
Assumptions of the Model Plausible Assumptions of the Model Plausible Given What is Known About the Given What is Known About the Process Generating this Data?Process Generating this Data?
Model Selection CriteriaModel Selection Criteria
Criterion 4Criterion 4:: Cost/Benefit Cost/Benefit ConsiderationsConsiderations
Can Analysis be Performed Using Can Analysis be Performed Using Widely Available Software?Widely Available Software?
Analyst Time vs. Computer Time?Analyst Time vs. Computer Time?
How Difficult to Describe to How Difficult to Describe to Junior Staff, Senior Management, Junior Staff, Senior Management, Regulators, Auditors, etc.?Regulators, Auditors, etc.?
Model Reasonability Model Reasonability ChecksChecks
Criterion 5Criterion 5: : Coefficient of Coefficient of Variation by YearVariation by Year
Should be Largest for Oldest Should be Largest for Oldest (Earliest) Year(Earliest) Year
Criterion 6Criterion 6: : Standard Error by Standard Error by YearYear
Should be Smallest for Oldest Should be Smallest for Oldest (Earliest) Year (on a Dollar Scale)(Earliest) Year (on a Dollar Scale)
Model Reasonability Model Reasonability ChecksChecks
Criterion 7Criterion 7:: Overall Coefficient of Overall Coefficient of VariationVariation
Should be Smaller for All Years Should be Smaller for All Years Combined than any Individual YearCombined than any Individual Year
Criterion 8Criterion 8:: Overall Standard Overall Standard ErrorError
Should be Larger for All Years Should be Larger for All Years Combined than any Individual YearCombined than any Individual Year
Model Reasonability Model Reasonability ChecksChecks
Accident Yr MeanStandard
ErrorCoeffient of
Variation1996 26,416 37,927 143.6%1997 26,216 38,774 147.9%1998 50,890 54,508 107.1%1999 90,705 74,824 82.5%2000 148,110 99,986 67.5%2001 186,832 117,230 62.7%2002 418,461 183,841 43.9%2003 638,082 268,578 42.1%2004 607,107 477,760 78.7%2005 1,521,202 1,017,129 66.9%Total 3,714,020 1,299,184 35.0%
Model Reasonability Model Reasonability ChecksChecks
Accident Yr MeanStandard
ErrorCoeffient of
Variation1996 25,913 37,956 146.5%1997 25,708 38,846 151.1%1998 50,043 54,780 109.5%1999 89,071 74,987 84.2%2000 145,388 100,373 69.0%2001 183,864 118,502 64.5%2002 411,367 185,211 45.0%2003 628,347 271,722 43.2%2004 1,113,073 229,923 20.7%2005 1,263,550 253,596 20.1%Total 3,936,326 599,048 15.2%
Model Reasonability Model Reasonability ChecksChecks
Criterion 9Criterion 9:: Correlated Standard Correlated Standard Error & Coefficient of VariationError & Coefficient of Variation
Should Both be Smaller for All Should Both be Smaller for All LOBs Combined than the Sum of LOBs Combined than the Sum of Individual LOBsIndividual LOBs
Criterion 10Criterion 10:: Reasonability of Reasonability of Model Parameters and Model Parameters and Development PatternsDevelopment Patterns
Is Loss Development Pattern Is Loss Development Pattern Implied by Model Reasonable?Implied by Model Reasonable?
Model Reasonability Model Reasonability ChecksChecks
Criterion 11Criterion 11:: Consistency of Consistency of Simulated Data with Actual DataSimulated Data with Actual Data
Can you Distinguish Simulated Can you Distinguish Simulated Data from Real Data?Data from Real Data?
Criterion 12Criterion 12:: Model Model Completeness and ConsistencyCompleteness and Consistency
Is it Possible Other Data Elements Is it Possible Other Data Elements or Knowledge Could be Integrated or Knowledge Could be Integrated for a More Accurate Prediction?for a More Accurate Prediction?
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Criterion 13Criterion 13: : Validity of Link Validity of Link RatiosRatios
Link Ratios are a Form of Link Ratios are a Form of Regression and Can be Tested Regression and Can be Tested StatisticallyStatistically
Exercise: Test the Validity of Exercise: Test the Validity of the 12 to 24 month age-to-age the 12 to 24 month age-to-age factors for the Mack data.factors for the Mack data.
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Graph the ratios of the 12 to 24 Graph the ratios of the 12 to 24 month factors (12 month values month factors (12 month values on X axis and 24 month values on X axis and 24 month values on Y axis).on Y axis).
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Calculate the linear regression of Y Calculate the linear regression of Y given X values, without an given X values, without an intercept.intercept.
Add a regression line to the Graph Add a regression line to the Graph of age-to-age ratios.of age-to-age ratios.
What is the slope of the regression?What is the slope of the regression?
Was it a good fit?Was it a good fit?
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Calculate the linear regression of Calculate the linear regression of Y given X values, with an Y given X values, with an intercept.intercept.
Add another regression line to the Add another regression line to the Graph of age-to-age ratios.Graph of age-to-age ratios.
What is the intercept & slope of What is the intercept & slope of the regression?the regression?
Was it a good fit?Was it a good fit?
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Criterion 13Criterion 13: : Validity of Link RatiosValidity of Link Ratios
Link Ratios are a Form of Regression Link Ratios are a Form of Regression and Can be Tested Statisticallyand Can be Tested Statistically
Criterion 14Criterion 14: : Standardization of Standardization of ResidualsResiduals
Standardized Residuals Should be Standardized Residuals Should be Checked for Normality, Outliers, Checked for Normality, Outliers, Heteroscedasticity, etc.Heteroscedasticity, etc.
Standardized Standardized ResidualsResiduals
The residual equals the actual value The residual equals the actual value for a variable minus its expected for a variable minus its expected valuevalue
In general, a standardized residual In general, a standardized residual equals the residual divided by the equals the residual divided by the standard deviation of the variablestandard deviation of the variable
Sometimes residuals for a weighted Sometimes residuals for a weighted model (for instance WAD model) is model (for instance WAD model) is divided by the square root of the divided by the square root of the weightweight
Standardized ResidualsStandardized Residuals
Plot of Residuals against Predicted
-2.0000
-1.5000
-1.0000
-0.5000
0.0000
0.5000
1.0000
1.5000
5.0000 6.0000 7.0000 8.0000
Predicted
Res
idua
ls
Standardized ResidualsStandardized Residuals
Plot of Residuals against Predicted
-0.8000
-0.6000
-0.4000
-0.2000
0.0000
0.2000
0.4000
0.6000
0.8000
4.0000 5.0000 6.0000 7.0000 8.0000
Predicted
Res
idua
ls
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Criterion 15Criterion 15: : Analysis of Analysis of Residual PatternsResidual Patterns
Check Against Accident, Check Against Accident, Development and Calendar Development and Calendar PeriodsPeriods
Standardized ResidualsStandardized Residuals
Plot of Residuals against Predicted
-0.8000
-0.6000
-0.4000
-0.2000
0.0000
0.2000
0.4000
0.6000
0.8000
4.0000 5.0000 6.0000 7.0000 8.0000
Predicted
Res
idu
als
Plot of Residuals against Development Period
-0.8000
-0.6000
-0.4000
-0.2000
0.0000
0.2000
0.4000
0.6000
0.8000
Development Period
Res
idua
ls
Plot of Residuals against Accident Period
-0.8000
-0.6000
-0.4000
-0.2000
0.0000
0.2000
0.4000
0.6000
0.8000
0 2 4 6 8 10 12
Accident Period
Res
idua
l
Plot of Residuals against Payment Period
-2.5000
-2.0000
-1.5000
-1.0000
-0.5000
0.0000
0.5000
1.0000
0 2 4 6 8 10 12
Payment Period
Res
idu
al
Standardized ResidualsStandardized ResidualsPlot of Residuals against Development Period
-600.00
-400.00
-200.00
0.00
200.00
400.00
600.00
800.00
Development Period
Res
idua
ls
Plot of Residuals against Accident Period
-600.00
-400.00
-200.00
0.00
200.00
400.00
600.00
800.00
Accident Period
Res
idua
l
Plot of Residuals against Payment Period
-600.00
-400.00
-200.00
0.00
200.00
400.00
600.00
800.00
Payment Period
Res
idua
l
Plot of Residuals against Predicted
-600.00
-400.00
-200.00
0.00
200.00
400.00
600.00
800.00
0 200000 400000 600000 800000 1000000 1200000 1400000
Predicted
Res
idua
ls
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Criterion 15Criterion 15: : Analysis of Residual Analysis of Residual PatternsPatterns
Check Against Accident, Check Against Accident, Development and Calendar PeriodsDevelopment and Calendar Periods
Criterion 16Criterion 16:: Prediction Error and Prediction Error and Out-of-Sample DataOut-of-Sample Data
Test the Accuracy of Predictions on Test the Accuracy of Predictions on Data that was Not Used to Fit the Data that was Not Used to Fit the ModelModel
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Criterion 17Criterion 17: : Goodness-of-Fit Goodness-of-Fit MeasuresMeasures
Quantitative Measures that Enable Quantitative Measures that Enable One to Find Optimal Tradeoff Between One to Find Optimal Tradeoff Between Minimizing Model Bias and Predictive Minimizing Model Bias and Predictive VarianceVarianceAdjusted Sum of Squared Errors Adjusted Sum of Squared Errors (SSE)(SSE)
Akaike Information Criterion (AIC)Akaike Information Criterion (AIC)Bayesian Information Criterion (BIC)Bayesian Information Criterion (BIC)
Goodness-of-Fit & Goodness-of-Fit & Prediction ErrorsPrediction Errors
Criterion 18Criterion 18: : Ockham’s Razor Ockham’s Razor and the Principle of Parsimonyand the Principle of Parsimony
All Else Being Equal, the Simpler All Else Being Equal, the Simpler Model is PreferableModel is Preferable
Criterion 19Criterion 19:: Model Validation Model Validation
Systematically Remove Last Systematically Remove Last Several Diagonals and Make Several Diagonals and Make Same Forecast of Ultimate Values Same Forecast of Ultimate Values Without the Excluded DataWithout the Excluded Data
Questions? Questions?
Suggested BibliographySuggested Bibliography
[3][3] Barnett, Glen and Ben ZehnwirthBarnett, Glen and Ben Zehnwirth. . 2000. 2000. Best Best Estimates for ReservesEstimates for Reserves. PCAS LXXXVII: 245-321. . PCAS LXXXVII: 245-321.
[15][15] The Analysis and Estimation of Loss & ALAE The Analysis and Estimation of Loss & ALAE Variability: A Summary ReportVariability: A Summary Report. 2005. . 2005. CAS Working CAS Working Party on Quantifying Variability in Reserve EstimatesParty on Quantifying Variability in Reserve Estimates. . CAS CAS ForumForum (Fall): 29-146. (Fall): 29-146.
[25][25] Shapland, Mark RShapland, Mark R. 2007.. 2007. Loss Reserve Estimates: A Loss Reserve Estimates: A Statistical Approach for Determining ‘Reasonableness’Statistical Approach for Determining ‘Reasonableness’. . VarianceVariance 1, no. 1 (Spring): 120-148. 1, no. 1 (Spring): 120-148.
[27][27] Venter, Gary GVenter, Gary G. 1998. . 1998. Testing the Assumptions of Age-Testing the Assumptions of Age-to-Age Factorsto-Age Factors. PCAS LXXXV: 807-47.. PCAS LXXXV: 807-47.
[30][30] Zehnwirth, BenZehnwirth, Ben. 1994. . 1994. Probabilistic Development Probabilistic Development Factor Models with Applications to Loss Reserve Factor Models with Applications to Loss Reserve Variability, Prediction Intervals and Risk Based CapitalVariability, Prediction Intervals and Risk Based Capital. . CAS CAS ForumForum 2 (Spring): 447-605. 2 (Spring): 447-605.
Contact InformationContact Information
Mark R. Shapland, FCAS, ASA, MAAAMark R. Shapland, FCAS, ASA, MAAAE-Mail:E-Mail:mark.shapland@milliman.commark.shapland@milliman.comOffice:Office: (636) 273-6428(636) 273-6428
Louis A. Francis, FCAS, MAAALouis A. Francis, FCAS, MAAAE-Mail:E-Mail: louise_francis@msn.comlouise_francis@msn.comOffice:Office: (215) 923-1567(215) 923-1567
top related