On Determining an Appropriate Rate StabilizationReserve for MPI: Critique of Methodology and

Related Issues

Derek Hum, Ph. D.Professor of Economics

Wayne Simpson, Ph. D.Professor of Economics

8 September 2006

Prepared for the Public Interest Law Centre

Table of Contents

1 Introduction

2 Purpose of the Rate Stabilization reserve

3 Setting the RSR Level: The Risk Analysis Approach

4 Setting the RSR Level: The Minimum Capital Test Approach

4.1 MPI’s Discussion of the Rate Stabilization Reserve4.2 Dynamic Capital Adequacy Testing

4.2.1 Single large catastrophe scenario4.2.2 Inflation scenario

5 Rate Stabilization Reserve: Forensic Assessment

5.1 On the Use of Correlation in Risk Assessment5.2 On Forecast Bias by MPI5.3 On Serial Correlation in MPI Forecasts5.4 Are Risks for MPI Getting Larger Over Time?

6 Summary and Conclusions

7 Disclaimer

Tables: Table 1. Actual, MPI Forecast, and Linear Trend Forecast Net Income with and

without Operating ExpensesTable 2. Forecast Errors (Differences), Their Squares and Test for

Heteroscedasticity for Linear Trend and MPI ForecastsFigures:

Figure 1. Differences (Actual Minus Forecast) Based on MPI and Linear TrendForecasts

Figure 2. Squared Differences (Actual Minus Forecast) Divided by Mean forLinear Trend and MPI Forecasts

2

1 Introduction

This report concerns the appropriate sum that Manitoba PublicInsurance (MPI) should set aside in its Rate Stabilization Reserve(RSR).

To investigate this issue, we examine past practices and calculations.These include: the Risk Analysis Approach (RAA) suggested by thePublic Utilities Board (PUB), and the Minimum Capital Test (MCT)suggested by MPI.

Specifically, the report includes the following elements:

(1) A review of the previous findings of the PUB together withevidence it considered in arriving at its decision with respect to theRSR level. We also provide a discussion for conceptual clarity;that is, each party’s conception of the purpose of the RSR andwhat level it ought to be.

(2) Analysis of the respective positions and relative merits of theevidence/approach contained in the Todd Report, and various MPIdocuments relating to the RSR and its appropriate level.

(3) Discussion of the relative merits and appropriateness of theMinimum Capital Test approach and the Risk Analysis Approach.We also consider the Dynamic Capital Adequacy Testing (DCAT)exercise by Ernst and Young (performed by James Christie).

(4) An independent forensic statistical assessment of the forecastingrecord of MPI and an evaluation of their performance, in order toprovide a set of recommendations for consideration by the PUB.

3

2 Purpose of the Rate Stabilization Reserve (RSR)

The starting point must obviously be: What is the purpose of the RSR?The PUB and MPI appear to agree on the stated purpose of the RSR:

“The basic insurance rate stabilization reserverelates to basic compulsory automobile insurance andis intended to protect motorists from rate increasesmade necessary by unexpected events and lossesarising from non-recurring events or factors.” (MPI,2005 Annual Report, 48)

“As a basic principle… the RSR should beutilized to protect motorists from rate increases thatwould otherwise be necessary because of lossesfrom unexpected and non-recurring events.” (PUBOrder No. 150/05, Nov 14, 2005)

The statements of purpose quoted above are deserving of comment. In anideal world with perfect foresight, the amount of premiums collected eachand every year would exactly match the amount of claims paid plusoperating costs. However, forecasts are always inaccurate, and there willinevitably be forecasting errors; that is, a discrepancy between premiumrevenue and claims paid. Forecast errors are expected; as well, theseforecast errors can be expected to recur year after year, despite all bestefforts to improve forecasts. In other words, deviations (or “variances”)between what is forecasted and what occurs is to be expected, ex poste,every year, and this will have to be taken into account in planning for thenext fiscal year.

The RSR is not meant to smooth rate increases from one year to thenext. If it is deemed desirable to have some “smoothing” of annual rateincreases, a buffer reserve may be useful whenever it is judged advisableto implement a necessary rate increase over more than a single year.Forecast errors are normal and to be expected, rather than unexpected,and “smoothing” of rate increases, if desired, should be considered anormal management practice rather than “an unexpected event” arisingfrom “non recurring … factors”. But a buffer reserve for this purposeimplies that the required rate increase based upon forecasts of costs for

4

the next year, in combination with forecast errors of the past year, maylead to a projected rate increase that is “too large to be acceptable”.

This begs the question of what rate of increase is considered“unacceptably large”. The PUB and MPI must make that determination inlight of management experience and general economic climate, but thePUB has gone on record as establishing an overall cap of 20%, whichmeans, “… no vehicle premium may be increased or decreased by morethan 20% in any year…” (2005, Order 150/05 p. 11) This establishes somerough guide as to what is “unacceptably high” and gives some numericalbasis for a (maximum) bound of rate increase that would trigger a need fordrawing upon the RSR amount. (Of course, the PUB can always approve adraw on the RSR for any lower rate of premium increase it judgesnecessary upon review of all relevant circumstances.) The point to bemade here is simply that the RSR is not intended for smoothing of rateincreases due to normal forecast errors.

In short, we interpret the purpose of the RSR (as stated by MPI and PUBabove) to be for the contingency of “large” (left undefined, for the moment)unforeseen events that do not recur on a frequent basis. The RSR is notmeant to smooth annual adjustments in rate setting. To repeat, the RSR isintended only for a “large” non-recurring unexpected event.1

3 Setting the RSR Level: The Risk Analysis Approach (RAA)

The PUB suggests that the RSR be determined through a Risk AnalysisApproach. It is currently the main tool used to determine the appropriatelevel of the RSR.2

Briefly, the procedure calculates “risk margins” for (1) operational risk1 The Immobilizer Fund was declared to be part of the RSR for calculating RSRadequacy in Order 150/05 (p 9). These (forecast) expenditures are “foreseen” and are,more properly, part of normal operational expenditures associated with a particularinitiative. It is difficult to understand how a deliberately planned initiative such as theImmobilizer Program can be regarded as an “unforeseen” “non recurring” event.

2 We do not provide an exhaustive description of the procedure since it is documentedelsewhere. We assume readers of this report will be familiar with the method.

5

and (2) investment risks combined by employing historical data on the“discrepancies” between forecast and actual amounts for revenues, losscosts,3 operating expenses and claim expenses. Correlations betweenrisk components may (or may not) be employed, two confidence levelsare adopted (95% and 97.5%) and operating costs are alternativelyincluded and excluded.4

The implicit thinking behind this approach is that (1) past historicalexperience is a good guide for assessing the distribution of forecasterrors in the future (2) the major risks to be considered involveoperational risk and investment risk combined and (3) a risk margin canbe calculated to provide for the contingency of a “large, unforeseen nonrecurring event”.

It is important to note that the definition of an “unforeseen nonrecurring event” is given substance by adopting a confidence level.For example, crudely put, a 97.5% confidence level would correspond toan event having a 2.5% annual chance of occurring, or an one-in-fortyyear event. The definition of “large” remains subjective, and can varywith context.

The PUB (Order 151/2000) recommends the RAA method, and hasordered calculations be made employing, alternatively, a 95.0% and97.5% confidence level. Further, it suggested that the procedurealternatively include, and exclude, operating costs. It has repeatedlyrejected the MCT test advocated by MPI.

If one were to take the view that financing forecast errors in OperatingCosts is not a legitimate goal of the RSR, then operating costs can be

3 Loss costs refer to “net claims plus financial adjustments” (A16, Exhibit A)4 Note that we substitute the term “discrepancy” for the term “variance” betweenforecast and actual amount to avoid confusion. The term “variance” is used inaccounting terminology to describe the difference between forecast and actualamounts. However, the term “variance” (meaning the square of the standard deviation)is also a precise technical term in statistics that has relevance for computing confidenceintervals for forecasts. Throughout this report, we employ the language and acronymsused in the various PUB and MPI documents when no confusion is likely.

6

excluded from the RAA.5

In any case, according to MPI’s own data, the main source ofoperational risk utilizing the RAA approach is the “loss cost”component. In their listing of sources of operational risk (BasicAutopac Operational and Risk Analysis, June 14, 2006, Table 2, the risk(measured by standard deviation) associated with Loss Costs is aboutten times larger than the components for either operation expenses orclaim expenses. In short, not surprisingly, the main operational “risks”derive from not being able to predict with accuracy the losses arising inany given year. Accordingly, given the stated aim of the RSR, theprimary design consideration might be restated to be “what sort of‘large’ ‘unforeseen (loss) event’ must the RSR level be able toaccommodate?

The Todd report (Sept 2005) provides a chronology of the issuessurrounding the RSR. While both the PUB and MPI appear to agreewith the purpose of the RSR (see the statement of purpose of the RSRby both side above), disagreement continues to exist between the PUBand MPI as to (1) the appropriate method to calculate the riskassociated, and therefore, (2) the appropriate level of the RSR toestablish.6

The PUB requested that MPI update the RAA, giving some specific

5 Mr Todd takes this view. (Comments of John Todd on MPI’s Rate StabilizationReserve and Related Issues, (hereafter, Todd Report, Sept 2005, 11). “… MPI hascarried out its risk assessment by examining the variances from one year to the next inloss costs, claim expenses, and operating expenses. The implication of this approachto assessing the relevant risk is that the purpose of the RSR is to smooth rate increasesfrom year to year resulting from the trend in the underlying cost factors. Put simply, thisapproach measures rate instability, whether that instability is forecast or not…. thisapproach is not consistent with the objective …that the RSR is intended to protectmotorist from rate increases made necessary by unexpected events or factors. Thisdefinition implies that foreseen non-recurring events or factors that impact on revenuesand costs should not be addressed by drawing on the RSR to smooth rate impacts.Rates should be cost based, which means that in each year rates should be set so thatforecast revenues will be sufficient to recover forecast costs.” 6 We shall accept the reporting of the chronology by Todd as accurate, since it is welldocumented with respect to the various PUB Orders.

7

guidance and instructions, including the confidence levels to employ, toinclude only PIPP data, and to consider various scenarios. MPI partlycomplied, and has provided new estimates of the required RSR,including adjustments for inflation and new conditions.

It should be noted at the outset that there should be little objection inprinciple to updating historical “numbers” to current nominalvalues if there is good reason for doing so. Economists typicallyrecommend converting all historical figures to their present dayequivalent, or alternatively, where the context warrants, convertingpresent day figures to equivalent historical dollars in some base year. Itdoes not make much difference which method is chosen since typically,what one is attempting to do with such an exercise is separate thechanges that are due to “quantity magnitudes” (often termed “realchanges”) from changes that are simply due to inflation.

If the objective is simply to estimate the deviations of actual fromforecast in each year, this can be accomplished by stating the “error offorecast” in percentage terms.7 Then any percentage estimate of theforecast error can be converted to dollar amounts simply by multiplyingby the total dollar amount forecast. Given that claims will not be paid inhistorical dollars, it would seem preferable for ease of interpretation tocalculate amounts in present day dollars rather than historical dollars.

At issue is the appropriate manner to adjust for historical data. We seeno rationale for the procedure used by MPI. Indeed, the MPI procedurehas the property of actually increasing the variance of past years. TheMPI procedure is discussed further in our forensic statistical examination(Section 5).

Frustration abounds with respect to the RSR issue as no closureappears to be in sight. The methodology favoured by PUB is not onedesired by MPI. The PUB favours the RAA and expects MPI to use thisapproach (Order 150/05, 17). MPI favours the MCT test, even though it“lacks the technical ability to compute MCT ratios on its own “)(Ibid., p19)and this approach has been definitely rejected by PUB (Order 150/05 ff20-22).

Among the reasons listed in support of the RAA approach are: MPIcannot go bankrupt (Order 150/05, p20); there is no merit for the 50%

7 This is often referred to as a canonical form; that is, an expression that isindependent of units. It is difficult to understand exactly how and why MPI made thevarious adjustments that they did.

8

MCT standard (ibid., p21); and MCT is not a better determinant of anacceptable RSR range (ibid., p21). The root of the matter lies in thepurpose for which the MCT is designed and the type of organizationalstructures to which the MCT is applicable; namely, non-monopoly,private sector institutions with some real risk of insolvency. For thesetypes of institutions, a ‘private insurer’ capital has to be sufficient toensure continued solvency to meet claims” (ibid., p 21). Since this risk isabsent for MPI, MPI proposes to adjust the MCT ratio to 50%. Yet, thisdoes not address the fundamental matter --- which is, that the MCT testis a determination of capital adequacy, and not a determination of theadequacy of the level of RSR for the purpose stated; namely, toreiterate, “to protect motorists from rate increases that would otherwisebe necessary because of losses from unexpected and non-recurringevents.” (PUB Order No. 150/05, Nov 14, 2005).

This latter point cannot be emphasized enough. The MCT favoured byMPI does not address the RSR level directly. Rather, it assesses thecapital requirements to forestall insolvency for a private sector insurer;MPI offers the MCT test

(1) as an “indicative analysis” tool only, [our emphasis].

(2) then claims the ”appropriate “level of the RSR can then be“associated with” the proper MCT ratio, and further,

(3) MPI argues that setting the target MCT at 50% is the “rightadjustment” for a monopoly crown corporation.

We consider these arguments at greater length next.

4 Setting the RSR level: The Minimum Capital Test (MCT) Approach

The MCT approach is the one championed by MPI. Two particulardocuments are important in elaborating the MPI position. The reasonsfor MPI preferring the MCT approach are succinctly summarized in AI.15Discussion of the Rate Stabilization Reserve (June 14, 2006). Theevidence offered in support of the MCT approach is largely contained ina report by James Christie of Ernst & Young: Universal CompulsoryAutomobile Insurance, 2005 Dynamic Capital Adequacy Testing,September 23, 2005. (hereafter, the Christie Report).

9

4.1 MPI’s Discussion of the Rate Stabilization Reserve

We first consider the rationale articulated by MPI in the AI.15document that was tendered in support of their 2007 rate application.The AI.15 document attempts two objectives:

(1) Rationalizing the use of the MCT (in one section), whilesimultaneously,

(2) Arguing (in another section) that the RAA should not be used.[Sections are not numbered in AI.15.]

MPI acknowledges that the MCT was “not designed to deal with risksrelated to variances [deviations] from forecast for a crown corporation,such as MPI, in providing mandatory insurance products on amonopoly basis” (p2)[our emphasis], and that the MCT is designed to “assess the key risks faced by the insurance industry”. It adds the nonsequitur that the “majority of the risks …do not disappear for MPIbecause it is in a monopoly situation” and proposes that a range of50% to 100% of MCT (as opposed to the federal minimal standard of100%) be adopted to account for its monopoly status.8 The documentalso claims an advantage for the MCT approach in its use of currentfinancial statement information rather than a small sample of data ofhistorical forecast errors. Further, it is suggested that the MCT

8 How the 50% figure is determined is not explained, nor do we believe that there is anytechnical way of determining the “right” percentage. It is a matter of managementjudgment. However, in the prefiled testimony of M.J. McLaren, President and CEO ofPMI, she states that MPI can work with an end target of 50% to

provide rate stability and predictability for Manitobans … solely because ofthe commitment to transfer excess retained earnings from the twocompetitive lines of business to help in the rebuilding of the RSR….

She goes on to testify:This year, the Corporation suspended these transfers to RSR because thePUB position in Order 150/05 was that the RSR was in excess ofrequirements. It was never the Corporation’s intention to transfer fundsearned from ratepayers in the two competitive lines of business to fund arebate to basic insurance ratepayers.

This testimony raises issues as to the appropriate use of any “excess” RSR monies,and the appropriate degree of cross subsidy financing of the RSR. We consider theseissues outside our terms of reference. But obviously revenue risk for Basic will beaffected and dependent on policies that management adopt between Basic and thecompetitive lines. The PUB has recommended review of the cost allocations betweenBasic and Extension, as well as the decision to include DVL within Extension. (Order150/05, p. 47) These latter issues remain unresolved.

10

approach is consistent with those used in other financial sectors.

The question in our context, sharply raised, is how much adjustmentshould be made for a monopoly crown corporation selling a mandatoryproduct. A monopoly incurs no restraint on setting “prices” (premiums)from competing suppliers although the PUB monitors the rates set. Amonopoly does not have to worry about losing “market share”.9

Additionally, purchase of the product is compulsory. Further, a crowncorporation such as MPI is unlikely to be allowed to declarebankruptcy (by the Manitoba government). That is, its continuedexistence is assured. Insolvency risk simply does not exist.

It ought to be conceded by all reasonable persons that anyadjustment of a MCT standard set for a private sector competitiveindustry applied to a monopoly situation of a crown corporationis arbitrary, and no useful purpose is served by attempting a definitivenumber.

However, another (perhaps whimsical) way of phrasing the situation isto ask: --- rhetorically, of course, --- how much lower should the MCTbe if a private sector company were to be suddenly guaranteed withcertainty

• that its market share would always be 100%,

• that it could set its prices at any level it wished without fear ofcompetition, and

• that it could never go bankrupt?10

9 Over and above its 100% market share with respect to “Basic”, MPI even enjoys amarket share in excess of 80% for its competitive lines of business. (Order 150/05, p.5)Some economists would call the latter a virtual monopoly. The point is simply that forpractical purposes, MPI enjoys near monopoly in all of its product lines combined. 10 Yet another perspective for those who prefer to view the matter in balance sheetterms is as follows. How much should the asset side of the balance sheet reflect itsthree intangible “goodwill and other such nothings” associated with (i) a monopolycharter, (ii) a co-signatory against default, and (iii) a legislative enactment requiring allof its citizens to purchase this product. On this interpretation, the MCT has seriouslyomitted three of the most significant capital assets of MPI from the asset side of theledger.

11

However, this would still leave the company open to the risk offinancing a “large, unforeseen non–recurring event”. Such would bethe purpose of a reserve requirement for this firm, and this should bethe proper and primary focus in determining the level of RSRrequired.

More important, however, is the appropriateness of the MCT approachfor this purpose. The MCT exercise may very well be a usefuldiscipline for management as “an indicative analysis tool”, but it is notdirectly “on point” for the purpose of establishing an appropriate levelof RSR. The MCT can usefully serve as an “indicative” tool, especiallyin the context of a dynamic capital adequacy test (considered shortly).

Another section of the AI.15 document then outlines some (five)problems with the RAA technique as currently applied. Most of thesedetails are technical, and can be easily accommodated by modernmethods at little cost. We briefly consider the five listed “problems” orpoints listed in the AI.15 document.

(1) Point one states that the operational risk calculation isbased on outcomes of the past twelve years withoutadjustment to reflect the current situation. The reportsuggests using average percentage differences rather thanabsolute dollar differences. Since the goal is to reflect morefully what management considers to be important currentconsiderations, it could consider weighting more heavilyrecent (or particular) past years in the risk calculations. Thisis standard statistical practice and allows managerialjudgment a significant role. In any case, operational riskcalculations need not be a mindless technical exercisedevoid of judgment. (See also footnote 7 for furtherdiscussion).

(2) Point two states that complications arise when a “rare”significant event --- “(e.g. 1 in a 100 year)”--- in the lasttwelve years is reflected disproportionately in allcalculations. Again, the standard statistical solution is to

12

adjust for this “outlier”.

(3) Point three states that “if the corporation improves itsforecasting process, it will have a lower operational riskmargin”, [implicitly considered a “bad” thing?] andconversely. One might well respond by saying thatimprovements to the forecasting process should always bewelcomed, and that one should always aim for better, ratherthan worse, forecasting. (See our forensic statisticalevaluation of MPI’s forecasting record in section 5.)

(4) Point four argues that

“using the actual correlations between thefour operational risk components (revenues,loss costs, claim expenses, and operationalexpenses) produces unreliable results. Thecorrelations have changed significantly sincethe last risk analysis in 2002.”

It is difficult to be generous towards this statement, sincethe meaning of “unreliable” is not made clear. In aforecasting context, ignoring correlation or co-variationin component variables offends against all acceptedstatistical methodology. (See fn 17 for technical detail.)The fact that correlations change over time is the reasonwhy re-calculations are periodically performed in light of newdata and circumstances. In short, the fact that correlationsamong components have changed is not an argument fornot using the information, or ignoring it (that is, assumingthat it does not matter). The statement that “correlationchanges” is not equivalent to the statement that “correlationdoes not exist”. To use a medical analogy, the fact that aperson’s weight has changed since the last medical checkup does not mean that the BMI (bio-mass index) should notbe measured to monitor the person’s health status.

13

(5) Point five states that the RAA is not “used by anyone else inthe industry”. In our view, the issue is not whether the RAAis used by others in the industry, but whether the RAA is theright approach for MPI. We are not in a position tocomment on this observation without further information.

The remainder of AI.15 attempts a comparison of the MCT with theRAA, with sparse details. The very legitimacy of this is questionablesince the two methods are entirely different, but the documentattempts to compare the RAA to the MCT categories. Why this isdone is not explained, and many technical claims are not elaborated.Other issues mentioned are tangential to the issue of the appropriatesum to establish for RSR.

As an example, in its argument for the MCT approach, the statementis made that the “operational risk analysis does not specificallycalculate the Corporation’s risk profile by class of insurance” (p8).This would appear to be a detailed issue of the structure of ratesrather than being absolutely essential to determining the overallrequirement of the RSR.

Again, the report notes that “The ‘risk’ for revenues and loss costs aremeasured on the historical difference between actual and forecastedresults. Therefore, the operational risk analysis only adds risks to theextent that deviations between actual and forecast have actuallyoccurred in the recent past”. It is trite to note that all forecasts rely onhistorical patterns together with the assumption that a roughly similarpattern will hold in the future. This is the usual starting point beforeinclusion of other information, including managerial judgment, or newinformation concerning future possibilities. A criticism of the RAA onthese epistemological grounds is non-fatal (and disingenuous) sinceeven the MCT approach requires use of past (balance sheet) results(that is, a historical pattern) to provide a starting point. In sum, thebasis for comparison is insufficiently developed to warrant confidencethat one has demonstrated the superiority of one method over anotherfor use in establishing the level of RSR.

4.2 Dynamic Capital Adequacy Testing (the Christie Report)

MPI engaged Ernst & Young to conduct a Dynamic Capital

14

Adequacy Testing (DCAT) analysis “to assess the future financialcondition of MPI Basic.” The report (hereafter the Christie Report)was delivered September 2005. The Christie Report is significantbecause it forms the evidentiary basis for MPI’s position concerningthe RSR level. Further, the Christie Report is well documented as toassumptions and procedures, and provides sufficient detail to examinecalculations.

The Christie Report’s mandate was to employ the DCAT method to“assess the future financial condition of MPI Basic” (p1). Briefly, andwith much violence to subtlety, the DCAT involves creating a BaseScenario (Fiscal 2005) similar to MPI’s financial statements, and theninvestigating various “shock” scenarios to measure the sensitivity ofRetained Earnings to unforeseen events, referred to as PlausibleAdverse Scenarios. Additionally, the Christie Report estimates levelsof Retained Earnings required by MPI Basic to achieve specifiedlevels of the MCT. The standard adopted is 50% of the amountrequired to achieve a MCT ratio of 100%.11 The DCAT employs acomputer model that generates financial statements for variousscenarios; it was employed to investigate five plausible adversescenarios: (1) unanticipated inflation (2) one time non-recurring rise inunderlying loss ratio (3) precipitous and ongoing rise in annual claimcosts (4) stock market crash and (5) single very large catastrophe.12

Before examining the findings of the Christie Report, it is appropriateto note two comments acknowledged in that Report.

(1) First, the DCAT was carried out under two confidence levelstandards --- 99% and 97.5%, the latter being the levelhistorically used by MPI. The Christie Report makes clear that“there is no ‘proper’ confidence level applicable whenestablishing a target level for risk; rather the confidence level is a

11 Recall that MPI Board of Directors determined that the minimum level of retainedearnings should reflect 50% of the level necessary to achieve a 100% MCT ratio. Seealso footnote 8.12 The details are given in the Christie Report, so we do not summarize them here.Since the detailed calculations are fully documented and provided in the ChristieReport, we also do not reproduce any results in this report.

15

function of risk aversion of management and board”. (ChristieReport, p2). It is important to emphasize this subjective element.

(2) Second, and more relevant to our focus, is the following point.Quoting from the Christie Report:

“In past years review, the DCAT analysis wasperformed considering only the Rate StabilizationReserve. During fiscal 2004/5, an ImmobilizerFund was created within the organization. TheMPI Basic’s balance sheet effectively containsRetained Earnings that are the sum of theRate Stabilization Reserve and theImmobilization Fund. Consequently, [Christie’s]DCAT analysis [was] performed based on MPIBasic’s Retained Earnings.” (Christie Report, p2)[Emphasis added].

The propriety of viewing the Immobilizer Fund in combination with the RSRcan again be raised. The Todd Report argues that the Immobilization Fundexpenditures are not properly part of the RSR. We are also of that view.

In sum, the DCAT exercise of the Christie Report strictly pertains toquestion of the adequacy of Total Retained Earnings to achieve agiven MCT ratio, and not the appropriate level of the RSR (excludingor including Immobilization Fund expenditures). We cannot emphasizethis point enough. The Christie Report addresses the issue of RetainedEarnings, and not the RSR per se. This is not a pedantic point. The RSR,by MPI’s own declaration, is to guard against “unexpected events” and“non-recurring factors”. Expenditures on the Immobilization Fund do notfulfill these conditions, and it is inappropriate to calculate a level for theRSR based upon standards adopted for retained earnings and itsrelationship to the MCT criterion. Therefore, the Christie Reportconclusions must be treated with great care for our focus. Recall that theMCT is recommended by MPI as only “an indicative analysis tool”.

With these preliminary remarks in mind, we now turn to an examination ofthe conclusions of the Christie Report.

16

The nature of any computer-generated projection is always sensitive to theset of assumptions employed, including the magnitude of the “stress” or“shock” contemplated. It is impossible to “disprove” assumptions. All thatcan be done is to ask whether the assumptions an analyst chooses are“reasonable” and “plausible”, and whether or not the interpretations of thecomputer–generated results are warranted. To phrase it differently: Howreasonable are the assumptions? How “plausible” are the “plausibleadverse scenarios” and the interpretation of the simulations?

We select for examination the two most extreme results of the ChristieReport --- (i) Single very large catastrophe, and (ii) Inflation withmanagement action.13

4.2.1 Single large catastrophe scenario

Consider first, the “plausible adverse scenario” of a single very largecatastrophe. The “catastrophe” (for narrative verisimilitude) consists of a“single extremely large hailstorm” coupled with MPI’s largest reinsurer,“Lloyd’s syndicates, … honouring [only] 50% of its hail reinsuranceobligations.” Further, MPI SRE is unable to transfer any funds to Basic.Finally, the loss amount is “judgmentally selected to equal twice thelargest event incurred by MPI.” [emphasis added].

Under this extreme adverse scenario, Retained Earnings decrease from aninitial $178 million in the Base scenario to $152 million --- a decrease of$26 million. The MCT falls from 94% in the Base scenario to 77% in 2006and to 70% in 2010. In other words, even when faced with a shock ofthe magnitude equal to twice the size ever experienced by MPI, theMCT ratio calculated by the Christie Report remains well above the50% level prescribed by the MPI Board as the minimum some 5 yearslater, without any extra action by management to replenish RSR.Additionally, bear in mind that the Christie Report calculates the DCAT for

13 We choose these extreme “plausible adverse scenarios” since they represent thestrongest case for a larger RSR as argued by MPI. Similar critiques could beconducted for each “plausible adverse scenario”. But if the case for larger reserves areweak for those plausible adverse scenarios that give rise to the largest reserverequirements, then the case for scenarios that are more moderate in impact are weakerstill. We already noted that the DCAT of the Christie Report actually applies to RetainedEarnings, a broader category than our concern with RSR.

17

retained earnings (a broader category than RSR itself). It is fair to suggestthat the minimum 50% MCT level is more than adequate for RSRpurposes. It is difficult to imagine a more stringent test for an“unexpected” and “ non-recurring event” --- the hallmark test for theRSR.

4.2.2 Inflation Scenario

Inflation is a fact of life and should be expected. However, unexpected orunanticipated inflation can create difficulties. The Christie Report considersthe adverse scenario of “a significant and sustained increase in the rate ofinflation amounting to an additional 3% per annum …, resulting in anincrease in projected losses and general expenses of 3% annually.” (p10).In other words, inflation is 3% greater than initially anticipated.14 No onewith experience of the Canadian economy during the 1980s can fail toappreciate the spectre of high inflation.

The Christie report assumes that investment income only increases slowlyto offset inflation. (No justification or discussion is provided on therelationship between inflation and investment rates of return.) However,projected claims and general expenses increase by 3%. Compared to thebase scenario, even with management response, the Christie Reportcalculates that net income is negative over the entire projection period, andRetained Earnings are negative as well at the end of the forecast period”(p12).15 This scenario is the most extreme of the five adversescenarios considered in the Christie Report, being the only one thatgives rise to (a) negative Retained Earnings by February 2010, and (b)

14 Technically speaking, inflation is the rate of increase of price levels; that is, a rate ofchange over time and expressed as a rate such as 3% per annum. An “increase” in therate of inflation is therefore an acceleration in the rate of change of the price level. Webelieve the language of the Christie Report did not intend this. We therefore interpretthe scenario as one in which inflation is simply 3% higher than what was expected, andthat this remains so through the projection period. Christie’s figures adopt thisinterpretation. As an aside, such a scenario assumed by Christie could not thengenerate the assumed forecasts by MPI according to the current forecastingprocedures. 15 Obviously, losses under the scenario with no management action at all are evenmore dramatic. We do not consider this case.

18

a projected MCT ratio that is negative. Indeed, on the basis of thisscenario, the Christie Report (p 20) concludes:

“With the inflation scenario we have identified aplausible adverse scenario where the RetainedEarnings become negative. Consequently we haveconcluded that the future financial condition of MPICBasic is not satisfactory.” [Emphasis in original].

Is this correct?

Given the strong conclusion drawn from what is clearly the centerpiece ofall the adverse plausible scenarios considered, further examination iswarranted.

Let us begin with the base scenario. The base scenario specifies a ratechange of 3.7% in the base year, and no further changes thereafter. Onthe other hand, projections are made to increase other components, (suchas vehicle upgrading, general expenses growth, etc) for every subsequentyear of the projection period. One would have thought a better basescenario would have been a situation in which rate changes are a constantpercentage each year throughout the projection period; that is, 3.7% in thiscase. A base case in which a rate change of 3.7% is implemented in fiscal2005, followed by a “rate moratorium” for the next five years seemsquestionable. Rate changes are necessary to accommodate “normalexpected factors”; therefore, specifying a constant percentage rate changeeach year would allow the DCAT to isolate the “pure” shock effect of anunanticipated inflation increase of 3%.

The net effect of the base scenario specified in the Christie Report is, ofcourse, to amplify the severity of any adverse stress, especially inflationthat occurs year after year during the projection period. (unlike the one-time shocks such as a single large catastrophe like a hailstorm coupledwith a partial failure by Lloyd's). In other words, the “inflation” adversescenario is particularly “stressful” because it cumulates year after yearthroughout the projection period, while the one-time shocks are confined toa single episode.

19

The Christie Report “assumes that management would recognize theimpact of inflation gradually over time, and would increase rates by 3% in2007, 6% in each of 2008 and 2009. (p 11). As indicated previously, onecannot prove or disprove assumptions. It is fair to characterize theassumed pattern of management response as “sluggish” and mechanical.

Consider the information provided by the DCAT from the Christie Reportitself. (MPI management would have this information available to thembecause they are employing the DCAT monitoring.) With no managementaction in the first year, (see p 11 – the no management action scenario)net income (- $10 m) is seen to be negative in 2006, and the MCT ratio hasfallen from 90.5% to 53.7%. The Christie Report assumes thatmanagement takes no action in 2006, but raise rates by 3% in 2007.

Now turn to the results of the Christie Report (p12 – the activemanagement scenario) under the assumption of management action.Here we see that net income has further fallen negative five fold (to -$51m), and the MCT ratio is actually negative (-6.5%). Management actionin the face of this “bankruptcy” situation is simply to raise rates by3%! Of course, the story gets worse each year, with the end resultreported by Christie that “Retained Earnings [become] negative at the endof the forecast”(p12) and that “the future financial condition of MPI Basic isnot satisfactory.” (p 20)

What should one conclude from this DCAT exercise? Alternative to theChristie conclusion that “the financial condition of Basic is not satisfactory”is one that would stress (1) the unsatisfactory specification of the basescenario, and (2) the simplicity and complete implausibility of the assumedpattern of managerial response.

Typically, economic models and system analyses employ a “reactionfunction” or “feedback rule” (terminology varies) in which decisions torespond are triggered by some critical value of a monitored indicator orparameter. In the present context, this could mean: “raise rates whenclaim losses exceed premium revenue by a given magnitude” or “replenishthe RSR whenever the level drops below a set value” etc. or “do somethingmore dramatic than raise rates by 3% when you are facing ‘bankruptcy’

20

conditions”. The issue, here, is not to argue that the Christie Reportshould have adopted an alternative set of assumed management actionspreferred by someone else; rather the point is to view the Christie DCATfindings as substantiating an interpretation that the management actionscenario assumed is wholly insufficient and non-instructive. It is actuallythe sluggish and inappropriate response by management built into thecomputer modeling of the DCAT that is faulty. To be clear, this is not evena criticism of the DCAT approach itself but the particular implementationand interpretation in the current application.

To give a pithy summary, the Christie Report does not so muchdemonstrate the inadequacy of the future financial situation of MPIthan it outlines what management absolutely should not do if itencounters unanticipated inflation of 3%.

This lesson would seem particularly instructive since it is more common tohave a bit of unanticipated inflation than, say, a large hailstorm with lossestwice the size ever experienced simultaneously with reinsuranceobligations not honoured by Lloyds syndicates. It would be distressing, tosay the least, to think that MPI could become insolvent simply in the faceof an unanticipated 3% inflation.

5 Rate Stabilization Reserve: Forensic Assessment

We now turn to an examination of the calculations of the amount to be setaside for the RSR.

MPI has provided a series of calculations of its operational and investmentrisk as part of its 2007 rate application. These calculations follow the PUBapproach outlined in Order 151/2000 but MPI also provided alternativescenarios that suggest that the PUB approach to risk assessment is tooconservative and that the Rate Stabilization Reserve (RSR) is too small. Inthis section we provide our assessment of the risk based on theinformation provided by MPI in “2007 Basic AutoPac Operational andInvestment Risk Analysis – AI.16” (June 14, 2006).

The PUB and the MPI both agree that the level of the RSR is to bedetermined by the anticipated size and prospect of unexpected and non-

21

recurring losses. The RSR is not intended to smooth rate increases; i.e., itis not a “smoothing” reserve but strictly a reserve to eliminate rateincreases that might arise from substantial, unanticipated expenses (orrevenue declines). Normal rate increases that arise from anticipated risesin expenses (or declines in revenue) are incorporated in the forecasts andrate applications submitted by MPI.

The starting point for the analysis of operational risk is theaccounting identity16:

Net Underwriting Income =

Net Revenue Earned – Loss Cost – Claims Expenses – OperatingExpenses.

• Net Revenue Earned includes Motor Vehicle and Drivers Premiumsplus Reinsurance Ceded plus Service Fees and Other Revenues;

• Loss Cost includes Net Claims Incurred plus Financial Provisions,and

• Operating Expenses includes Regulatory and Appeal Expenses.

Operational risk then arises from the risk inherent in forecasts of NetRevenue Earned, Loss Cost, Claims Expenses and Operating Expensesrelative to their actual outcomes each year, although the PUB hasstipulated that the analysis is to be performed both including and omittingOperating Expenses.

5.1 On the Use of Correlation in Risk Assessments

There is some discussion in the MPI rate applications concerning the useof actual or zero correlations in the operational risk assessment. The PUBhas wisely rejected the suggestion that correlations among the riskcomponents should be ignored and we would strongly support this positionwithout even glancing at the data from the rate application process.

Rates are set each year on the basis of forecasted Costs (Loss Cost +Claims Expenses + Operating Expenses), using historical experience aswell as management judgment. Thus all anticipated costs are alreadyincorporated in rate applications and, for the most part, in revenues for that

16 Here and there, MPI refers to this as a “restricting equation”.

22

year. From this rate-setting process, one should expect a large, positivecorrelation between Net Revenue Earned and Costs, and this cannot beignored without overestimating the risk associated with Net Income, sinceNet Income is the difference between these two amounts.17

In fact, when one examines the data, the correlation between NetRevenue Earned and Costs is very large, 0.9418, or nearly perfect.Thus, actual correlations must be used to provide any reasonableassessment of the risk associated with Net Underwriting Income.

It is clear that the operational risk is directly connected to the ability of MPIto forecast costs and revenues each year. From this perspective it isimportant to assess MPI’s forecasting record over the past 12 years, from1994/95 to 2005/06. Based on the evidence MPI has provided in 2007Basic AutoPac Operational and Investment Risk Analysis – AI.16 (June14, 2006, Exhibit A), we focus initially on the two series that represent theultimate operational risk concerns19:

(1) Net Underwriting Income (2) Net Underwriting Income without Operating Expenses.

For these series we provide the MPI forecast and actual amounts for thetwelve-year period from 1994/95 to 2005/06 and a simple alternative linearforecast based on a linear regression over the period; this amounts tofitting a linear time trend through the data and identifying deviations fromthe trend as forecast errors. These results are shown in Table 1 and theforecast errors (actual minus forecast for the MPI and Linear Trendforecasts) are shown in Figure 1.

(See Table 1 and Figure 1)

17 That is, if Net Earned Revenue is R and Costs are C, we know that the variance ofNet (Underwriting) Income is:

Var (R - C)=Var (R)+Var (C) - 2Cov(R,C)To assume that there is no correlation between revenues R and costs C is to assumethat Cov(R,C) = 0, which overstates the variance or risk inherent in Var(R - C), thevariance of Net Income.18Authors' calculations from Table 1 (found at Tab 3).19 Focusing on Net Underwriting Income with or without Operating Expenses amountsto using actual correlations among the components and corresponds to the directive ofthe PUB.

23

The results suggest that MPI’s prospective forecasts have done slightlybetter than a Linear Trend retrospective forecast over the period from1994/95 to 2005/06. The standard deviation of MPI forecast errors (actualminus forecast) for Net Underwriting Income has been about $31,988,000,slightly less that the Linear Trend forecast errors and the standarddeviation of MPI forecast errors for Net Underwriting Income excludingOperating Expenses has been about $31,353,000, again slightly less thatthe Linear Trend forecast errors.20

The operational risk assessment has concentrated on the size of theforecast errors and, in particular, the size of prospective negativeforecast error outliers that might exceed the RSR. It is this concernthat constitutes the primary rationale for the RSR.

Before turning to this question, however, we should ask some otherquestions about the forecast errors; in particular,

• the extent of forecast bias and• the extent of serial correlation in forecast errors.

5.2 On Forecast bias by MPI

Forecast bias occurs if, on average, forecast errors differ from zero. Thatis, put simply, is the forecast error just as likely to be too high one year andtoo low another year? Or is the forecast error always too high or too lowyear after year? The forecast errors from the linear trend, for example,produce no forecast bias since the average forecast error is zero (byconstruction) for Net Underwriting Income with and without OperatingExpenses.21 But this may not be necessarily so for the MPI forecastmethods.

The MPI mean forecast error is $846,000 per year for Net Underwriting

20 Better forecasts can very likely be obtained by fitting an autoregressive movingaverage (ARMA) model, but this is only an exercise in retrospective forecasting and notcomparable to the MPI exercise. We use the linear trend (LT) forecast because it hassimple and useful properties to serve as a basis of comparison for MPI forecastaccuracy.21 This is a basic property of the ordinary least squares estimator used to compute alinear trend.

24

Income. This implies that the MPI forecast has a positive bias that hassystematically underestimated Net Underwriting Income by anamount that has accumulated to $10,152,000 (12 time $846,000) overthe period from 1994/95 to 2005/06. This conservative forecastingoutcome offers additional protection against unforeseen losses, sincethese “unforeseen” gains will accumulate in the RSR.

Similarly, for Net Underwriting Income excluding Operating Expenses, theMPI mean forecast error is $579,000 which has resulted in anaccumulation of $6,948,000 since 1994/95. These forecast errors aresmall, only 5.2% of mean actual Net Underwriting Income and 2.7% ofmean actual Net Underwriting Income excluding Operating Expenses overthe period, but the cumulative effect over many years can be quitesignificant. The 5.2% growth in Net Underwriting Income, for example,exceeds the rate of growth of inflation during the period and, indeed,exceeds all inflation forecasts into the foreseeable future.

5.3 On Serial Correlation in MPI Forecasts

Another pertinent property of the forecast errors is their serial correlation;that is, is a negative forecast error (unanticipated loss) likely to be followedby another negative forecast error (a second unanticipated loss) or by apositive forecast error (an unanticipated gain)? This issue has designrepercussions for the amount that one would require for a RSR since thewithdrawal demands on a reserve fund will be much greater if the summust withstand a series of draws (losses) year after year than if it is onlyrequired to accommodate (normally) an occasional unanticipated event.

The MPI forecast errors for Net Underwriting Income have a negativeserial correlation of -0.16. This indicates that a negative forecast error ismore likely to be followed by a positive forecast error than anothernegative forecast error; that is, unanticipated losses are more likelyto be followed by unanticipated gains than by additionalunanticipated losses in the following year.

Similarly, the MPI forecast errors for Net Underwriting Income excludingOperating Expenses have a negative serial correlation of -0.14, indicatingagain that unanticipated losses are more likely to be followed byunanticipated gains than by additional unanticipated losses.

25

What are the implications of these properties of the forecast errors for theRSR?

• The presence of positive forecast bias provides the RSR withunanticipated gains over time, while

• the negative serial correlation in forecasts protects the RSR fromrepeated unanticipated losses.

Put starkly, the behaviour of the MPI forecast errors in Table 1 does notsuggest that a RSR in the current range ($50 to 80 million from 2000 to2006) is likely to be depleted even if no adjustments are made to the fundwhen it falls below its minimum value. Rather, the RSR will replaceunanticipated losses (negative forecast errors) with unanticipated gains(positive forecast errors) and, on average, the RSR will grow at a rate thatat least compensates for the growth in inflation.

From this perspective, then, the calculations of operational riskprovided by MPI according to the PUB approach provide an adequateRSR. The $50 million lower bound for the RSR corresponds to the 95%confidence limit suggested by the PUB, while the $80 million upper boundexceeds the 97.5% confidence limit suggested by the PUB22 andrepresents a 99.5% confidence limit.

5.4 Are Risks for MPI Getting Larger over Time?

Consider an “average” RSR level of $65 million (mid range of $50 million -$80 Million) which corresponds to a 97.5% confidence limit based on MPIforecasting experience. This protects MPI from a one-in-40-yearcatastrophic loss. Moreover, past experience suggest that the fund willgrow through time at least at the rate of inflation and that a one-in-40-yearcatastrophic loss, if it occurs, is very unlikely to be followed by a secondcatastrophic loss the following year.23 The prospect, instead, is forunanticipated losses to be followed by even larger unanticipated gains tobuild the RSR over time.

22 2007 Basic AutoPac Operational and Investment Risk Analysis – AI.16,” June 14,2006, Appendix A, Exhibit 4.23 Negative serial correlation suggests that the probability of successive one-in-40-yearlosses is less than one in 1600 (the probability of truly independent forecasting errors).

26

MPI produces four alternative scenarios, but only one directly concerns theoperational risk analysis.24 Their second alternative scenario (combinedwith the other scenarios in the fourth scenario) involves adjusting historicalloss costs, revenues and expenses to reflect current dollar values andvolume of business. They do this by adjusting past net revenues, losscosts, claims revenues and operating expenses pro rata to current levels.This is thought necessary because MPI argues that it is dealing withlarger magnitudes than in the past and historical figures need to beadjusted to current levels. We have previously commented onacceptable ways to correct for historical dollar figures collected in variousyears. Since standard deviations are sensitive to the units used, thestatistical effect of this adjustment by MPI is predictable. Pastforecast errors are adjusted upward resulting in a larger standarddeviation for the forecast errors and a higher operational riskassessment.

We make two objections to this procedure. First, there is no indication thatthe growth in the magnitudes of net revenues, loss costs, claims revenues,and operating expenses involves an unanticipated component. In fact,MPI forecasts have clearly done a good job of anticipating the growth inthese four components of net underwriting income and therefore thegrowth in the magnitude of net underwriting income itself, with or withoutoperating expenses. This leads us, then, to our second objection.

What evidence do we have, despite the growth in the magnitude of netincome, that the variation in forecast errors has risen over time? Thisquestion leads us directly to consider a third statistical property of the MPIforecast errors; namely,

• heteroscedasticity of the forecast errors.

Quite simply put, heteroscedisticity involves a change in the variance of theforecast errors over time.25 If the variance, and hence operational risk, of

24 The other scenarios involve shortening the investment time horizon and adjustinghistorical loss costs and revenues to reflect current reinsurance terms. We find neitherof these adjustments pertinent and do not discuss them here.25 Here, we are using “variance” in the technical statistical sense. We previously notedthat accounting use of the term “variance” for what we have termed “deviation”(meaning difference between forecast and actual amounts) may lead to confusion. Thetechnical statistical “variance” is the square of the accounting “variance” (deviation).

27

MPI forecasts is growing over time, the forecast errors will exhibit thisproperty of heteroscedasticity; if the variance is steady over time, theforecast errors will exhibit the property of homoscedasticity. A useful testfor heteroscedasticity is the Breusch-Pagan test26 which examines thepattern of the squared forecasting errors divided by the mean forecasterror. Since the test applies strictly only to linear regression, we first applythis test using the Linear Trend forecasting errors from Table 1 (beforealso applying it to the MPI forecasts). The raw forecasting errors fromTable 1, their square, and the squared forecasting errors divided by themean forecast error are provided in the left-hand side columns of Table 2for Net Underwriting Income with and without Operating Expenses. Sincethe Breusch-Pagan test uses the squared forecasting errors divided by themean forecasting error, we graph these in the top graph of Figure 2 for NetUnderwriting Income with and without Operating Expenses.

[See Figure 2 and Table 2]

What do we find?

There is no discernible pattern over time, and a trend line through thedata shows no significant trend. In fact, the Breusch-Pagan is based onthis idea; it is one-half the regression (or explained) sum of squares from aregression of the squared forecasting errors divided by the meanforecasting error on time. If the regression (a trend line through the topgraph in Figure 2) is significant, the test will be significant, indicating thatthe variance of forecasting errors is changing over time. This test iscarried out in Table 2 for the Linear Trend forecast errors and yields valuesof 0.008 for Net Underwriting Income and 0.011 for Net UnderwritingIncome excluding Operating Expenses. The critical (chi-squared) value forthis test is 3.84 and these values fall well short of that measure; they arehighly insignificant. This is a clear failure to reject that the forecasterrors are homoscedastic; that is, that the variances of the Linear Trendforecast errors are stable, neither growing nor declining, through time.

Stripped of technical language, the forecasts errors associated with netoperating income are not growing larger over time.

26 Our reference for the discussion of heteroscedasticity and the Breusch-Pagan test isthe authoritative textbook in econometrics by William Greene, Econometric Analysis:Fifth Edition, Prentice-Hall, 2003, chapter 11.

28

Since the MPI forecast errors are very similar to the Linear Trend forecasterrors (see Figure 1), we can expect similar results. These results areshown on the right-hand side of Table 2 and the squared forecasting errorsdivided by the mean forecasting error are graphed in the bottom graph ofFigure 2 for Net Underwriting Income with and without OperatingExpenses. Again there is no discernible trend in this graph and theBreusch-Pagan test confirms this visual impression: The test values are0.356 for Net Underwriting Income and 0.308 for Net Underwriting Incomeexcluding Operating Expenses, both well below the critical (chi-squared)value of 3.84. Thus, the best evidence we have from the historicalrecord is that the variance in forecast errors, whether from the LinearTrend or the MPI forecast, is not changing with time. Thus, we see nobasis for the MPI adjustment to reflect current magnitudes inAlternative Scenario 2. There has been growth in all components of NetIncome but these have not, according to the evidence available, producedchanges in the variance of the forecasting errors.

6 Summary and Conclusions

• The purpose of this report is to investigate the sum that ManitobaPublic Insurance Corporation (MPI) should set aside in its RateStabilization Reserve (RSR). Differences in approach, andrecommended sums, exist between the amount suggested by ThePUB and the amount suggested by MPI.

• We examine past practices and calculations. These include: theRisk Analysis Approach (RAA) suggested by the Public UtilityBoard (PUB), the Minimum Capital Test (MCT) approachsuggested by MPI, including the results of a DCAT (DynamicCapital Asset Test) by Ernst & Young (Christie Report) and resultsof Elenchus Research Associates (Todd Report).

• We provide an independent forensic statistical assessment of theforecasting record of MPI and evaluate their performance for thepast twelve years.

• Various documents of PUB and MPI are in agreement that the

29

purpose of the RSR is “to protect motorists from rate increasesmade necessary by unexpected events and losses arising fromnon-recurring events or factors.” (MPI 2005 Annual Report, 48).

• The RSR is not meant to smooth rate increases from one yearto the next. Forecast errors are expected, rather thanunexpected, and “smoothing” of rate increases, if desired, shouldbe considered a normal management practice rather than “anunexpected event” arising from “non recurring … factors”.

• We therefore interpret the RSR (as stated by MPI and PUBabove) as contingency funds against “large” “unforeseen” eventsthat do not recur on a frequent basis.

• The definition of an “unforeseen non recurring event” isgiven substance by adopting a confidence level. For example,a 97.5% confidence level would correspond to an event having a2.5% annual chance of occurring, or an one-in-forty year event.The definition of “large” remains subjective, and can vary withcontext.

• According to MPI’s own data, the main source of operationalrisk utilizing the RAA approach is the “loss cost” component.The risk (measured by standard deviation) associated with LossCosts is about ten times larger than the components for eitheroperation expenses or claim expenses. Not surprisingly, the mainoperational “risks” derive from not being able to predict withaccuracy the losses arising in any given year. Accordingly, giventhe stated aim of the RSR, the primary design considerationmight be restated to be “what sort of ‘large’ ‘unforeseen(loss) event’ must the RSR level be able to accommodate?

• The MCT favoured by MPI does not address the question of theRSR level directly. Rather, it assesses the capital requirements toforestall insolvency for a private sector insurer; MPI claims to offerthe MCT test as an “indicative analysis” tool only, [our

30

emphasis].

• However, MPI asserts that the ”appropriate “level of the RSR canthen be “associated with” the proper MCT ratio, and argues that atarget MCT of 50% is the “right adjustment” for a monopoly crowncorporation.

• One question for the MCT approach, sharply raised, is: Whatadjustment should be made for a monopoly crown corporationselling a mandatory product? A monopoly incurs no competitiverestraint on setting “prices” (premiums) and a monopoly does nothave to worry about losing “market share”. Additionally, purchaseof the product is compulsory. Further, a crown corporation suchas MPI is unlikely to be allowed to declare bankruptcy (by the ngovernment). That is, its continued existence is assured.Insolvency risk simply does not exist.

• It must be conceded that any adjustment of a MCT standard setfor a private sector competitive industry applied to amonopoly situation of a crown corporation is arbitrary, and nouseful purpose is served by attempting a definitive number.

• The MPI documents calculated RSR requirements according toseveral sets of instructions suggested by the PUB. Somecalculations ignored correlation among components comprisingNet Income. Ignoring correlation or co-variation in componentvariables offends against all accepted statisticalmethodology.

• The DCAT exercise (Christie Report) considered several“plausible adverse scenarios” and concluded that the financialsituation of MPI was wanting and, by extension, the RSR wasinadequate. Our evaluation of the results suggests that theseconclusions result from a poor choice of the Base Scenario andan unrealistic policy reaction pattern from management that canbest be characterized as “inaction” in the face of pending

31

bankruptcy.

• To summarize, the Christie Report does not so muchdemonstrate the inadequacy of the future financial situationof MPI as outline what management absolutely should not doif it encounters unanticipated inflation of 3%.

• At this point, it is useful to clarify the nature of the question thatthe DCAT asks, and contrasts it with the question asked by thePUB RAA approach. The RAA asks, essentially,

What is the level of reserves necessary towithstand an unanticipated loss by MPI, givenpast patterns of historical occurrences of aninfrequent event?

• The DCAT asks, essentially:

How much capital must MPI have in order notto suffer complete insolvency in the sense thatall liabilities must be covered from firm assetswithin one year?

The one year restriction is implicit in the emphasis on liquidassets, rather than total assets, including the intangible asset of agovernment guarantee against insolvency, and the market powerassociated with its monopoly status in marketing a mandatoryproduct.

• The distinction between the two questions addressed by the RAAapproach and the MCT approach should be kept firmly in view.

• The RAA approach is critically dependent upon the ability of MPIto provide reliable forecasts of its revenues and expenses,including claim losses in order to forecast its net underwritingrevenue. One can legitimately asks such technical questions as:

32

“Should correlation between components of net underwritingincome be used?” and, most important, a general question suchas” How well has MPI performed its forecasts in the past?

• This is the purpose of the forensic statistical review of MPI’sforecasting record.

• The correlation between Net Revenue Earned and Costs is verylarge, 0.94, or nearly perfect. Thus, actual correlations must beused to provide any reasonable assessment of the riskassociated with Net Underwriting Income.

• The MPI mean forecast error is $846,000 per year for NetUnderwriting Income. This implies that the MPI forecasts have apositive bias that has systematically underestimated NetUnderwriting Income by an amount that has accumulated to$10,152,000 (12 time $846,000) over the period from 1994/95to 2005/06. This conservative forecasting outcome offersadditional protection against unforeseen losses, since these“unforeseen” gains will accumulate in the RSR.

• The MPI forecast errors for Net Underwriting Income have anegative serial correlation of -0.16. This indicates that a negativeforecast error is more likely to be followed by a positive forecasterror than another negative forecast error; that is, unanticipatedlosses are more likely to be followed by unanticipated gainsthan by additional unanticipated losses in the following year.

• Put succinctly, the behaviour of the MPI forecast errors (see Table1) does not suggest that a RSR in the current range ($50 to 80million from 2000 to 2006) is likely to be depleted even if noadjustments are made to the fund when it falls below itsminimum value. Rather, the RSR will replace unanticipatedlosses (negative forecast errors) with unanticipated gains (positiveforecast errors) and, on average, the RSR will grow at a rate thatat least compensates for the growth in inflation.

• From this perspective, then, the calculations of operational risk

33

provided by MPI according to the PUB approach provide anadequate RSR. The $50 million lower bound for the RSRcorresponds to the 95% confidence limit suggested by the PUB,while the $80 million upper bound exceeds the 97.5% confidencelimit suggested by the PUB and represents a 99.5% confidencelimit.

• Finally, MPI argues that it is dealing with larger magnitudes than inthe past and historical figures need to be adjusted to currentlevels. We have commented on acceptable ways to correct forhistorical dollar figures collected in various years, when the issueis to separate “real changes” from “changes due to inflation”. Thecontext for MPI would tend to suggest that it should be interestedin forecasting the value of its future losses in nominal terms; thatis, in current dollars. As a matter of technical statistical procedure,standard deviations are sensitive to the units used, therefore, thestatistical effect of the adjustment by MPI is predictable. Pastforecast errors are adjusted upward resulting in a largerstandard deviation for the forecast errors and a higheroperational risk assessment.

• MPI's analysis implies that forecast errors are heteroscedastic:that is to say, growing over time, with the scale of revenues andcosts.

• Our forensic analysis showed that forecast errors are actuallyhomoscedastic. Stripped of technical language, the forecasterrors associated with net operating income are not growinglarger over time.

• In the final analysis, we believe there is a fundamental distinctionthat should be borne in mind when considering whether the MCTor the RAA is the more appropriate tool to employ in determiningthe level of reserves to hold for the enunciated purpose of ratestabilization; namely to provide for an unforeseen andunanticipated non-recurring contingency of a large nature.

o The MCT is a valuable indicator of the overall financial

34

health of a firm, and it is relevant to determining whethersufficient capitalization exist to avoid insolvency.

o The RAA is an indicator that speaks directly to the liquidityrequirement when an unforeseen event occurs, leading to a“spike” in claims. This liquidity requirement must be met,(and not indefinitely deferred), notwithstanding thepossession of adequate capital assets.

While there may be merit, on other grounds, for MPI to adopt theMCT procedure for management purposes, such as to monitor otherfinancial conditions, we see no merit to the MCT as a substitutemethodological approach for determining the appropriate level of theRSR. The MCT is designed to address an entirely different question;namely the issue of insolvency, rather than liquidity. Furthermore, itsproportional application to a crown monopoly is ad hoc andunsubstantiated by empirical evidence; the MCT – RSR link istenuous, and more importantly, the MCT simply does not address theprimary issue at hand, namely, how much liquidity is required tofinance a “unexpected, non recurring” loss event.

7 Disclaimer

We have accepted all figures and data in the MPI Reports as accurate; wehave not independently verified the accuracy of the data. Nor have wereplicated their Risk Analysis or MCT calculations. We have also acceptedall figures, data and calculations reported in the Todd and Christie Reportsas accurate. We have confined our focus to the issue of the better (of two)approach for determining the amount of the Rate Stabilization Reserve,assuming other major policies and MPI management practices currently inplace continue without significant change. We do not investigate otherpolicy or management practices that, if adopted simultaneously, mightreduce the Rate Stabilization Reserve requirements, including otherforecasting techniques. We regard these matters as beyond our presentmandate. Any errors remaining are our responsibility.

35