1. Introduction
1.1 The History of GRIT
1.1.1 The General Insurance Board of the Faculty and Institute of Actuaries set up a taskforce, known as GRIT (General insurance Reserving Issues Taskforce), in response to some of the criticisms raised in the Morris (Reference Morris2005) review and in the press by rating agencies and others, regarding the Profession's approach to actuarial reserving in general insurance. GRIT worked from 2004 to 2006 and produced a significant report, including some new professional content and recommendations for further areas of development and research (Jones et al., Reference Jones, Copeman, Gibson, Line, Lowe, Martin, Matthews and Powell2006).
1.1.2 GRIT discussed the differences in understanding of uncertainty in reserving between actuaries, and the different practices at the time in disclosure of uncertainty to the users of actuarial reserving products. Some actuaries would, at that time, not include any quantification of the uncertainty; others would discuss a range of reasonable estimates, and others, still, would discuss uncertainty in the final outcome, with or without some form of quantification.
1.1.3 GRIT highlighted the fundamental difference between the discussion of a range of reasonable estimates and the discussion of a range of possible outcomes for ultimate claims.
1.2 Range of Reasonable Estimates
1.2.1 A range of reasonable estimates is potentially talking about two things: the uncertainty in the process of selecting a mean or best estimate, and the tolerances around how to go about that process for a particular purpose. For some purposes, such as certain accounting rules, there may be a moderately high degree of tolerance around an actuary's ‘best estimate’, so that it might be reasonable to carry a reserve, say, 10% higher or lower. This might, therefore, describe a reasonable accounting basis for carrying a reserve. Conversely, the actuary may not be particularly referring to any accounting treatment, but rather discussing the range of estimates that are equally likely to represent a true mean.
1.2.2 Clearly, the choice of vocabulary here can potentially be misleading to users of actuarial reports – an actuary may be discussing a theoretical point relating to the uncertainty within the estimation process, but a reader might interpret this as an accounting point, and hence apply the uncertainty to the reserve amount. This situation can be complicated further by the treatment of the tail of the distribution of possible outcomes in the actuary's choice of best estimate. Depending on the purpose, actuaries may wish to exclude the most extreme potential outcomes, such as those arising from unexpected latent claims’ development, and so the best estimate that they produce may not be a mean at all, but, perhaps, nearer to a median, being a ‘mean’ of a truncated distribution.
1.2.3 In the United States of America, the normal practice in handling uncertainty is to discuss the range of reasonable estimates, but also to give an indication, where appropriate, of any material risk of an adverse deviation in an outcome. In other words, the normal uncertainty discussion is around the choice of booked number – the accounting tolerance – and the outcome uncertainty is addressed only where the potential for a severe deviation from the result given needs to be highlighted.
1.3 Range of Possible Outcomes
1.3.1 The range of outcomes is different from the range of reasonable estimates. This is a discussion of the whole spectrum of possible ultimate claims outcomes, and is not concentrating on a discussion of an accounting basis tolerance. To discuss this uncertainty of outcome, one has a number of obstacles to overcome.
1.3.2 Firstly, the ranges which one might produce for, say, the tenth to 90th percentiles of probability may be significantly larger than the user of the actuaries’ work would expect. Providing such ranges may create the impression that the principal aim of the work in forming a best estimate is being devalued through seeking a ‘get-out’, should things go wrong.
1.3.3 Secondly, the methods which we, as actuaries, use to describe uncertainty are mainly derived from the methods used to form best estimates and are arguably designed to illustrate uncertainty around the mean, rather than to demonstrate accurately a full range of outcomes. At the time when the GRIT paper was produced, the ‘uncertainty methods’ were only, typically, in use 20% of the time, and were generally not thought to be well understood by the Profession.
1.3.4 Thirdly, an additional cost would be added to the reserving work if actuaries were required to disclose an uncertainty measure, and the user of their services may regard this as a cost which adds little or no value.
1.3.5 Notwithstanding these arguments, because of the belief that the quality of actuarial work would improve, and a better service would result to our customers, one of the recommendations of GRIT was that general insurance actuaries should include some form of quantification of the uncertainty of outcomes in ultimate claims when they carry out reserving work.
1.3.6 This was a controversial recommendation, and there was a great deal of discussion in the Profession about its merits. However, the Board for Actuarial Standards 2006 version of GN12 included the guidance that quantification of uncertainty of outcome ‘should normally’ be provided in reserving reports.
1.3.7 Since then, through the GRIT successor body ROC, many working parties have been formed and many GIRO presentations and papers have been forthcoming, in an attempt to provide practical help and guidance for practising actuaries in addressing this issue. In particular, ROC produced a paper on the communication of uncertainty and laid out two approaches, involving percentiles or stress tests. Both are now in common use. In addition several ROC GIRO working parties have looked at the methods used to quantify uncertainty – finding that the commonly used methods tend to break down in the extremes of the distribution, which has implications for capital modelling purposes. Work done on the quantification of uncertainty is discussed in Section 3, and the communication issues are discussed in Section 4.
1.4 Best Estimates
1.4.1 One cannot get far in looking at uncertainty of outcome before returning to the question: “What is a best estimate?” This is something which ROC has explored, and some of the ROC GIRO working parties have carried out a great deal of work looking at this question and the methods which we use to form our best estimates. This is discussed further in the next section.
1.4.2 Under Solvency II, the potential ambiguity is removed over whether the whole distribution should be included in the set of outcomes against which the best estimate aims for a mean. There is no provision for excluding extreme outcomes – the best estimate should be a true mean of all outcomes. This, alone, will necessitate some changes in practice, setting aside the requirement for discounting for the time value of money.
2. Best Estimates
2.1 Introduction
2.1.1 The general thrust of the conclusions of the GRIT paper (Jones et al., Reference Jones, Copeman, Gibson, Line, Lowe, Martin, Matthews and Powell2006), as it related to best estimates, was that, although actuaries did a valuable job in terms of providing input into the process by which insurers set their reserves, there were a number of shortcomings in the actuarial process which could lead to mis-estimation of liabilities. The principal focus of concern was the potential for systemic under-reserving, particularly in a soft market, although it should be recognised that management of an insurance operation is not well served by over-reserving in a hard market either.
2.1.2 The main issues in relation to the best estimates identified by GRIT were as follows:
– Historically, actuarial projections have a tendency to be unduly influenced by underwriters (particularly in the London Market).
– An inadequate understanding of the changes in the terms and conditions is often brought to bear in the actuarial reserving process; more broadly, actuaries should invest more time in understanding the business better.
– Existing reserving methods are, in many cases, applied inconsistently:
– the chain ladder is often applied too mechanically;
– the application of the Bornhuetter-Ferguson method is less rigorous than it could be;
– diagnostic tests on reserving models are not used as often or as rigorously as they should be;
– the existing models do not deal adequately with issues arising from the underwriting cycle, which has the tendency to create a reserving cycle.
2.1.3 There was one key question upon which Jones et al. (Reference Jones, Copeman, Gibson, Line, Lowe, Martin, Matthews and Powell2006) chose to remain silent: that of whether the existing methodologies were appropriate or whether alternatives should be adopted.
2.1.4 In direct response to the issues raised by GRIT, ROC established three working parties to address the following questions:
– the implications of the underwriting and reserving cycles for reserving, which focused on the issues affecting the more common reserving methods, and sought to identify responses to these issues;
– making allowance for the terms and conditions and other coverage issues in reserving; and
– the effectiveness of reserving methods, which sought to address the point upon which GRIT had remained silent, by running large scale testing on a wide range of reserving methods.
2.1.5 In 2008, ROC commissioned a working party to address issues around understanding the business better, although, at the time of going to press, this workstream had not yet reached any firm conclusions.
2.2 Underwriting and Reserving Cycle
2.2.1 The existence of a reserving cycle has been acknowledged within the Profession for some time, at least in the form of booked reserves. Archer-Lock et al. (Reference Archer-Lock, Shah, Fisher, Hilder, Line, Wenzel and White2003) suggested that the reserving cycle was, at least in part, driven by actuarial projections, which, unless modified explicitly, would, in many cases, exhibit cyclical under- and over-estimation of liabilities. This tended to arise from two features common to many lines of business, namely:
– the tendency for tail length to vary according to the state of the underwriting cycle – for various reasons business written in a soft market tends to have a longer tail than hard market business; and
– the tendency for recorded rate movements to understate the amplitude of the underwriting cycle as it affects a particular portfolio of business.
2.2.2 Work carried out by the ROC working party, which reported to GIRO as an interim update (Hilder et al., Reference Hilder, Rivers, Rothwell, Sheaf and Toller2007), and final report (Paillot et al., Reference Paillot, Hilder, Rothwell, Sheaf and Toller2008), had supported these observations. The approach taken by this working party was to seek to address these issues working within the framework of the existing methodologies, essentially the chain ladder combined with the Bornhuetter-Ferguson method.
2.2.3 The view of the authors of this paper (Hilder et al., Reference Hilder, Rivers, Rothwell, Sheaf and Toller2007) is that significant steps could be made towards mitigating the reserving cycle simply by the actuary being aware of these issues and seeking to make some allowance for these features in the formation of assumptions for the projection models.
2.2.4 There are obviously limitations in relying solely on subjective inputs, not least because the application of these judgements may well meet with resistance from the management of the insurer, and require substantiation. With this in mind, some more structured approaches were proposed, including:
– the development of a framework to provide some structure, and, potentially, some analytical support, to the required subjective adjustments to both tail factors and rate indices; this may take the form of a checklist of issues to be considered;
– the development of explicit hard and soft market development profiles, based upon long-term claims experience, to make allowance for varying tail length;
– the explicit modelling of varying tail length by using parametric forms to define the development profile; and
– the use of exposure measures other than rate adjusted premiums, which can be a more robust basis for the Bornhuetter-Ferguson method in certain circumstances.
2.2.5 All of these approaches are, in effect, ways of dealing with the problems presented when reserving with the information which is generally available at the present time. The final issue stressed by the authors of the paper was the need to improve the quality of rate indices. The ‘perfect’ index would serve to remove one of the key drivers of the reserving cycle from the list of difficulties faced by the reserving actuary, and, although the perfect index may be hard to get to, there are still significant improvements which can be made.
2.2.6 In this context, it should be recognised that significant work has been carried out in recent years by the insurance industry, with considerable input from the Actuarial Profession, in improving the quality of rate indices. Although no-one is suggesting that this work has been completed, the progress made should mean that one of the drivers for the next reserving cycle has been scaled back. Although this must be seen as a positive development, it produces an additional challenge for the actuary seeking to use past experience to determine the level of correction to apply to present data.
2.3 Terms and Conditions
2.3.1 The objective of the ‘The Implications of the Underwriting and Reserving Cycles for Reserving’ working party was to compile a list of the principal terms and conditions whose changes had had a significant impact on the reserving projections in the recent past. This list would then serve as a check list for actuaries performing reserving projections. The thinking was that changes to the terms and conditions would go some way towards explaining the divergence between the recorded rate indices and the ultimate loss ratios, which could be observed in almost all classes of business.
2.3.2 The main conclusion of this working party was, however, that the changes in the terms and conditions were not a significant cause of this divergence. There were clear cases, from past experience, where changes in coverage had had a significant impact on profitability, although, as often as not, these had been one-off rather than cyclical changes. Also, there was a sense that the effect of certain changes, particularly those relating to the structure of the policies underwritten rather than extensions or reductions to coverage, could, in many cases, be reasonably easily quantified, and therefore be factored into rate indices. Whether this is indeed being done varies by insurer, and over the course of time, with such adjustments being less prevalent in rate indices from earlier underwriting years.
2.3.3 The working party did not wish to suggest that the effect of changing the terms and conditions can be ignored safely, and, indeed, a number of changes which might be expected to be cyclical were identified in the paper. Rather, the conclusion was that changes to the terms and conditions were only one of the issues which could lead to shortcomings in the rate indices.
2.3.4 In view of this conclusion, the working party extended its brief to consider what, other than changes in the terms and conditions, might be the cause for the divergence between the recorded rate movements and the ultimate results. Although no analysis was carried out to support this, the views of the group, based upon personal experience, were that the general reliance on renewal rate movements had a significant potential to understate the amplitude of the cycle. In particular, the loss of business to competitors – possibly targeted by competitors as being better than average business – and the discounting of rates to attract new business in a softening market, would have this effect. Neither of these elements factor into most ‘renewal rate’ calculations.
2.4 Effectiveness of Reserving Methods
2.4.1 Research in this area is ongoing, and has involved sample testing by general insurance actuaries of a range of projection methods on sample datasets. There are clear limitations in such an analysis, as it is extremely difficult to replicate the interrogative nature of a ‘real-life’ reserving analysis, where the actuary will seek ‘business’ explanations to unexpected features observed in the data. Also, in such an exercise, there is an absence of the commercial pressures which invariably influence the reserving exercise. Nevertheless, the analysis should give an indication of the relative efficacy of various methods, if it can be assumed that each one will be affected similarly by the absence of the real business context.
2.4.2 The working party is due to produce a final report for the General Insurance Conference (GIRO) 2009, and, although more work is needed before firm conclusions from the exercise can be formulated, the working party made some observations based upon the work carried out prior to GIRO in 2008. These were as follows:
1) possible, although not universal, evidence to suggest a tendency to be conservative amongst testers;
2) a wide divergence in the testers’ estimates, possibly due to the lack of real business information;
3) the premium is a better estimator than claims in the early development years; claims are better than the premium in later years;
4) no evidence is observed from this exercise that final selected estimates, based upon a range of methods, were better than the ‘standard’ combination of the incurred chain ladder and the Bornhuetter-Ferguson (CL/BF) approaches; and
5) outliers were more common under the CL/BF combination than under the other methods.
3. Reserving Uncertainty – Quantification
3.1 Introduction
3.1.1 The GRIT paper (Jones et al., Reference Jones, Copeman, Gibson, Line, Lowe, Martin, Matthews and Powell2006) highlighted many areas in relation to the calculation of reserve uncertainty where it considered that improvements were necessary. This was firmly placed in the context of the users of reserve estimates, with the key recommendation that the Profession should be: “Providing more transparency to our reserving methods and helping our stakeholders have more insight into the key reserving assumptions and decisions” (Abstract).
3.1.2 Although this does not mention uncertainty explicitly, the next recommendation could not be more explicit: “Providing more information on uncertainty in our reserve estimates. In particular we recommend that actuaries provide a quantitative indication of the range of outcomes for future claim payments.”
3.1.3 Further on, in ¶1.5.3, the GRIT paper underlined the need for further research: “More research needs to be done on the topic of quantifying uncertainty, and we recommend that the Profession commissions research in this area.”
3.1.4 Indeed, quantification of reserve uncertainty has been a key area of development in recent times. Although significant progress has been made in the technical aspects of this discipline, significant attention has also been paid to the practicalities of applying and assessing the accuracy of the methods.
3.2 Defining the Problem
3.2.1 Part of this progress has been in the area of making it easier for practitioners to discuss and to describe the terms and the concepts within the topic of reserve uncertainty; the GRIT paper noted in the abstract that: “we recommend that … our profession defines a common vocabulary for communicating uncertainty”.
3.2.2 This discussion relates in part to the use of ‘a range of reasonable estimates’ versus ‘a range of possible outcomes’. This distinction is discussed in more detail below, and is only referred to here to remind the reader of the potential variety in uncertainty calculations and the results which can be considered.
3.2.3 Barlow et al., (Reference Barlow, Copeman, Gibson, Hilary, Hilder, Jones and Winter2007) goes further than the GRIT paper and states that: “the Quantitative Illustration should be of the eventual outcome of ultimate claims. The actuary may of course also provide an illustration of the range of reasonable best estimates, or some other measure, if she feels it is appropriate. If she does so it is important that the actuary describes the distinction clearly”.
3.2.4 More important for this section of the current paper is the consideration of the area of the distribution in which the stakeholders are (or should be) most interested for the purpose of the reserving calculations. Reports relating to solvency measures or extreme adverse development protection contracts are likely to consider the extreme tail of the distribution, whereas uncertainty indications for quarterly reserving exercises may be concentrated better around the centre of the distribution of possible outcomes.
3.3 Choosing a Method
3.3.1 The work since GRIT has encompassed many methods of estimating uncertainty. These have generally been categorised into three types, as in Barlow et al. (Reference Barlow, Copeman, Gibson, Hilary, Hilder, Jones and Winter2007):
“GN12 is not prescriptive about the methods that should be employed by the actuary when quantifying uncertainty, however one, or some combination, of the following approaches should normally be used:
– Judgemental/Indicative Volatility
– Scenario/Stress Testing
– Statistical Methods
A statistical methodology is not always appropriate and a judgemental approach based on the actuary's knowledge of the account and experience of the relevant wider market issues may be the most practical approach.
In choosing the approach to quantifying uncertainty the actuary may also have regard to the costs and benefits involved. For example an approximate judgemental method may in some circumstances be preferable to a complex sophisticated and time consuming statistical approach, whereas in other cases the latter may be more appropriate.”
3.3.2 Both GRIT and the Best Estimates and Reserve Uncertainty Working Party, 2007, surveyed the Profession to investigate the use of a variety of methods.
3.3.3 The Best Estimates and Reserve Uncertainty Working Party report (Gibson et al., Reference Gibson, Archer-Lock, Bruce, Collins, Dunne, Felisky, Hamilton, Jewell, Lo, Locke, Marshall, Nicholson, Thomas, Wilcox, Winer and Wright2007) noted, in Section 3.4:
“The survey shows reliance on a small number of standard methods for producing estimates of reserve uncertainty, and the key requirements of such methods are to identify variability around the best estimate and identify the tail of the distribution.
…
Judgement and scenarios were more used commonly to calculate best estimates and reserving uncertainty used in reinsurance lines than was the case for direct writers.
Reinsurers also were more interested than primary insurers in identifying the complete distribution of reserve variability rather than just the tail of a distribution. The key requirement though was variability around the best estimate for both types.
Personal lines insurers chose a method to identify reserve uncertainty based more upon the quality of results than the quality of the data. Commercial lines and reinsurers chose a method based more upon the available data.”
3.3.4 The various methods considered in the Best Estimates and Reserve Uncertainty Working Parties, 2007 and 2008, have been investigated in terms of their applicability to different parts of the distribution. Gibson et al. (Reference Gibson, Archer-Lock, Bruce, Collins, Dunne, Felisky, Hamilton, Jewell, Lo, Locke, Marshall, Nicholson, Thomas, Wilcox, Winer and Wright2007) also commented on the determination of the uncertainty from various sources, specifically the distinction between process, parameter and model uncertainty.
3.4 Accuracy of Methods
3.4.1 The GRIT paper (Jones et al., Reference Jones, Copeman, Gibson, Line, Lowe, Martin, Matthews and Powell2006) also indicated that reserve uncertainty models may be restricted in their accuracy, due to the limited data available: “We believe that it can be instructive to appreciate how far out reserve estimates can be compared with the ultimate outturn. This understanding should be helpful when considering the variability of reserve estimates, since in our experience there is in reality significantly greater variability than is often indicated by statistical techniques based solely on the observed historical data at the time of estimating reserves.” It further recommended that backtesting of best estimates be undertaken to identify historical accuracy, and to give a more grounded view of the accuracy of previous actuarial calculations (¶1.9.11).
3.4.2 Work carried out by the Best Estimates and Reserve Uncertainty Working Parties, 2007 and 2008, has indicated a more pronounced area of potential error in the results of the more common methods. Work carried out using data that perfectly satisfied the methods’ assumptions tended to understate the extremes of the distribution, as seen in Table 9.4.1, which is replicated here.
Table 9.4.1 Proportion of simulations in which ‘true’ outcome exceeded 99th percentile
(Table replicated with original numbering).
3.4.3 Research contained in the 2008 paper (Bruce et al., Reference Bruce, Chen, Dunne, Hinder, Mcmurrough, Meyers, White and Wright2008) elaborated on the circumstances of this effect in Section 1.5:
“We find that the work last year implying that some stochastic reserving methods understate the extremities of the predicted distribution is correct. These methods as usually applied rely on Maximum Likelihood Estimator methods to derive parameter values, which seems to exacerbate the problems. We have found that the use of Bayesian methodologies helps to reduce this effect, although there remains underestimation in the research we and others have carried out.
We find that a hybrid method using the higher of Mack and ODP [Overdispersed Poisson] provides a consistently better result at higher percentiles of the reserve distribution than using either in all cases when the underlying data exactly meets the assumptions relating to the ODP method. Note that this method does not produce significantly better results in all cases.
When investigating the effects of changing the properties of the triangles under Mack and ODP, we discovered some apparently anomalous results. These indicate that shorter tail business can be more understated at higher percentiles than longer tail business (assuming a full run-off triangle is available).
This result combined with that indicating that applying these methods to classes with fewer expected claims also makes the estimation of higher percentiles worse led us to infer that it is the volatility in the development patterns that is the key driver to the estimation error.
In effect, shorter tail business may well have greater volatility in the early periods of development than longer tail business when using similar development period intervals. Similarly where fewer claims are expected, the development pattern will be more volatile than where a greater number of claims will give more statistical stability to a development pattern.
Thus we anticipate that accuracy of these methods can be improved by choosing development intervals that are appropriate for the length of tail of the business being modelled to ensure that the development pattern is as stable as possible. Where such stability is not achievable through either development intervals being too long, or low frequency of claims, we expect the methods to perform less well.
We therefore also expect that other methods are more suitable to the modelling of low frequency claims and hence methods based on transactional level data or operational time may be more suitable for such situations. We have investigated transactional methods briefly in this paper and note that they require extreme care when parameterising if sensible results are to be obtained.”
3.4.4 Further, Section 9.4 of Gibson et al. (Reference Gibson, Archer-Lock, Bruce, Collins, Dunne, Felisky, Hamilton, Jewell, Lo, Locke, Marshall, Nicholson, Thomas, Wilcox, Winer and Wright2007) noted a somewhat surprising result relating to non-perfect data:
“We should generally expect stochastic methods to perform worse than this in practice because their assumptions will never be perfectly satisfied. We have done some testing of robustness to violations of assumptions only for Mack's method and the ODP methods so far. The results for all variants of the ODP method indicate, as expected, a deterioration in performance if applied where the ODP assumptions are violated. Surprisingly, results obtained so far for Mack's method show it performs better where its assumptions are not satisfied than where they are perfectly satisfied”
3.4.5 Bruce et al. (Reference Bruce, Chen, Dunne, Hinder, Mcmurrough, Meyers, White and Wright2008) also noted in section 1.5 that:
“To put this paper into context, we note that any estimation of the uncertainty of a homogeneous book of business will only tell part of the story of the uncertainty relating to a wider portfolio.
Correlations between such books will almost certainly be a significant factor in assessing any overall portfolio based uncertainty, although the evidence presented in this paper and its predecessors indicate that the uncertainty within a single portfolio can be at least as great.”
4. Reserving Uncertainty – Communication
Traditionally, actuaries have struggled to communicate their results clearly to their various non-actuarial stakeholders; but when it comes to communicating the uncertainty in our estimates, our efforts range from: “There is no uncertainty in my estimate, I double-checked my calculations” to “There is so much uncertainty, I couldn't even begin to estimate it.”
4.1 Descriptions
4.1.1 The original GRIT paper (Jones et al., Reference Jones, Copeman, Gibson, Line, Lowe, Martin, Matthews and Powell2006) provides a thorough review of how uncertainty is dealt with in current actuarial practice, including how it is communicated. It says, in ¶6.6.2.12, that: “There are a number of examples where actuaries have referred to best estimate as a mean, and given a range of reasonable best estimates, and have talked about how the range captures uncertainty in the eventual outcome. This is illogical and implies a lack of understanding of the distinction between parameter uncertainty and process uncertainty. Greater clarity and care is required in this area and it is to be hoped that a common vocabulary would go some way to assisting the Profession in this area.” A survey of typical wordings for best estimates, uncertainty and ranges was undertaken and a selection of excerpts was provided in Section 6.6.
4.1.2 Common themes arising in the description of a best estimate are the use of phrases such as: “do not contain any margins for optimism or prudence” and “no deliberate bias towards over- or under-statement.” Most descriptions did include a reference to uncertainty, to some extent at least, when the best estimate is described. In many cases, it is highlighted that the amount which eventually turns out to be required may be more or less than the best estimate, perhaps significantly so. However, only in very few cases is it stated explicitly how often we might expect the best estimate to be exceeded. In fact, in only one of the examples is it actually clearly stated that, if a reserve is required which will be sufficient more than 50% of the time, an amount in excess of the best estimate must be held.
4.1.3 Most of the example wordings describing uncertainty are clear about process uncertainty and its various causes. Fewer focus on parameter uncertainty, and some fail to mention it at all. The paper mentions, in ¶6.6.1.6, that: “two of the examples discuss both types of uncertainty and the distinction between the types is made fairly clear. The phrase “inherent uncertainty” is sometimes unclear. Very few examples of sources of parameter uncertainty appear to be given.” Only one of the examples tries to quantify the uncertainty, although the quantification given is described as typical based on experience rather than a specific calculation.
4.1.4 In the vast majority of cases, it is specifically stated that uncertainty is increased by the fact that no allowance has been made for factors which are not apparent in the data. In other cases, there is no link made between the exclusion of new types of claim and the uncertainty in the eventual outcome.
4.1.5 When ranges are being discussed, in many of the examples, it is not clear as to which type of uncertainty the range refers, e.g. if it has been done using a bootstrapping approach, it probably refers to both process and parameter uncertainty. Sometimes the range is being defined in statistical terms, and sometimes in qualitative terms.
4.2 Professional Guidance
4.2.1 Guidance Note 12 contains certain requirements around communication (mainly in Section 4) around reserving uncertainty including the following four requirements:
1) an indication of the degree of uncertainty;
2) an indication of the sensitivity of the results to key assumptions;
3) the highlighting of abnormally high uncertainty areas and issues; and
4) the quantification of uncertainty when it is useful to the recipient of the report.
4.2.2 The GRIT paper suggests that, perhaps, our objectives in communicating about uncertainty should include the following:
(1) to be understood by the reader – clarity;
(2) to be consistent with the professional vocabulary usage;
(3) to emphasise the more material issues;
(4) to emphasise the unusual issues;
(5) to comment in the context of our scope and purpose; and
(6) to protect ourselves from litigation.
4.2.3 GRIT believed that we are currently failing on (1) and (2). While GN12 currently addresses (3), (4) and (5), compliance with this has varying degrees of thoroughness. Many reports seem to be fully mindful of (6).
4.2.4 One area that is unclear is the extent to which actuaries should comment on the usual issues in regards to uncertainty. GN12 implies that it is safer to cover all issues. Instead of including a standard description of a plethora of causes of uncertainty in reserving, it would be more helpful if the major, or more unusual, issues were covered fully, and the minor, more ‘standard’ issues were de-emphasised. GRIT believes that this is in keeping with the spirit of GN12.
4.3 Vocabulary
4.3.1 Inconsistent vocabulary creates difficulties in communication. Jones et al., (Reference Jones, Copeman, Gibson, Line, Lowe, Martin, Matthews and Powell2006) present a common vocabulary, and suggest that this clarifies what various types of estimates and ranges are meant to represent, which, in turn, should facilitate a more meaningful comparison. The paper lists a number of common terms, with suggested definitions for them.
4.3.2 The follow-up paper, Barlow et al. (Reference Barlow, Copeman, Gibson, Hilary, Hilder, Jones and Winter2007), by ROC, elaborates further on the difficulties of communicating the causes of uncertainty. This paper discusses how the communication of uncertainty should enable users of actuarial reports to understand the nature, as well as the size, of the uncertainty. ROC argues that helping the user of the report to understand the context and the implications of the quantitative measure of uncertainty is as important as the quantitative assessment itself.
4.4 Describing the Causes of Uncertainty
4.4.1 This paper (Barlow et al., Reference Barlow, Copeman, Gibson, Hilary, Hilder, Jones and Winter2007) recommends that actuarial reports should disclose sufficient information on the key drivers of uncertainty. To this end, normally the actuary should accompany the quantitative illustration with a description of the sort of event, events or trends which would need to occur to reach the lower and the upper limits of any ranges, specific points on a distribution, scenarios, or the illustrations produced.
4.4.2 Producing an overall measure of uncertainty at an aggregate level will require some adjustment for diversification, and so will require assumptions about the dependency structure which exists between portfolios. This will contribute to the uncertainty in the overall aggregate estimate, and should be included in the communication of uncertainty. It may be included in the quantitative illustration of uncertainty in a way in which the actuary considers appropriate.
4.5 Practical Approach to Communicating Uncertainty
4.5.1 Barlow et al., (Reference Barlow, Copeman, Gibson, Hilary, Hilder, Jones and Winter2007) goes on to discuss the following two approaches to communicating uncertainty:
(1) everyday English; and
(2) percentiles.
4.5.2 In particular, ROC note that ‘percentiles’ is a method of communicating uncertainty rather than a method of estimation, i.e. the actuary could use a judgemental approach to quantify the reserve uncertainty and then communicate this uncertainty using percentiles.
4.5.3 The paper goes on to warn that describing uncertainty with percentiles could be interpreted as implying that the uncertainty has been quantified very accurately. Everyday English could be a preferable way to communicate uncertainty, when the quantification is based on significant areas of judgement, but it runs a greater risk of ambiguity.
4.5.4 The paper also recommends against giving a range of outcomes by using the terms ‘high’ and ‘low’ without explaining the meaning of these, as the reader may draw erroneous conclusions regarding the degree of extremity of these points within the complete distribution of outcomes.
4.5.5 If the actuary wishes to communicate outcomes in the tail of the distribution, ROC believes that consistency in how we, as a profession, communicate with our stakeholders is important, and so the paper presents the following standard vocabulary table:
4.5.6 The paper describes the rationale for the wording in line 1 as relying on the dictionary definition of ‘likely’: ‘probable’, such as ‘might well happen’. This definition was then applied to the 90 percentile, and the other percentiles graduated accordingly. Similarly, for line 2 the dictionary definition of ‘possible’ is: ‘that can happen’.
4.5.7 The paper goes on to describe various combinations of wordings and suggests wordings, particularly when describing scenarios. It points out that, if the scenario approach is used in the quantification of overall uncertainty (as opposed to illustrating the causes of uncertainty, as described in Section 1.2), then actuaries should consider carefully how the scenario has been incorporated in their best estimate.
4.6 Summary
The fundamental nature of insurance business means that there will always be uncertainty in actuarial estimates. We should focus on the development and the usage of a common and clear vocabulary, the clear communication of uncertainty and sensitivity quantification. We must work towards better communication and the education of ourselves and our fellow general insurance professionals.
5. Current International Context
5.1 It is not only the United Kingdom Actuarial Profession that has been interested in the uncertainty in actuarial projections. The problem of how to assess the uncertainty in the estimates for reserves, and to report on the uncertainty, is currently engaging actuaries, accountants, regulators and other insurance professionals worldwide.
5.2 The International Actuarial Association issued an exposure draft from its Risk Margin Working Group (RMWG) in early 2007, entitled, “Measurement of Liabilities for Insurance Contracts: Current Estimates and Risk Margins” (IAA, 2007), where they state that the margin over the current estimate can be regarded as an additional amount ‘for uncertainty’. The working group posits that any acceptable method for calculating the margin should exhibit the following characteristics:
– “The less that is know about the current estimate and its trend; the higher the risk margins should be
– Risks with low frequency and high severity will have higher risk margins than risks with high frequency and low severity
– For similar risks, contracts that persist over a longer timeframe will have higher risk margins than those of shorter duration
– Risks with a wide probability distribution will have higher risk margins than those risks with a narrower distribution
– To the extent that emerging experience reduces uncertainty, risk margins will decrease, and vice versa.”
While the comments from this paper related largely to costing risks for pricing, the indications for margins can equally be applied to the reserving process.
5.3 In addition, the RMWG states that it is desirable for the risk margin methodology to have the following characteristics:
1) to have a consistent basis at issue and subsequent to issue, i.e. for the entire lifetime of the contract;
2) to use underlying assumptions consistent with those used in the determination of the corresponding current estimates;
3) to have a risk margin methodology that is consistent with other financial contracts; and
4) where possible, be determined in a manner consistent with the accepted economic and actuarial pricing methodologies.
5.4 It is the view of the RMWG that this risk uncertainty should, itself, reflect knowledge about the risk(s) assumed, including observable information concerning the uncertainty associated with:
1) the estimated current level of the risk(s);
2) the estimated future level or trend of the risk(s), as applicable;
3) the estimated future term of the risk(s); and
4) the estimated uncertainty associated with these three sets of risk.
5.5 The RMWG describes three basic approaches (sometimes referred to as methods) of determining risk margins which have been used in the past:
1) Explicit assumption approaches. These risk margin methods use ‘appropriate’ margins for adverse deviation on top of realistic ‘current estimate’ assumptions.
2) Quantile methods. These risk margin methods express uncertainty in terms of the excess of a percentile (quantile) for a given confidence level above the expected value for a given period, such as the lifetime of the coverage. They can be determined, based on an estimate of a probability of ruin in excess of a given percentile.
3) Cost of capital methods. These methods base risk margins on the cost of holding the capital needed to support the obligation.
Examples of the various methods are given in the exposure draft.
5.6 Solvency II
5.6.1 Of most importance to UK actuaries, at the moment, is the implementation of Solvency II. The Level 1 Directive addresses uncertainty in Article 76, where it states: “The value of the technical provisions shall be equal to the sum of the best estimate and a risk margin… The risk margin shall be such as to ensure that the value of the technical provisions is equivalent to the amount insurance and reinsurance undertakings would be expected to require in order to take over and meet the insurance and reinsurance obligations … the risk margin shall be calculated by determining the cost of providing an amount of eligible own funds to the solvency capital requirement necessary to support the insurance and reinsurance obligations over the lifetime thereof. The rate used in the determination of the cost of providing that amount of eligible own funds (cost-of-capital rate) shall be the same for all insurance and reinsurance undertakings.”
5.6.2 Solvency II states that the purpose of the risk margin is to provide sufficient funds for the orderly transfer of the liability to another insurer. The proposed calculation is mechanically driven by external factors, but remains actuarially complex. The cost-of-capital approach avoids the more complex calculations to measure the impact of volatility which are necessary at the technical provision level. In addition, it is hard to determine the level of the margin with any sort of confidence.
5.7 International Financial Reporting Standards
5.7.1 In May 2007, the International Accounting Standards Board (IASB) published a discussion paper setting out draft proposals for a new International Financial Reporting Standard for insurance contracts (IFRS Phase II) which is due to come into force around 2013.
5.7.2 One of the obvious areas of interest is the extent to which uncertainty will be recognised in financial reporting, and the paper issued by the IAA on Risk Margins is a key input to this process.
5.7.3 Another discussion paper will be issued towards the end of 2009, which will help clarify the IASB's thinking and the extent to which IFRS Phase II will be similar to, or different from, Solvency II.
5.7.4 The IASB has narrowed the margin requirement down to two possible approaches: the current entry value approach and the current exit value approach. The current entry value approach is based on the price that an insurer would demand from a policyholder to assume the risk, i.e. the value at initial recognition is the transaction price (premium) with no gain realised on day one. The current exit value approach is based on the price that the insurer would pay to lay off the same risk to another party, and is expected to be more similar to Solvency II's cost-of-capital approach. As with the current entry value approach, it is a discounted best estimate cash flow valuation, representing the cost of the remaining rights and obligations under the contract.
5.7.5 The main difference between the above two approaches is the risk margin. In a current entry value approach, the margin is adjusted to reflect the current quantity of risk and the original price of risk. In a current exit value approach the margin reflects the current quantity of risk and the current price of risk; that is that the price of risk will be adjusted at each reporting period to reflect current conditions.
5.7.6 There is also discussion about including other obligations in this margin, so that the margin covers items such as risk, expenses and profit.
5.8 United States of America
5.8.1 In response to criticism that the Statements of Actuarial Opinion were too vague, so that: “the average reader is left clueless about the potential magnitude of variability and can easily be misled”, the National Association of Insurance Commissioners took a number of steps to enhance insurance company financial reporting and financial controls. Some of the key changes are:
1) a requirement to state whether or not the appointed actuary believes that there is a significant risk of material adverse deviation;
2) the dollar amount of the materiality standard underlying the above; and
3) the introduction of the Actuarial Opinion Summary, where the comparison is made between the appointed actuary's range of reasonable estimates and the company's carried reserves.
5.9 Australia
5.9.1 In 2001, the Australian Prudential Regulation Authority (APRA) released new standards for the determination of the liability valuation and solvency for Australian general insurers. The regulations require that the risk margin of the insurance liabilities (i.e. the sum of the outstanding claim liabilities and premium liabilities) is at least equal to the difference between the 75th percentile of the total liabilities and the current estimate, subject to the risk margin being greater than, or equal to, half of the coefficient of variation. Under these standards, the Approved Actuary of an insurer is responsible for determining these risk margins.
5.9.2 However, the insurer's board still has ultimate responsibility for setting the provisions, and can override the actuary's advice, but must then disclose and explain this decision to APRA and to the market, through the published annual accounts.
5.9.3 It was intended that the calculation of the 75th percentile should not rely only on statistical techniques (with bootstrapping and Mack being the most popular), but also on the knowledge of the business and on judgement.
5.10 Switzerland
5.10.1 In 2003, the Federal Office of Private Insurance (FOPI) in Switzerland set out a new directive concerning the Swiss Solvency Test (SST). The SST became effective as of 1 January 2006, as a part of the new insurance supervision act.
5.10.2 This risk-based solvency standard is based on the actual risks run by the companies. Similarly to Solvency II, it puts the responsibility on the companies to investigate their own risk situations. In order for Swiss companies not to be at a competitive disadvantage to insurers domiciled in European Union member countries, it is the aim of the supervisor for the SST to be compatible with Solvency II.
5.10.3 The risk margin of an insurance portfolio is defined as the hypothetical cost of the regulatory capital necessary to run off all the insurance liabilities. This risk margin is calculated with the cost-of-capital approach.
6. The Future for Reserving and Uncertainty
6.1 A Unifying Method of Reserving
6.1.1 One remaining problem for us is that, typically, we use one set of methods for forming our best estimates and a different set of methods for quantifying uncertainty. When we bring the methods together, we find that they tend not to ‘match’. Many actuaries need to adjust their bootstrap or Mack method ‘mean’, so that it comes into line with their selected best estimate, and so the uncertainty measure then becomes relative rather than absolute.
6.1.2 The language of Solvency II seems to imagine a single set of methods which describe the uncertainty of the outcome and select a best estimate, although this is evidently not mandated in current articles.
6.1.3 Is the next stage of our progression, as a profession, to find a unifying method, which adequately handles the uncertainty of outcome, but also produces best estimates which make sense and in which we can believe? If such a method exists currently then it is yet to gain sufficient acceptance and understanding to be able to fulfil its potential; but, surely, this is where we must go. There is no fundamental reason why we cannot solve this problem; on the contrary, our supposedly ‘mean’ forecasts surely should be aligned to the universe of outcomes that we can see, if we are to ever truly attain ‘best estimates’.
