Healthcare systems have scarce resources and require choices to be made regarding the allocation of these resources to remain sustainable. Financial resource allocation can be made arbitrarily, through market mechanisms, or they may be guided through economic modeling and cost-effectiveness analyses (CEA). The latter approach aims to allocate finite financial resources such that health gain (generally at a population level) is maximized. In this way, scarce and constrained financial resources are used as efficiently as possible.
Economic models are valuable tools used to assist decision makers in the estimation of value for money of new health technologies. While empirical clinical research methods for determining the comparative effectiveness of new healthcare interventions are well developed, inevitably they do not capture all the details required to support a fully informed decision about the incremental cost-effectiveness of a new healthcare technology. While scientific experiments can conclude that there is not enough evidence available to support a hypothesis, decision makers cannot fail to act on the information available to them (Reference Weinstein1). Rejecting or postponing the reimbursement of a new technology is a decision to continue reimbursing current practice instead of the new technology. Therefore, economic modeling provides an important formal and transparent means by which decisions can be made in the face of inevitably incomplete data.
The results of economic evaluations of the same health technology in different jurisdictions can differ leading to seemingly incongruous reimbursement decisions across regions. Some of these variations simply reflect legitimate differences in the economic question being posed. For example, two economic analyses investigating the same healthcare technology may take different perspectives, one a societal and one a payer perspective, and may legitimately elicit different conclusions. In another instance, the cost of the comparator treatment may be different in one jurisdiction compared with another, again leading to different, but equally legitimate reimbursement decisions. In some situations there may be a paucity of good quality data available to populate an economic model which may lead to erroneous cost-effectiveness estimates. In some cases, differences between economic model predictions may simply be due to calculation errors. In others, there may be more subtle reasons why the results of economic models vary, including differences in the complexity of the models, different underlying modeling assumptions and the use of different modeling techniques.
For over a decade researchers have been discussing the comparative advantages and disadvantages of Markov cohort models (MM) and discrete event simulation (DES) when applied to questions of cost-effectiveness of new healthcare technologies (Reference Karnon and Brown2). This article seeks to consolidate these comparisons by providing a systematic overview of the practical differences the use of these different modeling techniques may have on the results of CEA and the allocation of resources in financially constrained healthcare systems.
The MM was named after Andrey Andreyevich Markov, a Russian mathematician that introduced the Markov “chain” for the first time in 1906 (Reference Kouemou and Dymarski3). The MM is characterized by the Markov assumption, or memorylessness, whereby the probability of a given transition in the system is independent of the nature or timing of earlier transitions (Reference Drummond, Sculpher, Torrance, O'Brien and Stoddart4). An MM has a time-horizon which is separated into fixed time periods referred to as cycles. During each of these cycles, the cohort may transition between a finite number of health states according to appropriate probabilities (Reference Sonnenberg and Beck5). The accrual of costs and utility in the MM are determined by the number of cycles (i.e., the period of time) and the proportion of the cohort that reside in each health state over the time-horizon of the economic model. MM currently dominate the healthcare economic evaluation literature.
In contrast, DES first emerged in the 1950s and is now recognized as the most commonly used simulation technique in the field of operational research (Reference Hollocks6). DES has been used in a wide array of industries including: manufacturing, travel, finance, and health care. DES is characterized by individual entities (e.g., patients) that undergo a series of processes (events) that affect the entities attributes (such as outcomes) over time (Reference Brennan, Chick and Davies7). Typically, a list of current or future events is constructed by randomly sampling time-to-event distributions for each event, placing them in the order in which the events are predicted to occur, and attaching them to each individual entity. In this way, each entity carries an event list describing the type and timing of current or future events which may occur to the entity over time. This event list may be updated after each event to reflect the impact the “history” of the entity has on current or future events. Unlike the MM, a DES does not have fixed equal cycle lengths; instead a DES is event driven and progresses according to the timing of these events. A DES also allows interaction between entities (e.g., patients) and resources (e.g., hospital beds) such that queues can form if the demand for resources exceeds supply.
The term MM is often poorly defined in the literature. This may lead to ambiguity in the interpretation of the relative benefits of MM when compared with other modeling techniques. To avoid such ambiguity, in this article, an MM is defined as a closed system Markovian, discrete-time state-transition model which calculates “expected value” results (i.e., a cohort analysis). This includes MM in which transition probabilities can change over the modeling period (e.g., by being indexed to the model cycle) and where temporary health states and tunnel states can be used. This definition excludes closed system discrete-time individual patient simulation (IPS) state-transition models, often referred to as a Monte-Carlo microsimulation, hereafter referred to as microsimulations. The term microsimulation used herein refers to microsimulation as generally defined in the health economics literature, which model individual patients one at a time, without interaction between patients. Like DES, microsimulation models allow individual patient history to be accrued during the simulation and for this history to inform a patient's future events and pathways through the modeled healthcare system.
METHODS
A systematic search of the literature was conducted capturing citations indexed up until the 5 December 2012. Medline records from 1950 onward and Embase records from 1947 onward were searched concurrently using EMBASE.com. The Cochrane Library, including the National Health Service Economic Evaluation Database (NHS EED), was also searched for pertinent literature. Only full publications reported in peer-reviewed journals which drew comparisons between MM and DES modeling techniques when used for CEA of healthcare technologies were included in the review.
The following literature search terms were used to identify pertinent literature: discrete event simulation (and synonyms such as DES, discrete event, event simulation), Markov, micro-simulation, Monte-Carlo, cost and economic and synonyms thereof. The title and abstract of the article were reviewed by L.S. to determine the relevance of the article to the current review. Abstracts were excluded from the review if they were duplicates of other abstracts, were not full publications (e.g., letters, editorials, conference abstracts), were not focused on health economics, were not cost-effectiveness analyses, did not compare Markov and DES methods or were not published in English. If a publication could not be excluded on the basis of its abstract then the full publication was retrieved for detailed review. Two authors (L.S. & T.C.) reviewed all included publications in full to identify the advantages and disadvantages reported for each modeling method. Any differences were reviewed by the third author (P.S.) and agreed by consensus.
RESULTS
A total of 584 citations were identified in the literature searches. Some thirty-five publications could not be excluded on the basis of their abstract alone and were retrieved in full. A total of twenty-two of these publications were pertinent to the final review. The reasons for excluding the remaining 562 citations are shown in Table 1.
Table 1. Inclusion and Exclusion of Publications Comparing Discrete Event Simulation and Markov Modeling
Table 2 presents a brief description of each of the included publications and the methods by which conclusions were drawn on the relative advantages and disadvantages of MM and DES models. The majority of publications drew conclusions on the advantages and disadvantages of MM versus DES modeling through a general discussion of the author's experience and understanding of the modeling techniques (17 publications (Reference Weinstein1;Reference Karnon and Brown2;Reference Brennan, Chick and Davies7–Reference Tran-Duy, Boonen, Van De Laar, Franke and Severens21), one study (two publications) described a consensus guideline on good modeling practices for DES (Reference Karnon, Stahl and Brennan22;Reference Caro, Briggs, Siebert and Kuntz23), one publication presented a comparison of a conceptual MM and DES model (Reference Le Lay, Despiegel, Francois and Duru24), and the remaining two publications compared the results of an MM and DES model empirically (Reference Karnon25;Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26). One of these studies compared the MM and DES model results with each other (Reference Karnon25) and another compared the results of each model with each other and with an actual data-set (Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26).
Table 2. Description of Included Publications
DES, discrete event simulation; MM, Markov model; TNF, Tumour necrosis factor.
The primary differences between MM and DES models as described in the identified publications are summarized in Table 3. Many of these differences are interrelated. The most commonly reported advantage of DES over MM was that DES modeling can track individual patient histories, such that each individual in the economic model can carry a large amount of information which can affect their future treatment options, risk of events and prognosis over time (Reference Weinstein1;Reference Brennan, Chick and Davies7–Reference Hollingworth and Spackman12;Reference Kamal, Miller, Kavookjian and Madhavan14;Reference Heeg, Buskens and Knapp17;Reference Karnon18;Reference Tran-Duy, Boonen, Van De Laar, Franke and Severens21;Reference Karnon, Stahl and Brennan22;Reference Le Lay, Despiegel, Francois and Duru24). The tracking of individual patient history also allows DES to capture individual risk factors that affect outcomes in a nonlinear manner (Reference Brennan, Chick and Davies7;Reference Tran-Duy, Boonen, Van De Laar, Franke and Severens21). Furthermore, the tracking of individual patient history facilitates the modeling of interaction between covariates (Reference Brennan, Chick and Davies7;Reference Heeg, Damen and Buskens11–Reference Hughes, Cowell, Koncz and Cramer13). In this way, a DES model may produce a realistic set of virtual patient histories (Reference Kamal, Miller, Kavookjian and Madhavan14). In comparison, MM has a more limited ability to capture patient history and nonlinearity. A solution to this in an MM can be to define more health states, use tunnel states or to further relax the Markovian assumption by using different transition matrices as time progresses (Reference Brennan, Chick and Davies7;Reference Caro, Moller and Getsios9;Reference Heeg, Damen and Buskens11;Reference Hollingworth and Spackman12). However, by definition, in an MM the “residents” of these health states are assumed to be homogenous patient groups. Therefore, an MM may need an enormous number of health states to represent complex systems and this may only be feasible for a limited number of items that need to be remembered (Reference Brennan, Chick and Davies7;Reference Caro, Moller and Getsios9;Reference Hollingworth and Spackman12;Reference Kamal, Miller, Kavookjian and Madhavan14).
Table 3. Description of Advantages and Disadvantages of Markov versus DES Models
DES, discrete event simulation; IPS, individual patient simulation; MM, Markov model; SR, systematic review.
Another commonly reported benefit of DES is its ability to allow the modeling of more complex systems than MM (Reference Karnon and Brown2;Reference Cooper, Brailsford, Davies and Raftery10;Reference Hollingworth and Spackman12;Reference Kamal, Miller, Kavookjian and Madhavan14;Reference Kim and Goldie15;Reference Le Lay, Despiegel, Francois and Duru24). This feature is partly related to the DES modeling methods ability to track individual patient attributes (Reference Cooper, Brailsford, Davies and Raftery10). However, the flexibility of DES is also supported its ability to model interactions between patients, or between patients and the environment (Reference Brennan, Chick and Davies7;Reference Cooper, Brailsford, Davies and Raftery10;Reference Hollingworth and Spackman12;Reference Kim and Goldie15;Reference Karnon, Stahl and Brennan22). This feature is often used when modeling queuing systems, where supply and demand unfolds probabilistically over time (Reference Weinstein1). This allows DES to capture situations when constraints on resources mean that the choice of treatment for one patient affects what can be given to another (Reference Barton, Bryan and Robinson8;Reference Karnon, Stahl and Brennan22;Reference Caro, Briggs, Siebert and Kuntz23). These changes may affect the system performance in a nonlinear manner which is accommodated by DES methods (Reference Brennan, Chick and Davies7). It is not possible to capture such patient interactions and queuing using an MM.
Several authors suggest DES is well suited to modeling situations where patients are subject to multiple competing risks (Reference Hollingworth and Spackman12;Reference Caro16;Reference Karnon, Stahl and Brennan22;Reference Caro, Briggs, Siebert and Kuntz23). A DES manages the competing and the sequencing of events by generating a future events list, then, for example, selecting the next closest time-to-event to ascertain which event occurs next in the process. Then this process is repeated and any impact the updated patient history may have on future events is captured. Alternatively, as recommended in the consensus guidelines by Karnon et al. 2012, the analyst may sample the time to one of a range of competing events or risks and then determine through another algorithm the event of interest that occurs (Reference Karnon, Stahl and Brennan22;Reference Caro, Briggs, Siebert and Kuntz23). In an MM, a transition probability is derived for each mutually exclusive competing health state and these competing health states must be exhaustive. The incorporation of complexity is further enhanced by the DES model's ability to accommodate multiple events simultaneously (Reference Caro, Moller and Getsios9;Reference Heeg, Damen and Buskens11;Reference Hollingworth and Spackman12;Reference Heeg, Buskens and Knapp17;Reference Le Lay, Despiegel, Francois and Duru24). In MM, only one transition can be modeled per cycle. However, the impact of this limitation on accuracy may be reduced by decreasing the cycle length of the MM (Reference Brennan, Chick and Davies7;Reference Heeg, Damen and Buskens11). Furthermore, authors note that the handling of time is explicit in DES (Reference Hollingworth and Spackman12;Reference Caro16;Reference Karnon, Stahl and Brennan22;Reference Caro, Briggs, Siebert and Kuntz23). In an MM, a fixed discrete cycle length is used, which is not required in DES modeling where time can be varied throughout the duration of the simulation (Reference Hollingworth and Spackman12;Reference Caro16;Reference Karnon, Stahl and Brennan22;Reference Caro, Briggs, Siebert and Kuntz23;Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26).
As DES models are able to accommodate more complexity a greater level of detail and variables can be captured in these models than in MM. This greater flexibility allows these models to capture more detail regarding uncertainty in the system being modeled (Reference Karnon and Brown2;Reference Heeg, Damen and Buskens11;Reference Le Lay, Despiegel, Francois and Duru24). The additional variables can then be subject to probabilistic analysis. The corresponding sensitivity analyses may then give a better understanding of the uncertainty that exists in the base case values than in simpler models where these assumptions may remain implicit, not quantified and consequently incontestable (Reference Heeg, Damen and Buskens11). DES also facilitates structural sensitivity analysis by allowing the incorporation of alternative model structures in the one simulation (Reference Brennan, Chick and Davies7;Reference Caro16).
Some authors have also suggested that DES predicts the course of a disease more naturally than an MM and may, therefore, give superior “face validity” with decision makers (Reference Heeg, Buskens and Knapp17;Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26). Many of the authors found that DES models were more complex and took more time to develop and to validate than MM (Reference Karnon and Brown2;Reference Cooper, Brailsford, Davies and Raftery10;Reference Kamal, Miller, Kavookjian and Madhavan14;Reference Kim and Goldie15;Reference Le Lay, Despiegel, Francois and Duru24;Reference Karnon25). Karnon and colleagues concluded that the slight benefit of DES, in terms of increased flexibility, were outweighed by the far greater time required to develop and evaluate the DES model (Reference Karnon25). Furthermore, computational time was increased for DES over an MM (Reference Heeg, Damen and Buskens11;Reference Kim and Goldie15). DES also has the additional disadvantage of potential computational slowness associated with inserting and removing events from the future event lists (Reference Brennan, Chick and Davies7).
Some authors note that the flexibility of DES may lead to model over-specification, whereby possible patient models may become more complex than necessary (Reference Karnon and Brown2;Reference Le Lay, Despiegel, Francois and Duru24). Furthermore, data requirements are generally increased with DES compared with MM (Reference Karnon and Brown2;Reference Caro, Moller and Getsios9;Reference Heeg, Damen and Buskens11;Reference Hollingworth and Spackman12;Reference Kim and Goldie15;Reference Xenakis, Kinter and Ishak20;Reference Le Lay, Despiegel, Francois and Duru24;Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26). Also, unlike MM, authors note that DES models require specialized and often expensive software, with archaic programming conventions (Reference Barton, Bryan and Robinson8;Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26). Other researchers also cite the need for specialized analytical knowledge as a disadvantage of DES models (Reference Karnon and Brown2;Reference Barton, Bryan and Robinson8;Reference Le Lay, Despiegel, Francois and Duru24). The need for this specialist analytic knowledge may reduce accessibility to DES models (Reference Hollingworth and Spackman12;Reference Le Lay, Despiegel, Francois and Duru24). However, other authors note that there are now several high-level software packages available that may overcome this problem (e.g., Arena®, Simul8®) (Reference Caro16;Reference Karnon, Stahl and Brennan22;Reference Caro, Briggs, Siebert and Kuntz23).
Empirical Studies
As previously stated, MM and DES were compared in two empirical studies (Reference Karnon25;Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26). The results of these comparisons are presented in Table 4. In the study by Simpson, the results of the MM and DES were similar when the time-frame was short (1-year) but the DES model had a slightly better long-term (5-year) predictive validity than the MM for HIV. However, the performance of the MM was good, with the clinical outcomes generated by the model falling within a 3 percent margin of error of the actual clinical data. The cost and effect estimates generated by the DES for both treatments (lopinavir/ritonavir versus atazanavir/ritonavir) were higher than those predicted by the MM. For the lopinavir/ritonavir regimen, the estimated cost-effectiveness ratio (CER) was around $2,000 per quality-adjusted life-year (QALY) higher for the MM compared with the result obtained through DES. For the atazanavir/ritonavir regimen similar differences in the CER from the two competing models were found. Both models predicted cost-savings and QALY benefits for the lopinavir/ritonavir regimen over the atazanavir/ritonavir regimen (i.e., lopinavir/ritonavir dominates the atazanavir/ritonavir regimen). The authors found that DES was able to accommodate direct input of a range of patient inputs (CD4+ and plasma viral load levels) and provide many more details about what may be expected to happen in a population than the MM. Due to the limitations of MM methods the authors state that researchers tend to use categorical groupings to represent complex interacting continuous measures which may cause short-term aggregation bias leading to long-term prediction errors. In contrast, DES allows the inclusion of individual variables without the need to create compound, aggregate health states, which improves model precision (Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26).
Table 4. Results of the Empirical Comparisons of Markov Modeling and Discrete Event Simulation
Note. NB. Rounding applies.
aCalculated post hoc.
Ata, atazanavir; CER, cost-effectiveness ratio; Cx, chemotherapy; DES, discrete event simulation; ICER, incremental cost-effectiveness ratio; IC, incremental cost; IE, incremental effect; Lop, lopinavir; MM, Markov model; QALY, quality-adjusted life-year; Rit, ritonavir; Tam, tamoxifen; USD, United States Dollar
These factors led the authors to conclude that DES represented the course of disease more naturally with fewer restrictions which may give the model superior face validity. Furthermore, they concluded that DES is better than MM in isolating long-term implications of small but important differences in crucial input data. However, the authors note that DES requires more data, data analysis, and programming time than an MM, although the authors acknowledge that the extra time required for the DES model implementation may be due to their lesser experience working with DES software. The authors also discuss the importance of transparency of the modeling methods to decision makers; however, on the basis of this study they could not conclude which of the two modeling methods was more transparent.
The second empirical study by Karnon and colleagues compared an MM and DES for the treatment of early breast cancer (Reference Karnon25). Many of the underlying assumptions used in the two models were similar (e.g., disease free interval calculations, toxicity, etc). A total of one hour computational time was required to run the MM while the DES model took 3 days to run. Like the findings in the publication by Simpson, the DES model generated higher cost and effect estimates for both treatments than the MM, but both modeling methods predicted very similar incremental cost-effectiveness ratio results. Nonetheless, the authors found that the closeness of these results disguised some potentially important differences in the models that appear to have balanced each other out. The comparison of the alternative modeling techniques identified two areas in which DES facilitated a more flexible representation of the available data (i.e., state entry-dependent probabilities and set survival times). However, the authors found that the process of verification and validation for DES was longer, taking weeks, compared to days for the MM. The authors acknowledged that when estimating the time required to build, verify, and validate a model, factors such as experience with each method should be accounted for.
The authors concluded that the increased flexibility of DES may only provide significant improvements in the accuracy of a modeling evaluation when the areas of increased flexibility apply to a large proportion of the model. In the example presented, the authors found that the closeness of the results of the MM and DES model suggested that it was unlikely that the use of one model's results over the other would lead to an alternative resource allocation decision, and posited that the slight benefits of DES, in terms of increased flexibility, were outweighed by the far greater time required to develop and evaluate the DES model (Reference Karnon25). Therefore, the authors concluded that, in this instance, the MM was the optimal technique for evaluation of alternative adjuvant therapies for early breast cancer. However, the authors also acknowledged that there may be circumstances in which a DES model would provide a more accurate representation of the available data.
DISCUSSION
The evidence comparing MM and DES suggests that DES is a useful addition to the modeler's armamentarium for use in CEA of healthcare interventions in certain situations. One of the key advantages of DES over MM is that, like microsimulation, it is an IPS technique. This allows individual patient history to be accrued during the simulation and for this history to inform a patient's future events and pathways through the modeled healthcare system. IPS techniques accommodate correlations, inter-related covariates and nonlinear effects of patient history on future events in a more flexible manner than MM. However, this increased flexibility may lead to model over-specification, where economic models may become more complex than is necessary to elicit an accurate result. Furthermore, while not a necessity, data requirements are generally increased with DES compared with MM. Also DES is generally more complex and may require more specialist skills and take more time to develop and validate compared with MM. These additional requirements may increase the time required for governments to independently verify the results of such analyses and may reduce the apparent transparency of such analyses to decision makers. In some circumstances, the potential for increased model complexity and data requirements may act as a barrier to the adoption of DES over MM.
In practice, the MM has evolved well beyond the strict confines of the Markovian assumption. Using additional health states, tunnel states, and temporary health states allows the analyst to manipulate the Markovian assumption to capture the history of the cohort. This is further enhanced through the use of transition probabilities that are indexed to the number of cycles the cohort has spent in the model or in a particular tunnel state. However, as the interplay between patient history and future events increases in complexity the number of the additional health states required to capture this complexity increases exponentially. In some cases, this will lead to a situation where an MM becomes unwieldy and ultimately unsuitable for the accurate modeling of the cost-effectiveness question at hand. In these cases, the use of a microsimulation or a DES model would appear appropriate.
DES uses a future events list for each modeled entity which highlights an important conceptual difference between a DES and MM. A DES manages the competing and the sequencing of events by generating a future events list, then, for example, selecting the next closest time-to-event to ascertain which event occurs next in the process. Then this process is repeated and any impact the updated patient history may have on future events is captured. The rationale for this approach is that the current event may alter the patient's probability of a subsequent event. This approach provides a relatively simple way to manage a multitude of competing events and schedule them. In contrast, in an MM and a microsimulation a transition probability is derived for each mutually exclusive competing health state and these competing health states must be exhaustive.
Another area which clearly distinguishes the MM and microsimulation modeling approaches from DES is that DES allows interaction between patients or between patients and the environment whereas MM and microsimulation does not (Reference Brennan, Chick and Davies7;Reference Cooper, Brailsford, Davies and Raftery10;Reference Hollingworth and Spackman12;Reference Kim and Goldie15). This allows DES to capture situations when constraints on resources mean that the choice of treatment for one patient affects what can be given to another (Reference Barton, Bryan and Robinson8). Furthermore, these changes may affect the system performance in a nonlinear manner which is accommodated by DES methods (Reference Brennan, Chick and Davies7). However, the analyst should consider whether any incremental change in demand introduced by the new intervention being modelled is likely to lead to the alleviation or exacerbation of supply shortages and changes in subsequent queuing, or diversion of patients through other, often less efficient, pathways in the health care system. Where this is not the case, or this is unlikely to be a driver of cost-effectiveness, incorporating queuing into the economic model is likely to be an unnecessary over-specification. In other situations, such as the modeling of demand for nursing home care for patients with Alzheimer's disease or other dementias in increasingly ageing populations, the ability of an economic model to capture the impact of competition for resources may be crucial. The choice of modeling method in this situation has the potential to result in very different resource allocation decisions that may alter the overall efficiency of the healthcare system markedly.
A MM is a cohort level analysis that only needs to be run once to generate expected values of cost and effect. This process is generally very rapid. In comparison, an IPS such as DES requires that multiple iterations of the model be calculated to generate accurate and precise estimates of the expected values of cost and effect. Clearly this leads to increased computational time. However, in theory, a DES model may be more computationally efficient than a comparable microsimulation. In a microsimulation, each patient must traverse all of the cycles of the model, if the model is constructed using a lifetime time horizon and the cycle length is short this may lead to a high computational burden. In contrast, in a DES model, only pertinent events are modeled and as time is explicit and stochastic a patient may traverse the model in a small number of calculations, thereby decreasing computational burden. In some situations, this may lead to the use of a DES over a microsimulation. However, the computational efficiency of DES, when compared with microsimulation, may be somewhat tempered as a DES must maintain a future event list where events must be inserted and removed as required (Reference Brennan, Chick and Davies7). Furthermore, in CEA, the economic model must ascertain when costs and effects are being accrued so that they may be discounted appropriately. This may require that the DES model is supplemented with additional calculations to assess the timing of cost and effect accrual and allow appropriate discounting calculations to be applied. Furthermore, the efficiency of the programming used should also be considered as this can have an impact on computational times. Regardless, as many authors discuss, the availability of relatively inexpensive computational power and efficient computer programs to run such analyses can somewhat alleviate these computational burden considerations.
This review has demonstrated the advantages and disadvantages of MM and DES. However, this review is not without limitations. While a substantial number of articles were identified which provide important insights into the advantages and disadvantages of DES compared with MM, the majority of publications identified compared MM and DES subjectively through the authors’ experience and understanding of the modeling techniques. Only three studies were identified that compared MM and DES models directly (Reference Le Lay, Despiegel, Francois and Duru24–Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26) and one of these studies was a comparison of conceptual models only (Reference Le Lay, Despiegel, Francois and Duru24). The remaining two studies compared DES and MM empirically (Reference Karnon25;Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26), however, only one of these studies attempted to calibrate these results back to an actual dataset (Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26). Both studies found that DES took longer to develop and validate than the MM. However, the authors acknowledged that when estimating the time required to build, verify, and validate a model factors such as experience with each method should be accounted for. In the two empirical analyses, both modeling techniques provided similar guidance on the incremental cost-effectiveness of the products being investigated. However, both studies found that the DES technique was able to accommodate higher levels of complexity than an MM and, in some situations, this may result in DES being more accurate than an MM (Reference Karnon25;Reference Simpson, Strassburger, Jones, Dietz and Rajagopalan26). Also, it is important to note that, while these studies explored the ability of DES and MM to accommodate complexity, neither of the studies focused on the potential differences in cost-effectiveness results that may be driven by queuing and competition for limited resources. This is an area that needs to be researched in more detail.
In summary, DES modeling, like other IPS methods, would be preferred over MM where individual patient history is an important and complex driver of future events. Furthermore, DES would appear to be superior to MM when modeling situations where supply shortages and subsequent queuing, or diversion of patients through other, often less efficient, pathways in the healthcare system are likely to be a driver of cost-effectiveness. Where this is not the case, incorporating queuing into the economic model is likely to be an unnecessary over-specification which is unlikely to be informative to decision makers. Additionally, DES may have some advantages over MM when modeling patients at multiple competing risks. However, where these are not major features of the cost-effectiveness question, it would appear that the MM remains an efficient, transparent, easily validated, parsimonious and accurate method of determining the cost-effectiveness of new healthcare interventions. Therefore, MM remains an important tool by which to guide decision makers on the efficient allocation of scarce resources in the healthcare setting.
CONTACT INFORMATION
Lachlan Standfield, BBiotech (Hons I) (lachlan.standfield@griffithuni.edu.au), Tracy Comans, PhD, Senior Research Fellow, Paul Scuffham, PhD, Professor and Director of Population and Social Health Research, Centre for Applied Health Economics, School of Medicine & Griffith Health Institute, Logan Campus, Griffith University, Meadowbrook, Queensland 4131, Australia
CONFLICTS OF INTEREST
The authors report no conflicts of interest.
