Introduction
Many jurisdictions have implemented health technology assessment (HTA) programs to support decision makers in managing a portfolio of clinically and cost-effective health services and interventions to improve the health of the population (Reference Kristensen, Husereau, Huic, Drummond, Berger and Bond1). HTAs are most commonly undertaken to assess often narrowly defined decision problems for single classes of technologies including drugs, medical devices, and screening programs. Processes of care, public health interventions, and other complex interventions are less commonly assessed; however, the value of such analysis is recognized (Reference Stojanovic, Wübbeler, Geis, Reviriego, Gutiérrez-Ibarluzea and Lenoir-Wijnkoop2).
A notable goal of HTA is to “inform decision-making in order to promote an equitable, efficient, and high-quality health system” [(Reference O'Rourke, Oortwijn and Schuller3): p. 2]. Despite this aspiration, the extent to which decisions around health technologies embrace a systems perspective is seemingly modest and the impacts of doing so are not well understood. Recently, Lopes and colleagues (Reference Lopes, Street, Carter, Merlin and Stafinski4) conceptualized HTA from a system lens to capture the diversity and complexity of HTA as a system with multiple actors. The authors recognized that there are complex interactions among agents in a system and that a small change in one part of the system is likely to impact another part (Reference Lopes, Street, Carter, Merlin and Stafinski4). A similar type of systems thinking could be used to consider to what extent the conduct of HTA takes a systems perspective and what the impacts of doing so are.
The World Health Organization (WHO) defined systems thinking as “an approach to problem solving” (Reference de Savigny and Taghreed5) that places a problem within a dynamic system that requires an understanding of the pieces of the system and the relationships between them (Reference de Savigny and Taghreed5). Systems thinking was identified as one of four simultaneous revolutions that are believed to transform health and health systems (Reference de Savigny and Taghreed5). Systems thinking offers both a conceptual and a methodological perspective for considering the interactions between the components of a system (Reference Trochim, Cabrera, Milstein, Gallagher and Leischow6) and its key tenets include: self-organizing, nonlinearity, constant change, dependence on history, tightly linked ideas, counterintuitiveness, feedback governed, and resistance to change (Reference de Savigny and Taghreed5). In the context of HTA, systems thinking has been recognized as a potentially under-recognized and underdeveloped area of exploration. These developments have come from several perspectives—for example, a recognition of the importance of capturing cross-sector effects and costs of health interventions (Reference Claxton, Sculpher and Culyer7) and the value of modeling the complexity involved with the implementation of an intervention (Reference Marshall, Burgos-Liz, Ijzerman, Osgood, Padula and Higashi8). There are several ways to define a system, including how the “agents” within the system are defined and what the interactions of interest are between the agents. Conceptual clarity around systems thinking in all disciplines, including health, is evolving, and there remains no “universally accepted theories and concepts” (Reference Hossain, Dayarathna, Nagahi and Jaradat9).
A systems-level perspective may afford advantages over narrower analytic approaches that do not account for interacting elements within the system that may offer more effective responses (Reference Atun10): evidence suggests that there is potential for improved health outcomes and savings with integration and collaboration within the health sector [e.g., alignment between hospital and community formularies (Reference Lapointe-Shaw, Fischer, Newman, John-Baptiste, Anderson and Rochon11), Medicare and Medicaid in the USA (Reference Chen, Yang and Gajadhar12)] and with other sectors (e.g., social services) (Reference Mattessich and Rausch13). In contrast to a systems-level perspective, a type of “silo-mentality” is evident in health care (i.e., a grouping of health expenditures into “silos”—e.g., drug, hospital care, etc.), which may have impacts on overall health system efficiency (Reference Garrison and Towse14). This mentality has been documented with respect to healthcare services in, for example, Canada (Reference Sutherland and Hellsten15) and Portugal (Reference Meneses and Caseiro16), and with respect to drug budgets in Germany, Italy, The Netherlands, Spain, and the UK (Reference Garrison and Towse14).
Objective
To our knowledge, no prior work has synthesized or conceptualized systems thinking within HTA. To address this gap, our aim was to assess how, and to what extent, a systems-level perspective is considered in decision-making processes relating to health interventions. Our objective was to illustrate how studies define the boundaries of the system in their analyses and to define the decision-making context in which a systems-level perspective is undertaken. This approach was anticipated to allow us to understand the current knowledge in the field and to identify relevant gaps in the literature in an effort to inform the field of HTA.
Methods
We conducted a scoping review following the Joanna Briggs Institute methodology (17) and the PRISMA-ScR reporting guideline (Reference Tricco, Lillie, Zarin, O'Brien, Colquhoun and Levac18) to answer the following research question: What is known in the literature about how systems thinking is considered in the decision-making process for the adoption or de-adoption of health interventions?
Information Sources
Literature searches of electronic bibliographic databases were conducted in Ovid MEDLINE, Ovid EMBASE, Wily Cochrane Library (which includes the HTA Database and the National Health Service [NHS] Economic Evaluation Database [EED]), and Ebsco EconLit in consultation with an information specialist. Preliminary searches were conducted to assess the relevance of the identified articles and iteratively revised to capture the literature that best reflected our research question. The final search string was developed for MEDLINE and adapted for use in the other databases. Key concepts included: decision making (e.g., decision planning, decision theory, etc.), system (e.g., systems theory, systems analysis, population health management, etc.), and integration (e.g., intersectoral collaboration, alignment, collaboration). MeSH, Emtree, and text words were identified and included in the search strategy. All databases were searched from the date of inception. Search filters included: English language, human studies, and no conference abstracts. An iterative search strategy was used whereby our search string was revised following the initial search and full-text screening. Articles that described healthcare decision making as “intersectoral action” programs were captured in the subsequent search. This revision was deemed reasonable given the iterative nature of scoping reviews as terminology becomes more clear (Reference Morris, Boruff and Gore19). The initial search was conducted on 16 October 2019 and the revised and updated search was conducted on 1 April 2020. See Supplementary Tables 1 and 2 for the Medline Search string and results for the initial and revised search.
Eligibility Criteria
Publicly available studies were eligible for inclusion if they were indexed in the electronic databases searched. A search of the grey literature was not conducted.
Articles were included if a systems-level perspective was considered in the analysis of the health technology (i.e., if there was consideration of other parts of the health system or nonhealth system in the analysis), if the analysis was focused on the adoption or de-adoption of a health intervention/technology (e.g., drug, device, public health policy, etc.), and if there was consideration for integration or impact beyond the focal organization for which the assessment was being conducted (i.e., that the adoption or de-adoption of the health intervention would have an impact on, or involve, other parts of the health system or other sectors). We operationalized a systems-level perspective as an explicit recognition that: (i) a policy or health intervention decision may have an impact on other parts of the system (for example, implementation of a technology in one part of the system may be offset by costs in another part of the system), (ii) that there may be multiple policy options to address a particular problem that may fall in other sectors (e.g., using simulation modeling to compare policy options according to their impact on health outcomes), or (iii) that there may be an opportunity to jointly influence health outcomes (e.g., joint financing of an intervention between two or more budgets).
We excluded articles that considered clinical-level decision making, described examples of collaborations between sectors (without assessing impacts between sectors), or focused on how to develop integrated or collaborative care models. Our analysis was not looking to capture studies that identified opportunities to collaborate or to integrate across sectors but to identify those studies that took a “step back” and to some extent took a “birds eye view” of the system (however the system was defined). Articles that focused on, or that were meant for, low- to middle-income countries (based on the World Bank Country and Lending Groups classification) (20) were excluded to focus our analysis on countries that are likely to have well-established HTA activities in place. Studies that considered human resource planning or information technology infrastructure were also excluded. Two reviewers independently screened titles, abstracts, and full text; conflicts were discussed until consensus was reached.
Data Extraction and Analysis
Extracted data included: country, study design, objective, methodology, patient population, therapeutic area, interventions assessed, outcomes assessed, the scope of the system (i.e., health sector or nonhealth sector, the specifics within health-sector subsectors), the decision-making context, whether the study was applied or theoretical, and if any frameworks or theories were discussed. Findings were summarized and categorized descriptively and visually. The scope of the “system” was variably, and often not explicitly, defined in the included studies. Consequently, based on the full-text review, we interpreted each study's “system” of analysis and classified the included studies according to the four-level system conceptualized in Reid and colleagues (Reference Reid, Compton, Grossman and Fanjiang21), which was adapted from the model developed by Ferlie and Shortell (Reference Ferlie and Shortell22). Reid and colleagues (Reference Reid, Compton, Grossman and Fanjiang21) defined the four levels of the health system, based on (i) the individual patient, (ii) the care team, (iii) the organization, and (iv) the political and economic environments. Furthermore, we interpreted and categorized the decision-making context described within each study's system of analysis as “an integrated care pathway” (i.e., how would the intervention impact other parts of the care-pathway) or as “an integrated environment” (i.e., looking at the influence of policy options on other policy options or on the system as a whole). Our classification system was established posthoc given the largely uncharted nature of systems thinking in decision making for health interventions and a need to extract or chart the data prior to establishing appropriate categorizations.
Results
A total of 2,664 articles were identified through database searching and screened based on title and abstract. Of these, 2,131 articles were excluded, and 189 full-text articles were reviewed. Of the full-text articles that were reviewed, 160 were excluded. The most common reason for exclusion in the initial search was that studies were not looking at decision making for the adoption or de-adoption of a health intervention (n = 90). For the revised and updated search, the most common reason for exclusion was that there was no consideration for system integration or impact (n = 11). See Supplementary Figure 1 for the PRISMA Flow and Supplementary Table 3 for the reasons for the exclusions. A total of twenty-nine studies were included for analysis (Reference Rosenheck, Leslie, Sint, Lin, Robinson and Schooler23–Reference Boudreaux and Vetter51).
Country and Year
Studies were most commonly conducted in the USA (n = 12; 41%) (Reference Rosenheck, Leslie, Sint, Lin, Robinson and Schooler23;Reference Gidwani, Goetz, Kominski, Asch, Mattocks and Samet26;Reference Loo, Shapiro, Tamaddon, Chien, Rhee and Jacobsen31;Reference Dulai, Singh, Ohno-Machado and Sandborn38;Reference Leonard, Thyagarajan, Wilson and Sekeres39;Reference Munar, Hovmand, Fleming and Darmstadt41;Reference Cheng and Solomon43–Reference Abrams, Graham, Levy, Mabry and Orleans45;Reference Boult and Pacala47;Reference Wheeler, Leeman, Hassmiller Lich, Tangka, Davis and Richardson49;Reference Boudreaux and Vetter51), Canada (n = 5; 21%) (Reference Santos, Gurling, Dvorak, Noonan, Fehlings and Burns25;Reference Steele Gray, Wodchis, Baker, Carswell, Kenealy and McKillop30;Reference Embuldeniya, Kirst, Walker and Wodchis33;Reference Tannenbaum, Farrell, Shaw, Morgan, Trimble and Currie40;Reference Best, Stokols, Green, Leischow, Holmes and Buchholz46), the UK (n = 3; 14%) (Reference Forder, Gousia and Saloniki28;Reference Maniatopoulos, Haining, Allen and Wilkes32;Reference Willmott, Womack, Hollingworth and Campbell34), and Australia (n = 3; 10%) (Reference Cheng24;Reference Skouteris, Huang, Millar, Kuhlberg, Dodd and Callaway42;Reference Freebairn, Rychetnik, Atkinson, Kelly, McDonnell and Roberts50). One study was conducted in each of Germany (Reference Zhang36), Europe (Reference Gallacher, de Reydet de Vulpillieres, Amzal, Angehrn, Bexelius and Bintener48), and The Netherlands (Reference Mooy and Gunning-Schepers27). In three studies, the analysis was not targeting a specific jurisdiction of interest (Reference Liu, Triantis, Zhao and Wang29;Reference McGuire, Vijayasingham, Vassall, Small, Webb and Guthrie35;Reference Isaranuwatchai, Bayoumi, Renahy, Cheff and O'Campo37). The earliest study included was conducted in 1999, with the number of studies increasing steadily over time with a peak of eight studies in 2018. Detailed data extraction can be found in Supplementary Tables 4 and 5.
Study Type and Methods
The studies included in our review were systematic reviews, nonsystematic reviews, commentaries, framework development, decision models, cost-effectiveness analyses, and qualitative analyses. Approximately one-third of studies were applied (i.e., informing a particular decision), one-third theoretical (i.e., consisting of hypothetical cases or ideas), and one-third both applied and theoretical.
Therapeutic Area and Intervention Type
The majority of studies assessed multiple interventions (n = 18; 62%) and most commonly in the context of public health (n = 7; 24%). Four studies assessed multiple interventions in multiple therapeutic areas (e.g., in assessing integration between primary care and long-term care services, cofinancing options between two or more budgets, approaches to integrating and organizing services for older adults, and the relationship between early childhood interventions and later life trajectory). Health policy options were assessed in five studies, pharmaceuticals in three studies, screening programs in two studies, and diagnostics in one study. The number of studies by area and intervention assessed is illustrated in Figure 1.
Figure 1. Number of studies by area and intervention assessed: public health (n = 8), perioperative care (n = 1), perinatology (n = 2), oncology (n = 2), neurological (n = 2), multiple areas (n = 9), mental health (n = 1), infectious disease (n = 1), gastroenterology (n = 1), chronic diseases (n = 1), cardiovascular disease (n = 1).
Using the four-level system described by Reid and colleagues (Reference Reid, Compton, Grossman and Fanjiang21), of the twenty-nine studies included in our review, ten studies conceptualized the system of analysis based on the policy environment only, three studies from an organizational system perspective, nine studies gave consideration to all four levels of the system (i.e., patient, care team, organization, and policy environment), five studies considered three levels of the system (four considered the patient, care team, and organization; one considered the care team, organization, and policy environment), and two studies considered two levels of the system (patient and policy environments).
Figure 2 illustrates the four-level health system with illustrations of how system impacts were considered in the included studies. Studies that conceptualized the system at the patient and care-team level followed a disease-pathway or life-trajectory approach for defining the decision context (i.e., how would the intervention impact other parts of the disease pathway or life trajectory). Studies that conceptualized the system at an organizational level considered the decision context as how an intervention would impact other organizations or departments (e.g., the impact between hospital and long-term care, primary and specialty care, etc.). At the policy level, studies conceptualized the system as an intersectoral-focused decision context (e.g., the impact on nonhealth sectors) or a financing-focused decision context (e.g., the impact on public and private financing systems, cofinancing options, etc.). Details of how impact or integration was assessed in the included studies for each level are shown in the boxes in Figure 2.
Figure 2. Conceptualization of the health system and the decision problem based on the four-level health system described by Reid et al. (Reference Reid, Compton, Grossman and Fanjiang21) (adapted from Ferlie and Shortell (Reference Ferlie and Shortell22)). Note 1: The number of studies in each category adds up to more than twenty-nine, because studies conceptualized the system at more than one level in their analyses. Ten studies conceptualized the system based on the policy environment only, three studies from an organizational system perspective, nine studies gave consideration to all four levels of the system, five studies considered three levels of the system (four considered the patient, care team, and organization; one considered the care team, organization, and policy environment), and two studies considered two levels of the system (patient and environment). Note 2: Care-pathway-focused studies assessed how the intervention would impact other parts of the care pathway (e.g., downstream outcomes and health service use). Organization-focused studies assessed how the intervention would impact other organizations or departments (e.g., the impact between hospital and long-term care, primary and specialty care, etc.), financing-focused studies assessed how the intervention would impact other funding systems (e.g., collaboration between public and private systems, cofinancing between sectors, etc.), and intersectoral-focused studies assessed how the intervention would impact other sectors (e.g., education, social services, employment, etc.).
Within each of the studies’ system, we defined the impacts between “agents” as either: (i) looking at the impact of interventions on other phases of care (i.e., the system as an integrated care pathway) or (ii) looking at the influence of policy options on other policy options or on the system as a whole (i.e., the system as an integrated environment). Approximately half (n = 15; 52%) of the studies looked at the influence of policy options on other policy options or on the system as a whole, and the other half looked at the impact of interventions on other phases of care (n = 14; 48%). The majority of studies (n = 25; 86%) assessed both health and resource use outcomes, and the remainder (n = 4; 14%) assessed health outcomes only. Several studies were conducted in the context of an existing integrated care setting (e.g., surgical care home, cancer control program, comprehensive care system in the USA, medical care homes, multihospital systems in the USA, bundled care in Canada, UK commissioning groups, US Kaiser Permanente in the USA, and US veterans affairs). In twelve studies (41%), nonhealth sector impacts were considered. These sectors included transportation, environment, education, social services, housing, agriculture, tax policy, justice, research and innovation, employment, marketing, community safety, and planning.
Figure 3 illustrates the relationship between how the study conceptualized the system (i.e., as an integrated care pathway or as an integrated environment) and if the study assessed single or multiple health interventions. Also illustrated is if the study considered nonhealth sectors and if the study described a system that was already part of an integrated care environment. Seven studies (24%) assessed single classes of health interventions (e.g., pharmaceuticals, screening programs, etc.) and only two of these studies considered the integration with nonhealth sectors. The majority (twenty-two studies; 76%) of studies assessed multiple health interventions or policy options, ten of which considered nonhealth sectors when describing their decision context. Of the twelve studies that considered multiple health interventions (within the health sector only), eight studies considered the system as an integrated care pathway—for example, in the context of inflammatory bowel disease (Reference Dulai, Singh, Ohno-Machado and Sandborn38), prostate cancer (Reference Loo, Shapiro, Tamaddon, Chien, Rhee and Jacobsen31), schizophrenia (Reference Rosenheck, Leslie, Sint, Lin, Robinson and Schooler23), and traumatic spinal cord injury (Reference Santos, Gurling, Dvorak, Noonan, Fehlings and Burns25). All studies that considered nonhealth sectors were situated in nonintegrated care systems.
Figure 3. Map of studies included in the review according to how the system of analysis was conceptualized and if the study assessed single or multiple health interventions. Note: The squares represent studies that described systems that were part of an existing integrated care environment (e.g., bundled care, medical care homes, multihospital care systems, etc.); the triangles represent studies that described systems that were not part of an existing integrated environment. Studies within the dotted circle consider only the health sector, whereas studies outside of the dotted circle (emphasized with darker shaded symbols) consider nonhealth sector(s) in their analyses. Studies on the left side of the y-axis include studies that look at the impact of the intervention(s) on other phases of care and studies on the right side of the y-axis are studies that look at the influence of policy options on other policy options or on the system as a whole.
Discussion
Our review highlights that decision making for the adoption or de-adoption of health interventions is made within complex and dynamic systems. This complexity was recognized in the included studies as a result of interactions between interventions within a system (e.g., Luke and Stamatakis (Reference Luke and Stamatakis44)) and from synergistic effects between screening policies (e.g., Wheeler et al. (Reference Wheeler, Leeman, Hassmiller Lich, Tangka, Davis and Richardson49)). Authors of several papers explicitly acknowledged that changes in one part of the system have implications on other parts of the system. For example, Cheng et al. (Reference Cheng24) showed that decisions regarding policy incentives for buying private insurance will have an impact on public health service use. Forder et al. (Reference Forder, Gousia and Saloniki28) showed that primary care physician service visits could influence the use of community-based long-term care services. Our results suggest that the determination of the most clinical and cost-effective intervention can, in part, be a function of how broadly, or narrowly, a system of analysis is defined.
Several studies included in our review were applied in nature and demonstrated that coordinated decision making could achieve cost savings and improve health outcomes. We classified types of coordination in these studies as either: (i) looking at the impact of interventions on other phases of care (i.e., the system as an integrated care-pathway) or (ii) looking at the influence of policy options on other policy options or on the system as a whole (i.e., the system as an integrated environment). Santos and colleagues (Reference Santos, Gurling, Dvorak, Noonan, Fehlings and Burns25), for example, recognized the importance of considering intervention effects beyond a specific phase of care; and other studies recognized the importance of considering the full disease spectrum in their analyses (i.e., prevention, screening, diagnosis, treatment) to improve care and reduce costs (Reference Gidwani, Goetz, Kominski, Asch, Mattocks and Samet26;Reference Loo, Shapiro, Tamaddon, Chien, Rhee and Jacobsen31;Reference Dulai, Singh, Ohno-Machado and Sandborn38), Across organizations, Leonard et al. (Reference Leonard, Thyagarajan, Wilson and Sekeres39) found cost savings with the coordination of formulary management across hospitals, and Gidwani et al. (Reference Gidwani, Goetz, Kominski, Asch, Mattocks and Samet26) identified opportunities for cost savings and improved health outcomes across hospital departments. Our findings suggest that an acknowledgment and integration of commissioning and decision-making efforts may help achieve improved outcomes and reduce costs.
The majority of studies included in our review assessed multiple health interventions or policy options; relatively few studies assessed single health interventions. Consideration of the impact of a single intervention on nonhealth sectors was also rare. The difference between the number of studies included that considered single health interventions versus multiple health interventions might suggest that there is a general acceptance of a reductionist approach to assessing the value of a single health intervention. Our findings may be a consequence of a focus on high-income countries, for which decisions are typically related to the incremental adoption of health interventions assessed as new technologies emerge (Reference Glassman, Giedion, Sakuma and Smith52). This is in contrast to the growing recognition of universal health coverage (UHC) for low- to middle-income countries, which collectively considers multiple health interventions in setting up a health benefits package (Reference Glassman, Giedion, Sakuma and Smith52). Our findings do not capture this work; however, the contribution of HTA methods to informing UHC should not be overlooked. Within the context of jurisdictions with well-established HTA agencies in place, it is unclear if a lack of consideration for the complexity of the system is reasonable.
Challenges with Decision Making in Complex and Dynamic Systems
The qualitative studies included in our review identified challenges associated with considering system impacts in decision making (i.e., broadening the scope beyond individual budgets or organizations)—in particular, the challenges associated with how health services are organized and financed. Willmott and colleagues (Reference Willmott, Womack, Hollingworth and Campbell34) identified the need to understand who achieves the savings, over what time frame, and how certain the savings are as an important component of making the case for the implementation of public health interventions. Another study highlighted the need for more integration in not only the delivery of care but also the commissioning of services (Reference Maniatopoulos, Haining, Allen and Wilkes32) and acknowledged that the new care models in development in the UK had the potential to redesign the current “poorly aligned financial incentives” across different stakeholders (p. 9). Embuldeniya and colleagues (Reference Embuldeniya, Kirst, Walker and Wodchis33) demonstrated successes that could be had with integrated funding models (in particular, bundled care) for improving patient outcomes.
Implications for HTA
HTA is a process that consists of several steps: scoping, assessment, appraisal, implementation, and monitoring (Reference Kristensen, Husereau, Huic, Drummond, Berger and Bond1). Most studies included in our analysis could arguably fall within the “assessment” stage of the HTA process; however, it is important to note that our search was conducted using traditional electronic bibliographic databases and not directly from HTA agency Web sites. Consequently, we consider the extent to which systems thinking is conceptualized within only some of the activities that HTA comprises and within the context of traditional peer-reviewed publications. It was not our intent to cover the full spectrum of HTA activities but to illustrate how systems thinking might inform HTA contexts and some of the steps that comprise HTA decision making.
Our review found that systems thinking is not common in the areas where HTAs are typically applied—for example, pharmaceuticals, diagnostics, and screening programs—that is, single technology assessments. The majority of studies included in our review were for the consideration of various policy options and in the area of public health. If there are interactions between interventions, including synergistic effects between interventions, and if the adoption or de-adoption of a health technology is going to have implications on other parts of the system—how should HTA ideally define the system of analysis? If HTA is most commonly used to assess drugs or medical devices relative to the standard of care and for a specific decision maker, is the analysis overlooking potential implications on other parts of the system? Does this translate into HTA truly meeting the needs of informing a high-quality health system?
Three studies assessed decision making for single interventions within the context of a patient care pathway outside of an existing integrated care setting (Reference McGuire, Vijayasingham, Vassall, Small, Webb and Guthrie35;Reference Tannenbaum, Farrell, Shaw, Morgan, Trimble and Currie40;Reference Gallacher, de Reydet de Vulpillieres, Amzal, Angehrn, Bexelius and Bintener48), two of which considered nonhealth sectors (i.e., education, social housing, justice, agriculture, research and innovation) (Reference McGuire, Vijayasingham, Vassall, Small, Webb and Guthrie35;Reference Tannenbaum, Farrell, Shaw, Morgan, Trimble and Currie40) in their analyses. Although not commonly considered in HTA, our review highlighted several methods that could be used to account for the dynamic and complex nature of the system to support decision making. Discrete event simulation, for example, to model integrated providers as a whole versus as single units (Reference Willmott, Womack, Hollingworth and Campbell34), the use of economic evaluation, portfolio analysis, multiple-criteria decision analysis, and program budgeting and marginal analysis to assess the joint impact of multiple interventions (Reference Isaranuwatchai, Bayoumi, Renahy, Cheff and O'Campo37), or participatory group model building to facilitate collaboration between decision makers (Reference Mooy and Gunning-Schepers27;Reference Munar, Hovmand, Fleming and Darmstadt41;Reference Skouteris, Huang, Millar, Kuhlberg, Dodd and Callaway42). A consideration of nonhealth sectors is rare in HTA, but it is recognized to have the potential to improve population health. McGuire et al. (Reference McGuire, Vijayasingham, Vassall, Small, Webb and Guthrie35) reviewed cofinancing models for intersectoral action interventions, and although they acknowledged that this type of integration is still in an exploratory stage, it has the potential to achieve improved health outcomes.
Limitations
There are several key limitations of our review. Firstly, it is possible that literature relevant to our review was missed. We excluded conference abstracts unless an associated full-text article could be identified. Consequently, this excluded work, for example, by Marshall et al. (Reference Marshall, Vanderby, Enns, Frank, Wasylak and Mosher53) on system dynamics for osteoarthritis care. We focused on high-income countries, which was likely to have omitted a growing literature base on UHC for which HTA has a key role. Furthermore, our initial search strings did not capture the decision methods that were identified in Isaranuwatchai et al. (Reference Isaranuwatchai, Bayoumi, Renahy, Cheff and O'Campo37). Although we conducted a supplemental search that was less restrictive to capture other relevant literature, it is possible that the inclusion of other keywords and concepts would yield additional results. The nature of systems-thinking and systems-level impacts in decision making for health interventions is not well defined in the literature, which made our search string challenging to operationalize to achieve our study aim. The infancy of the systems-thinking field in HTA is also likely to have influenced the study section process that is subject to the experiences of the authors. It is possible that other authors would have had differing opinions on what constitutes decision making and system impacts; however, study screening and data extraction were conducted in duplicate, mitigating this potential limitation. It is our hope that other researchers will build on, and/or refine, our methodology and findings to continue to move the conversation forward. Secondly, we did not restrict our selection of articles to peer-reviewed publications and did not assess study quality or funding sources in our review. It is possible that the inclusion of these limits would change our evidence base. Lastly, given the subjective nature of our review, the reproducibility of the results is likely to be an important limitation. For example, the system and decision-making contexts were often not well defined in the included studies and were subject to our interpretation of how the study authors presented their analyses. Although we classified studies according to the four-level model of the health system developed by Ferlie and Shortell (Reference Ferlie and Shortell22), our classifications may not reflect the authors' intention. We sought to be as transparent and rigorous in our literature search, screening, data extraction, and analysis as possible; however, future researchers should expand on our work to continue to optimize the search strings, review the grey literature, and explore other classification or conceptualizations of systems thinking in their analyses.
Future Research
Greater collaboration and integration are at the root of key health system reforms including integrated care, a shift toward population health management, alternative payment mechanisms, system coordination, and collective governance (Reference Aggarwal and Williams54). These reforms will have an impact on how we conceptualize evidence to inform decision making for health interventions. Future research should continue to assess the synergistic effects between interventions and the dynamic nature of the system—for example, at a disease pathway (patient and care team), organizational, and policy level. Clarity on what the boundaries of the “system” should be in order to achieve the greatest value to the system is needed. The development of methodology to explore how to work within, or change, existing funding structures to allow appropriate collaboration between organizations and providers—that is, to not let funding structures dictate the care that is provided—would be valuable and one in which HTA could play a role. Importantly, understanding what information decision makers need in order to implement the best care for patients without being limited by their own mandate and budget constraints is paramount.
Conclusion
Our study highlights that systems thinking is not common in the areas where HTAs are typically conducted—for example, pharmaceuticals, diagnostics, and screening programs. The boundaries of the system used in analysis are typically not explicitly defined in the literature, making the conceptualization of systems thinking in health intervention decision making difficult to characterize. Future research should explore different conceptualizations of systems thinking in HTA. Our review offers one such conceptualization and highlights important considerations for the field of HTA to facilitate a meaningful evolution to truly meeting the needs of informing a high-quality health system.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000428.
Funding
No project-specific funding was provided for this work. MR is supported by a Canadian Institute of Health Research (CIHR) Doctoral Research Award—Frederick Banting and Charles Best Canada Graduate Scholarship (Funding Reference Number 164205). BS is supported by a Canada Research Chair in Economics of Infectious Diseases (CRC-950-232429).
Conflict of Interest
MR is a CADTH employee. The current work was unrelated to her employment, and CADTH had no role in the funding, design, or oversight of the work reported.
