Introduction
The world has a rapidly ageing population. Associated with an ageing population is an increased prevalence of injuries, health conditions, and diseases. Furthermore, across the age spectrum, an estimated 20 percent of the world's population experiences difficulties with physical, cognitive, or sensory functioning, mental health, or behavioral health (1;2). These experiences may be temporary or permanent, acute or chronic, and may change throughout one's life span. If left unchecked, trends in chronic disease risk factors—combined with an ageing population—suggest that the delivery of affordable, timely, and quality care may become unsustainable. Preventative care models show potential as an alternative and proactive approach that aims to maintain the health of individuals by providing timely intervention prior to the onset of evident signs of ailment. Precision health (PH) is such a preventative health strategy that aims to support and maintain the health and well-being of individuals through the provision of predictive and personalized care (Reference Schüssler-Fiorenza Rose, Contrepois, Moneghetti, Zhou, Mishra and Mataraso3;Reference Silvera-Tawil, Hussain and Li4). PH requires a holistic understanding of an individual's health beyond their clinical physiological status including genome, lifestyle, and mental health factors. PH-based models of care have the potential to be more efficient and cost-effective to the healthcare system, because the nature of care provision is preventative, rather than reactive once an individual has already experienced an ailment (Reference Stangt5;Reference Spina, Huang, Vaes, Spruit and Amft6).
A range of health technologies exist that aim to assess, prevent, treat, and manage health conditions to promote better health and well-being (7). Through leveraging existing and emerging technologies—such as wearable sensors, smartphones, mobile applications (apps), internet-of-things (IoT), and artificial intelligence (AI)—new opportunities are positively shaping healthcare models and patient outcomes (Reference Bhavnani, Parakh, Atreja, Druz, Graham and Hayek8). To this end, there is a growing number of emerging technologies with a diverse range of functionalities and forms that have the potential to become PH innovations, including hardware (e.g., sensors), software (e.g., mobile and web platforms with data integration), and data analytics techniques (e.g., predictive models) to assess, diagnose, and manage health and well-being.
One of the main challenges of integrating emerging technologies into a PH solution is that most technologies do not have the required scientific evidence to support their claims and do not meet regulatory standards to be PH solutions—regardless of whether they were intended for health or designed for a PH context. With these factors in mind, a framework that supports a standardized approach for identifying and implementing PH products is essential. In response, we propose a framework in the form of health technology assessment (HTA), which can be used to accommodate a large range of products, technologies, and innovations for PH applications.
HTA and Opportunities for PH
HTA is the systematic evaluation of technology properties and their associated impact on health care (7). It encompasses a multidisciplinary decision-making process to determine the value of technology at different points across its lifecycle (Reference O'Rourke, Oortwijn and Schuller9). Global (e.g., World Health Organization) and national (e.g., FDA, Australian TGA, and Department of Health) health agencies engage in HTA and provide guidelines for the assessment of technologies in health care (7;10). These guidelines may cover criteria to assess the justification and rationale (e.g., clinical evidence, impact on efficiency and effectiveness), service provision (e.g., patient pathway and care model, safety requirements, and conflict of interests), and monitoring and evaluation of new technologies (10). Examples of assessment outcomes may include: out of scope, still in research, seek further advice, not recommended or recommended.
Although available frameworks provide an effective tool to assess a wide range of technologies in health care, the rapid emergence and quickly evolving nature of digital health technologies has attracted academic research and international agencies to develop new assessment guidelines and frameworks specific to digital health (11;Reference Rodriguez-Villa and Torous12). The basis of these frameworks consists of indicators that guide the assessment of digital health technologies to reach a recommendation of suitability for their use in health care. HTA frameworks in digital health generally include assessments related to technical requirements and functionalities; clinical, economic, organizational, ethical, and legal factors; and healthcare system characteristics and stakeholder determinants (Reference Vis, Bührmann, Riper and Ossebaard13).
Digital health frameworks can be classified into (i) staged, which apply a sequential approach for assessing measures relevant to the development phase of a technology service; (ii) dimensional, which contain a set of assessment measures grouped according to the expected impact of technology services, irrespective of the development phase; (iii) hybrid, which combine a sequential phased development approach and the impact of a technology service, and apply a varying set of criteria in a particular order for the assessment; and (iv) business modeling, which include an assessment of technology services in the context of economic viability and business models (Reference Vis, Bührmann, Riper and Ossebaard13).
Despite the availability of assessment frameworks for digital health, there is a growing demand for more specific guidelines and metrics to identify and assess current and emerging digital technologies. Of particular interest are sensing devices and related technologies that have potential in PH applications (Reference Silvera-Tawil, Hussain and Li4), even if their original purpose was not related to PH. Most work in this area has been in the form of siloed approaches largely focused on specific attributes (e.g., compliance to patient safety) grounded by technical, physical, and operational factors (Reference Angelis and Kanavos14–Reference Johnson, Sikich, Evans, Evans, Giacomini and Glendining25). The literature in terms of assessment measures to PH innovations is sparse. Existing assessment frameworks can provide guidelines on health-related technologies in their complete form (e.g., Ref. 7), but they are not necessarily effective when assessing technologies that were not originally intended for health care.
A lack of agreed measures poses challenges in ensuring whether a specific innovation aimed for PH is appropriate. Assessment models, such as the GRASP framework (Reference Khalifa, Magrabi and Gallego26), were developed to cover tools that recommend actions and decisions of specific clinical implementations, but they do not cover their potential as digital health technologies or innovations for PH. The proposed framework goes beyond the assessment of a technology for implementation. It provides advice to guide a decision maker toward including a piece of technology within a product and the impacts this may have. It can be used by a variety of stakeholders (e.g., innovators, software developers, clinicians, and researchers) working on PH solutions that require emerging technologies in the form of hardware, software, or data analytics techniques.
The fundamental novelty of this framework lies with its intend of use for PH applications, including preventative care models that go beyond traditional clinical care settings (e.g., home care monitoring and personalized/adaptive intervention). If such applications become part of an individual's care, there are technical aspects—such as accuracy of measures, safety, privacy, regulatory compliance—that need special consideration and are not captured comprehensively in the scope of existing health assessment frameworks, which impacts their utility for such purposes. As a wave of digital technologies enter the health domain, with specific potential for PH, this framework aims to empower an array of stakeholders with the ability to assess, evaluate, and implement digital innovations (including technologies repurposed to be used in PH applications) based on the suitability and impact they may have on the target application.
In line with previous work, the proposed framework aims to provide guidance on a standardized approach to support an improved implementation process of PH innovations, ensuring their suitability, compatibility, performance, reliability, interoperability, quality, and safety among a range of key considerations.
Methodology
A scoping review was undertaken of available academic literature (between the years 2005 and 2018) through PubMed and Google Scholar to identify articles that included general health informatics terms (i.e., telemedicine, medical innovations, eHealth, mHealth, and medicine) and keywords related to technology assessment or evaluation (i.e., technology, assessment, evaluation, criteria, decision, and framework) within paper titles. The search returned numerous articles across these databases (>1,000 in total). Duplicates of the retrieved studies, from both databases, were removed first. Then, studies were screened for relevance by one researcher, based on their titles and abstracts. Approximately 100 articles relevant to this work were shortlisted. A closer examination throughout the full text of the shortlisted articles identified those (n = 10) that included specific metrics, criteria, or frameworks for assessing or evaluating products, technologies, and health innovations applicable to the PH context. During this process, two additional articles (dated 1986 and 1995) were identified from references of the shortlisted articles and were included due to their suitability for the scope of this work. The final set of articles (n = 12, Supplementary Appendix 1) were then used to inform the development of the proposed framework.
In consolidating available frameworks for technology assessments and evaluations in health care, key metrics were extracted by one of the authors to develop a comprehensive process for assessing potential products, technologies, and innovations for PH. This was then revised and confirmed by the two other authors in an iterative process. Using the revised information, we designed an initial version of the framework and applied it to assess three different PH innovations: a wearable device, a mobile app, and a predictive algorithm. A revised version of the framework was then used by two independent researchers (with expertise in the development and implementation of digital health solutions) to assess emerging technologies with potential for PH (e.g., a novel sensor to monitor sleep patterns). Their feedback and recommendations were incorporated into the final framework. Figure 1 shows the flowchart of the technology assessment framework for PH applications.
Figure 1. Framework development flowchart.
Results
This section presents our health technology assessment framework for PH applications. The framework follows a hybrid framework approach that encompasses both: (i) a single piece of digital technology (e.g., a predictive algorithm, a biosensor, or a wearable sensor) and (ii) several integrated digital technologies. The framework identifies nine core criteria with subcriteria and groups them into four categories based on whether the criteria are related to technical, clinical, human (i.e., interaction), and implementation aspects. For each of the four groups, the framework provides a comprehensive list of metrics with suggested recommendations (Optional, Encouraged, Mandatory) to be considered when assessing PH innovations—referred to as the “technology or product.”
Not all aspects of the framework need to be considered when evaluating a product or technology; thus, the suitability of the core criteria, sub-criteria, and metrics will depend on the application and use case intended by the assessor. This will be referred to as appropriate for the target use case (ATUC).
Technical Assessment
Technical assessments cover Novelty, Adaptability, Information Management, and Performance aspects of the innovation (Table 1). Novelty refers to the value proposition, uniqueness, status, and readiness of the innovation. Adaptability considers criteria relating to the innovation's ability to be used with other innovations and/or expanded beyond its original purpose. Data collection and its handling by technical components fall under Information Management, whereas the standard of quality of the innovation is covered by Performance. These measures can be used to assess innovations that may or may not be complete PH solutions.
Table 1. Technical assessment metrics and guidelines for PH innovations
Clinical Assessment
Several factors can influence the appropriateness of an innovation when operated within clinical settings and environments or with existing clinical innovations. The clinical assessment of the framework (Table 2) covers End-to-end Intervention (aspects of intervention delivered by the innovation, including health assessment tools) and Quality Assurance (quality measures of clinical interventions, including health assessment tools). These measures can be used to assess the appropriateness and impact of an innovation when incorporated into clinical settings.
Table 2. Clinical assessment metrics and guidelines for PH innovations
Human Factors' Assessment
This section covers all aspects of user experience, including usability, interactivity, and engagement with the innovation (Table 3). These measures can be used to assess products and technologies that are directly used by people. These innovations may or may not be a final PH solution.
Table 3. Human factors and implementation assessment metrics and guidelines for PH innovations
Implementation
Implementation of innovations in health requires careful considerations of their efficacy, effectiveness, and implications on healthcare practice. Evidence-based, and Registration and Compliance, cover aspects regarding an innovation's assertion of its intended purpose and its availability and discoverability in the public domain (Table 3). This should be used to assess innovations that are close to a final PH solution.
Framework Implementation and Use Cases
This PH Innovation Assessment Framework is intended to be used by innovators and researchers developing or implementing PH solutions when leveraging current and emerging technologies in the form of hardware, software, or data analytics techniques. The framework aims to cover a comprehensive set of criteria as part of the assessment process of these technologies. For each of the criterion in the framework, guiding statements (or metrics in Tables 1–3) are used to complete the assessment using the following responses:
• True: The information is available, and the assessment is valid.
• False: The information is available, but the assessment is not valid.
• Unavailable: The information is not available.
• Partial: The information is incomplete or partially available.
• Not Applicable. The criterion is not applicable for the target use case
Only a subset of the criteria in Tables 1–3 may be appropriate based on an innovation's scope, intended use, development phase or maturity of the technology, and the health system requirements as part of the implementation (Reference Vis, Bührmann, Riper and Ossebaard13). From the overarching categories perspective, technical and clinical factors are essential. In most cases, the technical and clinical criteria should be assessed before an innovation is considered suitable for PH. Then, criteria related to human factors should be assessed for innovations that involve human interaction. Finally, implementation criteria should guide decisions when an innovation is to be incorporated into real-world settings.
Not every innovation put through the framework will successfully adhere to the full set of suggested criteria. This does not necessarily mean that an innovation should be excluded from its intended use, but may require careful consideration before implementation in practice. Thus, in this framework, recommendations are organized using three levels:
• Mandatory: Required to respond “True” pertaining to the guided statement; responding “False,” “Unavailable,” or “Partial” may indicate that an innovation is inappropriate for an intended use case.
• Encouraged: “True” response is recommended pertaining to the guided statements. Alternatively, proceed cautiously or conduct in-house evaluations for “False,” “Partial,” or “Unavailable” responses.
• Optional: The response should not inhibit the adoption of an innovation.
If an innovation is deemed unfit for use, contacting the developer or sponsor of the innovation may help validate the suggested recommendations against the core criteria and subcriteria in the assessment framework. Where possible, based on the use case and complexity of the innovation, additional development and/or evaluation might be considered as an alternative (Reference Savini, Ionas, Meier, Pop and Stormer22).
Conclusion
Technologies and innovations targeting PH solutions will play a fundamental role in shaping future healthcare models and supporting personal health, well-being, and patient outcomes. With the wave of established and emerging technologies showing the potential to support healthcare delivery through PH solutions, researchers and innovators need to consider safety regulatory standards and evidence-based implications, among many factors, associated with their innovation. Thus, a technology assessment framework for PH applications is essential to support the appropriate implementation of these innovations.
In this paper, we collated the components required during the assessment of PH innovations. We believe that the grounding of the assessment framework, formed from previous works, will be flexible and adaptable to a variety of PH applications, which will play a foundational role in supporting the growth of innovations in PH. Although the focus of this framework is on PH, the framework could be applicable to health technologies in general, particularly those where user data are collected and used to support decision making (e.g., medical assessment).
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000350.
Acknowledgment
This study was completed with funding from CSIRO.
Conflicts of Interest
The authors report no conflict of interest, no financial competing interests, and no nonfinancial competing interests.
