Hostname: page-component-745bb68f8f-l4dxg Total loading time: 0 Render date: 2025-02-04T15:10:52.045Z Has data issue: false hasContentIssue false
  • English
  • Français

A review of audiovisual telemedicine utilization and satisfaction assessment during the COVID-19 pandemic

Published online by Cambridge University Press:  20 December 2021

Raphael Agbali Open the ORCID record for Raphael Agbali [Opens in a new window] ,
Andrew E. Balas ,
Francesco Beltrame  and
Gianluca De Leo
Raphael Agbali*
Affiliation:
College of Allied Health Sciences, Augusta University, Augusta, GA, USA
Andrew E. Balas
Affiliation:
College of Allied Health Sciences, Augusta University, Augusta, GA, USA
Francesco Beltrame
Affiliation:
Department of Informatics, Bioengineering, Robotics and System Engineering University of Genoa, Genoa, Italy
Gianluca De Leo
Affiliation:
College of Allied Health Sciences, Augusta University, Augusta, GA, USA
*
Author for correspondence: Raphael Agbali, E-mail: ragbali@augusta.edu
Rights & Permissions [Opens in a new window]

Abstract

Introduction

The use of telemedicine has broadened as technology that both restores continuity of care during disruptions in healthcare delivery and routinely provides primary care alone or in combination with in-person care. During the Covid-19 outbreak, the use of telemedicine as a routine care modality further accelerated.

Methods

A review of scientific studies that used telemedicine to provide care from December 2019 to December 2020 is presented. From an initial set of 2,191 articles, 36 studies are analyzed. Evidence is organized and evaluated according to the country of study, the clinical specialty, the technology platform used, and satisfaction and utilization outcomes.

Results

Thirty-one studies reported high patient satisfaction scores. Eight studies reported satisfaction from both providers and patients with no uniformly accepted assessment instrument. Eight studies conducted a descriptive analysis of telemedicine use and patient adoption patterns. Less than one-third of studies were controlled before/after studies. Most studies were conducted in the USA followed by Europe.

Conclusions

Reported satisfaction rates are high, consistent with previously documented research, whereas utilization rates increased significantly compared with the prepandemic period. Future work in developing standardized uniform assessment instruments, embedded with each telemedicine system, would increase versatility and agility in the assessment, boosting statistical power and the interpretation of results.

Keywords

Covid-19TelemedicineOutcomesAssessmentAudiovisual consultation
Type
Assessment
Information
International Journal of Technology Assessment in Health Care , Volume 38 , Issue 1 , 2022 , e2
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Introduction

The outbreak of the highly contagious coronavirus disease 2019 (Covid-19) caused by the novel, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was first reported on 31 December 2019 (1). Health organizations and governments all over the world rapidly created strategies that effectively limited close interpersonal contact, including suspension of elective medical procedures and deferral of nonessential in-person clinic encounters (1). The United States Centers for Disease Control and Prevention (CDC) updated recommendations requiring health providers to identify alternatives to face-to-face visits, with optimization of telemedicine as a preferred modality (2).

According to the American Telemedicine Association (ATA), telemedicine is the use of medical information exchanged from one site to another via electronic communications to improve a patient's clinical health status (3).

Following the World Health Organization (WHO) declaration of the coronavirus disease (Covid-19) outbreak as a pandemic on 11 March 2020, adoption of strategies, guidelines, and relaxation of restrictions related to telemedicine assumed a global scale (1;Reference Greenhalgh, Wherton, Shaw and Morrison4–Reference Bashshur, Doarn, Frenk, Kvedar and Woolliscroft6).

Telemedicine had been shown to be helpful in previous outbreaks, including former coronavirus outbreaks such as SARS-CoV-1 and MERS-CoV (Middle East respiratory syndrome coronavirus), Ebola virus disease (EVD), and Zika viruses (Reference Ohannessian7). The benefits of video consultations had also been documented during case management of severe acute respiratory syndrome (SARS) to reduce the spread and exposure of providers and transmission (Reference Chang, Lee and Wu8). Telemedicine was also vital to assessment, diagnosis, treatment during outbreaks, and disasters in Somalia in 2011, Haiti in 2010, and Wenchuan, China, in 2008 (Reference Bonnefoy and Gionet-Landry9).

Recent reviews highlight telemedicine as one of the many indispensable components of e-health during the Covid-19 period (Reference Tebeje and Klein10).

Although increasingly used in many medical specialties pre-Covid-19, telemedicine may have also been perceived as an alternative modality that restores continuity of care in the medical home setting and hard-to-reach sites and at moments of disruption in healthcare delivery (Reference Hollander and Carr5;Reference Bashshur, Doarn, Frenk, Kvedar and Woolliscroft6;11). Governmental and institutional investments in scaling up, deregulating, and reimbursing telemedicine services during the Covid-19 outbreak supported an emergent role for telemedicine as capable of assuming “need of care” status or a necessity for routine stand-alone or hybrid use with in-person care delivery (Reference Bashshur, Doarn, Frenk, Kvedar and Woolliscroft6;12;Reference Perrin, Pierce and Elliott13). This emergent role during Covid-19 provided further opportunity to identify trends in both the application of telemedicine assessment frameworks that had been identified as limited in practice and the measurement properties of assessment instrument in use by researchers (Reference Vis, Bührmann, Riper and Ossebaard14–Reference Barsom, Van Heeds, Bemelman and Schijven16).

The purpose of this literature review is to identify and to summarize studies that report using telemedicine as a means to provide healthcare services during the Covid-19 pandemic. We are particularly interested in highlighting the satisfaction of patients and providers and the increase in the utilization of telemedicine.

Materials and Methods

We conducted a review of peer-reviewed published studies using inclusion and exclusion criteria related to our goals. The study was considered nonhuman research, not requiring ethics review.

Inclusion Criteria

Studies were considered eligible if they used synchronous audiovisual consultation with adults, adolescents, and children either as inpatients or outpatients. Studies with institutional scale-up evaluations were considered if they included actual patient telemedicine services derived from real-time provider–patient interface.

The major outcomes of our study are utilization measured by uptake of telemedicine services and patient–provider satisfaction assessment. The country of study and research design were also reported. The number of authors was not considered a criterion and audio or telephone-only consultation-based studies were excluded to enhance a comparison of only synchronous audiovisual technology, considered by review authors as the most optimal mode of telemedicine with real-time multiple communication cues.

Literature Search

We performed a database search in the Cumulative Index of Nursing and Allied Health Literature (CINAHL) and PubMed for peer-reviewed articles within the period 1 December 2019 to 1 December 2020, using the key terms “telemedicine” and Covid-19. Boolean operators “AND” with “OR” were used to limit and expand searches.

We applied the best match options in PubMed and used the filter functionality to limit search to publication date range. Initial query was conducted for title/abstracts using the key search terms. Our article selection process is presented in Figure 1.

Figure 1. Article selection process.

The results of the PubMed search (2,125 hits) were then manually screened to 231 articles by reviewing article titles to remove duplicates and articles whose titles included the search keywords but were, in fact, reports of protocols, opinions, guidelines, editorial letters, position papers, and comments. The search in CINAHL yielded sixty-six relevant-titled articles with twenty-nine duplicate articles that were removed in the next step using the “exclude Medline option.”

Finally, a detailed screening of abstracts was conducted on the 268 candidate title articles (231 PubMed and 37 CINAHL source) to remove single case reports and to identify only studies that presented telemedicine encounters that successfully allowed a provider to interact with a patient. A total of forty-five articles were identified. A further review excluded nine articles that were published within the study period but conducted before Covid-19 (Reference Cremades, Georgina, David, Navinés, Espin and Pardo17), focused on remote monitoring (Reference Rabuñal, Suarez-Gil, Golpe, Martínez-García, Gómez-Méndez and Romay-Lema18), utilized geographical mapping (Reference Khairat, Meng, Xu, Edson and Gianforcaro19), or utilized audio-only technology, resulting in thirty-six included studies for the final review. One article in German was translated using Google translate (https://translate.google.com/) and the survey scale translated from the German school grade scale to percentages using https://msingermany.co.in/german-grade-calculator/ (Reference Gerbutavicius, Brandlhuber and Glück20).

Two review authors (RA and GD) independently assessed the eligibility of each potentially relevant study. Disagreements were resolved by discussion among authors. Although we scrutinized studies for similarities in assessment instruments, heterogeneous contexts and diverse tools for patient satisfaction measurement were common, consistent with previously known limitations with telemedicine studies (Reference Whitten, Mair, Haycox, May, Williams and Helmrich21;Reference Batbaatar, Dorjdagva, Luvsannyam, Savino and Amenta22). Full texts of the retrieved articles were assessed, and a summary of their characteristics is reported in Table 1.

Using previously recommended steps (Reference Viswanathan and Berkman58), we applied consensus opinion to assess and to report believability and precision (likelihood of precise effects) at the whole study level as shown in Table 1. Rating was done separately for studies with outcomes related to satisfaction and utilization based on custom items from the University of North Carolina at Chapel Hill Evidence-based Practice Center® – RTI International item bank for risk of bias and precision of observational studies, and guidance from the US Agency for Healthcare Research Quality (AHRQ) Comparative Effectiveness Research Methods guide for risk assessment (Reference Viswanathan and Berkman58;Reference Viswanathan, Ansari, Berkman, Chang, Hartling and Mc Pheeters59).

Studies that reported satisfaction were rated as high in believability and precision when they used prospective design, comparison groups or controls, prior tested measures, explicitly defined inclusion/exclusion criteria, and response constructs. Studies identified to be of moderate believability and precision were typically pilot studies with retrospective design, without group comparison and analysis, or using self-developed, nontested outcome constructs or not clearly defined, inclusion criteria. The rest of the included studies applied pragmatic, context constructs with summary scales for satisfaction and no prespecified inclusion criteria and was rated as being low in believability and precision.

In studies that measure utilization, the temporal direction of patient recruitment (prospective or retrospective) was not taken under consideration in our rating decision because they mainly consist of descriptive and improvement studies focused on quantitative counts from chart reviews and are by design, retrospective.

In addition to telemedicine use and satisfaction measures, we also included study characteristics. The actual date period of data collection for each study is reported for the thirty-six studies in Table 1.

For ease of interpretation and readability, Table 2 reports satisfaction and utilization, displaying both percentages and the classified reported effects by ordinal categories. We categorized effect intervals into quartiles with nominal labels of trivial (0–25%), low/small (26–50%), moderate (51–75%), or high/large (76% and above) as presented in previous studies (Reference Cassar, Misso, Hopkins, Shaw, Teede and Stepto60;Reference Hopkins, Marshall, Batterham and Hanin61). Satisfaction scores reported quantitatively in other scales by authors of five studies (Reference Gerbutavicius, Brandlhuber and Glück20;Reference Negrini, Donzelli, Negrini, Negrini, Romano and Zaina32;Reference Shafi, Lovecchio, Forston, Wyss, Casey and Press36;Reference Yoon, Tong, Anton, Jasinski, Claus and Soo37;Reference Kalra, Williams, Commiskey, Bowers, Schempf and Sahel41) were also converted to percentages. Two studies solely reporting clinical outcomes are not included in Table 2 but reported under results (Reference Jones, Goley, Alexander, Keller, Caldwell and Buse28;Reference Lai, Yan, Yu, Tsui, Chan and Yee29).

Table 2. Satisfaction and utilization results

Results

Synthesis of Results

A total of thirty-six studies met predefined inclusion criteria for this review. Five studies reported utilization and quality measures solely while three studies simultaneously analyzed telemedicine institutional implementation measures along with patient or provider satisfaction (Reference Gilbert, Billany, Adam, Martin, Tobin and Bagdai27;Reference Madden, Emeruwa, Friedman, Aubey, Aziz and Baptiste31;Reference Mann, Chen, Chunara, Testa and Nov45). With regard to assessment of telemedicine user satisfaction, none of the other thirty-one studies reporting measures of satisfaction described adherence to domains of specific assessment framework or share a similar methodology and common assessment tool.

Design and selection bias vulnerabilities were detected commonly in included studies primarily due to a predominantly observational design, a lack of randomized clinical trials, or included controls. We rated ten satisfaction outcome studies as high in believability and precision (Reference Ashry and Alsawy23;Reference Fieux, Duret, Bawazeer, Denoix, Zaouche and Tringali25;Reference Garcia-Huidobro, Rivera, Valderrama, Bravo and Capurro26;Reference Negrini, Donzelli, Negrini, Negrini, Romano and Zaina32–Reference Zhang, Cha, Lynch, Cahlon, Gomez and Shaverdian38), thirteen studies as moderate (Reference Harper, Roof, Wadhawan, Terala, Turchan and Bagnato39–Reference Morisada, Hwang, Gill, Wilson, Strong and Steele46;Reference Ramaswamy, Yu, Drangsholt, Ng, Culligan and Schlegel48;Reference Byrne and Watkinson49;Reference Tenforde, Iaccarino, Borgstrom, Hefner, Silver and Ahmed51–Reference Zhu, Williamson, Lin, Bush, Hakim and Upadhyaya53), and the rest as low. We classified six utilization outcome studies as high in believability and precision (Reference Compton, Soper, Reilly, Gettle, List and Bailey24;Reference Gilbert, Billany, Adam, Martin, Tobin and Bagdai27–Reference Madden, Emeruwa, Friedman, Aubey, Aziz and Baptiste31), three studies in the moderate category (Reference Mann, Chen, Chunara, Testa and Nov45;Reference Punia, Nasr, Zagorski, Lawrence, Fesler and Nair47;Reference Siow, Walker, Britt, Kozy, Zanzucchi and Girard50), and one study as low.

Observations were outlined based on themes of adoption and implementation, measurement outcomes, and access/barrier issues identified in a previous publication as unmet potential areas for telemedicine at federally qualified health centers (Reference Lin, Dievler, Robbins, Sripipatana, Quinn and Nair62).

Technology Implementation, Clinical Specialties, and Assessment

The technology implementation and rate of adoption of telemedicine services by patients, providers, and institutions in the selected studies were extracted and reported in Table 1 with highlights.

Technology Implementation

Six studies utilized Zoom® as the technology platform for video consultations (Reference Garcia-Huidobro, Rivera, Valderrama, Bravo and Capurro26;Reference Lai, Yan, Yu, Tsui, Chan and Yee29;Reference Lonergan, Washington, Branagan, Gleason, Pruthi and Carroll30;Reference Shafi, Lovecchio, Forston, Wyss, Casey and Press36;Reference Tenforde, Iaccarino, Borgstrom, Hefner, Silver and Ahmed51;Reference Barney, Buckelew, Mesheriakova and Raymond-Flesch57). Five studies utilized Epic® (Reference Madden, Emeruwa, Friedman, Aubey, Aziz and Baptiste31;Reference Kalra, Williams, Commiskey, Bowers, Schempf and Sahel41;Reference Morisada, Hwang, Gill, Wilson, Strong and Steele46;Reference Ramaswamy, Yu, Drangsholt, Ng, Culligan and Schlegel48;Reference Siow, Walker, Britt, Kozy, Zanzucchi and Girard50), whereas four studies in each case applied WhatsApp® (Reference Lai, Yan, Yu, Tsui, Chan and Yee29;Reference Negrini, Donzelli, Negrini, Negrini, Romano and Zaina32;Reference Zhang, Cha, Lynch, Cahlon, Gomez and Shaverdian38;Reference Shenoy, Ahmed, Paul, Skaria, Joby and Alias56) and FaceTime® (Reference Negrini, Donzelli, Negrini, Negrini, Romano and Zaina32;Reference Zhang, Cha, Lynch, Cahlon, Gomez and Shaverdian38;Reference Harper, Roof, Wadhawan, Terala, Turchan and Bagnato39;Reference Layfield, Triantafillou, Prasad, Deng, Shanti and Newman42). WebEx® was the technology applied in three studies (Reference Compton, Soper, Reilly, Gettle, List and Bailey24;Reference Jones, Goley, Alexander, Keller, Caldwell and Buse28;Reference Zhu, Williamson, Lin, Bush, Hakim and Upadhyaya53).

Two studies each utilized Google® Meet (Reference Negrini, Donzelli, Negrini, Negrini, Romano and Zaina32;Reference Yoon, Tong, Anton, Jasinski, Claus and Soo37), InTouch®,(Reference Tenforde, Iaccarino, Borgstrom, Hefner, Silver and Ahmed51;Reference Peden, Mohan and Pagán55), Doximity Dialer® (Reference Zhang, Cha, Lynch, Cahlon, Gomez and Shaverdian38;Reference Layfield, Triantafillou, Prasad, Deng, Shanti and Newman42), and WeChat® (Reference Li, Chan, Huang and Cheng43;Reference Liu, Gu, Shao, Liang, Yue and Cheng44) for video consultations. Applications used in only one study were BlueJeans® (Reference Layfield, Triantafillou, Prasad, Deng, Shanti and Newman42), Vidyo® (Reference Mann, Chen, Chunara, Testa and Nov45), Skype® (Reference Negrini, Donzelli, Negrini, Negrini, Romano and Zaina32), Facebook® Messenger (Reference Ashry and Alsawy23), and Doctolib® (Reference Pinar, Anract, Perrot, Tabourin, Chartier-Kastler and Parra33).

Web-based (online) video consultation was applied in studies using ExpressCareOnline ECO (Reference Punia, Nasr, Zagorski, Lawrence, Fesler and Nair47), GCS Sara (Reference Fieux, Duret, Bawazeer, Denoix, Zaouche and Tringali25), Attend anywhere® (Reference Gilbert, Billany, Adam, Martin, Tobin and Bagdai27), and Artzkonsultation (Reference Gerbutavicius, Brandlhuber and Glück20). Audio formats, particularly telephone, were applied to supplement video consultation in four studies (Reference Compton, Soper, Reilly, Gettle, List and Bailey24;Reference Gilbert, Billany, Adam, Martin, Tobin and Bagdai27;Reference Jones, Goley, Alexander, Keller, Caldwell and Buse28;Reference Punia, Nasr, Zagorski, Lawrence, Fesler and Nair47). The type of technology used was not stated in five studies (Reference Gilbert, Billany, Adam, Martin, Tobin and Bagdai27;Reference Satin, Shenoy, Sheha, Basques, Schroeder and Vaccaro34;Reference Serper, Nunes, Ahmad, Roberts, Metz and Mehta35;Reference Haxhihamza, Arsova, Bajraktarov, Kalpak, Stefanovski and Novotni40;Reference Tenforde, Borgstrom, Polich, Steere, Davis and Cotton52).

Technologies rated as compliant for privacy protection based on the U.S. Health Insurance Portability and Accountability Act (HIPAA) requirements included Epic® MyChart, WebEx®, InTouch®, and Vidyo®. The use of other platforms in the United States were permitted under privacy waivers during the pandemic (Reference Perrin, Pierce and Elliott13).

Clinical Specialties Covered

Sixteen clinical specialties covered by included studies are reported in Table 1. Telemedicine use across multiple specialties was reported by three studies in which health system–wide evaluations were conducted (Reference Negrini, Donzelli, Negrini, Negrini, Romano and Zaina32;Reference Mann, Chen, Chunara, Testa and Nov45;Reference Peden, Mohan and Pagán55).

Assessment

Twenty-six of the thirty-one studies reporting user satisfaction utilized self-developed survey instruments based on summative or numeric rating scores designed by investigators. The other five studies conducted assessments using different prevalidated survey instruments, including the TUQ—Telemedicine Usability Questionnaire (Reference Layfield, Triantafillou, Prasad, Deng, Shanti and Newman42) and the TSQ—Telemedicine Satisfaction Questionnaire (Reference Pinar, Anract, Perrot, Tabourin, Chartier-Kastler and Parra33), whereas one study adapted individual questions from both the TUQ and the TSQ customized for study context (Reference Zhu, Williamson, Lin, Bush, Hakim and Upadhyaya53). The patient satisfaction questionnaire-18 (PSQ-18) was used to conduct a retrospective matched patient analysis (Reference Haxhihamza, Arsova, Bajraktarov, Kalpak, Stefanovski and Novotni40) and paired group comparative analysis between a telemedicine cohort and a face-to-face group (Reference Morisada, Hwang, Gill, Wilson, Strong and Steele46).

Satisfaction Outcomes

We highlight two satisfaction outcomes: one from the patient perspective and one from the provider. Satisfaction outcomes are presented in Table 2.

Patient Satisfaction Outcomes

Thirty-one studies (86%) reported patient satisfaction scores. In general, satisfaction with the use of telemedicine shares similar positive trends previously reported in the pre-Covid-19 period (Reference Cremades, Georgina, David, Navinés, Espin and Pardo17;Reference Ignatowicz, Atherton, Bernstein, Bryce, Court and Sturt63). We observed that, with the exception of two studies (Reference Haxhihamza, Arsova, Bajraktarov, Kalpak, Stefanovski and Novotni40;Reference Morisada, Hwang, Gill, Wilson, Strong and Steele46), none of the other thirty-one studies reporting measures of satisfaction share common survey instruments designed for telemedicine assessment. This persistent trend was previously explained by a lack of uniformly accepted standardized instrument (Reference Tenforde, Iaccarino, Borgstrom, Hefner, Silver and Ahmed51;Reference Zhu, Williamson, Lin, Bush, Hakim and Upadhyaya53).

Telemedicine in Covid-19-related care was reported in three studies (Reference Jones, Goley, Alexander, Keller, Caldwell and Buse28;Reference Liu, Gu, Shao, Liang, Yue and Cheng44;Reference Mann, Chen, Chunara, Testa and Nov45).

Provider Satisfaction Outcomes

Eight studies simultaneously report provider experience alongside patient satisfaction (Reference Ashry and Alsawy23;Reference Garcia-Huidobro, Rivera, Valderrama, Bravo and Capurro26;Reference Gilbert, Billany, Adam, Martin, Tobin and Bagdai27;Reference Madden, Emeruwa, Friedman, Aubey, Aziz and Baptiste31;Reference Pinar, Anract, Perrot, Tabourin, Chartier-Kastler and Parra33;Reference Serper, Nunes, Ahmad, Roberts, Metz and Mehta35;Reference Tenforde, Iaccarino, Borgstrom, Hefner, Silver and Ahmed51;Reference Luengo-Alonso, Pérez-Tabernero, Tovar-Bazaga, Arguello-Cuenca and Calvo54). These studies reported consistently high levels of satisfaction and acceptance of telemedicine among providers. Two studies reported satisfaction solely from providers' perspective (Reference Madden, Emeruwa, Friedman, Aubey, Aziz and Baptiste31;Reference Zhang, Cha, Lynch, Cahlon, Gomez and Shaverdian38).

Four studies reported clinical outcome measures: postsurgical visual acuity (Reference Gerbutavicius, Brandlhuber and Glück20), forced expiratory volume (FEV1) (Reference Compton, Soper, Reilly, Gettle, List and Bailey24), glycemic control (Reference Jones, Goley, Alexander, Keller, Caldwell and Buse28), and resilience in neurocognitive function (Reference Lai, Yan, Yu, Tsui, Chan and Yee29). In the latter studies, change in glycemic control was noninferior in the telemedicine group compared with face-to-face care in inpatient diabetes control (Reference Jones, Goley, Alexander, Keller, Caldwell and Buse28), whereas superior resilience to neurocognitive decline was reported in the audiovisual consultation group compared with telephone-only group in patients with Alzheimer's disease (Reference Lai, Yan, Yu, Tsui, Chan and Yee29). Others are reported in Table 2 because they also reported satisfaction (Reference Gerbutavicius, Brandlhuber and Glück20) and utilization (Reference Compton, Soper, Reilly, Gettle, List and Bailey24) as primary outcomes.

Barriers to Use

Barriers to telemedicine utilization were reported in narrative format in eight studies (Reference Compton, Soper, Reilly, Gettle, List and Bailey24;Reference Gilbert, Billany, Adam, Martin, Tobin and Bagdai27;Reference Madden, Emeruwa, Friedman, Aubey, Aziz and Baptiste31;Reference Serper, Nunes, Ahmad, Roberts, Metz and Mehta35;Reference Tenforde, Iaccarino, Borgstrom, Hefner, Silver and Ahmed51;Reference Zhu, Williamson, Lin, Bush, Hakim and Upadhyaya53;Reference Peden, Mohan and Pagán55;Reference Barney, Buckelew, Mesheriakova and Raymond-Flesch57). One study identified barriers from provider perspectives to include the environment, privacy concerns, nonacknowledgement by presenting clinic staff, and user-literacy (Reference Ashry and Alsawy23). Issues with operating equipment, low video quality, connectivity problems, and a lack of required technology were technology-related barriers (Reference Gerbutavicius, Brandlhuber and Glück20;Reference Compton, Soper, Reilly, Gettle, List and Bailey24;Reference Madden, Emeruwa, Friedman, Aubey, Aziz and Baptiste31). Two studies highlighted provider inability to perform physical examinations as barriers (Reference Serper, Nunes, Ahmad, Roberts, Metz and Mehta35;Reference Zhu, Williamson, Lin, Bush, Hakim and Upadhyaya53).

Three studies reported adverse outcomes related to the clinical diagnosis in the specific specialty (Reference Gerbutavicius, Brandlhuber and Glück20;Reference Compton, Soper, Reilly, Gettle, List and Bailey24;Reference Tenforde, Iaccarino, Borgstrom, Hefner, Silver and Ahmed51). Adverse effects in the two studies requiring follow-up admission were clinical in nature, for example, exacerbation of cystic fibrosis symptoms not directly attributable to inability to perform in-person care. One study reported a lone case of vasovagal syncope during audiovisual consultation which resolved at home (Reference Tenforde, Iaccarino, Borgstrom, Hefner, Silver and Ahmed51). In terms of geography, the highest proportion (61%; n = 22) of studies were from the USA, followed by Europe (22%; n = 8), with four studies from Asia and one study each from countries in Africa and South America.

Discussion

Telemedicine as a modality of health delivery has grown at a relatively slow pace in the last decade when compared with the current period. The advent of Covid-19 disease, the rapid sequence of policy, public health and practice changes, and relaxed restrictions served as catalysts for rapid telemedicine uptake.

Findings of this study support the view that in addition to its value in providing “continuity of care” during disruptions, the use of telemedicine has further broadened as a “need of care” delivery modality globally during the Covid-19 outbreak with potential for hybrid use with in-person care in the future.

The healthcare environment during Covid-19 continues to evolve rapidly with major health insurance providers in the USA beginning to roll back no-cost sharing for telehealth services (64), renewing earlier concerns about sustenance of the Covid-19 era reimbursement and relaxed restrictions (Reference Bashshur, Doarn, Frenk, Kvedar and Woolliscroft6;Reference Harper, Roof, Wadhawan, Terala, Turchan and Bagnato39). Averting a return to the pre-Covid-19 era should involve finding the optimal balance between needs for quality and value versus financial sustainability concerns for fraud, waste, and abuse in reimbursement. This would be boosted by integrating experiential evidence in the policy-making and political process.

Large, well-designed studies using uniform, generally accepted standardized assessment instruments to show investigatory evidence may be necessary to avoid relapse to status quo (Reference Bashshur, Doarn, Frenk, Kvedar and Woolliscroft6). Replacing the current diverse and fragmented assessment instruments is of necessity to enhance pooling of representative user samples and data based on uniformly accepted measures from provider and patient perspectives.

Technology Implementation, Clinical Specialties Covered, and Assessment

The vast majority of telemedicine use and outcome measurements during the Covid-19 pandemic are likely yet to be reported. Such unpublished but useful experience may not be captured in a review of peer-reviewed studies such as ours. Our choice of synchronous video telemedicine in our inclusion criteria similar to recent studies in the field is explained by its enhanced capacity for real-time visual cues important for rapport building, clinical observation, visual assessment, and sharing of resources or education materials (Reference Orlando, Beard and Kumar65). Other formats like telephonic, store, and forward, as well as digital remote monitoring, are also technology modes used in telemedicine with seemingly geographic variation to comparative levels of adoption. Although outside the scope of this study, we observed a trend toward video or audiovisual consultation increasingly becoming the dominant mode of telemedicine delivery in many countries primarily due to the above reason and improved communication infrastructure.

Included studies span neurology, surgery, endocrinology, sports medicine, family and emergency medicine, ophthalmology, outpatient, and inpatient care among other specialties. The number of studies showed a slight skew toward postsurgical units and internal medicine.

We found that assessment strategies, methodology, and tools varied widely with no commonly shared telemedicine assessment or survey instrument. In one notable exception, the PSQ-18 was used to conduct a retrospective matched patient analysis (Reference Haxhihamza, Arsova, Bajraktarov, Kalpak, Stefanovski and Novotni40) and paired group comparative analysis between a telemedicine cohort and face-to-face group (Reference Morisada, Hwang, Gill, Wilson, Strong and Steele46). However, like most adapted or general health encounter assessment instruments, the PSQ-18 is not designed specifically for telemedicine. This instrument retains redundant or ambiguous subdomain items (e.g., doctor attitude and location specific items) that potentially lower the instrument validity for assessing telemedicine sessions. For example, the technical aspects (e.g., audio, video quality) of the telemedicine encounter are not taken into consideration.

Most studies were also deficient in describing their foundational framework, even though a common aim was assessing telemedicine use in patient care. There are multiple frameworks previously found to be useful in conducting e-health and telemedicine assessments (12;Reference Ekeland and Grøttland15). The absence of reporting of adherence to health technology assessment (HTA) frameworks to guide preimplementation impact assessment weakens the evidence base underlying the study design (12).

Nonetheless, high rates of telemedicine uptake and satisfaction are reported across included studies. One quarter of included studies (25%; n = 9) are designed as pre–post studies describing how the service was implemented and the number of patients served.

Patient Satisfaction Outcomes

An aggregation of satisfaction scores for a meta-analysis was not feasible in this review due to the variety of assessment instruments and the lack of a uniform satisfaction instrument for telemedicine. Despite four studies reporting clinical outcome measures as secondary outcomes, there was a relative lack of experimental controls.

Provider Satisfaction Outcomes

The studies conducted on satisfaction from provider perspectives showed consistently high levels of satisfaction and acceptance of telemedicine among providers. However, providers also expressed low confidence with technology, a barrier that may be responsible for the low likelihood score for clinician return to video consultation with the potential to limit future use of telemedicine in a post-Covid-19 era, requiring future rigorous study.

Barriers

Barriers were only reported in narrative format in studies, and the foremost identified barrier by frequency was connectivity to the platform. This is at variance with a recent study that identified that broadband connectivity is responsible for less than 5 percent of attributable barriers to telemedicine use in federal qualified health centers (Reference Lin, Dievler, Robbins, Sripipatana, Quinn and Nair62).

Research Context and Quality

Due to the rapidly evolving nature of the pandemic, researcher control over sample selection was beyond the researchers in most included studies. This may also explain the predominance of retrospective observational type and cross-sectional surveys in our sample.

With regard to the overall believability and precision, we rated ten satisfaction outcome studies as high, thirteen studies as moderate, and the rest in the low category. In the utilization outcome study category, we rated six studies as high, three studies in the moderate category, and one study as low.

Overall, most of the studies measuring patient or provider satisfaction outcomes were smaller scale, observational, and cross-sectional studies using diverse assessment instruments. Utilization-based studies reported larger sample sizes but shared similar methodological deficiencies. These attributes had previously been identified as placing studies in a low-evidence classification based on the commonly used strength of evidence criteria, reducing the quality and ability to generalize results (Reference Guyatt, Oxman, Kunz, Vist, Falck-Ytter and Schünemann66).

As regards country of authorship, it was anticipated that a high proportion of included studies would be included from Asia, being the region reporting initial Covid-19 outbreak. Instead, the highest proportion (61%; n = 22) of studies reporting on video consultation was in the USA. The other dominant region with included studies was Europe (22%; n = 8), whereas Asia had four studies, and the African and south American regions had one included study each. We hypothesized that the prior existence of telemedicine framework(s) in countries like the United States and Europe may have given a head start to healthcare institutions in adapting to the need for virtual care.

Limitations

Our review was limited by the reliance on English language for article selection. Although a specific effort was devoted to translating articles with titles reflective of the inclusion criteria using Google Translate®, articles in other languages may still have been inadvertently omitted in the selection. In rating the overall quality of the studies, we applied a set of rules and consensus opinions rather than a rigid scoring system. We outlined our choice of criteria under methodology to support replication.

Overall, the strength of our review was also limited by the use of two databases, PubMed and CINAHL, increasing the likelihood of selection bias through exclusion of studies not indexed within the databases. Because different study settings adopted telemedicine delivery from different baseline levels of use prior to the Covid-19 period, reported percentage increases from baseline were ambiguous, and, hence, we categorized these into quartiles for ease of readability and outlined the categories under methodology. Studies that report a net negative utilization effect or poor user experience were also not found in the search, highlighting the possibility that reporting bias may have also limited our findings.

Conclusion

The reported satisfaction rates remain high and consistent in the direction of increased utilization and previously documented research. Measures to increase a priori use of HTA frameworks during the study design will be needed to strengthen evidence and its applicability, study barriers to telemedicine use, especially among providers and vulnerable patient populations, and extract evidence of effectiveness compared with in-person care.

This review also highlighted the continuing absence of a validated and standardized assessment tool to be used to assess the satisfaction of patients and providers with telemedicine visits. A future beneficial initiative that could emerge from our findings is the development of standard hands-on skills training course modules for the health force. This would embed proficiency in health teams and enhance sustenance of use of telemedicine in contrast to random response and implementation found in the course of this emergency.

Although methodological heterogeneity has been attributed to diverse contexts and the innovative nature of telemedicine, it is an enduring challenge in studies involving telemedicine in which evidence from small sample studies of variable methodological quality presents interpretation and generalization challenges (Reference Whitten, Mair, Haycox, May, Williams and Helmrich21;Reference Batbaatar, Dorjdagva, Luvsannyam, Savino and Amenta22).

Future research in developing a standardized, uniformly adopted assessment instrument to serve as a common data feed for telemedicine would increase the potential pooling, statistical power, and interpretation from small diverse studies. Such an instrument should be broad enough to assess diverse dimensions of telemedicine delivery while designed to capture telemedicine effectiveness data for a head-to-head comparison with in-person care. This would result in greater scientific inference, improving the potential to extrapolate results in broader contexts.

Acknowledgments

We would like to thank the staff of the Greenblatt Library at Augusta University for the use of library materials and assistance in obtaining full text articles.

Authors' Contribution

All authors contributed meaningfully to conceptualization, article selection, writing, reviewing, and editing this article.

Funding

This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.

Conflict of Interest

The authors declare no competing, personal financial interests or relationships impacting this study.

References

World Health Organization [Internet]. Archived: WHO Timeline Covid-19. Geneva. C2021 [updated 2020 Apr 27; cited 2020 Dec 30]. Available from: https://www.who.int/news-room/detail/27-04-2020-who-timeline---Covid-19.Google Scholar
Centers for Disease Control and Prevention [Internet]. Interim Guidance for Healthcare Facilities: Preparing for Community Transmission of Covid-19 in the United States. Atlanta. c2021 [updated 2020 Dec 22; cited 2020 Dec 30]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/healthcare-facilities/guidance-hcf.html.Google Scholar
American Telemedicine Association [Internet]. What Is Telemedicine, Exactly? Arlington (VA). c2021 [cited 2020 Dec 30]. Available from: https://www.americantelemed.org/ata-news/what-is-telemedicine-exactly/.Google Scholar
Greenhalgh, T, Wherton, J, Shaw, S, Morrison, C. Video consultations for Covid-19. BMJ. 2020;368:m998.Google ScholarPubMed
Hollander, JE, Carr, BG. Virtually perfect? Telemedicine for Covid-19. N Engl J Med. 2020;382:1679–81.CrossRefGoogle ScholarPubMed
Bashshur, R, Doarn, CR, Frenk, JM, Kvedar, JC, Woolliscroft, JO. Telemedicine and the COVID-19 pandemic, lessons for the future. Telemed J E Health. 2020;26:571–3.CrossRefGoogle ScholarPubMed
Ohannessian, R. Telemedicine: Potential applications in epidemic situations. Eur Res Telemed. 2015;3:95–8.CrossRefGoogle Scholar
Chang, TC, Lee, JD, Wu, SJ. The telemedicine and teleconsultation system application in clinical medicine. In: The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 2; 2004. pp. 3392–3395.CrossRefGoogle Scholar
Bonnefoy, A, Gionet-Landry, D. Humanitarian Telemedicine. Potential Telemedicine Applications to Assist Developing Countries in Primary snd Secondary Care. Vienna (Austria). European Space Policy Institute (ESPI); 2014. Report 48. [Cited 2020 Jan 6]. Available from: https://espi.or.at/publications/espi-public-reports/category/2-public-espi-reports?start=30.Google Scholar
Tebeje, TH, Klein, J. Applications of e-Health to support person-centered health care at the time of COVID-19 pandemic. Telemed J E Health. 2020. doi:10.1089/tmj.2020.0201.Google Scholar
Committee on Pediatric Workforce. The use of telemedicine to address access and physician workforce shortages. Pediatrics. 2015;136:202–9.CrossRefGoogle Scholar
Centers for Medicare & Medicaid Services (CMS) [Internet]. Medicare telemedicine health care provider fact sheet. Baltimore (MD). c2021 [updated 2020 Mar 17; cited 2021 Jan 4]. Available from: https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet.Google Scholar
Perrin, PB, Pierce, BS, Elliott, TR. COVID-19 and telemedicine: A revolution in healthcare delivery is at hand. Health Sci Rep. 2020;3:e166.CrossRefGoogle ScholarPubMed
Vis, C, Bührmann, L, Riper, H, Ossebaard, H. Health technology assessment frameworks for eHealth: A systematic review. Int J Technol Assess Health Care. 2020;36:204–16. doi:10.1017/S026646232000015X.CrossRefGoogle ScholarPubMed
Ekeland, A, Grøttland, A. Assessment of MAST in European patient-centered telemedicine pilots. Int J Technol Assess Health Care. 2015;31:304–11.CrossRefGoogle ScholarPubMed
Barsom, E, Van Heeds, E, Bemelman, W, Schijven, M. Measuring patient satisfaction with video consultation: A systematic review of assessment tools and their measurement properties. Int J Technol Assess Health Care. 2020;36:356–62.CrossRefGoogle Scholar
Cremades, M, Georgina, F, David, P, Navinés, J, Espin, F, Pardo, F, et al. Telemedicine to follow patients in a general surgery department. A randomized controlled trial. Am J Surg. 2020;219:882–7.CrossRefGoogle Scholar
Rabuñal, R, Suarez-Gil, R, Golpe, R, Martínez-García, M, Gómez-Méndez, R, Romay-Lema, E, et al. Usefulness of a telemedicine tool TELEA in the management of the Covid-19 pandemic. Telemed J E Health. 2020;26:1332–5.CrossRefGoogle ScholarPubMed
Khairat, S, Meng, C, Xu, Y, Edson, B, Gianforcaro, R. Interpreting Covid-19 and virtual care trends: Cohort study. JMIR Public Health Surveill. 2020;6:e18811. doi:10.2196/18811. PMID: 32252023; PMCID: PMC7162649.CrossRefGoogle ScholarPubMed
Gerbutavicius, R, Brandlhuber, U, Glück, S. Evaluierung der patientenzufriedenheit mit einer augenärztlichen videosprechstunde während der Covid-19-pandemie [Evaluation of patient satisfaction with an ophthalmology video consultation during the Covid-19 pandemic]. Ophthalmologe. 2020:1–8. doi:10.1007/s00347-020-01143-0 (German).Google Scholar
Whitten, PS, Mair, FS, Haycox, A, May, C, Williams, T, Helmrich, S. Systematic review of cost effectiveness studies of telemedicine interventions. BMJ. 2002;324:7351.CrossRefGoogle ScholarPubMed
Batbaatar, E, Dorjdagva, J, Luvsannyam, A, Savino, M, Amenta, P. Determinants of patient satisfaction: A systematic review. Perspect Public Health. 2017;137:89–101.CrossRefGoogle ScholarPubMed
Ashry, AH, Alsawy, MF. Doctor-patient distancing: An early experience of telemedicine for post-operative neurosurgical care in the time of Covid-19. Egypt J Neurol Psychiatric Neurosurg. 2020;56:80. doi:10.1186/s41983-020-00212-0.Google Scholar
Compton, M, Soper, M, Reilly, B, Gettle, L, List, R, Bailey, M, et al. A feasibility study of urgent implementation of cystic fibrosis multidisciplinary telemedicine clinic in the face of Covid-19 pandemic: Single-center experience. Telemed J E Health. 2020;26:978–84.CrossRefGoogle ScholarPubMed
Fieux, M, Duret, S, Bawazeer, N, Denoix, L, Zaouche, S, Tringali, S. Telemedicine for ENT: Effect on quality of care during Covid-19 pandemic. Eur Ann Otorhinolaryngol Head Neck Dis. 2020;137:257–61.CrossRefGoogle ScholarPubMed
Garcia-Huidobro, D, Rivera, S, Valderrama, CS, Bravo, P, Capurro, D. System-wide accelerated implementation of telemedicine in response to Covid-19: Mixed methods evaluation. J Med Internet Res. 2020;22:e22146. doi:10.2196/22146. PMID: 32903195; PMCID: PMC7541041.CrossRefGoogle ScholarPubMed
Gilbert, AW, Billany, J, Adam, R, Martin, L, Tobin, R, Bagdai, S, et al. Rapid implementation of virtual clinics due to Covid-19: Report and early evaluation of a quality improvement initiative. BMJ Open Qual. 2020;9:e000985.CrossRefGoogle ScholarPubMed
Jones, MS, Goley, AL, Alexander, BE, Keller, SB, Caldwell, MM, Buse, JB. Inpatient transition to virtual care during Covid-19 pandemic. Diabetes Technol Ther. 2020;22:444–8.CrossRefGoogle ScholarPubMed
Lai, FH, Yan, EW, Yu, KK, Tsui, WS, Chan, DT, Yee, BK. The protective impact of telemedicine on persons with dementia and their caregivers during the Covid-19 pandemic. Am J Geriatr Psychiatry. 2020;28:1175–84.CrossRefGoogle ScholarPubMed
Lonergan, PE, Washington, ISL, Branagan, L, Gleason, N, Pruthi, RS, Carroll, PR, et al. Rapid utilization of telehealth in a comprehensive cancer center as a response to Covid-19: Cross-sectional analysis. J Med Internet Res. 2020;22:e19322.CrossRefGoogle Scholar
Madden, N, Emeruwa, UN, Friedman, AM, Aubey, JJ, Aziz, A, Baptiste, CD, et al. Telehealth uptake into prenatal care and provider attitudes during the Covid-19 pandemic in New York City: A quantitative and qualitative analysis. Am J Perinatol. 2020;37:1005–14.Google ScholarPubMed
Negrini, S, Donzelli, S, Negrini, A, Negrini, A, Romano, M, Zaina, F. Feasibility and acceptability of telemedicine to substitute outpatient rehabilitation services in the Covid-19 emergency in Italy: An observational everyday clinical-life study. Arch Phys Med Rehabil. 2020;101:2027–32.CrossRefGoogle ScholarPubMed
Pinar, U, Anract, J, Perrot, O, Tabourin, T, Chartier-Kastler, E, Parra, J, et al. Preliminary assessment of patient and physician satisfaction with the use of teleconsultation in urology during the COVID-19 pandemic. World J Urol. 2020: 1–6. doi:10.1007/s00345-020-03432-4.Google ScholarPubMed
Satin, AM, Shenoy, K, Sheha, ED, Basques, B, Schroeder, GD, Vaccaro, AR, et al. Spine patient satisfaction with telemedicine during the Covid-19 pandemic: A cross-sectional study. Global Spine J. 2020;22:e2192568220965521.Google Scholar
Serper, M, Nunes, F, Ahmad, N, Roberts, D, Metz, DC, Mehta, SJ. Positive early patient and clinician experience with telemedicine in an academic gastroenterology practice during the Covid-19 pandemic. Gastroenterology. 2020;159:1589–91.e4.CrossRefGoogle Scholar
Shafi, K, Lovecchio, F, Forston, K, Wyss, J, Casey, E, Press, J, et al. The efficacy of telehealth for the treatment of spinal disorders: Patient-reported experiences during the Covid-19 pandemic. HSS J. 2020;16:1–7.CrossRefGoogle Scholar
Yoon, EJ, Tong, D, Anton, GM, Jasinski, JM, Claus, CF, Soo, TM, et al. Patient satisfaction with neurosurgery telemedicine visits during the coronavirus disease 2019 pandemic: A prospective cohort study. World Neurosurg. 2021;145:e184–e191.CrossRefGoogle ScholarPubMed
Zhang, H, Cha, EE, Lynch, K, Cahlon, O, Gomez, DR, Shaverdian, N, et al. Radiation oncologist perceptions of telemedicine from consultation to treatment planning: A mixed-methods study. Int J Radiat Oncol Biol Phys. 2020;108:421–9.Google ScholarPubMed
Harper, K, Roof, M, Wadhawan, N, Terala, A, Turchan, M, Bagnato, F, et al. Vanderbilt university medical center ambulatory teleneurology Covid-19 experience. Telemed J E Health. 2020. doi:10.1089/tmj.2020.0382.Google Scholar
Haxhihamza, K, Arsova, S, Bajraktarov, S, Kalpak, G, Stefanovski, B, Novotni, A, et al. Patient satisfaction with use of telemedicine in university clinic of psychiatry: Skopje, North Macedonia during COVID-19 pandemic. Telemed J E Health. 2020. doi:10.1089/tmj.2020.0256.Google ScholarPubMed
Kalra, G, Williams, AM, Commiskey, PW, Bowers, EMR, Schempf, T, Sahel, JA, et al. Incorporating video visits into ophthalmology practice: A retrospective analysis and patient survey to assess initial experiences and patient acceptability at an academic eye center. Ophthalmol Ther. 2020;9:549–62. doi:10.1007/s40123-020-00269-3.CrossRefGoogle ScholarPubMed
Layfield, E, Triantafillou, V, Prasad, A, Deng, J, Shanti, RM, Newman, JG, et al. Telemedicine for head and neck ambulatory visits during Covid-19: Evaluating usability and patient satisfaction. Head Neck. 2020;42:1681–9.CrossRefGoogle ScholarPubMed
Li, HL, Chan, YC, Huang, JX, Cheng, SW. Pilot study using telemedicine video consultation for vascular patients’ care during the Covid-19 period. Ann Vasc Surg. 2020;68:76–82.CrossRefGoogle ScholarPubMed
Liu, L, Gu, J, Shao, F, Liang, X, Yue, L, Cheng, Q, et al. Application and preliminary outcomes of remote diagnosis and treatment during the Covid-19 outbreak: Retrospective cohort study. JMIR mHealth uHealth. 2020;8:e19417.CrossRefGoogle ScholarPubMed
Mann, DM, Chen, J, Chunara, R, Testa, PA, Nov, O. COVID-19 transforms health care through telemedicine: Evidence from the field. J Am Med Inform Assoc. 2020;27:1132–5.CrossRefGoogle ScholarPubMed
Morisada, MV, Hwang, J, Gill, AS, Wilson, MD, Strong, EB, Steele, TO Telemedicine, patient satisfaction, and chronic rhinosinusitis care in the era of Covid-19. Am J Rhinol Allergy. 2020;35:494–9.CrossRefGoogle ScholarPubMed
Punia, V, Nasr, G, Zagorski, V, Lawrence, G, Fesler, J, Nair, D, et al. Evidence of a rapid shift in outpatient practice during the Covid-19 pandemic using telemedicine. Telemed J E Health. 2020;26:1301–3.CrossRefGoogle ScholarPubMed
Ramaswamy, A, Yu, M, Drangsholt, S, Ng, E, Culligan, PJ, Schlegel, PN, et al. Patient satisfaction with telemedicine during the Covid-19 pandemic: Retrospective cohort study. J Med Internet Res. 2020;22:e20786.CrossRefGoogle ScholarPubMed
Byrne, E, Watkinson, S. Patient and clinician satisfaction with video consultations during the Covid-19 pandemic: An opportunity for a new way of working. J Orthod. 2020:1465312520973677. doi:10.1177/1465312520973677.Google ScholarPubMed
Siow, MY, Walker, JT, Britt, E, Kozy, JP, Zanzucchi, A, Girard, PJ, et al. What was the change in telehealth usage and proportion of no-show visits for an orthopaedic trauma clinic during the Covid-19 pandemic? Clin Orthop Relat Res. 2020;478:2257–63.CrossRefGoogle ScholarPubMed
Tenforde, AS, Iaccarino, MA, Borgstrom, H, Hefner, JE, Silver, J, Ahmed, M, et al. Telemedicine during Covid-19 for outpatient sports and musculoskeletal medicine physicians. PM R. 2020;12:926–32.CrossRefGoogle ScholarPubMed
Tenforde, AS, Borgstrom, H, Polich, G, Steere, H, Davis, IS, Cotton, K, et al. Outpatient physical, occupational, and speech therapy synchronous telemedicine: A survey study of patient satisfaction with virtual visits during the Covid-19 pandemic. Am J Phys Med Rehabil. 2020;99:977–81.CrossRefGoogle ScholarPubMed
Zhu, C, Williamson, J, Lin, A, Bush, K, Hakim, A, Upadhyaya, K, et al. Implications for telemedicine for surgery patients after Covid-19: Survey of patient and provider experiences. Am Surg. 2020;86:907–15.CrossRefGoogle ScholarPubMed
Luengo-Alonso, G, Pérez-Tabernero, FG, Tovar-Bazaga, M, Arguello-Cuenca, JM, Calvo, E. Critical adjustments in a department of orthopaedics through the Covid-19 pandemic. Int Orthop. 2020;44:1557–64.CrossRefGoogle Scholar
Peden, CJ, Mohan, S, Pagán, V. Telemedicine and Covid-19: An observational study of rapid scale up in a US academic medical system. J Gen Intern Med. 2020;35:2823–5.Google Scholar
Shenoy, P, Ahmed, S, Paul, A, Skaria, TG, Joby, J, Alias, B. Switching to teleconsultation for rheumatology in the wake of the Covid-19 pandemic: Feasibility and patient response in India. Clin Rheumatol. 2020;39:2757–62.Google ScholarPubMed
Barney, A, Buckelew, S, Mesheriakova, V, Raymond-Flesch, M. The COVID-19 pandemic and rapid implementation of adolescent and young adult telemedicine: Challenges and opportunities for innovation. J Adolesc Health. 2020;67:164–71.CrossRefGoogle Scholar
Viswanathan, M, Berkman, ND. Development of the RTI item bank on risk of bias and precision of observational studies. J Clin Epidemiol. 2012;65:163–78.CrossRefGoogle ScholarPubMed
Viswanathan, M, Ansari, M, Berkman, N, Chang, S, Hartling, L, Mc Pheeters, L, et al. Assessing the risk of bias of individual studies in systematic reviews of health care interventions. Agency for Healthcare Research and Quality Methods Guide for Comparative Effectiveness Reviews; 2012 [Cited 2020 Mar 6]. Available from: https://www.ncbi.nlm.nih.gov/sites/books/NBK174881/.Google Scholar
Cassar, S, Misso, ML, Hopkins, WG, Shaw, CS, Teede, HJ, Stepto, NK. Insulin resistance in polycystic ovary syndrome: A systematic review and meta-analysis of euglycaemic-hyperinsulinaemic clamp studies. Hum Reprod. 2016;31:2619–31.CrossRefGoogle ScholarPubMed
Hopkins, WG, Marshall, SW, Batterham, AM, Hanin, J. Progressive statistics for studies in sports medicine and exercise science. Med Sci Sports Exerc. 2009;41:3–13.CrossRefGoogle ScholarPubMed
Lin, CC, Dievler, A, Robbins, C, Sripipatana, A, Quinn, M, Nair, S. Telehealth in health centers: Key adoption factors, barriers, and opportunities. Health Aff (Millwood). 2018;37:1967–74.CrossRefGoogle Scholar
Ignatowicz, A, Atherton, H, Bernstein, CJ, Bryce, C, Court, R, Sturt, J, et al. Internet videoconferencing for patient-clinician consultations in long-term conditions: A review of reviews and applications in line with guidelines and recommendations. Digit Health. 2019;5:e2055207619845831.Google Scholar
Healthcare Financial Management Association [Internet]. Telemedicine Blog Westchester (IL): Major insurers roll back no-cost sharing telehealth services. 2020 Oct 07. Mulvany CF, director. Washington, DC: HFMA. c2021 [cited 2021 Jan 4]. Available from: https://www.hfma.org/topics/technology/article/-major-insurers-roll-back-no-cost-sharing-telehealth-services.html?MessageRunDetailID=3486497604.Google Scholar
Orlando, JF, Beard, M, Kumar, S. Systematic review of patient and caregivers’ satisfaction with telehealth videoconferencing as a mode of service delivery in managing patients’ health. PLoS ONE. 2019;14:e0221848.CrossRefGoogle ScholarPubMed
Guyatt, GH, Oxman, AD, Kunz, R, Vist, GE, Falck-Ytter, Y, Schünemann, HJ. GRADE working group. What is “quality of evidence” and why is it important to clinicians? BMJ. 2008;336:995–8.CrossRefGoogle Scholar
Figure 0

Figure 1. Article selection process.

Figure 1

Table 1. Key characteristics of included studies

Figure 2

Table 2. Satisfaction and utilization results