Introduction
In the modern world, the internet is the dominant means of acquiring information. Patients frequently utilise online resources to learn about their disease processes and treatment options. Unfortunately, these resources are not always unbiased or of high quality.Reference Gualtieri1,Reference Lu, Zhang, Wu, Shang and Liu2 YouTube (Youtube.com) is one such resource.Reference Madathil, Rivera-Rodriguez, Greenstein and Gramopadhye3 There is considerable authorship diversity on YouTube. While most content is authored by patients sharing their own experiences, a minority is authored by knowledgeable medical professionals.
This study evaluated the quality of information that patients receive when searching YouTube for common paediatric otolaryngological procedures, and compared the information, based on the search terms used. No recent study has evaluated the quality of these YouTube videos as a source of information on common paediatric otolaryngology procedures.Reference Strychowsky, Nayan, Farrokhyar and MacLean4
Materials and methods
YouTube was searched on 23 May 2019 and a video list based on the following six terms was compiled: ‘adenoid removal’, ‘adenoidectomy’, ‘ear tubes’, ‘tympanostomy’, ‘tonsil removal’ and ‘tonsillectomy’. The terms were searched for using the default YouTube search option (based on ‘relevance’) and the ‘view count’ search filter. The top 10 video results were compiled for each search type and term, resulting in 120 videos. By eliminating non-English results and duplicate videos that appeared in multiple searches, we identified 91 unique videos. These videos were then rated using the Discern criterion for assessing the quality of health information.Reference McCool, Wahl, Schlecht and Apfelbacher5,Reference Charnock6
Four medical student reviewers were trained to rate videos utilising the Discern criterion using a standardised form. The Discern instrument includes 15 distinct questions to evaluate the reliability of information, and the inclusion or exclusion of key treatment choice details, as well as a final question (question 16), which prompts an overall quality rating for the video. The bias Discern score reports the mean results for question 6, which evaluates the impartiality of presented information. The overall Discern score reports the mean results for question 16, which prompts reviewers to use results from questions 1 to 15 to evaluate the overall quality of the video as a source of information on the treatment of interest. A Discern score of 1 indicates a low rating for the related question, while a score of 5 indicates a high rating.
The questions in the Discern criteria are: (1) are the aims clear?; (2) does it achieve its aims?; (3) is it relevant?; (4) is it clear what sources of information were used to compile the publication (other than the author or producer)?; (5) is it clear when the information used or reported in the publication was produced?; (6) is it balanced and unbiased?; (7) does it provide details of additional sources of support and information?; (8) does it refer to areas of uncertainty?; (9) does it describe how each treatment works?; (10) does it describe the benefits of each treatment?; (11) does it describe the risks of each treatment?; (12) does it describe what would happen if no treatment is used?; (13) does it describe how the treatment choices affect overall quality of life?; (14) is it clear that there may be more than one possible treatment choice?; (15) does it provide support for shared decision-making?; and (16) based on the answers to all of the above questions, rate the overall quality of the publication as a source of information about treatment choices.Reference Charnock6
Videos were categorised according to US board certification status and author type (Tables 1 and 2). Average overall and bias Discern scores were compared for videos including US board certified otolaryngologists against videos that did not, using a two-sample t-test.
Table 1. Discern score comparison of all unique videos with commentary from a US board certified physician*
Any video result from multiple search terms was included once.
* Confirmed using certificationmatters.org.
† Indicates a statistically significant difference between video scores (p < 0.0001)
Table 2. Discern score comparison of all unique videos classified by author type
Any video result from multiple search terms was included once. The video author is the person or entity that uploaded the video. MD = Doctor of Medicine
Results
For all videos, the mean bias Discern score was 3.18 and mean overall Discern score was 2.39 (Table 3). When comparing author type, the lowest bias was found in videos authored by otolaryngologists (3.59). The highest overall Discern score resulted from videos authored by companies or advertisers (2.90), followed by videos authored by otolaryngologists (2.76).
Table 3. Compiled data from top 120 YouTube videos by search term and search type
Videos including US board certified physicians had mean overall and bias Discern scores that were significantly higher (p < 0.001) than those which did not (3.00 vs 2.38 and 3.79 vs 1.55, respectively).
Videos resulting from a search of the term ‘adenoidectomy’ had the lowest bias score (3.90), while the term ‘ear tubes’ had the highest overall score (3.00). Video results for ‘ear tubes’ were the newest videos, with an average age of 3.83 years, and results for ‘tympanostomy’ were the oldest, with an average age of 5.94 years. Overall, the videos were viewed 176 769 549 times, with an average of 1 473 079.58 views per video. ‘Tonsil removal’ videos had the most views, with 113 864 388 views, and ‘tympanostomy’ had the least views, with 1 720 141 views.
Discussion
With over 1.9 billion monthly active users, YouTube is the second most visited website behind Google.com.7 The quality of YouTube videos that included US board certified physicians was significantly higher than that of videos without physicians. However, unbiased, high quality information is still lacking.
As patients are turning to internet sources to augment their knowledge, and may rely on this information when making medical decisions, it is important to evaluate and improve the quality of available content.Reference Gualtieri1 Otolaryngologists should be prepared to discuss videos that their patients see, and recommend search terms which result in higher quality videos, such as ‘ear tubes’ over ‘tympanostomy’ and ‘tonsillectomy’ over ‘tonsil removal’. Further evaluation is necessary to develop criteria that otolaryngologists should follow to ensure their videos are high quality and accessible.
• Patients are increasingly utilising video as a source of information on medical procedures
• YouTube videos on adenotonsillectomy, ear tube surgery and paediatric tonsillectomy often comprise low quality information and patient-experience testimonials
• This study categorises YouTube videos on paediatric otolaryngology procedures according to types of authors uploading
• It found that over one-third of the videos were published by US board certified otolaryngologists, and overall video quality was low
• Overall video quality and bias varies significantly depending on the chosen search term
• Otolaryngologists should be prepared to caution patients who may utilise YouTube to gain additional knowledge on procedures
Competing interests
None declared
