Introduction
Cancer is a word that can cause many emotions and feelings not only in those diagnosed with it but also in their families. It is expected that one in three people will be diagnosed with cancer during his/her lifetime and one in four of these will die from the disease.1
Radiation therapy is the treatment of choice for approximately four in ten people with cancer – with more than half a million people receiving radiotherapy worldwide each year.2
As decisions about treatment options are increasingly being made with the input from the patients themselves, the information needs of the patients are becoming greater too. Patients find information from a variety of sources – increasingly from the Internet. There are many Websites provided by charities, which offer patients information and forums to ask questions. Previous studies have shown that support groups or support intervention can improve quality of life for people with certain types of cancer.Reference Bui, Last, Bradley, Law, Maier and Smith3
CancerBACUP4 launched its Website in 1997 with over 1,500 pages of cancer related information and by 2002 the Website was being visited by 120,000 people a month. It is now established as a ‘valuable source of information and reference’4 showing that the Internet is being acknowledged as an important source of reference for patients and families. However the provision of information on the Internet via healthcare trusts is not as well developed.
This study evaluates the information currently provided by radiotherapy department Websites, and their readability level. It then provides a benchmark for radiotherapy Websites and makes suggestions for their improvement.
Background
Information and support – why do patients require information?
The development of multidisciplinary teams (MDTs) and the more clearly defined pathways that the patients follow mean that the patients are in greater control of decisions regarding their treatment.5 However, to be involved in making these decisions, the patients need to be properly informed with relevant, researched data.Reference Neufield, Degner and Dick6–Reference Edwards, Unigwe, Elwyn and Hood8
There are several factors influencing the growing awareness towards providing patient information. First, a strong political influence with government interventions such as the Patients Charter,Reference Dodd and Ahmed9 is encouraging the patients to become more actively involved in their treatment, and the NHS Cancer Plan1 has reiterated the need for NHS Trusts to provide ‘high quality’ information for patients. The provision of information can empower the patients and carers, meaning they are able to participate in decision-making and can improve their ability to cope with the illness and the treatment.Reference Bottomley and Jones10–Reference D’Haese, Vinh-Hung, Bijdekerke, Spinnoy, De Beukeleer, Lochie, De Roover and Storme13
Timing of information
The timing of information can influence the amount of information that is retained, which can in turn reduce the feelings of anxiety experienced by the patient.Reference Suominen14 However, each patient is different and has different needs, and the amount of information required and retained varies, demonstrating a need for a continual source of patient informationReference Vignos, Parker and Thompson15 and that the initial verbal information provided to the patient needs support from written information.Reference Dunkelman16–Reference Hinds, Streater and Mood19
How do patients receive information
Information is available in many different forms, and through many different media. It is thought that written information acts to reinforce verbal information.Reference Dunkelman16–Reference Webb18,Reference Cortis and Lacey20,Reference Hope, Williams and Lunn21
Studies have looked at patients’ sources of information and they have shown that patients do not obtain information from one source alone.Reference Luker, Bever, Leinster and Owens22 Those receiving radiotherapy treatment vary in age and this may also affect the way they gather information. Other studies show that sources of information for patients change as they progress through their care pathway. One studyReference Peterson and Fretz23 found that at diagnosis, hospital consultants, breast care nurses and written information (leaflets) were considered to be of greatest use. At the follow up-stage however, women’s magazines and television and radio were seen as more useful.Reference Peterson and Fretz23
World Wide Web
With the continuing expansion of the World Wide Web and Internet access, this media has become an increasingly popular source for health information.Reference Brown24 Current data regarding the use of the Internet to access health information is unclear and much of it is carried out in the US. American surveys show a variation in the number, and type of users that access the Internet and search for health information, from 11 to 25%,Reference Brown24 over 36%Reference Pastore25 and up to 75%.Reference Ademiluyi, Rees and Sheard26 While this shows variation in results, it does emphasize that the Internet is being used as a medium to obtain health information.
As a relatively new and major source of information, the Internet is replacing many other forms of media, such as television and books as the primary source for many patients and carers.Reference Oermann, Gerich, Ostosh and Zaleski27,Reference Jadad and Gagliardi28 The volume of information available is vast, available to everybody and can be sought at the necessary and appropriate time for the patients and their carers.
To be able to access information that patients feel is specifically relevant to their situation, at a time and stage of treatment that they feel necessary, can be beneficial as other forms of media may provide too much or too little information at times when it is difficult for patients or carers to comprehend it all.
Analysing tools
With all the information currently provided by the World Wide Web, being able to find a credible site is increasingly difficult. With no legal requirements for healthcare Websites to be authenticated, the public may unknowingly access incorrect information.
There are many methods for evaluating Website quality which range from very general to a detailed analysis. However there are few health specific analysis tools and those that do exist are not necessarily robust enough to meet the publics’ needs.29,Reference Davis30
The Health Information Technology Institute (HITI),Reference Davis30 developed criteria for evaluating health Websites in order to improve the quality of Internet health resources. These criteria include authority, content, links, design, interactivity and technical assessment. HITI assessed these criteria as being ‘desirable’, ‘important’ and ‘essential’ to a Website. The aim of HITI was to educate the user and the provider of Websites, via the analysis of the criteria mentioned.
Readability
The expectations of providing patients with written information are that the patients can read the information,Reference Arthur31 hopefully improving their experiences through informed decision-making and a greater understanding. However, written patient information is useful to the patient only if it is comprehensible.
In the UK there are few studies looking at the usability, readability and comprehensibility of this information to the patient.32 In 1993 the Audit CommissionReference Flesch33 investigated written communication and found written patient information to be of poor quality, stating patients ‘had difficulty with the content amount and quality of information’.32
The majority of research and studies have been undertaken in the US, using the common readability test – Flesch–Kincaid.Reference Kozol34 Using this readability test it was found that approximately 40% of American adults read at a level below eighth gradeReference Ley and Florio35 – age 12 years. However a British study suggests that it is unlikely that low reading ability is confined to the US and it is possible to assume that these findings would apply to other English-speaking countries.Reference Caspi, Block, Block, Klopp, Lynam and Stouthamer-Loeber36
Although other studies concluded that only 54% of the UK population could understand material written for a reading level of age of 15, they highlighted the fact that tabloid newspapers, accounting for 70% of newspapers sold in the UK, are written for a reading level of age of 12 with a Reading Ease score of 70 or greater – the ‘Tabloid Line’.Reference Schinka and Borum37,Reference Oermann and Wilson38 Although there is no research-based evidence to indicate the mean reading age of the UK population, it is generally accepted that to ensure comprehension, written information should be aimed at the reading level of an 8–9 year old.Reference Albert and Chadwick39 Further studies indicate that patient information may be aimed at people with a higher reading ability than is present in the general population.Reference Petterson40–Reference Croft and Peterson42
Methodology
Design
The design for this study was one of a quantitative approach as a structured data collection method in the form of an evaluation checklist was used. A measuring scale, which collected ordinal data and generated numerical data was utilised. An evaluation checklist provided a cheap and reasonably quick method of collecting the information from the whole target population. However, it is acknowledged that some author bias may have occurred.
Analysing tool
Likert scales were selected as an evaluation method due to ease of categorisation and analysis. For the purpose of this study, a ‘forced choice’ response scale was used, with the scale 0–3. This gave an even number of responses with no middle, neutral option. The reliability of Likert scales was considered to be good as participants were allowed a greater range of responses compared to other measurement scales. With respect to this study, the reliability or consistency of the individual completing the evaluation form was called into question. Piloting of the evaluation form attempted to address this.
Using five of the criteria suggested by HITIReference Davis30 and then tailoring the questions to meet the needs of radiotherapy patients, the questions in the checklist were formulated. The questions focused on what HITI saw as the ‘essential’ elements of a Website. For the purpose of this study, the five areas – content, authority, navigation, design and technical assessment – were described as criteria and the questions within each of these criteria were described as statements. Each statement received a score from 0–3 (0 = not covered in Website at all, 1 = area/criteria is mentioned but not described in Website, 2 = area/criteria is mentioned and discussed in Website, 3 = area/criteria is covered fully in Website to a high standard). The results of each criteria and Website, where appropriate, were compared.
The pilot study suggested that the statements maybe subjective depending upon the interpretation of the user. Therefore an attempt was made to make the statements more objective. Under the statements ‘site has been recently updated’ and ‘connection is quick’ time periods were given to increase reliability. All Website evaluations were carried out with Broadband connection to eliminate any variation here in connection speed.
Flesch–Kincaid readability tests
The Flesch–Kincaid spelling and grammar check of the document was chosen, as it is a recognised method of evaluation used in many studies.Reference Jadad and Gagliardi28,Reference Albert and Chadwick39,Reference McCall and Crabbs43 The readability formula of Flesch– Kincaid has been validated against a validation criterion of the McCall–Crabbs passages.Reference Sullivan44 The intra-text variability of the Flesch–Kincaid test was measured by taking several samples of text of 100 words, and then analysing them using Microsoft Word 2000. However, the reliability of this test did depend on the homogeneity of the text – meaning different samples of the same text may be written at different levels of difficulty.Reference Caspi, Block, Block, Klopp, Lynam and Stouthamer-Loeber36 To produce a more reliable predictor of readability the average results of more than one formula were taken – Grade level and Reading Ease formulas. Using one formula alone produced a reliability estimate of 0.76, while using a combination of formulas increased this reliability to 0.93.Reference Caspi, Block, Block, Klopp, Lynam and Stouthamer-Loeber36
Sample
Obtaining a list of all radiotherapy departments in the UK and Ireland from the Society of Radiographers, identified the target population (64 departments). Using three search engines Google, MSN and Yahoo! the keyword ‘radiotherapy’ was entered followed by the name of each hospital that had a radiotherapy department. From this search the hospital departments that provided a departmental Website were recorded (33 Websites). The three search engines selected were used as they were rated as three of the top five search engines by Nielsen//NetRatingsReference Hulse, Hunt and White45 and they also had the greatest coverage of search engines.Reference Hulse, Hunt and White45 The evaluations of the Websites were conducted over a 2 week period between February and March 2005.
Results
During the evaluation period it was not possible to evaluate 6 of the 33 Websites identified. Three Websites were under construction and three Websites could not be connected to.
The data obtained was input into the Microsoft Excel for Windows package. The total number of evaluations carried out can be seen in Table 1.
Table 1 Number of radiotherapy Websites and the number evaluated
Before evaluating the Websites, each was assigned a number, 1–33. Numbers 2, 9, 13, 16, 25 and 31 have not been evaluated for the reasons identified above.
Content
The range of results received for this criterion was 4–34 out of a maximum possible score of 48. The Website scoring 34 was Website 20, while two Websites (1 and 24) received the lowest score of 4. The mean value is 15.
Of the 27 Websites, 26 (96%) received a score of 3 for the statement ‘site uses correct spelling’ with the other remaining Website (Website 1) receiving no score at all. Twenty-three (85%) Websites received no score for the statement ‘other services in the department are explained’. Only one Website (Website 20) received a score of 3 for the statement ‘dictionary/glossary page is provided’.
Authority
The range of scores in this criterion was 3–18 out of a maximum possible score of 21. The Website receiving the highest score was Website 20. Of the 27 Websites, 5 (19%) scored the same lowest score of 3.
Twenty-five (93%) received a score of 0 for each of the statements ‘author is an authority in radiotherapy’, ‘author is identified as an authority in cancer’ and ‘number of hits displayed’. There was one Website that received a score of 3 for each of these mentioned statements, and that was Website 20.
Navigation
The range of scores in this criterion was 11–30 out of a maximum possible score of 30. This was the only criterion where the top score was achieved. The mean score was 26.
Of the 27 Websites, 5 (19%) received a score of 30. The results also show that 16 of the 27 (59%) Websites received a score of >90%, and it should also be noted that 26 of the 27 (96%) Websites achieved a score >70%. All 27 (100%) Websites received a score of 3 for the statement ‘easily readable font’ and 25 of the 27 (93%) Websites received a score of 3 for the statement ‘Web page background is clear and uncluttered’.
Design
The range of scores for the criteria was 4–18 out of a maximum possible score of 24. The Website receiving the highest score was Website 18. The mean score for the criterion was 12.
Twenty-seven (96%) Websites received a score of 3 for the statement ‘connection to the page is quick’. However 26 (96%) of the Websites received no score at all for the statement ‘illustrations are provided’.
Technical assessment
The range of scores for the criterion was 5–18 out of a maximum possible score of 27. The Website receiving the lowest score was Website 7 and the Website with the highest score was Website 20. The mean score for the criterion was 10.
Of the 27 Websites, 25 (93%) Websites received a score of 3 for the statement ‘paragraphs are short’ and 24 (89%) received a score of 3 for the statement ‘information headings are clear’. The results also show that 23 (85%) of the Websites scored a 3 for the statement ‘text is legible’. However all 27 received no score at all for the statement ‘references are complete’ and 24 of the 27 (89%) Websites received no score for the statement ‘technical terms are defined’ and ‘Web page is available in different language’.
Total scores
The total range of scores was 48–115 out of a maximum possible score of 150. The mean value was 69. The Website receiving the lowest score was Website 28, while the Website receiving the highest score was Website 20.
Websites
Website 20 had the top score in 4 of the 5 criteria – content, authority, navigation and technical assessment. Website 28 had the lowest overall score and also had the lowest score in 2 of the 5 criteria – authority and design (Figure 1).
Figure 1. Total score achieved by each Websites (lowest–highest) and the score achieved for each of the five criteria.
Flesch–Kincaid readability results
Flesch–Kincaid grade level
Each Website underwent a readability test using Microsoft Word 2000. Of the 27 Websites evaluated 2 Websites were not included in the analysis as they provided insufficient words for calculation of grade level. The range of scores was 8–12. The number of Websites achieving each grade level can be seen in Table 2. For this readability statistics, the lower the grade level, the easier the text is to read. A grade level of 8 is desirable.Reference Caspi, Block, Block, Klopp, Lynam and Stouthamer-Loeber36,Reference Albert and Chadwick39 This relates to the school year 8, and in UK terms, this means 13 years of age.
The two Websites achieving the desired grade level 8 were Websites 18 and 20, while the Websites achieving grade level 12 included Websites 11, 28 and 5.
Table 2 Flesch–Kincaid Grade Level results achieved by each Website
Flesch–Kincaid reading ease
Of the 27 Websites that underwent the Flesch–Kincaid Reading Ease test, 3 Websites were not included in the analysis as they provided insufficient words for calculation. The range of scores was 6.7–66.6%. The highest score was achieved by Website 18 while the lowest scoring Website was Website 11. Only 6 (25%) of the Websites achieved a reading ease score >50% and none of the Websites achieved the desired score of 70% (Figure 2).
Figure 2. Flesch–Kincaid Reading Ease results for all Websites.
Discussion
Issues and implications for the study
An unavoidable limitation of this study is that the results are already out of date, as the Internet changes so quickly46 and the data was collected during a specific time period. To increase the reliability and validity of the study it would be useful to re-evaluate all Websites, perhaps at a set period of every 6 months, and to allow a greater data collection period.
Although Nielson//Net RatingsReference Hulse, Hunt and White45 identified the search engines used as three of the top five, it is possible that these search engines used may not have been the preferred search methods of patients, and patients may also use different search terms and keywords to access the department Websites.
A pilot study of the evaluation checklist was completed, although only checked against one Website. The evaluation checklist did not provide the user with the opportunity to make any notes or points about each Website as they were being evaluated. This means that valuable information about Websites may have been lost.
The measures used in this study were very subjective and depended on the researchers’ judgement. However using a sole researcher to collect the data should improve the reliability of the study.
Criteria
The area relating to the last date of revision of the Website was rarely filled in (30%). This may suggest two things:
The information has been posted and then rarely updated, thus possibly the information could be out of date.
The departments and Web designers do not see revision date as an important or necessary part of the Web page, and, therefore it has not been included.
Content
The lack of provision of a forum for patients to ask questions by the majority of Websites means that there is a lack of interaction between the Website and user, and therefore, a lack of exchange of information among users.Reference Davis30 However, a contact e-mail address was posted for 70% of the Websites, allowing provision for the exchange of information.Reference Davis30
The majority of the Websites (74%) made no provision of information for children or adolescents. It is possible that these centres did not provide care for this population and therefore did not provide information for them, perhaps feeling that the specialised centre providing the treatment would give this information.
Late side effects were also neglected as a statement and when discussed, Websites often concentrated on a couple of specific treatment sites, such as the common cancers of breast or lung.47 Sources of information change at follow up,Reference Peterson and Fretz23 therefore the area of late side effects becomes very important.
Authority
The study raises issues about the authority of the Website and who is providing the information in the Website. All of the Websites could be identified as being either non-profit organisations, educational or most commonly having a domain name reserved exclusively for NHS organisations.Reference Pasching, Kaye and Hofbauer48 However, the domain name does not always guarantee institutional approval49 and those contributing to the Website are not always part of the organisation which means the user is unsure of the validity of the information they are being provided with.
Navigation
Cancer occurs primarily in the older population and the risk of developing cancer also increases with age.Reference Stoop, Van't Riet and Berg50 The Internet is a medium available to everybody and the usage of it by the older population is not known. Websites should be designed with this in mind and should be usable by the entire population.
The results indicate that the majority of Websites met the navigation criteria, and the design criterion was also one of the higher scoring criterion. Reasons for this may be that many of the Websites were provided/designed by Webmasters or Website designers, who would have much knowledge about the design and layout of a Website, but would only be able to include the content that they were provided with by the department itself. This may be a reason for the low score in the content criterion.
Design
The results show that 82% of the Websites identified could be accessed using the search ‘hospital name’ followed by radiotherapy. It is important that information can be accessed easily as brochures and leaflets can be picked up easily,Reference Wapenaar, Roling and Van den Ban51 but computer based information first has to be searched for. Therefore a Website difficult to find on a search engine is not providing the patient with information, no matter what its quality is. Also the more complicated the presentation of the information, the smaller the chance of the information being effective.Reference Wapenaar, Roling and Van den Ban51
Readability
The majority of Websites failed to achieve the desired readability levels. This may be due to the technical nature of the information presented and the homogeneity of the text analysed.Reference Caspi, Block, Block, Klopp, Lynam and Stouthamer-Loeber36 This may give further support to the need for a dictionary/glossary page, in order to aid the readers – only one Website provided this facility.
The fact that only 52% of radiotherapy departments provide a Website suggests that departments need to be made more aware of the different media that people use to gather information and the growing use, specifically, of the internet. Dissemination of the findings from this study could also be brought to all radiotherapy departments, particularly those already providing Websites, in order to see where improvements could be made.
It is also possible that some of the information included in the evaluation checklist was available on the Internet, but on other pages of the Website, for example the main home page, which can often be missed. This raises the issue of whether the patients should be provided with the Web address as part of their initial information session, allowing the patients to decide what form of media they wish to obtain information from and allowing them to access the Web page properly instead of through the search engine.
Conclusion
Information given to the patient is directly connected with patient care and the Internet provides the patient with information that can be accessed and recalled at times that are most suitable for the patient. How this information is presented can play an important part in how it is understood and retained by patients.
Although the search engines identified only 33 Websites from 64 departments, the results of this study suggest that there is an increasing awareness within the radiotherapy service, of the patients’ need and desire for information and also their use of the Internet. The study highlights that it is not enough to provide a Website that is easy to navigate; the content has to be good in order for it to benefit the patient.
The Websites evaluated were those identified using the methodology described. There may be Websites provided by radiotherapy departments that have not been evaluated in this study, however this does raise the issue of ‘keywords’ and the patients’ ability to access the information provided. Computer based information offers more possibilities but it can be more complicated finding the right information initially. Radiotherapy departments could overcome this by providing their Website address to users, in order for them to access the information directly at a time that they wish.
This study demonstrates the need for radiotherapy departments to pay attention to the content criteria of their Websites. To achieve this it may be necessary to have dedicated staff from within the department, who are trained in Web design to write the Website and facilitate a forum for patients.
It may be possible to take this a step further and involve patients to a greater degree in the Website design. The government wishes to make healthcare more patient-centred and collaborative1,5 and this would provide the patient with the opportunity to give personal experience, influencing the content of the site and also providing opinion on the design and navigation. This may sound like a similar situation to using a Web designer, but this would allow the therapy radiographer to guide the team, rather than be guided, as a Web designer would be.
If patients are not involved in the Website design it may be possible to provide them with a feedback page to enable an exchange of information, allowing the Web team to make constant updates and improvements to the Website.
The NHS Cancer Plan1 states the need for ‘high quality’ information, which is culturally sensitive, specific to local provisions and is related to specific cancers. While it is possible to say that some radiotherapy centres are providing this information, this study reflects the varying quality of the information and the lack of comprehensive information for all cancer types.
