What is ‘competence’ in Radiation Therapy?

There has been a dramatic shift in the manner in which health-care professionals are educated in the past decade. Radiation therapy education in Ireland was previously apprenticeship based. The bulk of student learning was through experience in radiotherapy departments with direct observation of techniques and procedures. Log books were kept, documenting how often each technique was observed and a pre-defined number had to be logged in order to be deemed competent. With the advent of the 4-year degree programme in our institution, teaching and learning strategies changed dramatically. Students are now taught considerable theory in all 4 years, with the bulk of clinical practice undertaken in the final 2 years. It is no longer apprenticeship based and some might therefore argue that this has initiated the ‘theory-practice gap’. In recent years, teaching and learning strategies have shifted from didactic, pedagogical methods to more androgogical approaches such as self-directed sessions with students taking accountability for their own learning.

What has become apparent from all of these changes is the difference in how ‘competence’ is now defined in radiation therapy. Prior to the academic programme, competence was solely related to technical ability as a radiation therapist. Although technical ability will always be an important aspect of the radiation therapist’s profession, other factors must be considered. These include factual knowledge, an ability to integrate this knowledge and apply it to practice, communication and critical thinking skills.

The distinction between ‘competencies’ and ‘competence’ will clarify the purpose of our clinical oral examination assessment strategy. ‘Competencies’ refers to the ability to meet a job’s requirements by producing certain outputs whereas ‘competence’ is seen as being ‘person-centred’; it is the underlying attributes of the student that lead to effective performances.Reference Zhang, Luk, Arthur and Wong¹ Helping students to become competent in such a holistic provision of radiation therapy underpins our teaching philosophy.

Structure of Clinical Oral Examination

Having passed the practical placement component of the clinical course, students then progress to the clinical oral examination and reflective essay component to determine their grade. The structure of the clinical oral examination requires a substantial number of preparation hours on the part of examiners. In our institution, three examination rooms are used to facilitate a 20-minute examination of 25 students. Three examiners sit on each panel, made up of clinical and academic radiation therapists. One potential disadvantage of the clinical interview is its lack of anonymity.Reference Race² The candidates will be known to the academic teaching staff. The clinical examiners are known to the students as ‘liaison radiation therapists’ in various radiotherapy departments, but in most cases they have not had any direct teaching involvement with the students being interviewed. In these cases, the clinical radiation therapist acts as an ‘external examiner’ on the panel as well as an expert on the local practice of their department.

A framework of questions is used to ensure that the learning outcomes of the clinical course are assessed in all examination rooms and maintains process objectivity and standardisation. A framework of questions has also been shown to overcome the risk of clinical interviews being either too structured or too formless.Reference Freeman and Lewis³ A common grading system is used by all examiners for the oral examination.

Supernumerary examiners are used as our internal audit process to ensure that the level of questioning is similar in all three examination rooms and this has previously been shown to refine and standardise questions.Reference Wakeford, Southgate and Wass⁴ All clinical oral examinations are recorded, in accordance with confidentiality guidelines of our institution, for multiple reasons. First, review of the recordings minimises the ‘halo effect’ where examiners focus on the student’s strength or weakness in one area to such a degree that they ignore all other topics.Reference Freeman and Lewis³ Reviewing the recordings allows comparison with other examiners to be drawn and ensures that all examiners adhere to the framework of questions. Second, review of the recorded interviews has the potential for correcting ‘errors of contrast’; where examiners’ judgements of one candidate are influenced by impressions of preceding candidates, despite the use of a criterion-based assessment tool.Reference Wakeford, Southgate and Wass⁴ Third, recorded clinical oral examinations are a permanent record of the assessment process. Therefore, should a query arise about any examination, a transparent account of the process is readily available. Finally, the recordings may also prove useful in subsequent student feedback sessions.

Advantages and Disadvantages of the Clinical Oral Examination

The use of a criterion-referenced assessment tool such as the structured framework of questions used in the clinical oral examination in our department has been considered didactic and pedagogical by some.Reference Chambers⁵ We recognise that this may well be at odds with the current andragogical approach to health sciences education, where students are self-directed and take responsibility for their own learning. However, safety to practice can be difficult to ascertain with andragogical approaches to assessment, where a paucity of prescribed standards or defined competencies may exist.Reference Bradshaw⁶ We believe that second year students are very much at the ‘novice’ level in the context of clinical practice and while andragogical approaches to assessment are used to great effect in other modules, they are inappropriate in assessing clinical competence at this stage.

Basic factual knowledge can easily be tested in the clinical oral examination. Ensuring clarity at this taxonomic level is required at this stage in order to effectively build on such knowledge in the subsequent years. We have found that effective use of open questioning can determine the level of understanding of the student on important concepts. A written paper would not afford the examiner the same opportunity to assess this comprehension. Use of higher order questioning can assess the students' ability to integrate knowledge from across the curriculum and their ability to apply it to clinical radiotherapy practice. Modules taught in the curriculum include radiation physics, cancer biology and epidemiology and counselling and communication skills. The ability to integrate knowledge from all of these modules and apply it to clinical practice is assessed. The application of basic physical concepts to practice is crucial at this stage in order to understand subsequent treatment techniques taught in the final 2 years and ultimately in becoming effective practitioners. Increasing numbers of cancer patients are being referred for radiotherapy, resulting in a pressurised and stressful working environment for future graduates. The ability to communicate clearly, both with patients and within the multi-disciplinary team under such circumstances must be instilled in students from early in their training. We, among others, have found that the clinical oral examination assesses both interpersonal competence and professional communication skills.Reference Bearn, Chadwick, Jack and Sackville⁷

Some disadvantages of the clinical oral examination are difficult to overcome. One example is the ‘leaking’ of the agenda from one student to the next.Reference Race² It is practically impossible to prevent this. However, we do not perceive this as problematic in our department as the students' understanding of the question asked and the ability to integrate theory and practice is very transparent in the oral examination. Similarly, as we provide clear learning outcomes of the clinical course at the start of the academic year, students are aware that what is taught will be assessed.

Our use of clinical oral examinations determines whether students understand the theory of radiotherapy practice. Students are marked on their practical ability by clinical radiation therapists in the radiotherapy departments, prior to the clinical oral examination and reflective essay components. We are conscious that it would be preferable to grade both theory and practice in the clinical setting, but currently this is not possible. Therefore, we have found clinical oral examinations are useful to ensure students have a sound understanding of the theory that underpins practice.

Finally, it has been our experience that the clinical oral examination enhances existing collaboration between academic and clinical radiation therapists. This indicates to students that academic learning is not to be viewed as a separate entity, but can be used on a daily basis as a clinical radiation therapist and thereby aids in bridging the ‘theory–practice gap’.

Summary

Assessment of clinical competence is complex and challenging and the clinical oral examination, as with most assessment tools, has acknowledged strengths and limitations. It has been our experience to date that the clinical oral examination is useful in assessing basic factual knowledge, checking the integration of theory from across the curriculum to practice, monitoring the development of critical thinking skills and assessing communication skills. It is recognised that some students will never perform well in oral examinations; therefore reflective essays are used as a written assessment tool in conjunction with the oral examinations. However, other methods of assessment, such as grading of practical skills would further improve our assessment strategy. Subjectivity in standardisation of questioning has been minimised with the use of supernumerary examiners and a structured framework of questions applicable to all clinical experiences. Finally, we have found that recording oral examinations, in line with confidentiality guidelines, does not add stress to students in the examination, but is viewed positively by students as providing more transparency to the examination process.

It is our intention that an evaluation of this process will occur in the near future to ensure that the clinical oral examination continues to be fit for the purpose of assessing clinical competence.