As many university exams shift towards remote formats, what implications does this have for clinical examinations like OSCEs? Should they be following the trend for remote exams, or are the benefits of running these in person still too compelling? Is there potential for organisations to integrate new technologies into OSCEs, and how could this enhance the experience for both institutions and candidates?
This blog will explore how clinical exams might evolve whilst keeping their integrity.
Delivering healthcare in the modern world
If you’ve recently made an appointment with a GP or sought a referral to another healthcare provider, it’s likely that your healthcare journey may have started (and possibly ended) with an online consultation.
If you attended an in person appointment, technology is increasingly being integrated into healthcare consultations: for instance, if you need to undergo surgery, your consultant may have utilised a 3D scan to enhance the discussion. Likewise in dentistry, 3D printing may have been employed to produce dental devices, including surgical guides, orthodontic aligners, and dental crowns. This raises an important question: as remote consultations and technological tools become essential in healthcare delivery, should OSCEs adapt to reflect this shift?
Implementing OSCEs
OSCEs play a crucial role in healthcare education, allowing organisations to assess candidates’ clinical skills and their response to situations they might face in the real world, when faced with actual people. However, delivering OSCE’s does bring its challenges:
Costs
Traditionally OSCEs have been delivered in person, requiring extensive organisation and significant resources including:
- Venue costs
- Equipment for each station
- Examiners for each station
- Actors for certain stations
- Onsite facilitators
For large cohorts, these resources may need to be secured over several days.
For organisations yet to transition to digital exams there are also additional costs and time involved in adding the candidate responses into their marking systems.
Training and Calibration
To ensure that OSCE exams are fair, examiners need to be carefully trained (adding to costs and the administration burden) to guarantee:
- Consistent scoring across all examiners, preventing disparities where some may mark more harshly than others—an issue commonly referred to as Hawk and Dove marking. Examiners require calibration to avoid disadvantaging candidates.
- Awareness of unconscious biases that might affect their marking of candidates – for example based on their appearance, gender or ethnicity.
What about remote OSCEs?
Some organisations choose to run their OSCEs remotely using a digital exam system. This approach can be a viable, and often much more cost-effective solution, depending on the specific competencies being assessed.
With advancements in technology and the integration of AI into assessments, the proportion of exams that could be delivered remotely or at least differently is likely to increase. You can imagine, for example, medical students interacting with an avatar as part of an OSCE – whether during an onsite or remote examination.
Barriers to delivering OSCEs remotely
1. Remote OSCEs are not always suitable
The suitability of remote OSCEs largely hinges on the nature of the exam.
For instance, when examining final-year medical and dental students, it is vital that candidates assess a patient as they would in real life —which includes performing a physical examination. Omitting this crucial aspect from an OSCE would be detrimental to both aspiring healthcare professionals and their future patients.
However, remote OSCEs may be more suitable for formative assessments or in fields like pharmacology.
2. Resistance from governing bodies
Governing bodies for healthcare degrees, rightly need to ensure that the aspiring doctors, dentists and other healthcare professional are at least as competent as the candidates who passed before them.
While this commitment to quality assurance is important, it can lead to resistance against new testing methods, as these approaches are often unproven. A potential solution could be to incorporate new question types within existing examinations, allowing organisations to develop their assessments over time.
3. Technology and parity of delivery
In order to take advantage of the developments in technology that could make remote OSCEs more realistic – for instance the use of video and interactive elements – candidates must have access to devices with adequate specifications and reliable Wi-Fi connections. Without this, the fairness of the examinations could be compromised, which is something no organisation would want.
Given all the above, how could OSCEs be modernised?
OSCEs play a crucial role in assessing healthcare students. In high-stakes healthcare exams, as previously discussed, it is likely that some form of in-person assessment will remain necessary for the foreseeable future.
This does not mean though that OSCEs need to continue exactly as they are. Rather, by being creative, there are opportunities to streamline processes and reduce the costs associated with these examinations.
We also believe that integrating technology goes beyond cost reduction; it can also significantly enhance the assessment of candidates. Adopting innovative methods could make OSCEs more relevant to real-world scenarios while maintaining the rigorous standards required to accurately identify competent healthcare professionals.
Streamlining and improving OSCEs
1. Reducing costs while maintaining integrity
Candidate led stations
If some questions do not require a human examiner, then why not deliver them without one?
This approach allows for the establishment of one or more stations using computers instead of relying on in-person examiners. Candidates can log in, interact with one or more ideally interactive questions, and log out when finished. The results from these stations are seamlessly integrated into the system and are analysed as part of the comprehensive examination process. Incorporating candidate-led stations can reduce overall costs by requiring fewer examiners and potentially less physical space and equipment.
Interacting with AI avatars instead of actors
As AI technology continues to develop, you can imagine that rather than interacting with real actors on every station, some could be manned by AI avatars trained to respond according to a candidate’s answers to questions. Currently the cost of this is high due to the initial development required, but in the future it is likely to become much more affordable and will reduce the need for actors.
An additional benefit of using Avatars could be guaranteeing consistency in the way they respond if a candidate answers in a particular way.
2. Employing technology to improve OSCEs
Conducting remote appointments as part of the OSCE
Conducting appointments remotely differs significantly from in-person interactions. Given the increasing integration of remote consultations in modern healthcare, it is essential for healthcare professionals to develop this skill. Shouldn’t educational providers be assessing this capability?
Recording live OSCEs
While recording live OSCEs might not save money, having these recordings can ensure fairness for all students—such as checking for hawk and dove examiners. Additionally recordings serve as a permanent record – this can be useful for both candidates and examiners if there is requirement for a review post-exam.
Integrating technology within stations
Technology can be employed to simulate real-life scenarios. For instance, healthcare professionals could explain to patients how a 3D scan operates and what it reveals, illustrating how this information influences their proposed treatment.
Conclusion
In conclusion, embracing technology in OSCE delivery may not only save costs but could also enhance the quality of assessments. While remote OSCEs might only be suitable for some situations, we believe that all organisations should explore innovative ways to incorporate new technologies and approaches to make OSCEs more relevant and effective.