Distributed exams, often also described as remote exams, are where the same exam is sat in a variety of different locations. For example, a candidate may sit an exam in their own home. This is particularly relevant at present due to Covid-19 and the social distancing measures prevalent across most of the world. Currently exams sat by large numbers of students in one location are just not feasible due to restrictions on travelling and gatherings.
For this blog we are looking specifically at computer-based distributed exams, which we offer using our Maxexam software.
Our experience is that when you conduct an internet search on distributed or remote exams, you will only see the positives of running them, but we wanted to take a more balanced view looking at both the pros and cons of this method of assessment.
The Pros
- Location: The most obvious pro, and the reason we are having so many conversations about distributed exams, is that they can be run anywhere as long as the candidate has a computer and an internet connection to download the exam.
- Timing: The exams can be set to run at any time, or even over a period of time, so that candidates can sit them when it is convenient for them. This means that if some of your candidates are ill, it is easy for you to arrange for them to be sat at a different time. The exam will not be visible until the candidate presses the start button, preventing them from looking at it in advance.
- Device familiarity: The candidate will run the exam on their own device which they will already be familiar with, making the process of sitting the exam potentially less stressful for them.
- Reduced costs for examining organisations: Running exams on a distributed basis means that the organisation administering the exam can significantly reduce their costs. Location booking, invigilation and administrating staff costs are all reduced.
- Reduced costs for students: Student costs are also reduced, due to the removal of the need to travel to the examining centre.
The Cons
- Parity of delivery: If distributed exams are sat online, then differences in students’ internet speed and any service interruptions can mean that some students are disadvantaged. However, if the Maxexam Exam App is used, internet connection is not required whilst the candidate is sitting the exam. It is required initially to download the app itself (which can be done in advance) and then the exam paper is downloaded onto the app at the point of logging in, meaning that once candidates press ‘start’ the paper is already downloaded and delivery of the exam will be equal for everyone.
- Exams need to be designed to be run without an invigilator: Letting candidates sit an exam in their own home does mean that the possibility for students to be able to look up answers (perhaps on another device) or even collude with each other is increased. However, there are many things that can be done to help reduce this, such as ensuring that questions are randomised and the time to sit the paper is limited (so making it difficult to look up answers and still finish the paper).
Running a distributed exam has many similarities to running an open book exam and when thinking about whether an exam can be run on a distributed basis, one option is to consider if it could be run as an open book exam. There has been a lot of research recently into the effectiveness of open book exams, suggesting that students do not tend to get higher marks in open book exams on average, and also that the students that do well are the same whether exams are open or closed book. The preparation for both types of exam was also found to be almost identical*.
Given proper consideration it would seem therefore that concerns about students being able to look up answers may be largely unfounded if the objective of the exam is to test knowledge and identify the best students.
As for the small percentage of students who may be tempted to collude during an exam, Maxexam has sophisticated cheat analysis built in including pairing candidates with the most suspicious behaviour according to the Harpp-Hogan index as well as timestamp and keystroke analysis. It may not stop candidates colluding, but it can help to identify those who may have done so, so that you can investigate further. If you’d like to understand more about this, please see this blog.
- Not immediately suitable for all exam types: Whilst distributed exams are easily used for written exams or MCQ’s, it is trickier and involves significant planning to use them for clinical exams such as OSCE’s where there is a requirement to interact with ‘patients’. We have been working with clients on the best ways to run clinical exams on a distributed basis, and while it is not easy it is an option that we believe is feasible with enough planning.
- Unequal student circumstances or technical knowledge may disadvantage some: Candidate circumstances are not all equal – most will have a quiet place to work, but some may not. Some students will be better with technology than others.The disparity in technical knowledge is unlikely to be a major barrier if the exam is sat on an intuitive accessible software such as Maxexam. The lack of a quiet place to work is harder to mitigate, however we would suggest asking the question in advance to discover if this is likely to be an issue and working together with candidates to try and find a solution.
In conclusion
The current lockdown due to Covid-19 has ignited the debate about whether running distributed exams is a feasible and reliable option for exams within the healthcare sector. We welcome the debate this has provoked, even if it has been caused by such unfortunate circumstances.
We believe that running exams on a distributed basis has many advantages, particularly at the moment when it is not feasible to gather candidates together in one location. However, there are some things to think about before jumping headlong into this way of running exams and these are outlined above. With careful planning, distributed exams can be run successfully for many (or we’d go as far as to say most) exams, but there are some exams such as clinical exams where they are certainly more difficult. Clients running high stakes qualifying exams may also need to think carefully before going down this route.
At Maxinity we have always taken the view that we will only encourage organisations to use Maxexam if it is right for them, and we have the same approach to distributed exams. If you would like to chat through with us whether Maxexam’s distributed exam offering might be able to work for your organisation, please do not hesitate to give us a call on +44 (0)117 428 0550.
*https://www.frontiersin.org/articles/10.3389/fpsyg.2019.00463/full, https://www.rug.nl/research/portal/files/14507189/02c2.pdf
