Background
Prior to the introduction of Maxexam the Medical Council of New Zealand (MCNZ) were using paper- based clinical exams. An external review of the examination in 2017 recommended that the Council move to ‘on-screen’ data capture to reduce the risk of human error in entering scores on paper, as well as the risk of scores being calculated incorrectly. In addition, the reduction in time that would be required for staff both to enter results and analyse them was perceived as a benefit for digital exams.
From a shortlist of 3 potential suppliers, Maxexam was chosen for the following reasons:
Reliability – Maxexam had a proven track record, having been used to run OSCE style examinations in the United Kingdom by organisations including the General Dental Council, King’s College London, and the University of Nottingham.
Psychometric analysis tools – Maxexam offered Borderline regression, Borderline Groups and Angoff analysis which were already used at MCNZ to determine cut scores and pass rates.
Resilience – Maxexam supports moving between online and offline operation seamlessly if there is a problem with the Wi-Fi in a venue.
Flexibility – Maxexam supports more than one examiner having access to a particular station.
Pricing structure – Maxexam’s per-candidate pricing was cost effective for the Council.
Prior to the introduction of the exam, MCNZ ran a couple of practice exams – one in the office and one at the venue using medical students as candidates, and on the back of the success of those they implemented Maxexam for the next set of clinical exams.
Results
Despite MCNZ and Maxexam being based on opposite sides of the World, the system was introduced efficiently and effectively. MCNZ were understandably nervous the first time they ran the exam, and Maxinity staff were available as remote back-up, but all ran smoothly.
As had been anticipated due to the building in which the exam was housed, the Wi-Fi went down during the day, but Maxexam handled these drops in connectivity brilliantly and most examiners did not even realise they had been working in offline mode until the administrators dropped into their station to reconnect their tablet to the Wi-Fi.
Reduced exam administration
When MCNZ ran the examination on paper, even with a significant number of staff attending the exam, MCNZ would be working hard throughout the day marking stations and double-entering scores into laptops.
With the introduction of Maxexam, they were able to take a reduced number of staff to the exam and the staff who did go were able to focus on ensuring the exam ran smoothly rather than on data-entry.
Increased control
The introduction of Maxexam has meant that MCNZ’s examination directors are able to monitor the progress of the exam and see how examiners are marking in almost real-time. This means that if there were to be any issues, these could be picked up and dealt with quickly. MCNZ were able to export and e mail the scores for every candidate to the examination directors within 30 minutes of the exam concluding, making them very happy.
Reliable exams
As well as being more streamlined, electronic marking with Maxexam now means the examination is more reliable. Risks such as errors in calculating scores, or examiners not recording a mark are almost entirely mitigated. Maxexam now handles the calculation of scores, and the rules built into the stations in the software mean it is impossible for examiners to submit an assessment without answering all required questions.
In the words of our client:
"The project to implement Maxexam at the Council has been an overwhelming success, in large part because of the dedication of Maxinity’s staff. We are not a large client, however Maxinity provided significant support and advice during the implementation, answering all of our questions, coming up with solutions to problems that are specific to our examination, and then being on stand-by overnight while we ran our first examination with real candidates. As a result of introducing Maxexam, the administration of our clinical exams has become more streamlined, the exam itself is more reliable and marking and analysis is both quicker and easier. We would absolutely recommend Maxinity and Maxexam to other organisations looking at a digital clinical exam system."
