For years, much of the discussion surrounding technology and assessment has focused on how to prevent students using it to cheat. The advent of AI has made that question more acute, but it also exposes a deeper problem: If a student can ask AI a conventional exam question and receive a plausible, well-written answer within seconds, perhaps it’s also time to look at the question itself.
This is particularly important in medicine. After all, we don’t just need students who can recall facts or produce a convincing explanation. We need people who can take information about a particular patient, interpret it, decide what matters, weigh different possibilities, prioritise what to do next and recognise when an answer doesn’t make sense.
So maybe we shouldn’t be aiming to make exams AI-proof. That is likely to become increasingly difficult and, ultimately, may turn out to be a futile exercise. A more useful aim may be to make them AI-resilient: assessments in which simply obtaining an answer from AI is not the same as demonstrating competence. That starts with the question itself. Put knowledge into a clinical context. Compare a question asking “” with a question that gives us a particular patient, their history, observations, medications and laboratory results, and asks what is the most likely diagnosis and what should be done next. Both require knowledge. But the second requires the student to use that knowledge in a particular situation. The answer cannot be separated so easily from the patient in front of them. That makes an important distinction: we want to know not just whether students have learned the information, but whether they can use it. Ask for the decision — and the thinking behind it. A single correct answer can tell us something, but it doesn’t necessarily tell us how the student arrived there. Asking “your decision?” begins to make the reasoning visible. It also allows us to see whether the student has noticed the clinically important information, as opposed to simply arriving at the right option. This doesn’t mean every exam question needs a long written explanation. Sometimes a short justification, prioritisation or identification of the key finding may tell us more than another recall question.
Let the case develop
There is another opportunity in digital assessment: we don’t necessarily have to give students the whole case at once. We can reveal information progressively. A student makes an initial decision; new observations or investigation results appear; they then have to decide whether to continue, change direction or reconsider their original interpretation. That is closer to the way clinical reasoning actually happens; it also changes the nature of the task. The student is responding to a developing problem rather than asking AI to generate an answer to a static question.
Ask students to interpret and prioritise
There are many ways to do this. Give students an ECG, a set of laboratory results, an image, a prescribing chart or part of a clinical record and ask them what they make of it. Ask which finding is most important. Which action should come first? Which of several plausible explanations is most likely? What additional information would they want before making a decision? These are not necessarily harder questions. They are simply different questions, ones that require students to apply knowledge, discriminate between possibilities and make judgements.
Make the problem less generic
Digital exams can also allow us to vary cases between students and, where appropriate, randomise clinical parameters. Follow-up questions can depend on what the student has previously decided. Again, the point is not simply to make cheating more difficult. Underlying everything is to make the assessment more closely reflect what is most important: the student’s ability to reason through a particular clinical problem.
Make the thinking visible – I think this is the heart of the matter.
If we only assess the final answer, we create an obvious opportunity for AI to do much of the work. If we ask students to interpret the evidence, explain a decision, identify uncertainty, prioritise actions or reconsider a previous judgement, we get closer to assessing their own thinking. There is an important caution here. We shouldn’t respond to AI by making exams artificially complicated, obscure or excessively time-pressured. Nor should we assume that an increasingly sophisticated AI detector will solve the problem.
The better response may be to step back and ask a more fundamental question:
What do we actually need a medical student to demonstrate in an exam? If the answer is mainly the ability to retrieve and reproduce information, AI presents a very serious challenge. If the answer includes clinical reasoning, interpretation, judgement and the ability to apply knowledge to a particular patient, then perhaps AI gives us a reason to rethink how we assess those things.
The problem has changed. Maybe our questions need to change too.
Part 2 will consider what happens next: should AI always be something we try to keep out of an exam, or could there be situations where using and critiquing AI becomes part of what we assess?