What is Borderline regression?
Borderline regression is an absolute, examinee-centred standard setting method that is widely used to standard set OSCE exams. Candidates are awarded a “global score” for a station in a circuit, based on the examiner’s judgement of their ability. This is usually awarded at the end of the station, after the actual marks for the station have been given. The global score is on a scale, generally around 3-5 options, where the examiner chooses the most appropriate reflection of how the candidate performed. An example of a global score scale may be clear pass, borderline pass, borderline, borderline fail, clear fail.
The definition of a borderline candidate is something that must be agreed upon by all of the examiners, but could be considered to be a just passing candidate. The candidate is awarded actual marks on a checklist of their performance and a global score for the whole station. The actual mark that all candidates scored for a station is plotted on a graph against the global score they were awarded for that station, and a best fit line (line of regression) is drawn. The point at which the line intersects with the “borderline” indicates the “cut-off” mark for the station.
Should you use Borderline regression?
Borderline regression is an extremely effective standard setting method when used in high stakes, practical exams such as OSCE’s. It can be argued that the best use case for borderline regression is in examiner led stations, as the examiner is face to face with the candidate and can make an honest and informed decision on the level of achievement the candidate should be awarded.
Borderline regression is not always appropriate. For example, in a station where the marker doesn’t actually see the candidate, it’s difficult to get an overall sense of the candidate’s performance level.
Advantages of using Borderline regression
Reliable results – This method uses real exam data. In Maxexam, the borderline regression method can be used to calculate the cut-off for a station using just the one exam, or multiple exams if available.
Effectiveness of station – Borderline regression can be an effective way of identifying problems with a station. For example, an unbalanced marking scheme can be indicated by poor discrimination in the borderline regression graph.
Easy for examiners – Awarding a global score for each candidate is not time consuming and the process is easy to understand as long as proper training is given.
Feedback – Borderline regression is a great tool for backing up other standard setting methods with real candidate results. For example, if Angoff was used for the initial standard setting, the borderline regression will indicate if this method was effective.
Disadvantages of using Borderline regression
Examiner training – The borderline method is simple to execute but it’s crucial that all examiners understand the definition of a “borderline” candidate, and are able to make a confident decision about whether a borderline candidate falls in to the pass or fail category.
Box ticking – This standard setting method heavily relies on the examiners using their overall judgement of the candidate’s performance to award a global score rather than counting marks.
Scale – This method requires a large number of candidates and a large number of examiners to be reliable, which relies on a scenario being used in multiple exams or over a large cohort to get the most reliable results. Due to the scale needed to produce reliable results, the station will usually need to be standard set using another method such as Angoff.
Coming soon…
Watch out for the next installment in our Standard setting simplified series… the Angoff method.
