I shudder at the mere thought of writing this post. The objective structured clinical examination or OSCE, is a phrase/abbreviation that will send chills down the spine of even the most hardened medic. Additionally now as GPs, we must complete the clinical skills assessment (CSA), in essence an OSCE before we can now complete our training to become a GP.
With my academic hat, it would be worth displaying some knowledge of the rhyme and reason behind an OSCE. I have to admit though, I will cheat a little by referring to an excellent article : ABC of learning and teaching in medicine: Skill based assessment (Smee, 2003) as it explains in detail the definition and academic values behind the examination. If however you do not wish to read this, the salient points articulate it as a process performing several short cases, normally about 10 minutes each, with the aim of testing the student’s ability to perform a history, skill or explanation in a scenario they are likely to meet in practice. It also outlines the need for reliability and validity from an OSCE, which are the true test of creating successful OSCE. Additionally it highlights the cost implications (examiners and subsequent remuneration, and simulated patients being the larger cost factors) as well as the time effort that is required to successfully run such an examination.
But at this point I would like you to look at this from a student’s perspective. I would ask you to cast your mind back to those thoughts you had in the run up to your OSCE. The anxiety that came with the prospect of an exam that makes or breaks your budding medical career. And I guess that is the key point, this is a nerve-racking experience. I can assure you it is not a pleasant one. When I sat my first OSCE, I distinctly remember during the exam after one station, I had a rest stop. What I most remember is the bending over and retching whilst trying to keep the vomit from spewing forth as my nerves, shot as they were, were culminating after what I thought was a horrific mistake. Thankfully I had a few minutes to compose myself before exposing myself to further sadism in the name of accomplishing my goals. In total I have sat through 6 summative OSCEs and countless formative and informal ones (created by myself and my colleagues). Without doubt the worse was the CSA, as it marked the culmination of my training, and significant cost. ( Just shy of £1500 on my shot).
Thankfully I have passed all my OSCEs. So it has been a humbling experience being on the other side of the process, as an examiner. Over the past year I have had the honour of being an undergraduate OSCE examiner. The training is thorough and instructive. The process, trying at times (especially to show no emotion…..a lot harder than you may think). Additionally this year I have also been a scrutineer (what a great word), where I have formatively assessed the assessors and the stations.
Conducting this process, what has been most interesting is the variety of students. I have witnessed students performing as me and my colleagues did several years ago as:
- ones that have excelled and known it
- ones that have excelled and not known it
- ones that perform adequately
- those that fail
- and those that fail from lack of knowledge or from allowing the process to take over.
The students that generally failed, allowed their nerves to get the better of them. This was most evident in a recent set, where I witnessed some students stutter and shake ( I for some reason have reggae music in my head when I thought of that phrase), while I prayed silently for their voices to stop quivering, ( as the only words an examiner can say in support are ‘please refer to your instruction sheet’,) and to say those few words with composure that meant they would get those vital marks. Unfortunately several didn’t, and I found myself having a possible hand in ending someone’s career. I will point out I do not know the results, but obviously failing one station though not meaning you fail the exam, would not bode well. Though knowing that you have put that fail mark, is still not an easy task, though justified (and I must add, was definitely justified with some cases).
But all this stress and anxiety surely can not be the only way. If we look at the CSA, the last examination to become a GP, I have known people reduced to tears at its mere mention. I have seen depression, elation and abject woe as a result of this test. Every GP trainee fears it. Recently there has been controversy over the exam itself, with a change in marking methods, and issues surrounding the difference seen by international medical graduates and home grown trainees. As a result some would argue that it is the test itself that needs to change. This is additionally highlighting the limitations of an OSCE:
- Stations often need trainees to do isolated aspects of the clinical encounter, which “deconstructs” the doctor-patient encounter
- OSCEs rely on task specific checklists, which tend to emphasise thoroughness. But with increasing experience, thoroughness becomes less relevant
- The limitations on what can be simulated constrain the type of patient problems that can be used
But what would be the alternative?
One could be change within the structure, to simulate true continuity by having a combination of the OSCE and the previous long case format, called observed structured long examination record (OSLER). Though this may help, with improved continuity, and settle some students as they can familiarise with the patient and setting, it still doesn’t take away the ‘fake’ setting and the anxiety that comes with a test.
An alternative would be to use continuous assessment as the driver, as this can take place in the ‘real’ world setting. While this may work, I believe the reality is that as GPs or doctors, we need to learn to cope with pressure, and also although continuous assessment may be useful, this would only work if you had a variety of assessors, as if not, you would be evaluated against one persons judgement of what a doctor should be.
A third option used abroad, is the basis of assessing outcomes and not observation. In this sense, students are given a clinical scenario, tools to enact the scenario, and assessed based upon the outcome. There is however no real observation. These are called safe and effective outcomes (SECO) clinics. While a useful idea and right for formal work, I do personally feel this structure would be better observed particularly if used in a summative environment.
Ultimately, the OSCE solution for assessing students, be they under-grad or post-grad is likely to stay. Nerve-racking and fear inducing as they are, as long as appropriate pre-work is done I do feel they are effective ways of assessment, and at present an appropriate solution in the pot of assessment.
I am sure others will disagree and I openly invite you to share your own thoughts and opinions in that matter. Especially as they say, there is always more than one solution to a problem.
References:
Picture: from pulsetoday.co.uk
http://www.rcgp-curriculum.org.uk/mrcgp/csa.aspx