Sociolinguistic factors affecting performance in the Clinical Skills Assessment of the MRCGP: a mixed-methods approach

Managing role player interruptions

No difference in RP behaviour was discovered between passing and failing case marks. However, RPs interrupted candidates more frequently than in real consultations. Earlier research on real consultations indicates that 33% of interruptions are patient initiated¹¹ compared with 50% in the CSA.¹³ The powerful position of RPs in the interaction could be seen in the alternating responses, as they probe and question explanations and clinical plans to get clarification of the candidate’s management (which is part of their role in the exam setting). They were more than twice as likely as candidates to highlight misunderstandings (61 occasions versus 27).

Interruptions in the explanation phase were a sign of poor performance, and were usually the result of inadequate and sometimes chaotic explanations. The combination of poor explaining technique and interrupted dialogue had an additive effect on the overall impression given to a listening examiner: both medically poor and behaviourally uncomfortable. The ‘explaining phase’ was a particularly difficult section of the case, where medical information needs to be tailored to the patient, with high-scoring candidates talking relatively more, and with fewer interruptions than in the rest of the consultation (72–95% of talk time in this phase).

Managing the simulation and examiner presence

This research identified a considerable amount of interactional ‘work’ to sustain the talk and illusion of a real consultation and to orient the examiner’s gaze; ‘the heightened mimicry’¹⁴ which can be rated as a convincing performance while simultaneously being alert to the examiner’s standards. For example, ‘talking about talk’ was a method of signposting both to the ‘patient’ and the listening examiner, where the consultation was heading and what the ‘patient’ was supposed to make of its meaning; for example, ‘tell me what you know about …, then I can build on your knowledge’. This ‘metacommunication’ involves more ‘work’ by the candidate than simply asking open-ended questions, providing a way for the patient’s ‘ideas, concerns and expectations (ICE)’¹⁵ to be expressed. It was one of many strategies of relationship-building or alignment skills used by successful candidates.¹⁹

The CSA setting

The exam setting affects the amount of talk. Candidates talked more in CSA consultations (68% of the time). Every case was a new patient and there were none of the usual practice structures or computers, than they have in ‘everyday’ GP consultations (50–60% of the time).¹⁶ The quality of talk was also affected, with candidates needing to make rapid inferences, deal with more questions, and metacommunicate more. CL analysis identified a ‘linguistic fingerprint’ to the CSA; candidates’ use of certain words and phrases indicative of the CSA when contrasted with real consultations. Comparing the dataset with larger reference corpora of general spoken English (Corpus of spoken GP interactions,J Skelton personal communication, 2016 ). showed that some phrases occurred more commonly in CSA consultations than in ‘everyday’ GP consultations.¹³ When case-specific words are filtered out, the remainder are oriented to relationship work in the consultation (for example, ‘OK right OK’/‘bit more about’/‘how can I help’/‘been going on.’¹³

As with the structuring of cases, there was similarity across all candidates in the amount they talked and their overall use of CSA phrases. However, successful candidates showed greater flexibility in the amount they talked in complex cases, which ranged from 45 to 76%. But most of the subtle differences between high- and low-performing candidates were in relationship work and in giving extended explanations.

Alignment and misalignment

CL analysis demonstrated that the most frequent CSA phrases were oriented to relationship work in the consultation, the business of creating and maintaining alignment: how to keep the interaction flowing smoothly, and showing some level of explicit affiliation with the RP patient.

Microanalysis of transcripts showed that virtually all candidates experienced moments of misunderstanding and/or misalignment, some measure of behavioural discomfort or confusion. However, overt misunderstandings were rare in the data collected, and there was no evidence that pronunciation caused misunderstandings. Table 2 shows two clear trends: typical CSA phrases were used more with successful candidates than unsuccessful ones and there were more misunderstandings and misalignments among unsuccessful candidates. These were often the result of missing cues which, as well as leading to misalignment, could result in missing the nub of the case.

With about one-third of candidates being IMG, it was pertinent to look for overt cultural misunderstandings that might affect performance in this group. Only one example of possible cultural misalignment was identified in the 40 cases and it involved an IMG candidate explaining genetic inheritance of a condition to a white RP who asked about first-cousin marriage. This example did not cause any overt misunderstanding.

Formulaic talk

Examiners commented that some candidates sounded: ‘formulaic’ or ‘clunky’, overusing the common ‘CSA phrases’ identified above. The fact that these CSA phrases were generally seen as formulaic, yet were used more by successful candidates (Table 2), raises the question of how they were used. Detailed microanalysis showed that similar phrases were located and delivered differently by high- and low-rated candidates. These phrases appeared formulaic when they appeared as isolated fragments, often with a rapid topic change to some aspect of clinical management. Successful candidates were able to tailor these phrases to the individual conversation and site them correctly in the talk.

Common CSA phrases were also seen as formulaic in the way they were delivered. Differences in how these phrases sounded depended on intonation and other features of speech delivery: pitch, rhythm, and stress, known collectively as ‘prosody’, which can affect shades of meaning in speech. Local English speakers’ use of prosody can differ systematically from that of English speakers from other countries.^8,9 Local English speakers routinely use what is called ‘an affective contour’ which sounds more natural to a British ear (which is instinctively more familiar with local intonation and voice modulation), while other speakers can sound non-fluent and incomplete, adding to the impression of formulaic consulting, especially if inserted unexpectedly into the conversation.

Extended explanations

Successful candidates talked more in the explanation phase (72–95% of the time), and there were fewer misunderstandings and misalignments in this phase. Eighty-one explanation phases were identified from the 40 cases for detailed analysis (Table 3).

Good explanations were logical, linked mini-explanations to causes, and used language smoothly and grammatically to take the patient on an explanatory narrative (Box 1). They also used understandable metaphors from patients’ everyday experiences to illustrate their points; for example, linking vascular problems with plumbing examples.