Using conversation analytic methods to assess fidelity to a talk-based healthcare intervention for frequently attending patients
Introduction
In randomised controlled trials (RCTs), the importance of assessing ‘implementation fidelity’ (whether an intervention has been implemented as planned) was first noted by Basch et al. (1985). Basch et al. (1985) highlighted the danger of incorrectly accepting a null hypothesis when it is the result of an (unknowingly) inadequate implementation, with the consequence of discarding a potentially effective intervention. The primary goal of assessing implementation fidelity is to increase scientific confidence that the planned intervention has been adequately tested, and that the measured outcomes are a reliable indication of its effectiveness.
The starting point for fidelity assessment is to have a clear description of the intended intervention in order to compare this to what was delivered. However, this is not always straightforward. In some trials the intervention is necessarily under specified. The main consideration from this perspective is that there may be more than one route to achieving the intended outcome. In such cases trialists may deliberately avoid specifying the precise form of an intervention due to an assumption that it may need to be adapted to the diverse contexts in which it is being evaluated e.g., in primary care settings and trials of ‘talk-based interventions’ (i.e. those geared towards stimulating interaction around particular topics or concerns, as opposed to physical treatments). Moreover interventions are often adapted by those who are delivering them, sometimes intentionally to accommodate them to the context of their delivery, but also unintentionally due to inadequate training, ‘drift’ from the original protocol, or ‘decay’ in provider skills over time (Bellg et al., 2004). Many protocols therefore describe interventions in terms of principles and intended functions rather than specifying too closely the form and/or detail of how those principles are to be implemented. Consequently, ‘fidelity of form’ (where the form of intervention to be implemented is precisely specified), has been distinguished from ‘fidelity of function’ (where the form of implementation is less important than it fulfilling the intended function) (Hawe et al., 2004).
There are a limited number of methods in use for assessing implementation fidelity and not all are ideally suited to assessing talk-based interventions. Conversation Analysis (CA) is widely considered to be the dominant contemporary method for the analysis of talk-in-interaction (Heritage, 2009). Indeed Robinson and Heritage (2014) have argued that CA methods may be a viable option for assessing the implementation fidelity of talk-based interventions. They give the example of a primary care trial evaluating the effectiveness of upfront agenda setting for the reduction of unmet patient concerns (Heritage et al., 2007). Doctors were trained in a tightly specified talk-based intervention to ask, following the problem presentation, if patients had “any” versus “some” further problems or concerns. All study consultations were recorded, and monitored to identify instances where doctors failed to perform the intervention correctly. In a retrospective analysis of 144 video-recordings of the delivery of the intervention during the trial, Robinson and Heritage (2015) demonstrated that despite high levels of provider implementation fidelity, patients’ misunderstanding of the action being implemented by the intervention question could cause them to withhold non-new problems in their responsive turns. In other words, their analysis demonstrated how in situ, fidelity of form could unintentionally impact on fidelity of function.
Pilnick and James (2013) have also argued for the utility of CA methods in addressing the way in which talk-based interventions are assessed. Focusing more on fidelity of function than form, they argue that some interventions are less easy to translate into discrete actions.
Pilnick and James report an assessment of a single video-recording of an encounter between a therapist and a parent of a child with a hearing impairment from a trial of a therapeutic intervention - Video Interaction Guidance. Through detailed description and analysis they demonstrate the scope of CA methods in unpacking the intervention process – how its guiding principles are enacted.
So for a variety of talk-based interventions, CA methods might enable a unique take on fidelity of form and function, and on delivery and responsiveness in situ. Offering more than a window into the extent to which an intervention is being delivered as planned, CA methods can also demonstrate why it may not be working, what else is happening, and how the intervention might be affecting routine practice i.e. other consultation tasks and goals. In other words, CA methods can help characterise the “real-world nature” of fidelity (Masterson-Elgar et al., 2014) in trials of talk-based interventions.
Rather than eliciting self-reports from providers or patients themselves, or using direct observation - the current ‘gold standard’ Bellg et al. (2004) - to judge the presence or absence of pre-specified intervention components, CA starts from observations made from the recorded data itself. Whereas the data generated from checklists are usually quantitative and separate the behaviour of the intervention provider and recipient, CA preserves the qualitative nature of recordings and the analytic focus encompasses all parties to the interaction. Therefore delivery and immediate responsiveness can be assessed together. CA methods allow for the identification of a range of linguistic and other resources that providers are drawing on to implement and integrate an intervention. Analyses can therefore provide an evidence base for the degrees of local tailoring and its impact on theoretical fidelity (i.e., whether local adaptations are consequential for how the underlying intervention theory predicts it should work). Working with recordings and detailed transcripts also means that analyses can be independently checked for agreement.
The idea for this study originated in a GP surgery where staff felt that improvements could be made regarding how it was caring for its most frequent attenders. This idea was developed into a Royal College of General Practitioners award-winning patient-focused intervention including training GPs in the “BATHE” technique (Stuart and Lieberman, 2015). BATHE is an acronym for Background, Affect, Trouble, Handling, and Empathy – see Box 1. It is a well-specified talk-based intervention based around a brief series of linked questions. Its function is to promote discussion of the psychosocial aspects of patients’ lives, to be an informal screen for emotional problems, to connect with the patient, and to support self-management (Stuart and Lieberman, 2015).
The study was designed as a 12 month feasibility cluster randomised trial (ISRCTN62939408) involving six GP surgeries (4 intervention, 2 usual care control). The aim was to explore the key uncertainties to a main trial to evaluate effectiveness and cost-effectiveness. One of the study objectives was to assess whether it would be possible to train other GPs to use BATHE and to assesses the extent of implementation fidelity.
Section snippets
Methods
South West - Central Bristol NHS Research Ethics Committee gave formal approval for the study. Eligible patients were determined by a search of practice records for those aged 18 or over falling in the top three percent of attenders over the last 12 months. GPs then reviewed each patient record and excluded: (i) patients whose level of attendance could be accounted for by a diagnosed physical or mental illness; (ii) patients with life-threatening illness such as cancer (iii) patients over 80
Results
All five BATHE components were delivered in only 58% (n = 15) of cases (range 33%–75% between practices). Four or more components were delivered in 85% (n = 22) of the recorded consultations. Partial implementations (less than four components) were noted in three cases and in one case, the GP failed to deliver the intervention altogether. In 19% (n = 5) of cases, two of the BATHE components were delivered out of order. In one case, four components were delivered out of order. Despite receiving
Discussion
In our study all participating GPs in intervention practices received the same basic BATHE training. Initial monitoring of a sample of recorded consultations showed most of the BATHE components to be present. However, when left to their own interpretative devices, some GPs adhered to the training less than others. Despite the ‘B’ question being asked in 24/26 of the recorded consultations, deviations in terms of position and adaptations regarding composition were often highly consequential for
Acknowledgements
This paper presents independent research funded by the NIHR under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-0613-31099). Rebecca Barnes is supported by funding from The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care West (NIHR CLAHRC West). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
This study was designed and conducted in
References (26)
- et al.
The rejection of advice: managing the problematic convergence of a 'troubles-telling' and a 'service encounter'
J. Pragmat.
(1981) - et al.
“I'm thrilled that you see that”: guiding parents to see success in interactions with children with deafness and autistic spectrum disorder
Soc. Sci. Med.
(2013) - et al.
Communicating a feeling: the social organisation of reported ‘private thoughts’
Discourse Stud.
(2007) - et al.
Avoiding type III errors in health education program evaluations: a case study
Health Educ. Q.
(1985) - et al.
Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium
Health Psychol.
(2004) The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials
J Public Health Dent. Winter
(2011)- et al.
Generating the topic: the use of topic initial elicitors
- et al.
The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit
Prev. Sci.
(2004) - et al.
Assessing usual care in clinical trials
West. J. Nurs. Res.
(2015) - et al.
Complex interventions: how “out of control” can a randomised trial be?
BMJ
(2004)
Conversation Analysis as an Approach to the Medical Encounter
Territories of knowledge, territories of experience: empathic moments in interaction
‘Some' vs 'any' medical issues: encouraging patients to reveal their unmet concerns.'
Cited by (14)
How student healthcare providers in a communication skills course respond to standardized patient resistance
2023, Social Science and MedicineStorytelling and affiliation between healthcare staff in Schwartz Round interactions: A conversation analytic study
2023, Social Science and MedicineCommunicating with older adults with long-term conditions about self-management goals: A systematic review and thematic synthesis
2021, Patient Education and CounselingCitation Excerpt :CA and DA studies have generated substantial knowledge about healthcare communication through fine-grained analysis of recorded interactions [39,40]. Such studies have demonstrated impact across multiple domains including informing policy recommendations [39], developing clinical practice guidelines [41], HCP training [42,43], and assessing the fidelity of interventions [44]. In CA research, audio or audio-visual recordings are subjected to repeated listening and/or viewing and transcribed in detail.
Methodological insights from ethnomethodology and conversation analysis
2019, Journal of PragmaticsFeatures of answers to questions about recent events by people with mild cognitive impairment and Alzheimer’s disease, and healthy controls
2023, Journal of Interactional Research in Communication DisordersCommunicative Practices Clinicians Use to Correct Patient Misconceptions in Primary Care Visits
2023, Health Communication