Using conversation analytic methods to assess fidelity to a talk-based healthcare intervention for frequently attending patients

Highlights

• Talk-based methods are key to assessing fidelity in talk-based interventions.

• Conversation analytic methods can assess intervention delivery and receipt in situ.

• For success with BATHE, how and where it is delivered in the consultation matters.

• Real examples are key when training providers to change their communication habits.

Abstract

The study aim was to assess implementation fidelity (i.e., adherence) to a talk-based primary care intervention using Conversation Analytic (CA) methods. The context was a UK feasibility trial where General Practitioners (GPs) were trained to use “BATHE” (Background, Affect, Trouble, Handling, Empathy) – a technique to screen for psychosocial issues during consultations – with frequently attending patients.

35 GPs received BATHE training between July–October 2015. 15 GPs across six practices self-selected to record a sample of their consultations with study patients at three and six months. 31 consultations were recorded. 21/26 patients in four intervention practices gave permission for analysis. The recordings were transcribed and initially coded for the presence or absence of the five BATHE components. CA methods were applied to assess delivery, focusing on position and composition of each component, and patients’ responses.

Initial coding showed most of the BATHE components to be present in most contacts. However the CA analysis revealed unplanned deviations in position and adaptations in composition. Frequently the intervention was initiated too early in the consultation, and the BATHE questions misunderstood by patients as pertaining to their presenting problems rather than the psychosocial context for their problems. Often these deviations resulted in reducing theoretical fidelity of the intervention as a whole.

A CA approach enabled a dynamic assessment of the delivery and receipt of BATHE in situ revealing common pitfalls in delivery and provided valuable examples of more and less efficacious implementations. During the trial this evidence was used in top-up trainings to address problems in delivery and to improve GP engagement. Using CA methods enabled a more accurate assessment of implementation fidelity, a fuller description of the intervention itself, and enhanced resources for future training. When positioned appropriately, BATHE can be a useful tool for eliciting information about the wider context of the medical visit.

Introduction

In randomised controlled trials (RCTs), the importance of assessing ‘implementation fidelity’ (whether an intervention has been implemented as planned) was first noted by Basch et al. (1985). Basch et al. (1985) highlighted the danger of incorrectly accepting a null hypothesis when it is the result of an (unknowingly) inadequate implementation, with the consequence of discarding a potentially effective intervention. The primary goal of assessing implementation fidelity is to increase scientific confidence that the planned intervention has been adequately tested, and that the measured outcomes are a reliable indication of its effectiveness.

The starting point for fidelity assessment is to have a clear description of the intended intervention in order to compare this to what was delivered. However, this is not always straightforward. In some trials the intervention is necessarily under specified. The main consideration from this perspective is that there may be more than one route to achieving the intended outcome. In such cases trialists may deliberately avoid specifying the precise form of an intervention due to an assumption that it may need to be adapted to the diverse contexts in which it is being evaluated e.g., in primary care settings and trials of ‘talk-based interventions’ (i.e. those geared towards stimulating interaction around particular topics or concerns, as opposed to physical treatments). Moreover interventions are often adapted by those who are delivering them, sometimes intentionally to accommodate them to the context of their delivery, but also unintentionally due to inadequate training, ‘drift’ from the original protocol, or ‘decay’ in provider skills over time (Bellg et al., 2004). Many protocols therefore describe interventions in terms of principles and intended functions rather than specifying too closely the form and/or detail of how those principles are to be implemented. Consequently, ‘fidelity of form’ (where the form of intervention to be implemented is precisely specified), has been distinguished from ‘fidelity of function’ (where the form of implementation is less important than it fulfilling the intended function) (Hawe et al., 2004).

There are a limited number of methods in use for assessing implementation fidelity and not all are ideally suited to assessing talk-based interventions. Conversation Analysis (CA) is widely considered to be the dominant contemporary method for the analysis of talk-in-interaction (Heritage, 2009). Indeed Robinson and Heritage (2014) have argued that CA methods may be a viable option for assessing the implementation fidelity of talk-based interventions. They give the example of a primary care trial evaluating the effectiveness of upfront agenda setting for the reduction of unmet patient concerns (Heritage et al., 2007). Doctors were trained in a tightly specified talk-based intervention to ask, following the problem presentation, if patients had “any” versus “some” further problems or concerns. All study consultations were recorded, and monitored to identify instances where doctors failed to perform the intervention correctly. In a retrospective analysis of 144 video-recordings of the delivery of the intervention during the trial, Robinson and Heritage (2015) demonstrated that despite high levels of provider implementation fidelity, patients’ misunderstanding of the action being implemented by the intervention question could cause them to withhold non-new problems in their responsive turns. In other words, their analysis demonstrated how in situ, fidelity of form could unintentionally impact on fidelity of function.

Pilnick and James (2013) have also argued for the utility of CA methods in addressing the way in which talk-based interventions are assessed. Focusing more on fidelity of function than form, they argue that some interventions are less easy to translate into discrete actions.

Pilnick and James report an assessment of a single video-recording of an encounter between a therapist and a parent of a child with a hearing impairment from a trial of a therapeutic intervention - Video Interaction Guidance. Through detailed description and analysis they demonstrate the scope of CA methods in unpacking the intervention process – how its guiding principles are enacted.

So for a variety of talk-based interventions, CA methods might enable a unique take on fidelity of form and function, and on delivery and responsiveness in situ. Offering more than a window into the extent to which an intervention is being delivered as planned, CA methods can also demonstrate why it may not be working, what else is happening, and how the intervention might be affecting routine practice i.e. other consultation tasks and goals. In other words, CA methods can help characterise the “real-world nature” of fidelity (Masterson-Elgar et al., 2014) in trials of talk-based interventions.

Rather than eliciting self-reports from providers or patients themselves, or using direct observation - the current ‘gold standard’ Bellg et al. (2004) - to judge the presence or absence of pre-specified intervention components, CA starts from observations made from the recorded data itself. Whereas the data generated from checklists are usually quantitative and separate the behaviour of the intervention provider and recipient, CA preserves the qualitative nature of recordings and the analytic focus encompasses all parties to the interaction. Therefore delivery and immediate responsiveness can be assessed together. CA methods allow for the identification of a range of linguistic and other resources that providers are drawing on to implement and integrate an intervention. Analyses can therefore provide an evidence base for the degrees of local tailoring and its impact on theoretical fidelity (i.e., whether local adaptations are consequential for how the underlying intervention theory predicts it should work). Working with recordings and detailed transcripts also means that analyses can be independently checked for agreement.

The idea for this study originated in a GP surgery where staff felt that improvements could be made regarding how it was caring for its most frequent attenders. This idea was developed into a Royal College of General Practitioners award-winning patient-focused intervention including training GPs in the “BATHE” technique (Stuart and Lieberman, 2015). BATHE is an acronym for Background, Affect, Trouble, Handling, and Empathy – see Box 1. It is a well-specified talk-based intervention based around a brief series of linked questions. Its function is to promote discussion of the psychosocial aspects of patients’ lives, to be an informal screen for emotional problems, to connect with the patient, and to support self-management (Stuart and Lieberman, 2015).

The study was designed as a 12 month feasibility cluster randomised trial (ISRCTN62939408) involving six GP surgeries (4 intervention, 2 usual care control). The aim was to explore the key uncertainties to a main trial to evaluate effectiveness and cost-effectiveness. One of the study objectives was to assess whether it would be possible to train other GPs to use BATHE and to assesses the extent of implementation fidelity.