A protocol for the development and piloting of quality measures to support the Healthier You: The NHS Diabetes Prevention Programme

RAND consensus methodology

The RAND appropriateness method uses a group judgment process to systematically and quantitatively incorporate expert opinion with scientific evidence, whereby panellists rate, discuss, and re-rate quality measures.²⁰ Benefits include the opportunity to discuss issues and maintain all potential information, as no indicators are dropped between meetings, as well as the use of scientific literature. The RAND panel will be a convenience sample of GPs. Recruitment to the RAND panels will be open to GPs from diverse backgrounds; for example, those with recent qualifications versus long-standing registered GPs, salaried versus partnered GPs. However, panels can be small, patients are rarely involved, and there is potential for dominant personalities to carry more influence.²⁰

In a systematic review by Kotter et al, a consensus method was used in all 35 included studies, 15 of which used a modified RAND method to limit stakeholder influence and incorporate expert opinion.²² The use of RAND in UK quality measurement has been outlined in research for conditions such as angina and non-insulin dependent diabetes in general practice settings.²³

The research panels will be undertaken with GPs and CCG commissioners which could result in missing out patient experiences, but there is a risk in combining patient and professional views as patients may be dominated by professionals.²⁴

The clarity, validity, and feasibility of identified measures will be evaluated using a modified RAND consensus methodology with groups of GPs and CCG level commissioners within Herefordshire CCG. A modified RAND methodology offers a unique systematic approach to synthesising the evidence base underpinning the quality measures, using existing literature as well as professional opinions.²⁵ It has been widely used to develop quality measures in the UK and internationally.²⁶

Each RAND panel will consist of two rounds of quality measure ranking (Figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1. RAND consensus methodology process for panellists.

Explore patient views

Qualitative research with patients lends itself to the enhancement of the validity and clarity of measures, not only for providers but also patients who face the consequences of being identified with the condition and of being part of the healthcare system.

There is a diverse range of behaviour change theories that can help develop an understanding of constructs regarding people’s behaviour.²⁷ The COM-B model provides three factors — capability, opportunity and motivation — to develop behavioural targets as a basis for intervention design.²⁸ The model encompasses a variety of psychosocial factors that can determine initiation and maintenance of behavioural change.²⁹ The model has been applied successfully in a number of contexts to understand behaviour change.²⁸ By investigating patient views and behaviours in relation to health management, researchers can identify potential barriers to or facilitators for changing behaviour once an identification of chronic disease risk has been made.

Although patient involvement in measure development has been carried out, there is a limited number of works in this field, especially regarding details of involvement and representativeness.²²

Patient views will be explored through a minimum of four focus groups, each consisting of 8–10 participants. Braun and Clarke suggest 3–6 focus groups can provide sufficient data for a medium sized project.²⁸ Data collection will be an iterative process where focus group recordings will be transcribed and analysed to monitor data saturation and provide additional topics for subsequent focus groups.³⁰ Incorporating patient views will further strengthen understanding of quality measure suitability. Furthermore, the use of the COM-B model of behaviour change to guide data collection, analysis, and interpretation will provide findings that can be understood from different healthcare perspectives, for example, for health interventions considering patient views on preventative care.²⁹

Patients will be identified through direct recruitment from GP practices or through local support networks as eligible for the focus groups if they:

• have been recorded as having NDH or their latest HbA1c was between 42–47 mmol/mol in the last 6 months (Table 1); and

• are aged ≥18 years

The practice will be asked to send patients information sheets about the study and an invitation to participate. No personal information will leave the practice/CCG. Patients will be contacted if they are at risk and inclusion will not be limited to patients taking part in the diabetes prevention programme.

The focus group will begin with an initial discussion of people’s experiences of diabetes prevention care and then each potential measure will be presented to the group with the reason for its development.

The topic guide will be used to gather data on participant's views and experiences, and is structured based on the three integral components of the COM-B model — namely, capability (both psychological and physical), motivation, and opportunity (physical and environmental)²⁹ (Box 3).

Participants' experience of risk assessment can parallel the experience of illness and associated social implications.³¹ Participants will therefore be encouraged to make suggestions not only to improve clarity and acceptability of the potential measures for diabetes prevention, but also to shed light on their experiences and perception of having an elevated blood sugar level and their capability, motivation, and opportunities in relation to it. Analysis of participants’ comments will be an iterative process and run concurrently through data collection.

Quality measure piloting and evaluation

A convenience sample of five GP practices will be recruited in NHS Herefordshire CCG to a maximum of 24 (whole health economy).

Practices which agree to take part will be expected to:

1. Work with the quality measure for the 6 months of the pilot.

2. Complete baseline and end of pilot computerised data extraction of quality measure achievement;

   • electronic data will be extracted for participating practices using EMIS web queries at the beginning and end of the pilot period in order to assess how performance against the measure changed over time. Demographic data will be extracted to explore potential differences as a result of age, sex, and ethnicity.

3. Complete a workload diary for 1 month of the pilot;

   • these consist of completing a spreadsheet that monitors how much time was spent on each quality measure.

4. Complete an end of pilot interview.

The lead GP at each practice and any other staff, including diabetic nurses who were actively involved in piloting, will be interviewed at the end of the pilot.

All interviews will be audiorecorded and transcribed verbatim. Data will be analysed thematically.³²

Cost-effectiveness analysis

For any QOF indicators developed from the agreed measures, their cost-effectiveness will be determined using a net (monetised) benefit approach.³³ In summary, the method applies the following calculation to an indicator:

Net benefit = (monetised health benefit – delivery cost) – QOF payment.

QOF payments are an incentive paid to GP practices when certain quality thresholds relating to the indicators are achieved.

The costs and benefits for each indicator are estimated through a rapid review of the available economic evidence. Evidence of delivery costs and benefits, expressed as quality-adjusted life years (QALYs), is sought. The opinions of those involved in delivering the indicators at pilot GP sites are also sought.

The costs associated with an indicator include service delivery costs; for example, the costs of additional GP or nurse consultations to monitor a patient’s health status over a period of time, the initial cost of implementing the intervention, and, if relevant, secondary care service usage. Unit costs will either be extracted from the literature review or derived from published sources such as NHS Reference Costs.

The benefits refer to health benefits which might be gained by a patient as a result of the introduction of the indicator. Effectiveness will be derived from the literature in the form of utility values, which can be converted to QALYs. The monetised health benefit of the indicator refers to the value of the health improvements associated with achieving the predicted benefits of the indicator. These benefits are presented in terms of the changes in QALYs as a result of introducing the new indicator compared to standard practice without the new indicator.

Sensitivity analysis is performed through simple scenario analysis rather than probabilistic sensitivity analysis.

Practical issues

The focus groups will be conducted with two members of the research team and all focus group interviews will be analysed and verified among the team to limit research subjectivity or bias of one researcher.³⁴

Participants are free to withdraw from the research at any time without giving any reason. There is no intervention, manipulation or deception taking place.