Article Text
PDF
Abstract
Background The Consumer Quality Index Palliative Care (CQ-index PC) is a structured questionnaire for measuring the quality of palliative care from the perspective of care users. CQ-indices assess which care aspects need quality improvement by relating answers about actual care experiences to answers about the importance of certain aspects of care.
Methods To improve the chance that the new instrument has good content validity, a literature study and individual and group discussions were performed, and a steering committee was consulted to establish the instrument's face and content validity. The questionnaire was administered to patients with a life expectancy of 6 months or less and/or who were receiving palliative treatment. Descriptive analyses were carried out on the items about actual care experiences and the importance of care aspects, and on ‘need for improvement’ scores.
Results 15 care organisations participated. 133 patients met the inclusion criteria (net response n=85). Patients considered the following aspects the most important: ‘offering help in good time in acute situations’, ‘caregivers having the necessary expertise’ and ‘caregivers taking the patient seriously’. The three care aspects with the highest ‘need for improvement’ scores were: ‘support when the patient feels depressed’, ‘support when the patient is anxious’ and ‘support when the patient has shortness of breath’.
Conclusions The CQ-index PC provides opportunities for care organisations to assess which care aspects have the highest priority for quality improvement within their organisation. Further research is needed to assess whether the instrument has enough discriminative power to assess differences between organisations.
Statistics from Altmetric.com
Background
Providing patient-centred care is the main aim of palliative care, and so profound insight into the quality of care experienced by patients is of paramount importance. A Consumer Quality Index (CQ-index) is a structured questionnaire for measuring the quality of care from the perspective of care users.
CQ-indices assess care users’ actual care experiences but do not ask directly about their satisfaction with care. A definition of satisfaction is ‘fulfilling expectations, needs or desires’.1 2 Satisfaction about care implies that healthcare users compare their expectations against the actual care delivered and that this leads to either a positive or negative feeling. Hence satisfaction is a result of expectations and actual care experiences. Consequently, variations in satisfaction scores can be a result of differences in either expectations or experiences,3 which means ambiguity is an important problem in satisfaction measurement.4 ,5 In addition, the fact that healthcare users often feel dependant and are grateful to their care providers may result in socially desirable answers and skewed distributions (the majority report being satisfied or very satisfied). Measuring care users’ experiences and relating these to their expectations may therefore better reflect the quality of care and may also provide clearer action points for quality improvement purposes.3 ,6 ,7
These days, a new generation of quality instruments is available that is based on this approach of measuring actual care experiences.8–11 This new generation includes the CQ-indices. They have been developed and used for a number of target groups; for example, patients who are suspected of having breast cancer, patients with rheumatoid arthritis and patients receiving long-term care.12–14 CQ-indices are practical instruments for measuring quality indicators from the care users’ perspective.15 They can provide useful quality information for healthcare organisations, and often also for patients, relatives or external parties such as the Health Care Inspectorate.
In 2007, the Dutch Ministry of Health decided that a CQ-index Palliative Care (CQ-index PC) should be developed that could be used in all settings in which adult patients receive palliative care. The development of such a quality instrument is in line with the policy of the Council of Europe recommending that ‘member states, within a consistent health-policy approach for the specific strategy of improving palliative health-care provision, identify practical indicators that can be used to check what progress has been made in patient care over a given period’.16
The CQ-index PC differs from instruments developed in the past, which ask—often in a traditional, direct way—about satisfaction with palliative care.17–22 Moreover, existing instruments often focus on very specific target groups, for example, relatives of patients in the final stage of dementia,22 persons involved in palliative cancer care23 or persons in a hospice setting.24 Finally, some existing instruments (eg,25) are not questionnaires, but semistructured lists for interviews, and are therefore less appropriate for use in large samples. Thus, the need for a valid questionnaire that could be used to assess the quality of palliative care in various settings prompted the decision to develop a CQ-index PC.
This new instrument consists of a patient version and a version for bereaved relatives. In this paper, the focus is on the CQ-index PC version for patients. The details about the version for relatives have been described elsewhere.26
The CQ-index PC combines questions on actual experiences with regard to certain aspects of care and questions about how important respondents find certain care aspects. By relating ‘experience scores’ to ‘importance scores’, insight can be gained in which care aspects should have the highest priority in quality improvement.
This paper aims to present this new approach to measuring patient experiences in palliative care. In particular, it highlights patients’ care experiences, quality aspects to which patients attach high importance and quality aspects with the highest priority for improvement.
Methods
Several stages can be distinguished in the development of a CQ-index27 (see box 1): (1) questionnaire construction, (2) assessment of psychometric properties and (3) assessment of discriminative power. These stages also fit with general approaches in the development of a measurement instrument.28
Stages in the development of a Consumer Quality Index
Stage 1. The construction phase, consisting of:
-
1a. Literature study.
-
1b. Focus groups and/or individual interviews.
-
1c. Development of the draft questionnaire, which is tested for face and content validity in a steering committee and/or other expert groups.
-
1d. Pretest, in which the draft questionnaire is tested among a small group of respondents to investigate whether the questions are understandable and complete.
Stage 2. The psychometric test.
The questionnaire is tested among a larger group of respondents. The aim is to shorten the instrument and assess its psychometric properties. Sometimes (like in this paper) the available data are also used to obtain a first impression of the quality of care provided and the care aspects that have the highest priority for improvement.
Stage 3. The test of discriminative power.
The questionnaire is tested among a larger number of respondents to assess whether the instrument is able to detect quality differences over time or between healthcare providers.
The discriminative power of the CQ-index PC will be investigated in a future project, and therefore this paper only addresses stages 1 and 2.
Construction of the questionnaire (stage 1)
Literature study
To develop a CQ-index addressing the construct ‘the patient perspective on the quality of physical, psychosocial and spiritual aspects of palliative care’ first a literature study of existing relevant instruments was performed. No existing questionnaires were found that were appropriate in every respect for measuring the quality of palliative care from a user perspective in various palliative care settings.26 However, some parts of existing questionnaires (see, eg,29) provided input for the development of the CQ-index PC.
In addition, relevant studies in which Dutch patients or relatives were asked about crucial elements in the quality of palliative care were studied. In a study by Borgsteede and coauthors, patients considered the following aspects of general practitioner (GP) care crucial at the end of life: the availability of the GP, medical competence and good cooperation with other professionals, and attention and the continuity of care.30
The conclusion of this part of the literature study was that different existing studies mentioned different quality aspects of palliative care, but almost all mentioned respect for autonomy, personal attention and privacy.26
Interviews and focus groups
To improve the chance that the instrument has good content validity, we used information from a previous interview study by our research group among 19 patients and 23 relatives.31 These interviews focussed on what the patients and relatives thought was important for good palliative care. Various inpatient and outpatient settings were represented in the interviews.
In mid-2008, we conducted additional interviews to refine and validate the earlier interview data. These consisted of three individual interviews with patients, one interview with a relative and one focus group interview with three relatives of patients who had recently died after a long period of illness. In addition, two focus group interviews were held in 2008 with a total of 14 professionals and four voluntary care providers to investigate important quality aspects further.
Development of the draft questionnaire and testing in the steering committee
The draft questionnaire consisted of 100 items divided into questions about actual care experiences (experience items), how important certain care aspects are for respondents (importance items) and background characteristics. For each question regarding the patient's experience with a certain aspect of care, a corresponding question was included about how important this care aspect was for the patient.
To reduce recall bias, the experience items mainly concerned the care received in the previous week. Another reason for asking about care experience in the previous week was that many transitions can take place in the palliative phase, and asking about a longer period therefore increases the risk that the patient has difficulty in choosing an appropriate answer. Most of the experience items had answers on a 4-point scale (‘never’, ‘sometimes’, 'usually’ and ‘always’, sometimes combined with the option of ‘not applicable’). A few experience items had answer options of ‘yes’ or ‘no’.
The importance items all had answers on a 4-point scale (‘not important’, ‘fairly important’, ‘important’ and ‘extremely important’).
The face validity and content validity of the first version of the CQ-index PC were established in the steering committee, which consisted of 16 experts in the field of palliative care, including care providers, and representatives of patient organisations, national professional organisations and the Ministry of Health.
Pretesting among patients
The completeness and comprehensiveness of the first version was tested in a so-called pretest with nine patients in various palliative care settings. Patients were also asked to respond to some questions about the comprehensiveness and completeness of the questionnaire and the time needed to complete the questionnaire.
The recommendations of the steering committee and the results of the pretest among patients resulted in some minor revisions to the items, for example, changing the answer options for some items to ‘yes’ or 'no’, instead of ‘never’, ‘sometimes’, 'usually’ and ‘always’. This was necessary because some items referred to an experience that could only occur once.
Psychometric test (stage 2)
Sample and data collection
At the end of 2008, the CQ-index PC was tested further among a larger group of patients. Organisations were recruited via existing contacts of the Netherlands Institute for Health Services Research and EMGO Institute for Health and Care Research. In every organisation there was a central contact person who was responsible for the recruitment of patients. All patients who met the inclusion criteria were approached. Patients were eligible for inclusion if they had a life expectancy of 6 months or less and/or were receiving palliative treatment. Furthermore, they had to be physically and mentally able to answer the questions in the CQ-index PC.
Fifteen organisations participated in the measurement of the CQ-index PC for patients (five high-care hospices, three hospitals, one nursing home, one home for the elderly, four home care organisations and one mental healthcare organisation). The patients were asked to complete the CQ-index PC based on their experiences with care delivered in the previous week by the care institution from which they received the questionnaire.
In inpatient or hospice settings, the CQ-index PC was completed during an interview conducted by one of the researchers or another trained interviewer. Patients at home received the questionnaire by post and had to complete it in writing. The decision to send the questionnaire to these patients by post was made mainly for practical reasons: it was not feasible to visit a large number of patients living at home all over the country within a short time frame and with a limited budget.
Analysis of the psychometric qualities
We aimed to shorten the CQ-index in order to create a manageable instrument. We analysed the distribution of scores, performed item-response analyses, analysed the scores for the importance items and performed Pearson's correlation analyses. Decisions about omitting or reformulating items were largely based on the methodological considerations common in the development of CQ-indices26 paying critical attention to the items with the characteristics listed in box 2. However, decisions to omit or reformulate items were influenced by the methodological considerations listed in box 2 and by the recommendations of the steering committee and remarks made by respondents.
Characteristics of items requiring critical attention when shortening a Consumer Quality Index
-
Extremely skewed items (>90% of answers in extreme category; in extreme category means >90% of the answers in the most positive category (‘always’ or ‘yes’) or in the most negative category (‘never’ or ‘no’)).
-
Items with a high percentage of missing values (>10% missing values) and or ‘not applicable’ answers.
-
Items with relatively low scores for the associated importance items (if the mean score for an importance item was in the ‘bottom 10’ of the lowest scores).
-
Items with a strong mutual relationship (Pearson's correlation r>0.70).
Analysis of actual care experiences, importance of care aspects and ‘need for improvement’ scores
Descriptive analyses were carried out of the items asking about actual care experiences, the importance of care aspects and the ‘need for improvement’ scores.
To obtain an indication of which quality aspects have highest priority for improvement, ‘need for improvement’ scores were calculated. ‘Need for improvement’ scores are calculated by multiplying the mean score for a question about the importance of a care aspect by the percentage of respondents with a ‘negative experience’ with this aspect, and dividing this number by 100. A ‘negative experience’ means that respondents answered ‘never/sometimes’ or 'no’ to a question about experiences with that aspect of care.
A high ‘need for improvement’ score means that respondents generally attach considerable importance to a care aspect, but at the same time often have a relatively negative experience.
The higher the ‘need for improvement’ score, the higher the priority for improvement.
Ethics
The research protocol was approved by the Medical Ethics Committee of the VU University medical center in Amsterdam. Before inclusion, the patients received verbal and written information about the content and aim of the CQ-index PC, and written informed consent was received from all patients who completed the CQ-index PC in the form of an interview.
Results
Response
There were 133 patients in the 15 organisations participating in the measurements using the patient version of the CQ-index PC who met the inclusion criteria and were given the questionnaire. The net response was 63.9% (n=85).
Over a third of the 85 participants were living at home (n=30, 35.3%); the rest were receiving care in a hospice facility (n=18, 21.2%) or a hospital (n=16, 18.8%), were receiving hospital day care (n=12, 14.1%), were in a nursing home or home for the elderly (n=8, 9.4%) or were in a mental health institution (n=1, 1.2%).
The mean age of the 85 patients was 68, more than half (n=49; 57.6%) were female subjects and the majority (n=75; 88.2%) had cancer. A total of 59 patients (69.4%) completed the CQ-index PC in an interview, and 26 patients (30.6%) completed it in writing.
Psychometric results
The psychometric analyses showed that no items were extremely skewed, and only three items had more than 10% missing values. For 15 items, 10%–60% of the respondents had answered ‘not applicable’. However, this was not always a reason for deleting the item because most of these items concerned support from professionals in case of pain or other symptoms; as not all patients suffered from these symptoms, answering ‘not applicable’ will often be appropriate. Such items were not omitted since the management of pain and other symptoms is a crucial element of palliative care.
There were 10 items with low scores for importance (ie, a mean score for the importance varying between 2.58 and 3.24). This prompted the decision to omit some of these items (eg, an item concerning the meals).
There were also some items with strong correlations (Pearson's correlation r>0.70), which was a reason for omission in some but not all cases. For example, the Pearson's correlation coefficient between the item on support when a patient was feeling depressed and the item concerning support when the patient was feeling anxious was 0.92. In this case, neither item was omitted because the strong correlation does not necessarily imply a large overlap in the content of the item.
The foregoing analyses resulted in a reduction in the number of items from 100 to 88 items (32 experience items, 32 importance items and 24 on background characteristics).
Patients’ care experiences
Table 1 shows all the experience items, as well as the relatively ‘negative’ answer categories of ‘no’ or ‘never/sometimes’. The most frequent negative experiences were for the items ‘receiving information about the advantages and disadvantages of various types of treatment’, ‘receiving support when being constipated’ and ‘receiving support when feeling depressed’. However, a feature of the CQ-index approach is that scores on experience items are related to scores on items about how important care users find certain care aspects in order to gain insight into which care aspects should have the highest priority for quality improvement. In the following sections we therefore discuss the importance scores, and ultimately the ‘need for improvement’ scores.
Content and scores of experience items in the Consumer Quality Index Palliative Care
Patients’ scores on importance items
The 10 importance items with the highest mean scores are presented in table 2.
Top 10 importance items with the highest mean scores
Patients consider ‘help is offered in good time in acute situations’, ‘caregivers have the necessary expertise’ and ‘caregivers take you seriously’ the most important items.
Quality aspects with ‘need for improvement’
Table 3 shows the 10 aspects that have the highest priority for improvement. The corresponding percentage of negative scores (percentage of respondents who answered ‘never’, ‘sometimes’ or ‘no’) for the experience items and the mean scores for the importance items are also presented in table 3. The items concerning professional support when the patient feels depressed, is anxious or has shortness of breath have the highest ‘need for improvement’ scores.
Ten highest ‘need for improvement’ scores with corresponding scores for experience items and importance items
The relatively high ‘need for improvement’ scores in table 3 indicate that healthcare providers should give priority to these aspects if they wish to improve palliative care. The scores for the experience items can give healthcare providers more detailed insight into the need for improvement. For example, the negative experience score of 45.95% for the item about ‘support when feeling depressed’ indicates that almost half of the patients answered that they ‘never’ or ‘sometimes’ received support from care providers when they were feeling depressed. At the same time, the importance score of 3.04 (within a range of 1–4) indicates that patients consider it ‘important’ to ‘extremely important’ that they receive this kind of support. The other ‘need for improvement’ scores in table 3 should be interpreted in the same way.
‘Politeness of the caregivers’, ‘respect for the patient's life stance’ and ‘receiving medical aids soon enough’ are examples of care aspects with a relatively low priority for quality improvement. The 10 lowest ‘need for improvement’ scores are shown in table 4.
Ten lowest ‘need for improvement’ scores with corresponding scores for experience items and importance items
There are, of course, also items that are not in the top 10 or in the bottom 10 of the ‘need for improvement’ scores.
Discussion
The main aim of this paper was to present a new approach and a new questionnaire for measuring patient experiences in palliative care. It highlighted aspects of palliative care to which patients attach high importance and aspects of palliative care that have the highest priority for improvement. This study provides an indication that the questionnaire is suitable for use in various palliative care settings.
To reduce recall bias, most of the items in this questionnaire concerned the care experienced in the previous week. Because respondents had to be physically and mentally able to answer the questions, patients in the terminal phase were excluded. This means that the patient version of the CQ-index PC does not provide insight into the quality of care in the very last days of life. We have therefore also developed a CQ-index questionnaire for bereaved relatives with a specific focus on the care provided in the last week of the patients’ life, as well as on support and after-care for the relatives themselves. This questionnaire for relatives has been described in detail elsewhere.26
An important characteristic of the CQ-index PC is that it assesses actual care experiences, not satisfaction with care. Therefore, there will be fewer socially desirable answers than with traditional satisfaction measurements. Two different data collection methods were used. Inpatient and hospice patients were interviewed to complete the CQ-index PC questionnaire, while patients who lived at home received a postal questionnaire. Since only one method of data collection was used in a given setting, no conclusions could be drawn about the impact of different methods of data collection on, for instance, socially desirable answers. This may be further explored in future research.
Another feature of the CQ-index PC is that it covers all aspects of palliative care: physical, social and spiritual well-being. Besides, a distinctive characteristic of this new instrument is that it can be used to gain insight into the patient perspective on which care aspects have the highest priority for quality improvement. In managers’ decision making and prioritising of needed quality improvements costs and complexity also will be taken into account. However, in our opinion it is important to look first at the patient perspective, since meeting patients’ needs is crucial for high-quality palliative care.
The ‘need for improvement’ scores in this initial study of the CQ-index PC indicate that support for patients who feel depressed or anxious, or who have shortness of breath are the areas with the highest priority for improvement in the participating Dutch care organisations. However, these organisations were not randomly selected, and the under-representation of some care settings (eg, homes for the elderly or nursing homes) may have biased the results. Future research with more representative samples will provide more insight into the quality of palliative care and the priorities for quality improvement.
In this initial study, the face validity and content validity of the instrument have been taken into account. At present we are setting up a study with a larger sample for more rigorous psychometric analyses, such as factor analyses and analysis of discriminative power. It would also be interesting to compare different settings and different organisations using a larger sample and to see whether there are differences in quality. For future comparisons, it is also important to gain more insight into the patient characteristics for which statistical adjustments need to be made (case-mix adjustment) in order to obtain valid comparisons of quality scores. The future research project will provide more insight into these issues.
Conclusions
The CQ-index PC is a new approach to measure the quality of palliative care from the perspective of patients and to determine which care aspects have the highest priority in quality improvement.
At the moment, the CQ-index PC has only been tested in Dutch. However, an English translation has been made to inform foreign researchers (available on request). It is recommended that the CQ-index PC is also tested and validated by researchers in other countries who wish to measure the quality of palliative care from a user perspective and who are interested in priorities for quality improvement.
Acknowledgments
The authors would like to thank all those who participated in the test phase of the CQ-index PC.
References
Footnotes
-
Contributors SJJC, ALF: drafted this manuscript in cooperation with AJEdV, HJS and LD. ALF, AJEdV and LD: were responsible for the design of this study. SJJC and ALF: were involved in the data collection. SJJC, ALF and HJS: were involved in the analysis and interpretation of the data. All authors read and approved the manuscript.
-
Funding This project was funded by the Netherlands Organization for Health Research and Development (ZonMw).
-
Competing interests None.
-
Patient consent Before inclusion, the patients received verbal and written information about the content and aim of the CQ-index PC, and written informed consent was received from all patients who completed the CQ-index PC in the form of an interview.
-
Ethics approval Ethics approval provided by the Medical Ethics Committee of the VU University medical centre Amsterdam.
-
Provenance and peer review Not commissioned; externally peer reviewed.