Article Text

Download PDFPDF

Accuracy, user-friendliness and usefulness of the Cobas h232 point-of-care test for NT-proBNP in primary care
  1. Chiel Hex1,
  2. Miek Smeets1,
  3. Joris Penders2,
  4. Viviane Van Hoof3,
  5. Jan Verbakel4,
  6. Frank Buntinx1,
  7. Bert Vaes1
  1. 1 Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
  2. 2 Department of Clinical Biology, ZOL Genk Hospital, Genk, Belgium
  3. 3 Department of Clinical Chemistry, Antwerp University Hospital, Edegem, Belgium
  4. 4 Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
  1. Correspondence to Dr Bert Vaes, Department of Public Health and Primary Care, KU Leuven, Leuven B-3000, Belgium; bert.vaes{at}kuleuven.be

Abstract

Aims N-terminal pro-B-type natriuretic peptide (NT-proBNP) has been shown to be useful for ruling out heart failure in primary care. In this study, we examined the accuracy of the Cobas h232 point-of-care (POC) instrument in primary care compared with an in-hospital measurement. Furthermore, we investigated the user-friendliness and usefulness of the POC device.

Methods Five general practitioner (GP) groups were asked to evaluate adult patients who were suspected of having heart failure and to test NT-proBNP with the Cobas h232. The samples were subsequently delivered to and analysed at a central hospital laboratory by the Cobas e602 using conventional transport and storage. Difference between the paired measurements was analysed using a percentage difference plot, and correlation was assessed using Passing-Bablok linear regression analysis. User-friendliness and usefulness were assessed using semistructured questionnaires.

Results Nineteen GPs studied 94 patients. Passing-Bablok analysis showed a slope of 1.05 (95% CI 1.00 to 1.11) (R2=0.97). The percentage difference plot showed a mean difference of 15.7% (95% CI −46.0% to –77.4%). User-friendliness and usefulness had median scores of 4 or 5 on a five-point Likert scale. Eighteen out of 19 GPs confirmed that the device influenced their clinical practice. During the study, GPs’ confidence in using NT-proBNP increased significantly from a mean score of 4.4 (95% CI 3.2 to 5.6) to 7.6 out of 10 (95% CI 7.1 to 8.2).

Conclusions The Cobas h232 NT-proBNP POC test proved to be an accurate, user-friendly and useful test in primary care. Nearly all participating GPs were convinced that the test could benefit clinical decision making.

  • cardiovascular
  • poc testing
  • diagnosis
  • heart

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Diagnosis of heart failure in young people is relatively unequivocal. However, diagnosis of heart failure in old age, especially slow-onset heart failure, is particularly difficult as the specificity of typical signs and symptoms decreases drastically.1 2 The prevalence of heart failure in the general population is 1%–2%, but increases progressively with age.3 The Rotterdam study showed a prevalence of 1% in the age group of 55–64 years, 3% in the age group of 65–74 years, 7% in the age group of 75–84 years and more than 10% in the age group of 85 years and older.4

Echocardiography remains the standard approach for establishing the diagnosis and aetiology of heart failure. However in recent decades, research has shown that N-terminal pro-B-type natriuretic peptide (NT-proBNP) is an accurate and cost-effective marker in the diagnostic work-up of heart failure, even in older adults.5–8 NT-proBNP is the inactive fragment of proBNP, which is synthesised in the ventricles in response to myocardial wall stress.9 Different guidelines recommend the measurement of NT-proBNP in the hospital setting to identify patients who are eligible for echocardiography as well as for ruling out heart failure in primary care.10

However, implementation of NT-proBNP in primary care faces several limitations. Routine laboratory test takes time, health insurance does not always reimburse the cost of the test and general practitioners (GPs) are often not familiar with the test. A point-of-care (POC) test offers the potential of implementing NT-proBNP in primary care. This test provides a rapid result, and its use could help educate GPs about its clinical utility. However, to date, there is limited data on the user-friendliness and usefulness of the POC test for NT-proBNP in primary care.11 12

Therefore, we assessed the Cobas h232 (Hoffman-La Roche, Roche Diagnostics, Basel, Switzerland) providing the instrument to five GP surgeries in Belgium. The participating GPs were asked to complete a survey about the user-friendliness and usefulness of the device. The Cobas h232 results were compared with results from an in-hospital laboratory test to assess its analytical accuracy.

Methods

Study population

Five GP surgeries, all located in the northeastern region of Belgium, were included in this pilot study: four of these practices were group practices and one was a solo practice. In total, 24 general practitioners and 5 medical practice assistants agreed to participate. The study started in August 2016 and lasted until January 2017. The GPs were asked to enrol consecutive patients suspected of having heart failure. No specific exclusion criteria were set. Consequently, patients with comorbidities, such as chronic renal failure, chronic obstructive pulmonary disease, atrial fibrillation and obesity were not excluded. All participating patients gave informed consent.

Analytical methods

The measuring range of the Cobas h232 for NT-proBNP is 60–9000 ng/L. The test is performed with 150 µL of heparinized venous blood using a Roche Cardiac proBNP test strip (Hoffman-La Roche, Roche Diagnostics, Basel, Switzerland) and takes 12 min.11 Previously, the Cobas h232 showed a sensitivity of 82% and specificity of 99% for values <60 ng/L, sensitivity of 97% and specificity of 83% between 60 and 3000 ng/L and sensitivity of 82% and specificity of 99% for values >3000 ng/L.13 Two age-dependent cut-offs, as recommended by the Belgian Health Care Knowledge Centre, were used.14 The cut-off value of 125 ng/L was used for patients <75 years and 450 ng/L for patients aged 75 years and older. The US Food and Drug Administration also recommends these separate cut-off values for the central laboratory proBNP assay on Elecsys (Hoffman-La Roche, Roche Diagnostics).15 After performing the POC test, blood samples were transported to the Laboratory of Clinical Biochemistry of Ziekenhuis Oost-Limburg (ZOL) in Genk, Belgium; stored at room temperature; and analysed independently within a maximum of 8 hours after collection using the proBNP II Elecsys Cobas e100 assay (Hoffman-La Roche, Roche Diagnostics) on a Cobas e602, a modular analytical instrument of the core unit Cobas 8000.

Legal framework of POC testing

In Belgium, evidence-based use of POC testing outside the hospital is subject to a proposed legal regulation. Collaboration with a central laboratory, organised training and certification, quality management, working according to a protocol and recording the POC test results in the electronic patient record are fundamental elements in this proposal. This proposed regulation would allow physicians who use the POC test outside the hospital to expect reliable, accurate and reproducible results. This pilot study was designed in adherence with this proposal.

Educational training

A POC co-ordinator, employed by the Laboratory of Clinical Biochemistry of ZOL in Genk, Belgium, was responsible for educating the users. Before the study started, the GPs were invited to a first training session. This training consisted of an introduction to the theory and use of the Cobas h232 device by the POC co-ordinator. A second training session was organised at the time the Cobas h232 was provided to the GP groups. During this training, led by the POC co-ordinator, every GP and practice assistant had to perform a dummy test and quality controls. Additional training was offered if the users experienced problems.

Quality control management

The POC co-ordinator supervised the timely execution of the quality controls. Quality control on the Cobas h232 device was performed at three levels. First, the device automatically executed an internal quality control when switched on. Second, was an instrumental quality control and third was a liquid quality control, which had to be executed by the users every week. The POC co-ordinator aliquoted the control samples in volumes of 150 µL, which were stored frozen and defrosted by the users when analysed.

Protocol

Heparin-containing tubes were used. Users identified themselves by scanning their personal barcode using the built-in scanner of the instrument. Patients were identified by scanning their eID (electronic identity card) barcode. Alternatively, when no e-ID was available, a preprinted barcode was scanned that was later linked to the patient by the POC co-ordinator. After identification, a test strip was placed in the device. A sample of 150 µL of blood was taken from the tube using a specifically marked pipette from Roche. The result displayed as <60 ng/L, >9000 ng/L or a number in between. Meanwhile, the user informed the central laboratory and a taxi service was sent to collect the blood tube with the remaining blood. Transport was arranged within a maximum of 8 hours.

Connectivity

The connection between the Cobas h232 devices and IT middleware Cobas IT1000 in the central laboratory was established with a secure VPN connection. The Cobas IT1000 transmitted the incoming results to the laboratory informatics system where the patient’s information was linked to his/her national insurance number. Thereafter, these data were sent to the hospital informatics system. The results of the POC test were integrated in the electronic patient record of the GP using secure communication software.

Outcome measures

User-friendliness and usefulness were assessed using a semistructured questionnaire (see online supplementary appendix 1). Both the GPs and medical practice assistants were asked to answer questions about the instrument’s user-friendliness. Only the GPs were questioned about its clinical usefulness. For each question, a five-point Likert scale was used. GPs completed a questionnaire before the study started and estimated their level of confidence in using NT-proBNP testing on a 10-point scale. This question was repeated in the final questionnaire.

Supplementary file 1

Statistical analysis

Continuous data are presented as means and SD and compared among groups with Student’s t-test. Categorical data are presented as numbers and percentages and compared among groups using the χ2 test. The correlation between the Cobas h232 results and Cobas e602 results was analysed and plotted using linear regression according to the Passing-Bablok method. The difference between the POC test and reference laboratory test was analysed using a percentage difference plot and an estimated bias plot.16 User-friendliness and usefulness were examined by reporting the scores on five-point Likert scales.

Data analysis was performed using SPSS V.22.0 (SPSS, Chicago, Illinois, USA), Excel for Mac 2011 (Microsoft, Redmond, Washington, USA), XLStat V.19.02 (Addinsoft, Paris, France) and R 3.2.4, (https://www.r-project.org/).

Results

In total, 24 GPs and 5 practice assistants were invited to participate. Twenty GPs and all of the assistants attended the training session, and 19 GPs and 5 assistants participated in the study. Figure 1 shows the distribution of the users across the different practices. The characteristics of the users are summarised in table 1.

Figure 1

Distribution of the participants and the test performed in the different GP surgeries. Assist, assistant; GP, general practitioner.

Table 1

User characteristics (n=29)

In total, 94 patients with a mean age of 72.0±11.1 years (range: 40–91 years) were included. The proportion of patients aged 75 years and older was 50%. The characteristics of the patients are presented in table 2.

Table 2

Patient characteristics (n=94)

Ninety-two out of 94 blood samples were reanalysed with the Cobas e602 instrument. In two cases, the GP forgot to send the blood sample to the central laboratory. Figure 2A shows the percentage difference between the NT-proBNP test results in 92 patients analysed with the Cobas h232 and Cobas e602. The mean relative difference was 15.7% (95% CI −46.0% to –77.4%) The mean absolute difference was 22.1 ng/L (95% CI −607.7 to –651.9 ng/L). Figure 2B shows the estimated bias for NT-proBNP measurements on the Cobas h232 compared with measurements on the Cobas e602. The estimated bias decreased from low NT-proBNP measurements (13.8% (95% CI 8.2% to 20.4%) at 100 ng/L) to higher measurements (7.1% (95% CI 0.39% to 14.2%) at 500 ng/L and 5.9 (95% CI −1.7% to –13.9%) at 2000 ng/L).

Figure 2

(A) Percentage difference plot of all tests (n=92). The plot compares the relative difference between the POC test result on the Cobas h232 and the result of the control test on the Cobas e602 and the mean of both measurements. (B) Estimated bias plot (n=92). The plot shows the estimated bias (and 95% CI) of the POC test result on the Cobas h232 compared with the control test on the Cobas e602. NT-proBNP,  N-terminal pro-B-type natriuretic peptide; POC, point of care.

Passing-Bablok linear regression analysis showed a slope of 1.05 that was not significantly different from 1 (95% CI 1.00 to 1.11) and intercept of 0.71 that was not significantly different from 0 (95% CI−6.80 to –13.26) (y=1.05 x+0.71) with an R2 of 0.97, as shown in figure 3.

Figure 3

Passing-Bablok regression plot comparing the results on the Cobas h232 and the results on the Cobas e602. NT-proBNP, N-terminal pro-B-type natriuretic peptide; POC, point of care.

The results of the questionnaire regarding usefulness showed median scores of 4 or 5 out of 5 for all of the items evaluated. Eighteen out of 19 responding GPs said that the device influenced their clinical practice. Additionally, 18 out of 19 GPs would use NT-proBNP testing in the future, of which 15 (78.9%) thought it was useful as a POC test. Almost all of the items about user-friendliness were evaluated as positive by at least 75% of the users based on 23 responses, of which four responses were from medical practice assistants. Only the number of process steps had a lower score, with only 65.2% of positive answers. Evaluation of the different questions is summarised in figure 4.

Figure 4

Distribution of the responses on the questionnaires about usefulness and user-friendliness. GP, general practitioner; NT-proBNP,  N-terminal pro-B-type natriuretic peptide;  POC, point of care.

Before the study started, GPs’ confidence in using NT-proBNP testing had a mean score of 4.4 out of 10 (95% CI 3.2 to 5.6) based on the responses of 15 GPs. Re-evaluation by these GPs after the study showed an estimated confidence of 7.6 (95% CI 7.1 to 8.2) (P<0.001).

Discussion

The Cobas h232 NT-proBNP POC test proved to be an accurate test in a primary care setting. However, a minor overestimation of the results with the Cobas h232 compared with the reference laboratory test needs further investigation. The users rated the device as satisfactorily in terms of usefulness and user-friendliness. Almost all of the GPs confirmed their intention to use NT-proBNP in the future, and a large majority believed in the value of using a POC test for NT-proBNP determination.

Accuracy

To our knowledge, only one published study has evaluated the accuracy of the Cobas h232 NT-proBNP POC test in a primary care setting. In this recent study, a discrepancy between the primary care and hospital NT-proBNP results was shown (mean bias 15.7%, range: 14.3% to 23.7%). No significant bias was found when the Cobas h232 was only used in a hospital setting (bias range: −4.9% to 7.0%).12 On the other hand, SKUP (Scandinavian evaluation of laboratory equipment for primary healthcare) reported that the Cobas h232 results in primary care were 14% to 24% higher than the comparison method.11 These results are consistent with our findings. A possible explanation for the bias might be the delay between both analyses, although NT-proBNP is assumed to be stable for at least 8 hours. This bias might also be due to the difference between the analysis of NT-proBNP from a whole blood specimen in the case of the Cobas h232 and from centrifuged blood in the case of the Cobas e602. Another explanation could be the accuracy of the sampling, for instance, the volume of blood applied on the test strip, but for this explanation, equal bias is expected in both directions. Further research is needed to determine the cause of this bias. However, an overestimation is safer than an underestimation for a test used in a primary care setting that focuses on ruling out heart failure. It is important to note that typical heart failure symptoms (dyspnoea, fatigue and oedema) occurred with comparable frequency in patients with a positive NT-proBNP test as in patients with a negative test.

User-friendliness and clinical usefulness

Concerning the user-friendliness and usefulness of the instrument, the Cobas h232 scored good to excellent. The score for user-friendliness is roughly comparable to the result of the SKUP report, which evaluated the device at three primary care surgical practices.11 12 According to our results, improvement is needed regarding the number of procedural steps. A major comment of the GPs was the time needed for the execution of the different steps and different quality controls. Together with the reported increase in confidence in using NT-proBNP as a diagnostic test, the test was considered to be user-friendly and useful in assisting in cases with diagnostic uncertainty.

Use of NT-proBNP as a POC test

Questions can be raised as to whether NT-proBNP should be available as a POC test. For comparison, much more evidence is available about the value of C reactive protein as a POC test.17–19 However, the majority of the GPs in our study considered the NT-proBNP POC test to be useful for their practice. This observation is consistent with an international study of 2770 primary care doctors. NT-proBNP testing was desired as a POC test by 59% of GPs.20 A critical consideration is the immediacy of the result in cases of slow-onset heart failure. For chronic illnesses, Hardy et al mentioned that knowing the result might not be advantageous when the patient is sitting across from you.21 However, HbA1c POC testing has been described as improving the patient–doctor relationship as well as the level of patient satisfaction and leads to an increased sense of provider responsibility.22 23

The future of NT-proBNP POC testing may consist of being part of a panel of cardiovascular POC tests that can be used for patients with dyspnoea or chest pain. Tomonaga et al emphasised the clinical value of 3-in-1 POC testing, including cardiac Troponin T, NT-proBNP and D-dimer, for the diagnosis and risk stratification of patients with presumed cardiovascular symptoms in primary care. Furthermore, the diagnoses were more accurate in the POC testing group.24

Implementation in daily practice and future research

All of the GPs mentioned the cost as the major drawback in using NT-proBNP testing, although it has been shown to be a cost-effective test.7 25 26 The second drawback was the effect of the workflow and time requirement. Most of the GPs believed that the best way to implement the POC test in their practice was to outsource the testing to an assistant, as was done by different practices in our study. The same issue regarding outsourcing was described for C-reactive protein (CRP) POC testing.27 However, whether the GP or assistant should perform the test remains a point of discussion.21 Moreover, according to the American Association of Clinical Chemistry (AACC), the main issue of POC testing is non-laboratory personnel performing the test, as this applies to both GP and assistant. Therefore, the AACC regulatory requirements of POC testing focus on two areas: training and competency of the personnel performing the test and verification of strict adherence to the test procedure.28 A POC co-ordinator should supervise this training. In this study, the support from the POC co-ordinator and laboratory was greatly appreciated by the users. Other than the controversy regarding the test performer and his/her training, it should be emphasised that more regulatory requirements are needed in a legal framework. Establishing a quality management system, working according to a protocol and the availability of the POC test results in the electronic patient record are other important issues. Larsson et al emphasised that this framework is essential for the implementation of POC testing.29

Future research should focus on the impact of NT-proBNP POC testing on the diagnosis and monitoring of heart failure in primary care and continued implementation of a legal framework for POC testing. Regarding connectivity, a reliable and possibly cloud-based IT solution, compatible with all different electronic health record programmes should be available. Concerning quality management, training and co-ordination, the role of central laboratories and their POC co-ordinators should be further investigated.

Strengths and limitations

The major strength of this study was the integration of all elements of a legal framework for POC testing in a realistic study protocol. The protocol was designed to fully implement NT-proBNP POC testing in primary care. It included a realistic sampling and testing setting, collaboration with a clinical laboratory, POC co-ordinator, training of users, connectivity and conventional transport and storage. A major limitation was that usefulness was not assessed using objective measures, as only the subjective opinion of the participants was questioned. Another limitation was the lack of comparing the usefulness of the POC test with the lab test. The relatively small numbers of participating GPs and tests performed could also be seen as a limitation of the current study. Another limitation was possibly the lack of reminders for the GPs to use the device. This might also be one of the responsibilities of a future POC co-ordinator. Finally, we were not able to assess clinical effectiveness, as we did not focus on morbidity or mortality outcomes.

Conclusion

The Cobas h232 NT-proBNP POC test proved to be an accurate test in a primary care setting. Potential overestimation of the results with the Cobas h232 requires further investigation. Considering the subjective impression of the study participants, the device was deemed to be useful and user-friendly. Almost all of the GPs expressed interest in using NT-proBNP testing in the future, and a large majority saw benefit in the use of a POC test for NT-proBNP.

Take home messages

  • The Cobas h232 N-terminal pro-B-type natriuretic peptide (NT-proBNP) point-of-care (POC) test proved to be an accurate test in a primary care setting.

  • A minor overestimation of the results with the Cobas h232 compared with the reference laboratory test needs further investigation.

  • The users rated the device as satisfactorily in terms of usefulness and user-friendliness.

  • A large majority of the general practitioners believed in the value of using a POC test for NT-proBNP determination.

Abstract translation

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Acknowledgments

The authors thank the participating GPs and assistants.

References

Footnotes

  • Handling editor Tahir S Pillay.

  • Contributors CH, MS and BV designed the study and BV was responsible for the conduct of the study. JP was responsible for the laboratory analyses. CH and JV performed the statistical analyses. CH, JV and BV drafted the manuscript. All authors participated in the critical revision of the manuscript. All authors read and approved the final version of the manuscript.

  • Competing interests The Department of Public Health and Primary Care, KU Leuven designed and initiated the study. Roche Diagnostics had no role in the study design or initiation. Roche Diagnostics provided material support, in particular 5 Cobas h232 point-of-care instruments. The cost of the material, transportation and analysis was borne by Roche Diagnostics. The authors declare to have had full independence during data analysis, without interference by Roche. We hereby state the independence of researchers from funders.

  • Patient consent Obtained.

  • Ethics approval The ethics review board of the University Hospitals/KU Leuven, Belgium approved the study (reference mp11149).

  • Provenance and peer review Not commissioned; externally peer reviewed.