Research

Recommendations for clinical research in children presenting to primary care out-of-hours services: a randomised controlled trial with parallel cohort study

Anouk AH Weghorst, Gea A Holtman, Pien I Wolters, Heleen A Russchen, Freek Fickweiler, Henkjan J Verkade, Johan Post, Karin M Vermeulen, Boudewijn J Kollen, Irma J Bonvanie and Marjolein Y Berger
BJGP Open 2021; 5 (2): bjgpopen20X101154. DOI: https://doi.org/10.3399/bjgpopen20X101154

Abstract

Background Research in primary care is essential, but recruiting children in this setting can be complex and may cause selection bias. Challenges surrounding informed consent, particularly in an acute clinical setting, can undermine feasibility. The off-protocol use of an intervention nearing implementation has become common in pragmatic randomised controlled trials (RCTs) set in primary care.

Aim To describe how the informed consent procedure affects study inclusion and to assess how off-protocol medication prescribing affects participant selection in a paediatric RCT.

Design & setting A pragmatic RCT evaluating the cost-effectiveness of oral ondansetron in children diagnosed with acute gastroenteritis (AGE) in primary care out-of-hours services and a parallel cohort study.

Method Consecutive children aged 6 months to 6 years attending primary care out-of-hours services with AGE were evaluated to assess the feasibility of obtaining informed consent, the off-protocol use of ondansetron, and other inclusion and exclusion criteria.

Results The RCT's feasibility was reduced by the informed consent procedure because 39.0% (n = 325/834) of children were accompanied by only one parent. GPs prescribed ondansetron off-protocol to 34 children (4.1%) of which 19 children were eligible for the RCT. RCT-eligible children included in the parallel cohort study had fewer risk factors for dehydration than children in the RCT despite similar dehydration assessments by GPs.

Conclusion The informed consent procedure and off-protocol use of study medication affect the inclusion rate, but had little effect on selection. A parallel cohort study alongside the RCT can help evaluate selection bias, and a pilot study can reveal potential barriers to inclusion.

How this fits in

Research in primary care is essential for evidence-based health care, yet 40% of paediatric RCTs are discontinued prematurely. A pilot study can help to identify potential recruitment barriers and prevent many reasons for recruitment failure. However, based on this study, the authors further advocate: 1) reconsidering the need for written informed consent from both parents in trials where the risk to the child is low; and 2) improving communication with all stakeholders to prevent off-protocol prescribing of study medication. Finally, selection bias can be identified by using a parallel cohort study.

Introduction

Outcomes from RCTs are essential for GPs to provide evidence-based health care.1 However, recruiting sufficient numbers of representative participants can be difficult, especially for acute paediatric management.2 This is illustrated by the fact that 40% of paediatric RCTs are discontinued prematurely owing to poor recruitment.2–4 Besides that, GPs who are aware of the effectiveness of the intervention may use this intervention, which can cause selection bias.5

Primary care is not an easy place to conduct research.6 Although GP involvement in case recruitment can decrease the chance of successful inclusion,1 not involving them is not always feasible and can be costly. In out-of-hours primary care centre (OOH-PC), GPs must also evaluate patients with whom they are unfamiliar, which may further decrease their willingness to recruit children into a trial and worsen the inclusion rate.1 When trials are discontinued, authors rarely report how and by whom participants were recruited, which prevents any lessons learnt being applied when planning trials in other settings.2 Ideally, authors would report the recruitment process of their trial in sufficient detail to help avoid the repetition of mistakes.5 Pilot studies can be used to uncover reasons for recruitment failure.5

A pragmatic RCT was performed to investigate the cost-effectiveness of adding a single dose of oral ondansetron to care-as-usual (CAU) in an OOH-PC on the frequency of vomiting in children aged 6 months to 6 years with AGE. Despite a pilot study, child recruitment was challenging. In this report, the recruitment efforts are described, focusing on how the informed consent procedure and the use of off-protocol prescribing affected the inclusion rate and child selection, respectively.

Method

Study design

In the pragmatic RCT, participants were enrolled from December 2015 to January 2018 at three OOH-PCs in the north of the Netherlands (Groningen, Zwolle, and Assen). After a pilot study (NL4700) from December 2015 to October 2016, in agreement with the medical ethical committee (METc), the primary outcome was changed from ‘referral rate’ to ‘proportion of children who continued vomiting in the first 4 hours after randomisation’ because this was considered a more patient-oriented outcome by both the METc and the parents involved. The primary and secondary outcomes of the RCT are detailed in Supplementary Box 1. The written informed consent procedure was also adapted because it could not be feasibly obtained from both parents, severely restricting inclusion. The METc agreed that children could be included from the pilot study in the amended RCT (NL5830), and a parallel cohort study was added. The parallel cohort study provides insight into the representativeness of the trial population and helps to assess the external validity in a non-invasive manner.7,8 Follow-up was for 7 days after randomisation.

Participants

Inclusion criteria: RCT

Children aged 6 months to 6 years were included who presented at the OOH-PC with vomiting and for whom the GP diagnosed AGE. Specifically, children were included if: 1) they reported at least four episodes of vomiting 24 hours before presentation; 2) they reported at least one episode of vomiting 4 hours before presentation; and 3) written informed consent was obtained from both parents. If the child was accompanied by one parent, that parent could give written informed consent and the second parent could give oral informed consent via a telephone call in the presence of the research assistant (RA). The written informed consent from the second parent had to be sent by post.

Exclusion criteria: RCT

Children were excluded if: 1) they had used or been prescribed antiemetics in the previous 6 hours; 2) they had renal failure or hypoalbuminaemia; 3) they had diabetes mellitus or inflammatory bowel disease; 4) they had a history of abdominal surgery that could explain the current symptoms according to the GP; 5) they had sensitivity to 5-HT3 receptor antagonists; 6) they had a prolonged QT interval or were using QT-prolonging medication; or 7) they had previously been enrolled.

Inclusion criteria: parallel cohort study

Children aged 6 months to 6 years and diagnosed with AGE, but whose parents did not give written informed consent for the RCT, were asked if they were willing to participate in a parallel cohort study in which written informed consent was only needed from one parent.

Patient recruitment and baseline assessment

Parents were informed about the study by an RA before the GP consultation. If parents were interested, the RA started baseline assessments (Supplementary Box 2) and the GP then assessed the child (confirming or refuting a diagnosis of AGE and assessing the degree of dehydration) and their suitability for participating based on the inclusion and exclusion criteria. The RA asked the parents of eligible children for written informed consent (Figure 1).

Figure 1.
Figure 1. Study design. AGE = acute gastroenteritis. CAU = care as usual. OOH-PC = out-of-hours primary care. RA = research assistant.

Randomisation and blinding

After obtaining at least one written and one oral informed consent from parents, children were block randomised (1:1 allocation) to intervention groups by a computer programme and were stratified by age (6–24 months or >24 months) and dehydration status (see Table 1: ‘at risk’ if no alarm symptoms, or ‘dehydrated’ if ≥1 alarm symptom). Allocation was not generated before inclusion to ensure concealment. Treatment allocation was not blinded to the parents, the child, the GP, or the RA. The researcher who performed the statistical analyses was blinded to treatment allocation.

View this table:
Table 1. Alarm symptoms and risk factors for dehydration

Interventions

Control group and parallel cohort study: CAU

Children received CAU (Figure 1) that comprised instructions to buy oral rehydration solution and how to use it, as described in the acute diarrhoea guideline of the Dutch College of General Practitioners.9 That involved 10 ml/kg compensation when at risk of dehydration (that is, all children) and 15 ml/kg for 4 hours when assessed as dehydrated. Parents were informed about the expected course, alarm symptoms, and when and how to contact their GP.

Intervention group: ondansetron plus CAU

Children in the intervention group received CAU plus a single weight-based dose of oral ondansetron syrup (0.1 mg/kg) according to the Dutch Paediatric Formulary.10 If the child vomited within 15 minutes, the dose was given a second time only.

Follow-up assessment

Parents used a structured diary to record symptoms (that is, diarrhoea, vomiting, and fever), oral rehydration therapy and fluid intake, medication use, adverse reactions, healthcare use, hours missed from work, and satisfaction with treatment during follow-up. The diary was to be completed every hour for the first 4 hours and daily thereafter for 7 days. Parents could return the diary on paper with the enclosed envelope. If parents did not return the diary after multiple requests, information was collected about the primary outcome by telephone.

Sample size

Based on a systematic review, it was estimated that 85% and 64% of children in the CAU and ondansetron groups would continue vomiting after 4 hours,11 indicating that a difference of 21% in the proportion of children with persistent vomiting was clinically relevant. Therefore, 89 children per arm needed to be included for a power of 90% and an α of 0.05, allowing for a 10% loss to follow-up.

Statistical analysis

The following baseline characteristics were assessed: age, sex, AGE symptoms (weight, duration or frequency of vomiting and/or diarrhoea), dehydration alarm symptoms, dehydration risk factors (see Table 1),9 and degree of dehydration assessed by the GP (0–100 scale). To evaluate the impact of the informed consent procedure on the inclusion rate, the number of children with AGE who visited the OOH-PC with only one parent was reported. To evaluate the impact of the off-protocol prescription of ondansetron on the selection of children in the RCT, baseline characteristics of RCT-eligible children included in the parallel cohort study and for whom the GP prescribed ondansetron were non-statistically compared with children included in the RCT. To understand the overall level of selection bias, the baseline characteristics of RCT-eligible children included in the parallel cohort study were statistically compared with the baseline characteristics of children included in the RCT. Continuous variables were compared with non-parametric tests (Mann–Whitney U test) and dichotomous variables were compared by logistic regression. Statistical significance was defined by a two-tailed P value of <0.05, and all analyses were performed using SPSS (version 25).

Results

Study population

The eligibility of 1061 children meeting the enrolment criteria were assessed. Of these, 834 had a GP diagnosis of AGE, with 194 ultimately randomised in the RCT and 201 included in the parallel cohort study. In total, 70 children were eligible for both the RCT and parallel cohort study and were included in the study analysis (Figure 2).

Figure 2.
Figure 2. Study flow diagram. RCT = randomised controlled trial. aOnly one parent was present (n = 325). bGP objected to ondansetron use (n = 16) or did not agree with inclusion (n = 9)

Informed consent

In 39.0% (n = 325/834) of cases, only one parent accompanied the child with AGE. Adapting the consent procedure increased the inclusion rate from seven to 10 cases per month. Of the 194 randomised children, 42 were accompanied by only one parent (21.2%) and 16 (8.3%; eight from each RCT group) did not send the second written informed consent after giving verbal permission, so were excluded (Figure 2).

Off-protocol ondansetron prescribing

GPs prescribed ondansetron off-protocol to 34 (4.1%) of the 834 children with AGE, and 19 of these were eligible for participation in the RCT. There were no clinically relevant differences in baseline characteristics compared with children in the RCT, except that GPs estimated the degree of dehydration to be almost twice as high in these 19 cases compared with those in the RCT (38 versus 20; Table 2).

View this table:
Table 2. Baseline characteristics of all children, and those in each analysed subgroup of children

Selection bias

Compared with children in the RCT, the baseline characteristics of the 70 RCT-eligible children in the parallel cohort study did not differ with statistical significance except for the risk factors of dehydration. Children in the parallel cohort study had less risk factors for dehydration compared to children in the RCT (odds ratio = 0.22; 95% confidence interval = 0.11 to 1.00), but the median GP-assessed dehydration level did not differ with statistical significance (P = 0.302) (Table 2).

Discussion

Summary

Almost 40% of all children attended with one parent, making it difficult to obtain informed consent as required and, thereby, complicating inclusion. GPs also prescribed off-protocol ondansetron if they suspected more severe dehydration, but this did not correspond with known dehydration risk factors or alarm symptoms. Children in the parallel cohort study had fewer additional risk factors for dehydration compared with children in the RCT.

Strengths and limitations

The authors are aware of no prior research assessing the pitfalls of the trial recruitment process in an OOH-PC. Consecutive children were screened presenting to one of the three OOH-PC over a period spanning more than 2 years, making this study highly representative of the population. Lessons have been learnt from the pilot, and a parallel cohort study has been added in which the children included have been evaluated. However, a limitation is that the RCT-eligible children in the parallel cohort study — to whom GPs prescribed ondansetron — was small (n = 19), precluding statistical testing with the children in the RCT. It should also be noted that the recommendations are based on the data from one RCT and a parallel cohort study.

Comparison with existing literature

Informed consent

Based on guidance for research involving humans, the METc decided that both parents must sign a parental consent form.12 However, the risk for children in this RCT was deemed low to moderate, with sufficient evidence of effect in referred children and extremely low risk of adverse events.13–15 In this study, almost 40% of children diagnosed with AGE at the OOH-PC were accompanied by only one parent, but the acuteness of the clinical problem meant that inclusion and randomisation could not be delayed to obtain the second consent, potentially resulting in study exclusion. This problem with obtaining informed consent from both parents has been reported in other paediatric RCTs.16,17

Based on the results of a pilot study that confirmed the above, the METc agreed to an adapted procedure that increased the inclusion rate by three children per month. The adaptation allowed for one parent to give written informed consent and the second to give initial oral informed consent by telephone, with confirmation of written informed consent obtained by post. A second written informed consent for 16 children was not received despite calling repeatedly; but, these children were randomised, eight received the study medication, and some even returned their diary. Nevertheless, they were excluded for protocol deviation, raising ethical concerns given that they had been randomised, had completed study activities, and had received the study medication.

Off-protocol ondansetron prescribing

Implementation of the study protocol created more awareness of the potential efficacy of ondansetron for children with AGE. In 34 children, GPs prescribed ondansetron off-protocol despite not being recommended in national guidelines.9 After the pilot study, fearing that the effect of ondansetron would be diluted if prescribed in both study arms, it was decided to not include eligible children from the RCT if a GP prescribed ondansetron before randomisation. Their follow-up was monitored in the parallel cohort study instead.

A clinically relevant difference existed in the level of dehydration estimated by the GP between groups, but owing to the small group size (n = 19), this was without statistical significance. Among children receiving off-protocol ondansetron, the GP estimated dehydration to be almost twice that in children in the RCT (38 versus 20). Given that it was intended to assess the effect of ondansetron in children at increased risk of dehydration, excluding these may have resulted in an underestimation of the true effect of ondansetron. However, the study demonstrated no differences in risk factors or alarm symptoms of dehydration. Therefore, the level of dehydration estimated by the GP alone should be interpreted with caution.

Selection bias

The baseline characteristics of RCT-eligible children in the parallel cohort study did not differ from those of children in the RCT, except that those in the parallel cohort study had fewer risk factors for dehydration. This could imply that the parents of more severely dehydrated children were more willing to participate in the RCT. However, the median dehydration level assessed by the GP did not differ statistically between groups. Other studies have shown that there is no structured way of assessing dehydration in children with AGE,18,19 meaning that determining the course of AGE and the risk of dehydration remain important challenges. Further research is needed to evaluate the prognostic value of risk factors and GP-based assessments of dehydration. Given the number of additional risk factors and the GP estimates, the authors are confident that they included children at mild-to-moderate risk of dehydration, as intended.

Implications for practice

The authors' experiences indicate that there are four areas in which study designs like theirs need improving. First, there is a need to reconsider if written informed consent is required of both parents in pragmatic RCTs involving low risk to the child, especially in acute settings. There is need for future observational research to see how often both parents visit the OOH-PC with their child, as often the other parent stays at home for other caring responsibilities. By identifying the exact numbers for multiple childhood diseases the need to reconsider this ethical decision increases even more. Consistent with this study's approach, it is thought that obtaining the written informed consent of one parent and the oral consent of the second parent, in the presence of the first parent and an RA, is ethically more responsible than excluding a child who otherwise engages in the study but for whom a second written consent form is not received. Second, GPs should receive information that starting with the intervention (or on a paediatrician’s recommendation) during an RCT can seriously bias the outcomes. Communication with all stakeholders, to assess the barriers to protocol compliance, should be routine when performing (pragmatic) RCTs. Third, the authors recommend initiating a parallel cohort study to run alongside RCTs to ensure follow-up when parents do not want to participate in the RCT. This allows comparison of the characteristics of children who did and did not participate, and gives an opportunity to assess if the right population was included. Finally, initiating a pilot study before an RCT offers an invaluable opportunity to evaluate potential barriers to study inclusion and patient selection.

Notes

Funding

This project has received funding from the Netherlands Organization for Health Research and Development, ZonMW (ZonMW project number: 836031001), with additional financing from the department of General Practice and Elderly Care Medicine.

Ethical approval

This study was conducted according the principles of the Declaration of Helsinki and in accordance with the Medical Research Involving Human Subjects Act and other guidelines, regulations, and acts. The Medical Ethical Committee of the University Medical Centre of Groningen approved this trial and it has been registered in the Dutch Trial Register (number NL5830).

Trial registration number

Dutch Trial Register number: NL5830 (https://www.trialregister.nl/trial/5830)

Provenance

Freely submitted; externally peer reviewed,

Acknowledgements

For the screening, inclusion, and randomisation of children at the OOH-PC, the authors would like to thank their research assistants. They also thank the OOH-PC in Groningen, Zwolle, and Assen for their cooperation. For advising in the design of the study, the authors thank Dr MH Blanker, Dr JP Rake, and Dr T Keizer. Finally, thanks to Dr Robert Sykes (www.doctored.org.uk) for providing editorial services.

Competing interests

The authors declare that no competing interests exist.

Patient consent

Parents of the included children signed written informed consent forms. All patient data were coded and the results are anonymous and will be presented in two papers on effectiveness and cost-effectiveness.

  • Received July 14, 2020.
  • Accepted August 20, 2020.

This article is Open Access: CC BY license (https://creativecommons.org/licenses/by/4.0/)

References

  1. 1.↵
    1. van der Wouden JC,
    2. Blankenstein AH,
    3. Huibers MJH,
    4. et al.
    (2007) Survey among 78 studies showed that Lasagna's law holds in Dutch primary care research. J Clin Epidemiol 60(8):819–824, doi:10.1016/j.jclinepi.2006.11.010, pmid:17606178.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Schandelmaier S,
    2. Tomonaga Y,
    3. Bassler D,
    4. et al.
    (2017) Premature discontinuation of pediatric randomized controlled trials: a retrospective cohort study. J Pediatr 184:209–214, doi:10.1016/j.jpeds.2017.01.071, pmid:28410086.
    OpenUrlCrossRefPubMed
  3. 3.
    1. Furyk JS,
    2. Lawton LD,
    3. Ting JY,
    4. et al.
    (2017) Informed consent in emergency care research: an oxymoron? Emerg Med Australas 29(1):110–112, doi:10.1111/1742-6723.12642, pmid:27469986.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Kasenda B,
    2. von Elm E,
    3. You J,
    4. et al.
    (2014) Prevalence, characteristics, and publication of discontinued randomized trials. JAMA 311(10):1045–1051, doi:10.1001/jama.2014.1361, pmid:24618966.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Briel M,
    2. Olu KK,
    3. von Elm E,
    4. et al.
    (2016) A systematic review of discontinued trials suggested that most reasons for recruitment failure were preventable. J Clin Epidemiol 80:8–15, doi:10.1016/j.jclinepi.2016.07.016, pmid:27498376.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Wilson S,
    2. Delaney BC,
    3. Roalfe A,
    4. et al.
    (2000) Randomised controlled trials in primary care: case study. BMJ 321(7252):24–27, doi:10.1136/bmj.321.7252.24, pmid:10875830.
    OpenUrlFREE Full Text
  7. 7.↵
    1. Olschewski M,
    2. Schumacher M,
    3. Davis KB
    (1992) Analysis of randomized and nonrandomized patients in clinical trials using the comprehensive cohort follow-up study design. Control Clin Trials 13(3):226–239, doi:10.1016/0197-2456(92)90005-K, pmid:1320559.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Goldenberg NA,
    2. Tripputi M,
    3. Crowther M,
    4. et al.
    (2010) The "parallel-cohort RCT": Novel design aspects and application in the Kids-DOTT trial of pediatric venous thromboembolism. Contemp Clin Trials 31(1):131–133, doi:10.1016/j.cct.2009.11.006, pmid:19941974.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Belo J,
    2. Bos M,
    3. Brühl F,
    4. et al.
    (2014) NHG-Standaard acute diarree (derde herziening). Huisarts Wet 57(9):462–471.
    OpenUrl
  10. 10.↵
    1. Kinderformularium [Dutch Paediatric Formulary]
    [Ondansetron] Geneesmiddel ondansetron (in Dutch). https://www.kinderformularium.nl/geneesmiddel/30/ondansetron. 19 Nov 2020.
  11. 11.↵
    1. Carter B,
    2. Fedorowicz Z
    (2012) Antiemetic treatment for acute gastroenteritis in children: an updated Cochrane systematic review with meta-analysis and mixed treatment comparison in a Bayesian framework. BMJ Open 2(4):e000622–11, doi:10.1136/bmjopen-2011-000622, pmid:22815462.
    OpenUrlAbstract/FREE Full Text
  12. 12.↵
    1. Lepola P,
    2. Needham A,
    3. Mendum J,
    4. et al.
    (2016) Informed consent for paediatric clinical trials in Europe. Arch Dis Child 101(11):1017–1025, doi:10.1136/archdischild-2015-310001, pmid:27226526.
    OpenUrlAbstract/FREE Full Text
  13. 13.↵
    1. Tomasik E,
    2. Ziółkowska E,
    3. Kołodziej M,
    4. et al.
    (2016) Systematic review with meta-analysis: ondansetron for vomiting in children with acute gastroenteritis. Aliment Pharmacol Ther 44(5):438–446, doi:10.1111/apt.13728, pmid:27401959.
    OpenUrlCrossRefPubMed
  14. 14.
    1. Freedman SB,
    2. Ali S,
    3. Oleszczuk M,
    4. et al.
    (2013) Treatment of acute gastroenteritis in children: an overview of systematic reviews of interventions commonly used in developed countries. Evid Based Child Health 8(4):1123–1137, doi:10.1002/ebch.1932, pmid:23877938.
    OpenUrlCrossRefPubMed
  15. 15.↵
    1. Freedman SB,
    2. Adler M,
    3. Seshadri R,
    4. et al.
    (2006) Oral ondansetron for gastroenteritis in a pediatric emergency department. N Engl J Med 354(16):1698–1705, doi:10.1056/NEJMoa055119, pmid:16625009.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Nelson DK,
    2. Skinner D,
    3. Guarda S,
    4. et al.
    (2013) Obtaining consent from both parents for pediatric research: what does "reasonably available" mean? Pediatrics 131(1):e223–e229, doi:10.1542/peds.2012-1278, pmid:23266920.
    OpenUrlAbstract/FREE Full Text
  17. 17.↵
    1. Lopez T
    (2008) Who can give consent to treatment on behalf of a child? Prescriber 19(17):55–60, doi:10.1002/psb.281.
    OpenUrlCrossRef
  18. 18.↵
    1. Duggan C,
    2. Refat M,
    3. Hashem M,
    4. et al.
    (1996) How valid are clinical signs of dehydration in infants? J Pediatr Gastroenterol Nutr 22(1):56–61, doi:10.1097/00005176-199601000-00009, pmid:8788288.
    OpenUrlCrossRefPubMed
  19. 19.↵
    1. Freedman SB,
    2. Vandermeer B,
    3. Milne A,
    4. et al.
    (2015) Diagnosing clinically significant dehydration in children with acute gastroenteritis using noninvasive methods: a meta-analysis. J Pediatr 166(4):908–916, doi:10.1016/j.jpeds.2014.12.029, pmid:25641247.
    OpenUrlCrossRefPubMed
Back to top

In this issue

BJGP Open
Vol. 5, Issue 2
April 2021
Download PDF
Download PowerPoint
Email Article
Citation Tools
Recommendations for clinical research in children presenting to primary care out-of-hours services: a randomised controlled trial with parallel cohort study
Anouk AH Weghorst, Gea A Holtman, Pien I Wolters, Heleen A Russchen, Freek Fickweiler, Henkjan J Verkade, Johan Post, Karin M Vermeulen, Boudewijn J Kollen, Irma J Bonvanie, Marjolein Y Berger
BJGP Open 2021; 5 (2): bjgpopen20X101154. DOI: 10.3399/bjgpopen20X101154
del.icio.us logo Twitter logo Facebook logo Mendeley logo

Jump to section

Keywords

Intended for Healthcare Professionals

 
 

British Journal of General Practice