Figures
Abstract
Background
Frequent attenders are patients who visit their general practitioner exceptionally frequently. Frequent attendance is usually transitory, but some frequent attenders become persistent. Clinically, prediction of persistent frequent attendance is useful to target treatment at underlying diseases or problems. Scientifically it is useful for the selection of high-risk populations for trials. We previously developed a model to predict which frequent attenders become persistent.
Aim
To validate an existing prediction model for persistent frequent attendance that uses information solely from General Practitioners’ electronic medical records.
Methods
We applied the existing model (N = 3,045, 2003–2005) to a later time frame (2009–2011) in the original derivation network (N = 4,032, temporal validation) and to patients of another network (SMILE; 2007–2009, N = 5,462, temporal and geographical validation). Model improvement was studied by adding three new predictors (presence of medically unexplained problems, prescriptions of psychoactive drugs and antibiotics). Finally, we derived a model on the three data sets combined (N = 12,539). We expressed discrimination using histograms of the predicted values and the concordance-statistic (c-statistic) and calibration using the calibration slope (1 = ideal) and Hosmer-Lemeshow tests.
Results
The existing model (c-statistic 0.67) discriminated moderately with predicted values between 7.5 and 50 percent and c-statistics of 0.62 and 0.63, for validation in the original network and SMILE network, respectively. Calibration (0.99 originally) was better in SMILE than in the original network (slopes 0.84 and 0.65, respectively). Adding information on the three new predictors did not importantly improve the model (c-statistics 0.64 and 0.63, respectively). Performance of the model based on the combined data was similar (c-statistic 0.65).
Citation: Smits FT, Brouwer HJ, Zwinderman AH, van den Akker M, van Steenkiste B, Mohrs J, et al. (2013) Predictability of Persistent Frequent Attendance in Primary Care: A Temporal and Geographical Validation Study. PLoS ONE 8(9): e73125. https://doi.org/10.1371/journal.pone.0073125
Editor: Jerson Laks, Federal University of Rio de Janeiro, Brazil
Received: January 31, 2013; Accepted: July 17, 2013; Published: September 5, 2013
Copyright: © 2013 Smits et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was financed by a grant from the Netherlands Organization for Health Research and Development (Zon-Mw); Alledaagse ziekten no. 42011002. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Some patients visit their general practitioner (GP) relatively often. This frequent attendance is mostly defined as an age- and sex-adjusted attendance rate ranking in the top 10% within a time frame of 1 year [1], [2] Frequent attenders (FAs) are responsible for 39% of all face-to-face consultations of their GPs and persistent frequent attenders (those 1.6 percent who frequently attend during three consecutive years or more; pFA) are responsible for about 8% of face-to-face consultations. [3] Frequent attenders and, in particular, persistent frequent attenders have relatively many somatic, psychiatric and social problems [3].
Although longitudinal studies on frequent attenders are scarce, it is known that most frequent attenders frequently attend their GPs for a short period of time only [3]–[7]. It seems neither reasonable, nor efficient to target extensive diagnostic assessment, monitoring, and intervention at transient 1-year frequent attenders. However, patients who continue to attend frequently may require special attention, and potential effective interventions should probably be targeted at this group. Prediction of persistent frequent attendance may therefore be clinically useful if effective treatment of underlying medical problem and (thereby) prevention of this persistence is available. Scientifically a prediction model for pFAs may be useful to help select more homogeneous high-risk populations for future randomized trials or support efficient subgroup analysis [8], [9].
A review on the effects of (mainly psychiatric) interventions on morbidity and attendance rates has shown conflicting results. [10] One out of a total of 5 trials showed that a depression management program improved quality of life and the number of depression-free days of patients frequently attending the GP during two years. [11], [12] None of the included trials showed an effect on healthcare utilization of frequent attenders. All trials, except the one mentioned above, included patients that attended frequently during just one year. [12] Therefore, some negative findings may have been due to strong regression to the mean and spontaneous ‘recovery’, making it difficult to detect any effects. A more recent Spanish study in 1-year FAs showed that a 15 hours' training of GPs, which incorporated biopsychosocial, organizational, and relational approaches resulted in a reduction of attendance rates (mean number of annual contacts in the intervention group 13.1 against 19.4 in the usual care group) [13].
Using information that was readily available in GPs’ electronic medical records (EMR), we developed a prediction rule to help GPs identify, among 1-year frequent attenders, those at extremely low or high risk of becoming persistent frequent attenders (see table 1). [14] With the indicators available in the EMR presented in our previous study, our rule was modestly effective in selecting those at risk of becoming persistent frequent attenders (AUC 0.67; CI 0.64–0.69).
Since many diagnostic indices perform worse in a different population, (external) validation in a different primary care population is warranted before the use in clinical practice is advocated. [15] In this study we temporally (another time frame) and geographically (another area) validated our previously derived prediction model for pFA-ship using information solely from GPs’ EMR and looked for opportunities to improve it with extra patient information.
Because they are more prevalent in pFAs and theoretically likely to increase persistence of frequent attendance, we added 4 extra variables (sex, presence of medically unexplained symptoms, prescriptions of psychoactive drugs and antibiotics) to the original model and tested it for improvement on the 3 cohorts (including original cohort) [3], [16], [17].
We also explored building a more robust model based on the combined data of all three datasets.
Methods
Ethics Statement
The study was conducted according to the Dutch legislation on data protection (Ministry of Justice, the Netherlands). Ethics approval was provided through the Medical Ethics Review Committee of the Academic Medical Center of the University of Amsterdam (letter W 12_259#12.27.0295), stating that “the Medical Research involving human subjects Act (WMO) does not apply to this study and that an official approval of this study by our committee is not required”.
Patient Population
To validate our prediction rule we used two primary care cohorts:
1. Temporal validation; Six primary healthcare centres in Amsterdam and Diemen provided data for the second cohort. These centers participate in the GP-based continuous morbidity registration network of the Department of General Practice, Academic Medical Center - University of Amsterdam. [14] This cohort was an enlarged version of our original cohort (4,032 vs. 3,045 adult patients) in a more recent time frame (2009–2011). See our original article for more details [14].
2. Geographical and temporal validation; All primary healthcare centres of the Eindhoven Corporation of Primary Health Care Centres in Eindhoven provided data for the first cohort. These centers participate in the GP-based Study on Medical Information and Lifestyles Eindhoven (SMILE) of the Department of General Practice of Maastricht University. [18] The patients studied were of average socioeconomic level, of more western descent, and slightly older than the general Dutch population. This cohort consisted of 5,462 patients who were FA in 2007 and we followed them to 2009.
In both networks, EMR data are extracted for research purposes. The participating GPs use a problem-oriented registration method. This study used the numbers of face-to-face consultations with the GPs, the lists of current medical problems as registered and coded by the GPs using the International Classification of Primary Care (ICPC), and a selection of prescriptions of all patients [19].
Selection of One-year Frequent Attenders and Persistent Frequent Attenders
In all cohorts, frequent attenders were defined as those adult patients whose attendance rates ranked nearest to the top 10th centile of their age group (15–30, 31–45, 46–60, ≥61 years) separate for men and women. [1], [2] Persistent frequent attenders were defined as those patients who were both registered and frequently attending during 3 consecutive years. Only face-to-face consultations with GPs (consultations in the office and house calls) were included. Consultations with other practice staff were excluded because, in the practices involved, such consultations are mostly initiated and planned by GPs or their staff and cover mainly chronic disease programs.
Definition and Extraction of Predictor Information
We considered potential predictors of persistent frequent attendance which were easily obtainable in all three cohorts (see table 2). In the problem-oriented approach to medical record keeping, a patient may have a list of current medical problems, also called a problem list. In the Netherlands, a current medical problem is defined as:
- Any disease or complaint which, according to the GP, needs continuing medical attention or monitoring and/or
- Any disease or complaint present for more than 6 months and/or
- Recurrent medical problems (more than 4 episodes per half year) [20].
Every problem on this list was coded by the GPs using the ICPC. [19] The prevalence of each medical problem was calculated at the end of the year. Medically Unexplained Symptoms (MUS) were defined according to Robbins et al. [21].
See the supporting file (table S1) for a list of the selected ICPC-codes.
Statistical Analysis
Validation of model to predict persistent frequent attendance.
The prediction model, to be validated in the present analysis, has been derived in our previous study using (bootstrapped) multivariable logistic regression analysis. This model included the variables: age (Odds ratio (OR) 0.99 per year), number of active problems (OR 1.13 per additional problem), presence of any chronic somatic problems (OR 1.55), any psychological problems (OR 1.72) and the monthly number of analgesic prescriptions (>4: OR 2.06). [14] See Table 1.
We first validated the original prediction model through the assessment of its discrimination (predictive values, c-statistic and corresponding 95% confidence intervals) and calibration (slope and corresponding 95% confidence intervals, and Hosmer Lemeshow plots and tests comparing predicted versus observed risk). A calibration slope of 1 indicates that predicted probabilities match observed risks perfectly (100%), a slope <1 indicates overprediction [22]–[24]. We repeated this analysis for an extended model with four extra variables.
Recalculation of the prediction model in the combined cohorts.
Recalculation of the original model was performed using all pooled subjects from the three cohorts (update cohort, n = 12,539). Regression coefficients were obtained using logistic regression with persistent frequent attendance as the dependent variable and age, number of active problems, any chronic somatic problem, any psychological problem and the monthly number of analgesic prescriptions as predictors. Analyses were performed using SPSS for windows, version 20.
Results
Prediction of Persistent Frequent Attenders
Table 2 and figure 1 show the general characteristics of the three cohorts. The persistent frequent attenders comprised 15–20% of all registered adult patients in all cohorts. Compared with both Amsterdam cohorts, patients in the SMILE cohort were relatively more often pFAs, had less loss to follow-up, fewer active problems, more psychological problems and fewer medically unexplained symptoms. In SMILE the mean number of prescriptions for analgesics and antibiotics was higher, but lower for psychoactive medication. In the SMILE cohort patients who changed GP within the same primary care organization were not registered as having moved house. Unfortunately, no distinction was made between moving house or death. We corrected for loss to follow (LFU) up by measuring the prognostic index with and without correction for LFU. The results did not materially change. See the supporting file (Table S2).
Table 3 shows the results of the original prediction rule in the Amsterdam II cohort and the SMILE cohort. Using the original regression weights with shrunken coefficients the c-statistics were 0.67 (95% confidence interval (CI) 0.64–0.69) in the original database, 0.62 (CI: 0.60–0.65) in the Amsterdam II cohort and 0.63 (CI: 0.61–0.65) in the SMILE cohort. Re-estimation of the regression weights did not change the results much (table 3). As expected in low-prevalence settings, negative predictive values were high, but all other indices were of moderate size.
Figure 2 shows the predictive values (predicted probabilities of becoming pFA) based on the prediction model in the three cohorts. Predictive values on external validation (AMC II and SMILE) lay between 7.5 percent and 54 percent, slightly more conservative than those in the original cohort (3.3 and 59%).
Histograms showing the predicted values based on the model predictions for the three cohorts, Amsterdam I (the original (derivation) cohort) and the two external validation cohorts, Amsterdam II and SMILE. The graphs illustrate the slight overoptimism of the original model and the shrinkage of the distributions, that is, the tails of the AMC II and SMILE cohort distributions are slightly closer to the center and predicted values smaller than 7 percent or greater than 54% no longer occur on external validation. Y-axes are frequencies.
The effect of clustering on the health center level was negligible with intraclass correlations of 0.02–0.06 and almost no change of the regression weights. See the supporting file (Table S2).
Adding three predictors (any MUS, any psychoactive medication and the mean monthly number of antibiotic prescription) hardly changed the performance (c-statistics Amsterdam II 0.65; CI 0.62–0.67; SMILE 0.65; CI 0.63–0.66) (see table 3).
The Hosmer Lemeshow plots showed modest calibration with underestimation of the probability to become pFA (see Figure 1, upper right circles which tend to fall below the line y = x). This was confirmed by the small p-values indicating large discrepancies between predicted and observed risks. (See fig. 3).
In these Hosmer-Lemeshow calibration plots, each circle represents the observed mean probability of becoming a persistent frequent attender (pFA) within a decile of patients after all patients were ordered from lowest to highest predicted probability. As usual, the Hosmer Lemeshow calibration plot in the top left hand corner shows a good match between predicted and observed risks in the derivation cohort (Amsterdam I) as all circles are close to the diagonal of perfect calibration. On external validation in the Amsterdam II cohort (top right hand graph), eight out of ten predicted values were higher than those observed and those in deciles 5, 8 and 10 (extreme right hand circle) in particular. On external validation in the SMILE cohort, predicted probabilities matched the observed ones well, except for the two highest deciles, 9 and 10. The small p-values are also partly caused by the large sample size so that small mismatches become statistically significant. Note that the vertical distance to the diagonal represents the mismatch between predicted and observed pFA probabilities.
Discussion
Summary of Main Findings
External validation in time and place of an existing prediction model for persistent frequent attendance in primary care showed that its discrimination remained stable while calibration was reduced for the higher predictions in particular. Model extension with three plausible predictors not previously included hardly improved model performance.
Strength and Limitations of the Study
Important strengths of this study are the size and the longitudinal character of the datasets and the experience of the participating GPs in recording and coding the problem lists. [20] The SMILE cohort had relatively little loss to follow-up (mostly because fewer patients ‘moved house’ because of registration limitations) and more pFAs. Knowing the way GPs and practice staff cooperate in the Netherlands, replacement of GP consultations by practice staff consultation (off-utilization bias) will be very limited. [25] Prescriptions are extracted from the electronic medical record and reflect the number of actual prescriptions. Prescription data in general practice may generally be considered to be of higher quality than diagnosis-oriented data. [26] The higher prescription rates of analgesics and antibiotics and the lower rates for psychoactive medication are therefore difficult to understand and may reflect the different patient populations in both cities and/or different prescription habits of the local GPs. Wennberg showed that everyday clinical practice is characterized by wide geographical variations that cannot be explained by illness severity or patient preference. [27], [28] The present retrospective study was based on routinely collected data and therefore reflects everyday general practice in the Netherlands. Because we wanted to predict patient behavior, we only used GP-patient consultations and not planned monitoring consultations for chronic diseases with other primary care staff.
However, there are also some limitations. First, there are differences between the Amsterdam and SMILE cohort which may have influenced the predictive performance of our rule. In general, within a General Practice Research Network, one distinguishes four categories of factors to explain morbidity and prescription differences: “healthcare system”, “methodological characteristics of the network”, “general practitioner”, and the “patient”. These factors and sub-factors are often interrelated. [29] The differences between the cohorts in Amsterdam and Eindhoven may partly be explained by methodological differences (the shorter existence of SMILE (fewer active problems) and coding agreements (fewer MUS, but more psychological codes in SMILE)), different populations of general practitioners (shorter experience in coding problems in SMILE) and patient factors (more stable population; more females in SMILE). However, socio-demographic characteristics of populations cannot explain the differences in morbidity estimations among these cohorts [30].
The problem lists may suffer from overreporting (by not removing resolved problems, for instance, depression) and underreporting (for instance personality disorder). This may have diminished the predictive power of “any psychological problem”. Moving out of practice was a reason for exclusion, as follow-up of these patients was not possible. Compared with SMILE the loss to follow-up rate was higher in Amsterdam, but in the original study this did not result in selection bias. [14] Finally, the presumed higher registration quality in these academic networks may diminish the generalizibility of a prediction rule derived in these networks to other practices.
Formal external validation of prediction models while important is still scarce. If the original prediction model had had excellent performance, one may expect worse performance on external validation. External validation usually reveals the so-called over-optimism of the original model. [31] Our original model performed moderately well and its discrimination performance remained largely intact, although the predicted risks did not match the observed risks very well for the more extreme risk predictions. Using shrunken coefficients of the original model had a very limited effect and there was no sign of selective loss to follow-up. Finally, accounting for clustering within health care center had little impact.
Comparison with Existing Literature
The few longitudinal studies about frequent attendance showed that attendance rates tend to regress to the mean over time, with only 20–30% of frequent attenders continuing to attend frequently in the following year [4]–[6], [32] However, these studies of persistent frequent attendance used different definitions of frequent attenders and lacked the power to detect factors associated with transient frequent attendance becoming persistent. Vedsted found that psychological distress, as measured with two psychometric scales, increased the risk of future daytime frequent attendance of adult patients in family practice. [16] Another, small, prospective study (85 primary healthcare patients of working-age) detected as risk factors for persistent frequent attendance female sex, body mass index above 30, former frequent attendance, fear of death, alcohol abstinence, low patient satisfaction, and irritable bowel syndrome. [33] Smits et al showed that with the indicators currently present in Dutch electronic medical records, a rule performed modestly in selecting those more likely to become persistent frequent attenders [14].
Implications for Clinical Practice
After validation and updating the existing rule only predicts persistence of frequent attendance moderately. For clinical use this rule has some significance to predict which 1yFAs have more risk to become a persistent FA. We are currently prospectively following a cohort of 623 frequent attenders for three years and hope to be able to improve predictions by incorporating better and more patient-based information, such as socioeconomic status, body mass index, health anxiety/illness behavior, depressive complaints and anxiety. This has to be weighed against increased time and costs to collect such information.
Implications for Future Research
The model may be useful to select populations for RCTs with a higher likelihood of becoming pFA. In addition, prediction models can be used in RCTs to characterize the trial arms at baseline in a multivariable way, thus enhancing the assessment of baseline comparability. [9], [34] Finally, subgroup analyses using the scores from prediction models may serve to move beyond multiple univariable subgroup analyses [9], [35].
Supporting Information
Table S1.
Selected problems and diseases with ICPC-code1.
https://doi.org/10.1371/journal.pone.0073125.s001
(DOCX)
Table S2.
Effect of loss to follow-up and clustering on the health centre level on the prognostic index.
https://doi.org/10.1371/journal.pone.0073125.s002
(DOC)
Acknowledgments
We thank the GPs involved in the Network of General Practitioners of the Academic Medical Centre/University of Amsterdam (HAG-net-AMC) and of the Academic Medical Centre/University Maastricht (SMILE) for their continuous efforts to keep the electronic medical records updated and their support.
Author Contributions
Conceived and designed the experiments: FS GtR AZ. Performed the experiments: FS AZ HB GtR. Analyzed the data: FS HB AZ MvdA BvS JM AS HvW GtR. Contributed reagents/materials/analysis tools: AZ GtR. Wrote the paper: FS HB AZ MvdA BvS JM AS HvW GtR.
References
- 1. Vedsted P, Christensen MB (2005) Frequent attenders in general practice care: a literature review with special reference to methodological considerations. Public Health 119: 118–137.
- 2. Smits FT, Mohrs J, Beem E, Bindels PJ, van Weert HC (2008) Defining frequent attendance in general practice. BMC Fam Pract 9: 21.
- 3. Smits FT, Brouwer HJ, ter Riet G, van Weert HC (2009) Epidemiology of frequent attenders: a 3-year historic cohort study comparing attendance, morbidity and prescriptions of one-year and persistent frequent attenders. BMC Public Health 9: 36.
- 4. Ward AM, Underwood P, Fatovich B, Wood A (1994) Stability of attendance in general practice. Fam Pract 11: 431–437.
- 5. Botica MV, Kovacic L, Tiljak MK, Katic M, Botica I, et al. (2004) Frequent attenders in family practice in Croatia: Retrospective study. Croatian Medical Journal 45: 620–624.
- 6. Carney TA, Guy S, Jeffrey G (2001) Frequent attenders in general practice: a retrospective 20-year follow-up study. Br J Gen Pract 51: 567–569.
- 7.
Andersson SO, Lynoe N, Hallgren CG, Nilsson M (2004) Is frequent attendance a persistent characteristic of a patient? Repeat studies of attendance pattern at the family practitioner. Scand J Prim Health Care: 91–94.
- 8.
Wagner M, Balk EM, Kent DM, Kasiske BL, Ekberg H, et al.. (2009) Subgroup analyses in randomized controlled trials: the need for risk stratification in kidney transplantation. Am J Transplant: 2217–2222.
- 9.
Kent DM, Hayward RA (2007) Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA: 1209–1212.
- 10.
Smits FT, Wittkampf KA, Schene AH, Bindels PJE, Van Weert HCPM (2008) Interventions on frequent attenders in primary care. Scandinavian Journal of Primary Health Care Vol 26 (2), –116.
- 11. Katzelnick DJ, Simon GE, Pearson SD, Manning WG, Helstad CP, et al. (2000) Randomized trial of a depression management program in high utilizers of medical care. Arch Fam Med 9: 345–351.
- 12. Simon GE, Manning WG, Katzelnick DJ, Pearson SD, Henk HJ, et al. (2001) Cost-effectiveness of systematic depression treatment for high utilizers of general medical care. Arch Gen Psychiatry 58: 181–187.
- 13. Bellon JA, Rodriguez-Bayon A, de Dios LJ, Torres-Gonzalez F (2008) Successful GP intervention with frequent attenders in primary care: randomised controlled trial. Br J Gen Pract 58: 324–330.
- 14. Smits FTM, Brouwer HJ, ter Riet G, van Weert HC (2009) Predictability of persistent frequent attendance. A historic 3-year cohort study. Br J Gen Pract 2–2009: 114–119.
- 15.
Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, et al.. (2010) Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 128–138.
- 16. Vedsted P, Fink P, Olesen F, Munk-Jorgensen P (2001) Psychological distress as a predictor of frequent attendance in family practice: a cohort study. Psychosomatics 42: 416–422.
- 17. Jyvasjarvi S, Joukamaa M, Vaisanen E, Larivaara P, Kivela S, et al. (2001) Somatizing frequent attenders in primary health care. J Psychosom Res 50: 185–192.
- 18.
van den Akker M, Spigt MG, De Raeve L, van Steenkiste B, Metsemakers JF, et al.. (2008) The SMILE study: a study of medical information and lifestyles in Eindhoven, the rationale and contents of a large prospective dynamic cohort study. BMC Public Health: 19–20696755.
- 19.
Lamberts H, Wood M, eds (1988) International classification of primary care. Oxford: Oxford University Press.
- 20. Brouwer HJ, Bindels PJ, van Weert HC (2006) Data quality improvement in general practice. Fam Pract 23: 529–536.
- 21. Robbins JM, Kirmayer LJ, Hemami S (1997) Latent variable models of functional somatic distress. J Nerv Ment Dis 185: 606–615.
- 22.
Hosmer DW, Lemeshow S, Sturdivant RX (2013) Applied logistic regression, 3 rd edition. New York: John Wiley & Sons.
- 23.
Lemeshow S, Hosmer DW Jr (1982) A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol: 92–106.
- 24.
Paul P, Pennell ML, Lemeshow S (2012) Standardizing the power of the Hosmer-Lemeshow goodness of fit test in large data sets. Stat Med 2012: Jul.
- 25. Bellon JA, Delgado-Sanchez A, de Dios LJ, Lardelli-Claret P (2007) Patient psychosocial factors and primary care consultation: a cohort study. Fam Pract 24: 562–569.
- 26. Thiru K, Hassey A, Sullivan F (2003) Systematic review of scope and quality of electronic patient record data in primary care. BMJ 326: 1070.
- 27. Wennberg JE (1987) The paradox of appropriate care. JAMA 258: 2568–2569.
- 28. Wennberg JE (1985) Practice variations: why all the fuss? Internist 26: 6–8.
- 29. van den Dungen C, Hoeymans N, Gijsen R, van den Akker M, Boesten J, et al. (2008) What factors explain the differences in morbidity estimations among general practice registration networks in the Netherlands? A first analysis. The influence of population characteristics on variation in general practice based morbidity estimations. BMC Public Health 14: 53–62.
- 30. van den Dungen C, Hoeymans N, Boshuizen HC, van den Akker M, Biermans MC, et al. (2011) The influence of population characteristics on variation in general practice based morbidity estimations. BMC Public Health 2011: 887.
- 31. Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 2000: 453–473.
- 32. Andersson S-O, Lynoe N, Hallgren C-G, Nilsson M (2004) Is frequent attendance a persistent characteristic of a patient? Repeat studies of attendance pattern at the family practitioner. Scandinavian Journal of Primary Health Care 22: 91–94.
- 33. Koskela TH, Ryynanen OP, Soini EJ (2010) Risk factors for persistent frequent use of the primary health care services among frequent attenders: a Bayesian approach. Scand J Prim Health Care 28: 55–61.
- 34.
Hayward RA, Kent DM, Vijan S, Hofer TP (2006) Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis. BMC Med Res Methodol: 18.
- 35. Kent DM, Lindenauer PK (2010) Aggregating and disaggregating patients in clinical trials and their subgroup analyses. Ann Intern Med 2010: 51–52.