Can complex health interventions be evaluated using routine clinical and administrative data? - a realist evaluation approach

Rationale, aims and objectives: Interventions aimed at improving chronic care typically consist of multiple interconnected parts, all of which are essential to the effect of the intervention. Limited attention has been paid to the use of routine clinical and administrative data in the evolution of these complex interventions. The purpose of this study is to examine the feasibility of routinely collected data when evaluating complex interventions and to demonstrate how a theory-based, realist approach to evaluation may increase the feasibility of routine data.

Methods: We present a case study of evaluating a complex intervention, namely, the chronic care model (CCM), in Finnish primary health care. Issues typically faced when evaluating the effects of a complex intervention on health outcomes and resource use are identified by using routine data in a natural setting, and we apply context-mechanism-outcome (CMO) approach from the realist evaluation paradigm to improve the feasibility of using routine data in evaluating complex interventions.

Results: From an experimentalist approach that dominates the medical literature, routine data collected from a single centre offered a poor starting point for evaluating complex interventions. However, the CMO approach offered tools for identifying indicators needed to evaluate complex interventions.

Conclusions: Applying the CMO approach can aid in a typical evaluation setting encountered by primary care managers: one in which the intervention is complex, the primary data source is routinely collected clinical and administrative data from a single centre, and in which randomization of patients into two research arms is too resource consuming to arrange.