Article Text
PDF
Statistics from Altmetric.com
Background
Performance measurement (PM) and management for quality have become ubiquitous in 21st-century healthcare. Numerous entities have independently developed measures for assessing mortality, quality of chronic-disease care, access and patient satisfaction. Consequently, measures have mushroomed; for example, the National Clearinghouse for Quality Measures houses nearly 1100 active measures.1 Despite this proliferation, those whose performance is being measured have had little input in measure development. Research consistently shows when performance measurement systems are implemented by leadership divorced of the evidence-based motivational component that induces goal commitment and facilitates behaviour change, these do not accelerate performance improvement.2 3
For example, traditional PM systems like the Healthcare Effectiveness Data and Information Set have focused on disease prevalence yet lack a conceptual model of the clinical-performance criterion domain. Outside of healthcare, performance measurement systems such as Total Quality Management (TQM), Lean/Six Sigma and Balanced Scorecards are based on conceptual models about the nature of quality and performance, yet lack bottom-up motivational features, leading to mixed success and sustainability.3 4 For example, TQM and Lean methods assume that people inherently want to contribute to quality, though quality must be driven by committed senior leaders responsible for quality.5 However, psychological research indicates goal-directed behaviour change is strongest when people are committed to their goals.6 Thus, TQM and other systems often lack the motivational component that facilitates the transition from receiving performance information to inducing desired behaviour change. Indeed, recent research suggests systems such as Lean have had mixed success and long-term sustainability.2 3
For example, although increases in productivity and reduction of waste (a signature outcome of Lean) associated with using Lean Six Sigma have been well documented in healthcare, implementation and sustainment has been challenging due to tensions between “the need to demonstrate efficiency and achieve performance targets… and the need to invest time and resources in embedding a culture of continuous improvement”.7 The result has been a host of unintended consequences such as denominator management, ‘box-checking’, clinician demotivation and increased facility costs due to the cost of measurement (especially among unactionable measures), especially among unactionable measures such as those with little variance due to ceiling or floor effects (ie, when performers consistently reach the highest or lowest possible score on a measure, and thus measurement no longer provides any useful, actionable information).8–14
Clinician motivation plays as important a role as scientific validity in eliciting behaviour change through performance measurement. Without engaging those doing the clinical work in articulating the objectives of a clinical specialty and the work required to achieve them, it is difficult to design motivating PM systems. Thus, what is needed is a system for eliciting what the clinic values; tracking progress towards those valued goals; and using that information to inspire, rather than oblige, clinicians to deliver high-quality care.
Applying motivational theory to performance improvement
The idea that organisations can influence individual motivation to improve performance has grounded the last 50 years of motivational theory,15 including approaches such as expectancy, goal-setting and team-motivation theories. Related paradigms such as behavioural economics16 and the Ikea effect (the idea that individuals place more value on products or initiatives they helped create)17 concur with these notions. For example, according to the Naylor–Pritchard–Ilgen theory of motivation,18 the perceived relationship between applying energy to actions and the resulting need satisfaction influences how much energy is devoted to that action. Thus, clinicians are more likely to deliver high-quality care when they see a clear connexion between what they do to deliver care and the value to the organisation of the outcome of their work. This is consistent with other perspectives on individual work motivation aiming to explain the drivers of direction and effort of energy, such as goal setting,6 expectancy theory,19 team motivation perspectives20 and behavioural economics.16 21 The goal is to give clinicians the information they need to improve the quality of their care by making explicit the value of given outcomes, and providing clinicians feedback about both their level of performance and the value of performing at that level.
An example of a PM methodology that capitalises on motivational theory to clarify objectives and their value is the Productivity Measurement and Enhancement System (ProMES).22 ProMES is a structured, focus group–based approach using intact work teams to identify values-based performance objectives, objective-based performance indicators and value assessment for given levels of performance. It uses a ‘bottom-up’ strategy for developing performance objectives and indicators. Work-unit (eg, clinic) members identify performance objectives for the units, within the context of the larger organisation’s objectives. For example, patient-centredness has not always been a central organisational objective in the past. With patient-centredness garnering more attention at the organisational level, primary care teams consider patient-centredness (an organisational objective) as they decide what their team’s performance objectives will be. The team’s performance objectives then become the compass for creating performance measures and identifying the relative value of each; the team co-creates these in partnership with senior leadership, capturing all important aspects of daily work. This alignment helps teams prioritise their work, improve engagement and productivity, reduce stress and mitigate waste. Users then receive performance feedback through facilitated debriefs, where staff collectively problem-solve to optimise performance. Table 1 summarises the ProMES process; more details on the ProMES process are available in online supplementary appendix A.
Supplemental material
Description of the ProMES5 methodology in the healthcare context
ProMES has been used in numerous work settings including healthcare; a meta-analysis found significant increases in productivity effect sizes 
after ProMES implementation compared with baseline.23 In addition, they found several effect modifiers associated with greater effect sizes than found by the main effect: (1) Type of organisation: ProMES had best results in service organisations (eg, healthcare, d=1.46) compared with others such as sales or manufacturing; (2) Type of worker: greater effect sizes were observed for academic and managerial (d=2.02), and technical workers (d=1.78) than for blue collar (d=1.06) or clerical workers (d=0.33); and (3) Centralisation: ProMES worked better in organisations that were centralised rather than decentralised (r=0.39). More recently, a randomised controlled trial demonstrated ProMES can be used effectively for stress prevention in primary care,24 and other work has shown effectiveness in sports settings.25 A recent review of the last 100 years of performance appraisal and performance management literature concluded ProMES was one of the few performance management systems with documented, empirical, peer-reviewed evidence of effectiveness.26 The alignment between performance and motivation helps individuals and teams to more clearly focus their effort on the most important aspects of their work resulting in greater productivity, reduced stress and less waste of effort. Many of the characteristics examined by the meta-analysis are present in healthcare settings; thus, a theory-based, motivationally centred performance system such as this one shows potential to yield good success when applied to healthcare.
Case example from the Veterans Health Administration (VHA)
Although ProMES’s primary product is a PM system, ProMES can also be used to impact other organisational needs including employee engagement, behaviour change and quality improvement. Below, we illustrate such applications using a research study of coordination in primary care teams as a case example.
Study overview: coordination within primary care teams
Many national-level organisations note the need for deliberately coordinated care, hence the widespread adoption of primary-care teams in healthcare organisations. To deliver well-coordinated care, however, teams that coordinate patient care must successfully coordinate among themselves: that is, teams must be able to successfully sequence and route interdependent tasks to the correct team members, while still maintaining collective situational awareness.27 We partnered with two VHA facilities to determine the extent to which ProMES could be used to improve coordination via three objectives, each associated with a different potential application of ProMES: (1) develop measurable, prioritised point-of-care criteria for effective team coordination (application: performance measurement); (2) identify information needs at the point of care to optimise coordination (application: quality improvement); and (3) assess the effect of adopting the aforementioned criteria on team members’ coordination behaviours (application: behaviour change). A complete protocol describing the study’s methods, including how ProMES was used in the study, is published elsewhere28; method highlights and deviations from the published protocol are described below.
Application 1: PM
To develop the PM system described in steps 1–4 of table 1, we partnered with an intact primary care team consisting of one of each of the following roles: physician, registered nurse, licensed practical nurse, a medical support assistant, a dietician, social worker, pharmacist, nurse practitioner, psychologist, health promotion coordinator and clinical information technology coordinator. Two research team-members served as facilitators. To generalise the system beyond this team’s needs, an advisory team of clinicians and leadership from other VHA sites provided feedback after each major ProMES step. The team identified two objectives specific to coordination and eight corresponding indicators (see box 1). One interesting outcome of these efforts resulted from the contingency-development process. All performance indicators identified already existed within VHA, though usually as part of a far larger set of unit-weighted metrics. The contingencies process helped the design team identify metrics of greater value to the facility than others, and were worth greater time and effort to improve than those of lower value (contingency curves for the final indicators appear in figure 1). Through contingency development, which made the metrics directly comparable, the design team also identified measures that added little new information, and thus could be combined or deleted from the system. Finally, the process highlighted the important point that performance and effectiveness or value to the organisation are not necessarily linearly related, and thus the teams’ priorities can change depending on their performance level. These efforts yielded a prioritised set of metrics, clearly linked to VHA’s top-value priorities.
Resulting objectives and coordination indicators
Objective 1: Support and foster patient engagement in their wellness by being patient-centred
Percentage of patients enrolled in electronic secure messaging portal. Enrolment.
Percentage of patient education classes filled (education offerings use).
Score on patient satisfaction survey.
Percentage of new patients attending new patient orientation (new patient orientation utilisation).
Objective 2: Ensure quality and efficient care is provided to the patient
Percentage of patient appointments that start on time.
Percentage of clinical reminders completed.
Percentage of timely recall appointment scheduling.
Reliance on emergency room care by current Patient Aligned Care Team (PACT) patients.
Contingency curves for coordination indicators identified by design team.
Application 2: informing quality improvement initiatives
After creating the PM system, we conducted three focus groups with the design and advisory teams to attempt to answer the question: “what are the team’s information needs at the point of care to optimise coordination?” This step is not part of the original ProMES methodology but rather a special application: Having just identified coordination objectives, indicators and contingencies, answering the question of information needs in a systematic way became a much more straightforward task for the design team than using a traditional focus group approach. The design and advisory teams reviewed each indicator and identified information and process-change needs to improve performance in each metric. Reviewing the design/advisory teams’ responses across indicators revealed some commonalities (see figure 2). Several metrics required basic patient information, such as contact and literacy information, as well as their history summary. However, the remaining information needs (eg, knowing what a patient’s available options are) centred on a need to improve common understanding within the team, between the team and the patient, and between the team and other providers outside the team. The teams’ recommendations for improvement followed similar themes; additionally, the design team recommended improving performance on those indicators serving as intermediate outcomes, as they predicted it would likely have a spillover effect on the remaining indicators.
Design and advisory team recommendations for improving coordination at the point of care.
Application 3: behaviour change
Once a ProMES-style measurement system is complete, it can be used as a source of audit and feedback and motivation to foster clinician engagement and facilitate behaviour change in healthcare teams. In our study, we created ProMES-style feedback reports for 34 primary care teams housed at four VHA healthcare locations in the Midwest and Southwestern USA; we monitored their performance on their ProMES indicators over 7 months, along with performance from a matched control arm of 34 teams representing five facilities who received no intervention (performance was followed passively). The original design team did not receive the intervention, as we were interested in exploring the scalability of ProMES beyond the design team’s contributions.
The report comprised a given team’s score on their performance metrics, delivered monthly (consistent with the indicators’ natural periodicity). Online supplementary appendix B illustrates the dashboard presented to participating primary care teams. Several evidence-based features of this feedback report are important to its effectiveness, including their current score and velocity (ie, change from previous period) and historical information as well as goal-specific targets for the upcoming period.29 Most importantly, the metrics are displayed in decreasing order of priority based on maximising gain in value to the organisation; contingency curves show, for each indicator, how much value is gained for each additional percentage point of improvement on each indicator. Once trained on the report features, users can assess previous and current performance, and prioritise where they most need to focus their efforts moving forward. Readers interested in an illustration of the feedback report are encouraged to review online supplementary appendix B, which presents a complete example with interactive annotations. Each team discusses their report in a monthly meeting aimed at improving coordination, identifying tasks the team should start, stop and continue doing in their work so that coordination improves. Research indicates regular debriefing such as this is one of the most powerful tools for improving team performance, in part because it provides opportunity for teams to reflect and self-correct, thereby eliciting behaviour change.30
Supplemental material
Application 4: employee engagement and team performance
Because ProMES is grounded in motivational theory,18 several components foster employee engagement; involving job incumbents rather than relying solely on managers and leadership creates a foundation for engagement from the beginning. As an example, in our study one of the design team members cancelled her vacation day voluntarily in order to attend the design team session and help develop the indicators being worked on that day.
Another opportunity for engagement is through feedback and debriefing, and is particularly useful for teams. Regular debriefing effectively improves team performance because it helps teams reflect and self-correct30–32 and also because teams who reflect together are more likely to develop shared mental models, which have been shown to improve engagement and performance by helping establish shared understanding, increasing information sharing and creating a focused strategy, thus improving engagement and performance.33 In our study, teams debriefed monthly for 15–20 min with a facilitator, focusing on team needs for successful coordination and identifying two practices to start, stop and continue doing. This focused engagement provided the motivational satisfaction of making meaningful contributions to the team and specific guidance for improving performance. For example, we initially observed resistance from several teams regarding the content of the feedback report. Team members wanted greater detail about how the measures were operationalised and attempted to defend their respective performance levels on specific indicators by critiquing different aspects of the measure definitions. During the first two debriefings, the facilitator spent approximately 15–20 min each session over and above the originally allotted debriefing time of 15–20 min building psychological safety and facilitating team engagement by walking the participants through the feedback report in detail, explaining the developmental nature of the report, and reminding participants that it was intended as a developmental tool for improving coordination broadly rather than administrative monitoring of those specific indicators. After approximately the third monthly meeting, the facilitator observed an unprompted shift in these teams’ behaviour and discussions: questions about the report began to decrease, and discussion shifted towards indicator-independent tasks that addressed coordination as a whole rather than action items to address indicator-specific performance.
In addition to these motivational shifts, we also observed improvements in team coordination. Compared with baseline, we saw improvements in clinical reminder completion, secure messaging enrolment and a protective effect against increased emergency room use. These effects, however, were dose dependent: we observed these effects only in those teams who consistently attended their debriefings, and only in teams whose members were not stretched out thin across multiple teams.34
Discussion and conclusion
We illustrated how a systematic, motivationally based approach to ground-level participation in performance measurement can be used successfully to identify clinical performance indicators that align with healthcare's organisational goals, help prioritise where to focus, and provide motivational impetus for clinicians to improve and work as a team. We also presented four applications of this approach as part of an applied effectiveness and implementation test of this system in a large primary care setting.
Our experience with this approach in clinical settings, as illustrated by this case example, suggests to us it is both feasible and desirable. We have already begun to see examples of greater engagement in some participating teams; for example, teams have begun to make tangible changes to their workflow processes, such as empowering individual team members to complete relevant clinical reminders as they appear rather than waiting for physician orders. However, it is important to note that fidelity of implementation has been closely associated with the effectiveness of this system23; in particular, the early work of the design team is thought to play an important motivational role in aligning organisational priorities and values with the daily work of workers. Thus, questions remain as to whether elements of a system such as ProMES can be adapted onto an existing performance measurement system and whether this method is associated with teamwork and improved workplace culture, and is a topic for future research.
In summary, when implemented successfully, motivationally based, bottom-up PM can be used successfully to align clinical performance indicators with healthcare’s value goals, prioritise effort, and inspire clinical teams to work together and improve. We believe healthcare must tame the proliferation of independent clinical performance measures and find ways to engage clinicians using a small, meaningful and motivating set of indicators that make sense for the situation. Such a change is consistent with the principles of system innovation,35 36 which advocates for making innovations to the relationships, rather than the parts, of a complex system in order to make large, system-level change: empowering the people who do the work to drive how they are evaluated, what they are held accountable to, and how they their work aligns with the larger goals of healthcare fundamentally shifts healthcare’s relationship with performance measurement systems from one of oversight to one of continuous learning and innovation. Without such a transformation, we foster excessive oversight such as the current proliferation of measures and administrative busywork such as responding to electronic alert notifications to ensure administrative credit for their work.37 We risk spending more time documenting care than providing it and risk compelling clinicians to leave healthcare entirely.38
References
Footnotes
Funding This study was funded by Health Services Research and Development grant nos. CRE 12-035 and CIN 13-413.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement This viewpoint manuscript reports no findings of any data. A complete study protocol from the larger trial described in this manuscript (ISRCTN15412521) is published elsewhere (DOI: 10.1186/s13012-015-0335-9). Data from the study described in this protocol are available in a form compliant with the funder’s data-sharing policies, by written request to the corresponding author.