Teaching Quality Improvement: Title and subTitle BreakThe Devil Is in the Details

From the list of 14 characteristics he assembled in 1980, Cyril Houle, a serious student of professional education, singled out self-development as professionalism's one unchanging feature; as he put it, “[A]s an occupational group raises its level of performance . . . its right to call itself a profession increases.”¹ Houle's choice of words warrants serious attention: a profession is defined by what it does, not just what it knows, and by doing what it does better all the time, not just doing it well.

Medicine's enormous and highly specialized body of knowledge inarguably makes it a learned profession, but delivering care is performance, not scholarship. In its most basic form, the delivery of science-based care consists of applying generalizable scientific knowledge to the problems of particular patients who are ill, with the result that those patients are better off. Learning to deliver care therefore means going beyond conceptual knowledge (“knowing that”) to the acquisition of working knowledge (“knowing how”).²

Experiential learning, the main source of working or “how to” knowledge, is a 4-element cycle (or spiral) in which learners move from involvement in new experience, to reflection on that experience, integration of those observations with sense-making concepts and mental models, then back out to try again; or more briefly: experience, observe, conceptualize, and retry.³ Experiential learning differs from conceptual learning in fundamental ways. It is case-based rather than concept-based and requires hands-on practicum (or simulation) experiences in addition to written end objectives. Moreover, experiential learning involves expert coaching rather than knowledge transfer from experts, requires repeated experiences and evaluations over time (practice, practice, practice) rather than initial 1-shot exercises, and enables learners to acquire the skill of reflection-in-action rather than a memory full of abstract principles.⁴

Since William Osler's introduction of clinical clerkships more than 100 years ago, medical education has gotten quite good at providing the kind of fully realized experiential learning that prepares clinicians to deliver competent clinical care; there is little doubt therefore that medicine satisfies Houle's performance criterion of a profession.⁵ But although science-based medicine usually is successful in doing what is needed to make patients better off—sometimes spectacularly so—the dark side of medicine is all too often in evidence: interventions are overused, underused, or used inappropriately; care is frequently inaccessible, inefficient, even harmful.^{6 - 7} Medicine, it seems, has been less successful in meeting Houle's second criterion: continuously raising its level of performance.

Practice experience does provide clinicians with the opportunity to improve progressively at translating clinical evidence into practice,⁸ but it is increasingly clear that the quality of many individual clinicians' performance decreases with time rather than improving, as measured against specific diagnostic and therapeutic standards.⁹ These sobering findings underscore the point that experiential learning is effective only when it includes all 4 elements of the cycle. Experience without adequate reflection and conceptualization is a recipe for stagnation; reflection and conceptualization without repeated, hands-on experience fosters pedantry.

Medical education has responded in important ways to the challenge of helping trainees learn to raise their own individual performance, not just how to perform. For example, over the past 40 years problem-based learning has largely replaced passive knowledge transfer (the time-honored way of teaching basic biological concepts) with active, case-based, individual and small group learning facilitated by faculty, with the expectation that in the process students will learn as much about effective self-directed and team-based learning as they learn about biology.¹⁰ Teaching the practice of evidence-based medicine (systematic and explicit linkage of specific clinical questions about individual patients and groups with the best available research evidence) is intended to make reflection on experience and conceptualizing better management as much a part of every clinical encounter as taking a history and performing a physical examination.

The teaching of evidence-based medicine addresses only one of the many challenges involved in raising the level of clinical performance.¹¹ Most importantly, all care of individual patients by individual practitioners is embedded in a complex, nested set of human organizations¹² ; and because (as is often said) every system is perfectly designed to get exactly the results it gets, medicine's many shortcomings can be attributed as much to inadequate human organizations (dysfunctional systems) as to the deficiencies of individual clinician performance.

Not all changes are improvements but all improvement involves change. Changing the systems that deliver care has thus become the cornerstone of the movement that is now referred to as medical quality improvement (QI). In essence, raising medicine's level of performance by changing care systems is a process of experiential learning. However, QI often involves groups of people rather than individuals, and therefore requires a special and relatively formal version of the experiential learning cycle that includes an initial planning step and in which the observe and conceptualize steps are combined. The result is the plan-do-study-act cycle that is now widely used in implementing small, repeated tests of change in care systems. The working knowledge involved in implementing useful tests of change is complex. It requires knowing how to recognize when a system is dysfunctional; understand the particular practice setting or context, which makes it possible to diagnose the sources of dysfunction; set realistic aims; find or devise innovations; adapt innovations to the local context; redesign the system; execute changes; measure the impact of the changes; and use outcome measurements to refine and adapt the changes further.¹³

The systematic review by Boonyasai and colleagues¹⁴ in this issue of JAMA examines published reports of efforts to help learners acquire the working knowledge needed for effective QI. Most systematic reviews and meta-analyses are designed to strengthen the evidence on the efficacy of a particular intervention (usually a sharply defined diagnostic or therapeutic tool) by aggregating data from multiple studies of the same (or similar) intervention; heterogeneity (of participants, study settings, and exposure to the intervention) is therefore undesirable, and its effects are minimized by using statistical techniques such as random-effects models.¹⁵ The present review stands the usual review method on its head by specifically focusing on heterogeneity; it asks the study question of what learning principles and curricular features influence the effectiveness of efforts to teach QI.

Boonyasai et al found that many of the educational programs studied appeared to be effective in improving knowledge about QI, attitudes toward health systems, and participation in QI activities. But they also noted that on the whole, programs improved knowledge about QI (a surrogate variable) more often than clinical outcomes; that fewer controlled studies than noncontrolled studies showed only positive (rather than mixed or negative) outcomes; that clinical benefits do not occur when learner behaviors do not change (not exactly a surprise); and (somewhat surprising) that the use of adult learning principles apparently does not improve educational outcomes.

More interestingly, the authors found that clinical outcomes were more likely to improve with teaching methods appropriate for experiential learning (providing learners with a package of QI tools and individualized coaching in QI methods) than with other interventions (such as audit or feedback and more limited QI training). Moreover, providing learners with robust outcomes data (access to preexisting performance data, particularly when curricula are set in data-rich inpatient environments) can improve outcomes. Particularly important was the finding that exposing learners to the specialized QI experiential learning cycle (implementing and testing interventions through several small cycles of change) results in better outcomes than when learners undertake single comprehensive interventions.

In short, the aggregate of published evidence, while not strong, is consistent with the view that learning how to do QI and actually carrying out QI are essentially one and the same; both are special forms of experiential learning. The authors make the important point that future assessment of educational effectiveness in QI will require published reports of educational rationales, curriculum features, teaching settings and methods (including learner attendance), and tools for assessing educational outcomes that are more complete, accurate, and transparent than they are now. Indeed, Boonyasai et al's discussion of these issues provides an excellent preliminary outline for a set of publication guidelines for such educational work analogous to those recently proposed for reporting original studies of QI interventions.¹⁶

The review by Boonyasai et al also highlights professional education as an essential element in the broader improvement landscape. In dealing with QI, physicians and others who work in QI have developed the unfortunate habit of being concerned with better patient (and population) outcomes, better system performance (the effectiveness and efficiency of care processes), and better professional development as separate and distinct entities. But unless everyone, including those who hold the norms of health care professionalism—health professional faculties, accreditors, certifying boards, and professional societies—can bring all of these elements together seamlessly to reinforce and enrich each other; unless everyone who works in health care recognizes that they have 2 jobs when they come to work every day, ie, doing the work and improving it,¹³ medicine is likely to have difficulty meeting Houle's second criterion for judging a profession: continuous movement toward new levels of performance.