Effectiveness and Efficiency of Root Cause Analysis in Medicine

Preventable mistakes are common in medicine. For example, at 1 hospital, a patient received patient-controlled analgesia (PCA), a combination of local anesthetic and narcotic. The medication was intended to be infused into the epidural space. Instead, a nurse inadvertently connected the tubing to an intravenous catheter, delivering potentially lethal anesthetic into the patient's bloodstream. What followed were the nurse's anguish and guilt and, almost as inevitably, the hospital's root cause analysis (RCA). In the last decade, this process has become the main way medicine investigates mistakes and tries to prevent future mistakes. But like many innovations in medicine, RCA has never been evaluated for effectiveness.

In the case mentioned above, the team identified flaws in the design of the epidural catheter, but thought that fixing those flaws was beyond their scope. Therefore, they made a recommendation they could implement: reeducating staff about the equipment's use. In the end, despite a significant investment of resources, this solution did not remove the underlying hazard and had little effect outside the institution. No one had confidence that things were safer. Indeed, since 1999, the US Pharmacopeia has received 1600 reports of epidural-to-intravenous misconnection (MEDMARX data in file, USP 2007). Many of these incidents undoubtedly received their own RCAs, but the mistake continues to occur.

Root cause analysis was originally developed in psychology and systems engineering to identify “the basic and causal factor(s) that underlie variation in performance.”¹ It provides structure to the retrospective analysis of errors and has been used successfully for decades to uncover latent errors in high reliability organizations, such as aviation and nuclear power.^{2 - 3} Root cause analysis is now a familiar tool for hospitals and health care organizations and has helped to identify many problems and solutions.^{4 - 5} The RCA process is designed to answer 3 basic questions: what happened, why did it happen, and what can be done to prevent it from happening again?⁶ What is missing in medicine is a fourth question: has the risk of recurrence actually been reduced? The fact that it generally is not known whether risk has been reduced is causing concern that some of the considerable resources and efforts expended on RCA are being wasted.

Pioneers, including Bagian at the US Department of Veterans Affairs (VA) and Croteau at the Joint Commission, first introduced RCA to the medical community in the mid-1990s. The VA and Joint Commission each developed their own programs, replacing older review methods with a more systematic approach. The Joint Commission now requires organizations to perform an RCA for every sentinel event. In the VA system, facilities submit RCA reports for serious adverse events to the National Center for Patient Safety. Currently, 25 states require reporting of adverse events to the state health department.⁷

However, among those states, definitions of adverse event vary, as do reporting requirements. The components in an RCA report also vary, both across and within organizations conducting the analysis.⁷ The VA requires that each RCA contain recommended corrective actions and a plan to verify that the action has the intended effect.³ Follow-up on these corrective action plans is left to the individual facilities. The Joint Commission asks health care organizations to create their own definition of sentinel event and voluntarily report such events to the commission. Reports must contain an action plan and measurement strategy and, in some cases, the Joint Commission follows up on outcomes. Ten states mandate RCA, and all require a concurrent action plan, but there is little routine follow-up on these plans. Within individual health systems, requirements vary widely.

To date, more than 7000 RCAs have been performed at the VA, nearly 4100 submitted to the Joint Commission,^{6 ,8} and countless more submitted to state health departments and health care systems nationwide. Experts estimate that each RCA requires 20 to 90 person-hours to complete.

At the VA, comparison of RCA with the previously used focused review showed a shift in the root causes identified, blaming individuals less and increasingly attributing the problem to systemic causes like communication and policies or procedures. However, the study did not assess the effect and outcomes of corrective actions.⁹ In studies of what happened following recommendations, full implementation occurred in 61.4%¹⁰ to 68.1%¹¹ of the cases with partial implementation in another 20%.¹⁰ In a survey of professionals conducting RCAs, 69.2% felt their recommendations were at least partly implemented.¹²

Not all actions aimed to mitigate risk are equal. Some actions, like redesigning a product or process, are strong and have a high probability of reducing harm. Other actions, like reeducation or writing a policy, the 2 most common recommendations in health care RCA, are weak and have a low probability of reducing risk. The study by Mills et al¹⁰ estimated that strong actions accounted for less than half of all actions taken.

Although a structured approach to investigating and mitigating hazards is desirable, there are problems with RCA. Many RCAs are performed incorrectly or incompletely and do not produce usable results. Experience suggests that in many organizations, emphasis is placed inappropriately on uncovering the single “most fundamental reason”⁴ for error.² Organizations tend to approach each RCA independently, rather than drawing lessons across investigations. Anecdotally, officials in state health departments observe that the quality of the RCAs they receive varies widely. Examination of RCAs from 7 practice regions in England suggested exemplary practice in 2, less rigor in 3, and “scant evidence of recognizable features of RCAs”¹³ in 2. Practitioners conducting RCAs report barriers, including a lack of time, resources, data and feedback, uncooperative colleagues, difficulty with teams, interprofessional differences, and unsupportive management.¹²

Formulating corrective actions is more difficult than finding problems, and follow-up on outcomes is rare. A sign of the incomplete adoption of recommendations is that despite having recently completed an RCA for a specific incident, hospitals commonly experience repeat events, which is a reminder of words attributed to Einstein, “Insanity is doing the same thing and expecting a different result.”

There are no studies in peer-reviewed literature on the effectiveness of RCA in reducing risk or improving safety, and there are no evaluations of the cost or cost-effectiveness of the procedure compared with other tools to mitigate hazards. Although the VA, Joint Commission, and others teach standard methods and have developed tools to help standardize reports,⁶ best practices have not been established for recommendations for action, follow-up, and analyzing results. Evaluation is hindered further by lack of validated measures for risk reduction in risk or safety improvement.

Recommendations may be aimed at the wrong level of the health care system. If events are common across hospitals, the remedial action should be designed at the health system rather than individual hospital level. In the PCA case, for example, in which human interaction with the device results in patient harm, it would be more effective to change the device than to attempt to educate the clinicians in more than 6000 US hospitals that use PCAs.

But without a collaborative effort of stakeholders, including manufacturers to correct the problem, as well as a higher oversight body that could enforce such an effort, hospitals often can only address the problem within their institution, using weaker interventions. Ironically, the resulting costs may be greater than for a higher-level solution, despite a lower probability of success.

A high-level intervention in the PCA case described above would have several requirements. First, the extent of the problem would have to be documented, using evidence of similar incidents across different institutions. Currently, tools to help raise the quality of RCAs and facilitate aggregation are available, but are underutilized.¹⁴

The second requirement would be reaching consensus among relevant stakeholders on the level and type of intervention. For example, for epidural catheters, after identifying a range of potential solutions at different levels of the system, it would be evident that the most effective and efficient solution would be to redesign the equipment. However, the framework is lacking by which to identify which types of incidents lend themselves to which types of interventions, and at which level of the health care system to intervene.

The third requirement is a forum to convene representatives of manufacturers, professional societies, health care organizations (especially hospitals), and end users to agree on appropriate redesign. This group would need sufficient purchasing power to entice manufacturers, as well as technical and clinical expertise to redesign wisely. Steps are being taken to develop such a group of stakeholders (the “orange-wire group”) that could implement this model.¹⁵ When an RCA is conducted as intended, these requirements can be achieved within the VA system. An important function of the National Center for Patient Safety is to review and aggregate individual reports.¹⁶ When this is done for a specific problem, it is then possible to enforce recommendations or disseminate advisories and alerts throughout the VA,¹⁷ or to initiate negotiations with suppliers. Still, only a handful of alerts are issued each year and negotiations with more than 1 manufacturer at a time are unusual. Although the Joint Commission can potentially offer similar oversight, it operates with a less unified group of organizations and suppliers.

To follow through completely, additional steps would be needed. It would be critical to pilot test the intervention, monitor for unintended consequences, revise and broadly implement, and evaluate the effect and outcomes of the program.

The model described provides a good fit for incidents and interventions related to medical devices and other manufactured products. There are likely to be different solutions for problems related to performance of specific tasks, communication, or training and education, although the same principles may apply. For example, substantial effort is required to develop a training program for performing a procedure, such as central line placement.¹⁸ It is inefficient for each individual organization to develop its own programs. It would be better for professional societies to develop these programs for widespread use. National and international leadership is needed to organize this effort.

Root cause analysis has been widely adopted as a central method to learn from mistakes and mitigate hazards. Although there have been some benefits, including increased awareness of faulty processes and fixes to specific problems, there is an undercurrent of sentiment that this approach has limited effectiveness. Many health care organizations, particularly smaller ones, may be spending a substantial portion of their quality improvement resources on interventions that have little chance of diminishing risks or harms. The next step is to evaluate RCA processes for effectiveness and utility, and make RCA more useful. More emphasis should be placed on understanding variations in the implementation of RCA and developing a greater evidence base for the best way to conduct them. Follow-up for implementation and outcomes should become a standard element of the process. At a minimum, better measures are needed to evaluate RCAs directed at specific, prevalent problems. It is imperative to develop mechanisms to implement the intervention with the highest probability for success. A national oversight body would have the latitude and leverage to ensure best practices for conducting investigations, instigate high-level discussions and negotiations, and track results to ensure that clinicians and health care organizations learn from errors and adverse events.