Comparison of the Canadian CT Head Rule and the New Orleans Criteria in Patients With Minor Head Injury FREE

Each year, physicians in Canadian and US emergency departments (EDs) treat more than 8 million patients with head injury, representing approximately 6.7% of the 120 million total ED visits.¹ Although some of these patients have sustained moderate or severe head injury leading to death or serious morbidity, the vast majority of patients are classified as having minimal or minor head injury.^2- 3 Patients with minimal head injury have not experienced loss of consciousness or other neurological alteration. Minor head injury or concussion is defined by a history of loss of consciousness, amnesia, or disorientation in a patient who is conscious and talking, ie, has a Glasgow Coma Scale (GCS) score of 13 to 15.^2- 6 Although after a period of observation most patients with minor head injury can be discharged without sequelae, a small portion deteriorate and require neurosurgical intervention for intracranial hematoma.^7- 8 The key to managing these patients is early diagnosis of intracranial injuries using computed tomography (CT) followed by early craniotomy.^2,9- 10

Current use of CT for minor head injury is increasing rapidly, is highly variable, and is inefficient. Between 1992 and 2000, use of CT imaging for all conditions in US EDs has increased from 2.4% to 5.3% of all visits, a 120% increase.¹ In 10 large Canadian hospitals, the use of CT has increased 165%, from 30% to 80%, specifically for patients with minor head injury.¹¹ We previously demonstrated large variation among similar Canadian teaching hospitals in ordering of CT for minor head injury.¹¹ Our data show that 90% of CTs are negative for clinically important brain injury.¹² Inefficient use of CT adds significantly to health care costs and adds to the burden of overcrowding in EDs.^13- 15 Rural centers without CT scanners must arrange for costly and time-consuming transfers of patients to larger urban centers for a CT scan.

Substantial potential for improving the efficiency of minor head injury management appears possible through the application of clinical decision rules. A clinical decision rule is derived from original research and is defined as a decision-making tool that incorporates 3 or more variables from the history, examination, or simple tests.^16- 21 Two decision rules have been independently developed to allow more selective ordering of CT scans, more rapid discharge of patients with minor head injury, and significant health care savings. The New Orleans Criteria (NOC) include 7 items (Box 1) that were developed for a study of 1429 patients with minor head injury and a GCS score of 15,²² and have been widely disseminated in the United States.²³ Our group derived the Canadian CT Head Rule (CCHR) in a study of 3121 patients with minor head injury and a GCS score of 13 to 15.^12,24- 25 This rule is based on 5 high-risk and 2 medium-risk criteria (Box 2).

Computed tomography is required for patients with minor head injury with any 1 of the following findings. The criteria apply only to patients who also have a Glasgow Coma Scale score of 15.

Computed tomography is only required for patients with minor head injury with any 1 of the following findings: Patients with minor head injury who present with a Glasgow Coma Scale score of 13 to 15 after witnessed loss of consciousness, amnesia, or confusion.

*The rule is not applicable if the patient did not experience a trauma, has a Glasgow Coma Scale score lower than 13, is younger than 16 years, is taking warfarin or has a bleeding disorder, or has an obvious open skull fracture.

†Signs of of basal skull fracture include hemotympanum, racoon eyes, cerebrospinal fluid, otorrhea or rhinorrhea, Battle’s sign.

‡Dangerous mechanism is a pedestrian struck by a motor vehicle, an occupant ejected from a motor vehicle, or a fall from an elevation of 3 or more feet or 5 stairs.

Decision tools may not perform as well in a validation setting as they did in the initial derivation phase.²⁶ Therefore, the goal of this study was to prospectively compare the accuracy, reliability, and potential impact of the CCHR and the NOC in minor head injury patients. Such a validation study is an essential step prior to the implementation of a decision rule for patient care.

We conducted this prospective cohort study in 9 Canadian tertiary care teaching hospital EDs. We considered for enrollment consecutive adults who had sustained acute minor head injury. Eligibility was based on the patient’s having all of the following: (1) blunt trauma to the head resulting in witnessed loss of consciousness, definite amnesia, or witnessed disorientation; (2) initial ED GCS score of 13 or greater as determined by the treating physician, and (3) injury within the previous 24 hours. Patients were ineligible if they were younger than 16 years; had minimal head injury, ie, no loss of consciousness, amnesia, or disorientation; had no clear history of trauma as the primary event; had an obvious penetrating skull injury or obvious depressed skull fracture; had focal neurological deficit; had unstable vital signs associated with major trauma; had a seizure prior to assessment in the ED; had a bleeding disorder or used oral anticoagulants; had returned for reassessment of the same injury; or were pregnant. The hospital research ethics boards approved the protocol without the need for informed consent. Patients who were followed up through a telephone interview by the study nurse had an opportunity to give verbal consent at that time.

All patient assessments were performed by emergency medicine attending or resident physicians, who were trained by means of a 1-hour lecture session given by the principal investigator. Although some physicians had participated in the CCHR derivation study, none had previously used either decision rule in clinical practice. After assessment and prior to CT, physicians recorded their findings and rule interpretations on standardized data collection forms. The CCHR and NOC rules were printed one on top of the other in large boxes on the data collection form.²⁷ When feasible, some patients were independently assessed by a second emergency physician to judge interobserver agreement.

The primary outcome was “need for neurosurgical intervention” and the secondary outcome was “clinically important brain injury” on CT. Need for neurosurgical intervention was defined as either death within 7 days secondary to head injury or the need for any of the following procedures within 7 days: craniotomy, elevation of skull fracture, intracranial pressure monitoring, or intubation for head injury (demonstrated on CT).

Clinically important brain injury was defined as any acute brain finding revealed on CT and that would normally require admission to hospital and neurosurgical follow-up. This definition was standardized based on the results of a formal survey of 129 academic neurosurgeons, neuroradiologists, and emergency physicians at 8 study sites and has been used by our group previously.^12,28 All brain injuries were considered clinically important unless the patient was neurologically intact and had 1 of the following lesions on CT (1) solitary contusion of less than 5 mm in diameter, (2) localized subarachnoid blood less than 1 mm thick, (3) smear subdural hematoma less than 4 mm thick, or (4) closed depressed skull fracture not through the inner table. After the clinical examination, patients underwent standard CT of the head according to the judgment of the treating physician. The CT scans were interpreted by qualified staff neuroradiologists who were blinded to the information on the data collection sheet. We have previously demonstrated the reliability of these CT interpretations.¹²

We could not demand routine CT for all patients with minor head injury because current practice at the study hospitals was that only 70% of eligible patients currently undergo CT imaging. Consequently, all enrolled patients who did not have imaging underwent a structured 14-day proxy outcome measure administered by a registered nurse over the telephone. According to this tool, patients were classified as having no clinically important brain injury if they met all of the following explicit criteria at 14 days: headache absent or mild, no complaints of memory or concentration problems, no seizure or focal motor findings, and returned to normal daily activities. The assessment of these criteria was made by a registered nurse who was unaware of the patient’s status for the individual clinical predictor variables. Patients who did not fulfill these criteria were recalled for clinical assessment and CT. Patients could only be classified as having “brain injury” based on their CT findings. We have previously validated the proxy outcome measure in a study of 172 patients.²⁹ Patients who could not be reached were excluded from the final analysis. We searched for the names of these latter patients in the ED records of the study hospitals during the ensuing 30 days to ensure that no adverse outcomes were missed.

The 2 rules were assessed for their classification performance (ie, sensitivity and specificity) for need for neurosurgical intervention and for clinically important brain injury. The CCHR and the NOC were compared in the cohort of patients who presented with a GCS score of 15. In addition, the CCHR was assessed for all patients in the study, ie, those with presenting GCS scores of 13 to 15. The “criterion interpretation” of the rules, ie, positive or negative for the outcome measures, was made by an adjudication committee that reviewed patient records and physician data form responses. Interobserver agreement for each variable and for interpretation of the 2 rules was measured with the weighted κ coefficient. We evaluated accuracy of the interpretations of the rule by the treating physician vs the criterion interpretation by the investigators. We also descriptively presented data regarding the physicians’ theoretical comfort and perceived ease of use of the rules. Finally, we estimated the potential referral fractions for CT based on the percentage of patients who would require CT according to the 2 rules. The referral fraction is an important measure of the usefulness of a decision rule and reflects the potential impact on practice.

All P values were 2-tailed. Ninety-five percent confidence intervals (CIs) were calculated for estimates, where appropriate. Proportions were compared between the CCHR and NOC with unadjusted χ² analysis. We estimated that a sample size of 2500 patients with minor head injury and 1800 patients with a GCS score of 15 would be required to provide a sufficient number of head injury cases to allow a 95% CI of 97% to 100% around a sensitivity of 99%. All analyses were conducted using various statistical software programs including SAS, version 8.2 for Unix (SAS Institute Inc, Cary, NC) and P<.05 was considered statistically significant.

From June 2000 to December 2002, 4248 eligible patients were seen in the study hospitals. Of these, 2707 patients had data forms completed by the physicians and had complete outcome assessments (80.2% had CT; 19.8% had the proxy outcome assessment tool). In addition 1330 eligible patients did not have data forms completed by the physicians, and another 211 patients had data forms but no outcome assessments, ie, no CT and could not be reached for the proxy outcome. A total of 307 different physicians participated.

Table 1 compares the characteristics of the entire patient group and the subgroup with a GCS score of 15. In these 2 cohorts, 1.5% and 0.4%, respectively, required neurosurgical intervention and 8.5% and 5.3% of cases, respectively, had clinically important brain injury (Table 2). All patients with clinically important brain injury were identified in the ED without the need for the proxy outcome assessment tool. Among the 97 patients with clinically unimportant injury, none required neurosurgical intervention.

Among the eligible patients not enrolled, demographic and clinical characteristics were very similar to those of the enrolled patients (data not shown). In addition, the 211 patients who had no outcome assessment were similar to the enrolled patients other than having undergone no imaging (data not shown). In 13 cases, patients with acute brain injury were discharged from the ED without the physician identifying the lesion on CT; however, only 2 of these patients had clinically important brain injury. These patients were subsequently identified by routine radiological review 1 or 2 days after the initial ED visit. No patients experienced an adverse outcome and neither of the 2 missed clinically important brain injury cases required hospitalization.²⁷

Table 3 compares the performance of various clinical predictors between patients with and without clinically important brain injury, showing both univariate association and interobserver agreement. Three of the 7 NOC criteria and 7 of 7 CCHR criteria showed statistically significant association. Other variables associated with brain injury included an initial GCS score of 13, any decrease in GCS score after arrival in the ED, and an object recall test score of less than 3 out of 3. The κ values for interobserver agreement of these variables ranged from 0.18 to 1.0.

The accuracy of the 2 rules is compared in Table 4 for the 1822 patients with a GCS score of 15. The sensitivities for need for neurosurgical intervention were 100% (95% CI, 63%-100%) for the CCHR and the NOC and their respective specificities were 76.3% (95% CI, 74%-78%) vs 12.1% (95% CI, 11%-14%; P<.001). The CCHR and NOC were both 100% (95% CI, 96%-100%) sensitive for clinically important brain injury and their respective specificities were 50.6% (95% CI, 48%-53%) and 12.7% (95% CI, 11%-14%; P<.001). For all 145 cases of brain injury, including the 48 cases of clinically unimportant injury, the sensitivities were 93.1% for CCHR and 98.6% for NOC and the specificities were 51.4% for CCHR and 12.9% for NOC.

Table 5 summarizes the classification performance of the CCHR for all 2707 minor head injury patients in the cohort, ie, those with a GCS score of 13 to 15. For 41 cases of neurosurgical intervention, using only the 5 high-risk criteria, CCHR had a sensitivity of 100% (95% CI, 91%-100%) and a specificity of 65.6% (95% CI, 64%-67%). For clinically important brain injury using the 7 high- and medium-risk criteria, the sensitivity was 100% (95% CI, 98%-100%) and the specificity was 41.1% (95% CI, 39%-43%).

The weighted κ value for physician interpretation of the overall rules in 49 cases was 0.85 (95% CI, 0.58-0.92) for the CCHR and 0.47 (95% CI, −0.13 to 1.0) for the NOC. A value greater than 0.6 is generally considered to reflect reasonable agreement.

Clinical acceptability was assessed in 2 ways. Using a 5-point scale, physicians indicated that they would have been uncomfortable or very uncomfortable in applying the CCHR for 9.5% (95% CI, 8%-11%) of cases compared with 11.7% (95% CI, 10%-13%) for NOC (P=.03). Physicians misinterpreted the rules as not requiring imaging on the data forms, in contrast to subsequent investigator interpretation that imaging was indicated, for 4.0% (95% CI, 3%-5%) of cases according to CCHR and 5.5% (95% CI, 5%-7%) according to NOC (P=.04).

Potential impact on CT ordering was evaluated by estimating the proportion of patients who would require CT imaging according to the rules. Among patients with a GCS score of 15, the rate was 52.1% (95% CI, 50%-54%) for the CCHR and 88.0% (95% CI, 86%-89%) for the NOC (P<.001). For the entire cohort of 2707 patients, the CT imaging rate according to CCHR would have been 62.4% (95% CI, 61%-64%); the actual CT rate for these cases was 80.2% at the 9 study sites. The potential impact on ED crowding was assessed by measuring the mean ED length of stay among the 1884 patients (GCS 13-15) whose CT scan showed no injury. Patients who did not undergo CT (n = 447) spent approximately 2.5 fewer hours in the ED (180.8 vs 323.9 minutes; P<.01) than patients who had undergone CT imaging (n = 1437).

In this large prospective comparison of clinical decision rules for the use of CT in minor head injury, both the CCHR and the NOC were highly sensitive for 2 important outcome measures, need for neurosurgical intervention and clinically important brain injury for patients with a GCS score of 15. On the other hand, the specificity of the CCHR was higher than that of the NOC and application of the CCHR would result in lower use of CT imaging. In addition, the reliability of physician interpretation of the CCHR was higher, reflecting better interobserver agreement. Clinician acceptance of both rules appeared to be similarly high. The NOC were only developed for use in patients with GCS score of 15. As measured in the larger cohort of patients with a GCS score of 13 to 15, the CCHR had 100% sensitivity for identifying clinically important brain injury and patients requiring neurosurgical intervention.

This prospective validation study was designed and conducted according to strict methodological standards,^{16- 18,30- 31} similar to our previous approach of comparing 2 decision rules for cervical spine radiography.^19,32 The outcome measures of clinically important brain injury and need for neurosurgical intervention were carefully defined and are clearly of clinical importance. Patients were selected for the study according to explicit and transportable inclusion criteria rather than on the subjective decision of physicians to order CT imaging. A large number of patients with a wide spectrum of injury severity were enrolled. We did, however, exclude children as we believe that pediatric cases require unique criteria and should be investigated in a large separate study. Also evaluated were other important measures besides accuracy, including interobserver agreement, acceptability, and potential impact on practice.

This study has potential limitations although most apply equally to the evaluation of both rules. While not all eligible cases were enrolled, no selection bias could be detected and the characteristics of enrolled and nonenrolled patients were similar. Patient enrollment required ED physicians to assess patients and voluntarily complete data forms in the midst of their busy clinical responsibilities. In many instances, this was not feasible. Some patients could not be reached for follow-up but we believe it is highly unlikely that any experienced important missed injury because such patients were completely intact neurologically and were considered to be very low risk by the physicians. In addition, none of these patients returned to the treating hospital or any other regional neurosurgical center within 30 days after their initial ED visit with minor head injury. Although some patients were transferred from other hospitals, only a few of those patients had undergone CT imaging prior to arrival and the treating physicians did not know the status of the patient’s brain injury prior to their assessment.

Some may be concerned about the use of clinically important brain injury as one of the outcome measures although it was applied equally to both rules. There has been very good acceptance of this definition by Canadian academic neurosurgeons, neuroradiologists, and emergency physicians who see this as a pragmatic and safe outcome measure.²⁸ Practice in most Canadian centers is to neither admit nor have a neurosurgeon follow up the patients with clinically unimportant lesions on CT. Furthermore, the primary outcome, need for neurosurgical intervention, was tightly defined and is clearly of clinical importance. Finally, a secondary analysis found similar performance of both rules for all brain injuries, regardless of clinical importance.

Not all study patients underwent CT imaging because the Canadian clinicians in the study occasionally do not order imaging when they consider patients to be low risk. Cases were classified as having no clinically important injury only if they satisfied all criteria on the 14-day proxy outcome tool, which has been validated and which was used equally for both rules.²⁹

One potential concern is whether Canadian physicians, who are more familiar with the CCHR, could have unconsciously biased the response to favor the CCHR. We believe this is unlikely. The NOC were accurately and prominently presented on the data forms and physicians were well informed of these criteria. In addition, our data demonstrated that few errors were made by the physicians in interpreting the NOC.

Several factors might account for our findings of lower specificity for the NOC compared with the CCHR. First, NOC was originally derived in a sample of 520 patients and then validated in a sample of 909 patients, with a total of only 6 patients who required neurosurgical intervention. Our study found NOC to have a specificity of 12.1% for brain injury even though this rule was designed only for use in patients with a GCS score of 15. We also found low specificity for the NOC when we conducted a retrospective review performed on our phase 1 database of 3121 patients. Second, we believe that a number of the NOC criteria lack accuracy or reliability when used to evaluate patients with minor head injury. The data presented in this study show that only 3 of 7 NOC criteria have a significant association with clinically important brain injury. Only 2 of the NOC criteria displayed κ values exceeding 0.60. In contrast, all 7 of the CCHR variables showed a statistical association with clinically important brain injury and 5 of 7 displayed κ values exceeding 0.60.

The true test of a clinical decision rule is whether clinical behavior is affected.^33- 36 Toward this end, we are now conducting a randomized multicenter implementation trial that compares strategies using and not using the CCHR as a guide for the use of CT imaging. Only through such real-life implementation studies can the potential impact of these decision tools be determined.^37- 39

In summary, this study confirmed the high sensitivity of both the CCHR and NOC rules for imaging in patients with minor head injury with a GCS score of 15. The CCHR had higher specificity and reliability and would have greater potential impact on CT ordering rates than the NOC. In addition, the CCHR performed equally well in a larger and more severely injured cohort of patients presenting with GCS scores ranging from 13 to 15. These data suggest that the CCHR has the potential to improve efficiency in the use of CT imaging for patients with minor head injury.