Monitoring Sedation Status Over Time in ICU Patients:  Reliability and Validity of the Richmond Agitation-Sedation Scale (RASS) FREE

Increased scrutiny has recently been placed on appropriate titration of sedative and analgesic medications in critically ill patients, especially those being treated with mechanical ventilation.^1- 5 Patient comfort should be a primary goal in the intensive care unit (ICU), including adequate pain control,^6- 7 anxiolysis, and prevention and treatment of delirium.⁸ However, achieving an appropriate balance of sedation and analgesia is challenging.^9- 11 Without rational and agreed on target levels of sedation, different members of the health care team will have disparate treatment goals,^12- 14 increasing the chance for iatrogenic complications and potentially impeding recovery.

The clinical practice guidelines of the Society of Critical Care Medicine emphasize the need for goal-directed delivery of psychoactive medications.^8,15 Although the Ramsay Scale (RS)¹⁶ was not originally intended for use as a clinical monitoring tool, it has been used for decades in both clinical practice and research. Still, most patients are not monitored with any scale to guide delivery of sedative medications. Objective, goal-directed sedation therapy is now the recommended standard to avoid oversedation and to promote earlier extubation.^{2,8,14,17- 19} As pointed out by others,^1,20- 22 very few of the available sedation scales have been appropriately tested for reliability and validity. Even among available instruments that have been tested for reliability and validity,^23- 26 none discretely separates verbal from physical stimulation (ie, the potency of the stimulus) in generating scores at pivotal levels of sedation. Two recent systematic reviews concluded that goal-directed sedative and analgesic administration would be enhanced if such instruments were shown to detect variations in level of consciousness over time and according to delivery of psychoactive drugs.^1,26 Other investigators observed cardiac surgery patients over time, yet the duration of monitoring was only 6 hours, the sample size was small (only 14 patients followed up to extubation), and correlations with drug doses were not published.²⁷

The Richmond Agitation-Sedation Scale (RASS)^28- 29 was developed by a multidisciplinary team at Virginia Commonwealth University in Richmond. It is a 10-point scale that can be rated briefly using 3 clearly defined steps and that has discrete criteria for levels of sedation and agitation. A unique feature of the RASS is that it uses the duration of eye contact following verbal stimulation as the principal means of titrating sedation. Hence, this scale's validation could be linked to both arousal and content of thought—the 2 components of consciousness.³⁰ We determined that the duration of eye contact could be easily measured with minimal training, allowing reproducibility and increased acceptability of the instrument by bedside physicians, nurses, and researchers alike. The RASS has been demonstrated to have excellent interrater reliability in a broad range of adult medical and surgical ICU patients and to have excellent validity when compared with a visual analogue scale and selected sedation scales.²⁹

The current investigation was designed to extend the reliability and validity testing of the RASS in novel ways to include assessment of sedation over time, correlation of the RASS with independent neuropsychiatric experts' measures of level of consciousness and formally measured content of consciousness (ie, inattention and delirium), doses of sedative and analgesic medications, and objective measurement of brain function using bispectral array electroencephalography.

This investigation was conducted in the adult medical and coronary ICUs at Vanderbilt University Medical Center, a 641-bed tertiary-care, academic medical center. The institutional review board approved the study, and written informed consent was obtained from the patients or proxies. Reliability and validity testing was performed in 2 phases. We enrolled patients into the reliability testing cohort from July 21, 1999, to September 7, 1999, and into the validity testing cohort from February 1, 2000, to May 3, 2001.

While none of the data in this report have been previously published, other data from this cohort of patients have been published as would be expected from prospective cohort investigations that have the capacity to address different issues. Specifically, the 38 patients from the RASS reliability testing cohort were those reported in the first Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) study,³¹ and 96 of the patients from this RASS validity testing were those reported in the second CAM-ICU study cohort.³²

Patients receiving and not receiving mechanical ventilation were screened during reliability testing to ensure reliability in verbal and nonverbal patients. Any adult admitted to the ICU who did not meet the following a priori exclusion criteria was eligible for enrollment: a history of severe dementia, psychosis, or neurologic disease (n = 12); patient or family refusal to participate (n = 8); and admission to the ICU after the predefined cap of 10 study patients per day (because of research staffing limitations) had been reached (n = 18). Consecutive patients receiving mechanical ventilation were enrolled into the validity testing cohort (and followed up until ICU discharge), because the primary challenge and use for sedation scales in the ICU are for patients who are nonverbal and are intubated. Patients were excluded if they had a history of psychosis or neurologic disease (n = 16), were non–English-speaking or deaf (n = 5), were extubated or had died before nurses' screen (n = 15), were previously enrolled in reliability cohort (n = 5), or because of patient or family refusal to participate (n = 9).

During this investigation, no protocol to guide analgesia, sedation, or neuromuscular blockade existed in our ICU, and no objective target levels of sedation were routinely identified according to disease state or ventilator settings. All doses of narcotics, benzodiazepines, propofol, and neuromuscular blocking agents were recorded prospectively in 8-hour intervals throughout the investigation. Administered narcotics were either morphine or fentanyl. Administered benzodiazepines were either lorazepam or midazolam (the midazolam dose was converted to lorazepam equivalents by dividing by 3 to achieve equipotent dose³³). If neuromuscular blocking agents had been given within 8 hours of RASS assessment, patient observations were excluded from our pharmacological analyses.

Prior to this investigation, raters had no experience with the RASS. Sessler et al²⁹ from the Virginia Commonwealth University provided a 1-page handout with the RASS description and the Procedure for RASS assessment (Table 1) to the investigators. No other formal training was received or required. All raters performed the RASS using the same sequence of 3 steps as outlined in Table 1. If the patients were alert or agitated prior to stimulation, they were scored 0 to + 4 accordingly. If patients were not spontaneously alert, they were then called by name to look at the rater, with the duration of eye contact measured, at which time a positive response was scored accordingly as −1 to −3. If the patients did not respond to verbal stimulation, they were then physically stimulated (ie, shoulder shake and/or sternal rub) and scored according to their response as –4 or –5. If calm and not alert prior to verbal and physical stimulation, patients were then rated as −1 to −5 (as standardized in Table 1) even if they became agitated on stimulation. Assessments required less than 20 seconds.

Two critical care study nurses, an intensivist, and a neuropsychiatric expert performed daily, independent RASS ratings during each patient's ICU stay. Interrater reliability assessments were conducted in the afternoon, and none of the raters had access to the others' scores at any time. The study nurses performed simultaneous ratings (one interacted with and rated the patient while the other observed and rated the same patient), while the intensivists and neuropsychiatric experts performed their independent RASS ratings within 4 hours of the nurses' ratings. In addition, both nurses performed independent RS¹⁶ ratings and Glasgow Coma Scale (GCS)³⁴ ratings for each patient.

Validity, the extent to which the instrument measured what it was intended to measure, was tested in 3 ways according to standard definitions.^35- 36Criterion validity, the extent to which a measure relates to a set of externally derived criteria, was tested on the first 96 patients of the validation cohort by comparing patients' RASS levels against reference standard evaluations performed by neuropsychiatric experts (ie, a geriatric psychiatrist and a geriatric neuropsychologist) who rated patients' levels of consciousness as normal, delirious, stuporous, or comatose using standardized definitions that did not incorporate the RASS in any way.^31- 32,37 The neuropsychiatric experts were blinded to the study nurses' RASS ratings and the 2 ratings were performed within 4 hours of one another. To address 2 areas of sedation monitoring that have not been studied adequately to date,^21,26 we analyzed data for patients who were and were not receiving mechanical ventilation and the ability of the RASS to identify changes in level of consciousness over time (as rated by the neuropsychiatric expert).

Construct validity, the extent to which a measure relates to the other measures that would theoretically support the concept (or construct) being measured, also should be measured whenever no universally accepted criterion exists.^35,38- 39 Thus, construct validity was tested 5 ways in patients with normal and abnormal consciousness: (1) RASS scores were compared with abnormalities in the "content of consciousness" as rated by the nurse using a screening examination for attention,³¹ which served as a measure of content of consciousness since this is the pivotal feature of delirium³²; (2) comparisons were made against ratings of the GCS, a standard instrument widely used in neurologic monitoring throughout the world; (3) RASS scores were correlated with the quantity of sedative and analgesic drugs administered to patients in the 8- and 24-hour periods prior to their assessments; (4) outcomes of planned extubation were compared with concurrent RASS scores; and (5) bispectral-XP electroencephalography (BIS-XP EEG) (Aspect Medical Systems Inc, Newton, Mass) was measured and correlated with RASS scores as described below. Both the GCS and BIS-XP EEG measurements were recorded at the same time as the RASS ratings, while the ratings by the neuropsychiatric experts occurred within 4 hours of the study nurses' ratings. The decision to extubate was made by the attending physician according to the results of spontaneous breathing trials with no knowledge of the RASS scores and without a predetermined neuorologic status. Extubations were considered successful if the patient remained extubated and did not receive mechanical ventilation for 48 hours. Detailed reasons for unsuccessful extubation were not recorded.

Face validity, the extent to which an instrument appears to measure what it is intended to measure, was tested by surveying critical care nurses about whether or not the RASS was an appropriate and clinically useful measure of agitation and sedation. The survey included 4 relevant questions each worded 2 different ways and embedded within other questions regarding patient management. The survey asked nurses to rate the following 4 statements using a 5-point Likert scale (1 [strongly disagree] to 5 [strongly agree]): (1) RASS levels for agitation (+1 to + 4) are clinically relevant and easy to score; (2) it makes sense clinically that the RASS assessment begins with verbal stimulation (−1 to − 3) and then moves to physical stimulation (−4 to − 5); (3) use of the RASS improves communication among the health care team; and (4) use of the RASS provides a team consensus for target-level sedation.

Bispectral index uses a nonlinear signal processor to measure brain wave activity in the form of raw electroencephalography (EEG) and to create a score ranging from 100 (awake) to 0 (no cortical activity). Both raw EEG and BIS-XP EEG data were recorded using a fronto-temporal montage with disposable sensors that were connected to a portable EEG monitor (A1050, Aspect Medical Systems Inc).^40- 41 As opposed to earlier versions of the BIS-XP EEG used in other investigations designed for the anesthetized surgical patients in the operating room,⁴² this version has been specifically designed for use in ICU patients who are sedated and receiving mechanical ventilation to reduce electromyographic interference.^43- 44 Electrode impedance values were acceptable if they were no greater than 5 kΩ and if the threshold for acceptable signal quality index was greater than 80%. Raw EEG data was sampled at 128 samples per second and recorded continuously in real time; processed variables were downloaded and recorded every 5 seconds. The data sampling rate was 256 times per second with filter settings of 70 Hz for the high frequency (70 Hz) and 2 Hz for the low frequency. The study nurses were blinded to both the raw EEG and the BIS-XP EEG data. The continuous EEG recording was later reviewed by a separate investigator blinded to the clinical assessment ratings to identify the reported baseline value (ie, a stable portion of the tracing that the EEG would return to after the sensor was attached).

Patients' baseline characteristics were presented using means and SDs for continuous variables, and frequencies and proportions for dichotomous variables. Interrater reliability was determined for the RASS, RS, and GCS by comparing ratings between raters using weighted κ indices and 95% confidence intervals. For criterion validity, RASS scores were compared with the neuropsychiatric expert rating using Wilcoxon rank sum tests. As part of criterion validity, and to account for dependency among observations within an individual patient, proportional odds regression analysis with generalized estimating equations was performed to assess the neuropsychiatric expert ratings vs RASS ratings, and odds ratios were presented. Similar analysis using Wilcoxon rank sum tests assessed the difference between observations within a level of consciousness of patients receiving and not receiving mechanical ventilation. For construct validity testing, the Spearman correlation coefficient (r) was calculated for the associations between RASS scores and onset of inattention, GCS score, daily dosage of sedative drugs, and BIS-XP EEG results. Wilcoxon rank sum tests were used to assess significant differences in GCS scores according to RASS levels, and proportional odds models were used with the generalized estimating equations to analyze the relationship over time among the RASS score, GCS score, and daily dosage of sedative drugs. Statistical analyses were performed using SAS version 8.02 (SAS Institute Inc, Cary NC) and STATA version 7.0 (STATA Corp, College Station, Tex).

Eighty-six patients were admitted to the medical and coronary ICUs during the reliability testing phase, of whom 38 (44%) were enrolled. During the validity testing phase, 325 consecutive patients receiving mechanical ventilation were admitted to the ICU, of whom 275 (84.6%) were enrolled. Baseline characteristics from all 313 patients are presented in Table 2. At the time of enrollment, 22 (46%) of the reliability cohort received mechanical ventilation, while all 275 of the validation cohort received mechanical ventilation. The 2 cohorts had a high baseline severity of illness as measured by Acute Physiology and Chronic Health Evaluation II (APACHE II) scores (mean [SD], 17.1 [8.7] and 25.0 [8.0]), and had a wide variety of medical diagnoses.

Patients were evaluated on multiple occasions during their ICU stay. The mean (SD) RASS scores for each rater were as follows: nurse 1, −1.60 (2.16); nurse 2, −1.88 (2.20); intensivist, −1.61 (2.17); and neuropsychiatric expert, −1.50 (2.25). In 290 paired observations by nurses, both the RASS and the RS demonstrated excellent interrater reliability (weighted κ, 0.91 and 0.94, respectively), which were superior to GCS (weighted κ, 0.64; P<.001 for both comparisons). Using only the first observation for each patient (n = 38), the weighted κ values for the RASS, RS, and GCS were unchanged at 0.95, 0.95, and 0.65, respectively. The interrater reliability of the RASS was very high across nurses, intensivists, and neuropsychiatric experts (Table 3). Because reliability testing was expected to be most challenging for patients receiving mechanical ventilation, we conducted another analysis of interrater reliability restricted to the 22 patients who were intubated. The weighted κ for the RASS between the 2 nurse ratings for patients who were intubated was 0.88 (95% confidence interval, 0.78-0.97).

The relative frequency that the validation cohort spent in each major arousal category for 1833 observations in 275 patients indicated that the patients who received mechanical ventilation spent about one third of their time either unarousable or in a deeply sedated state (RASS score, −5 or −4; n = 548 observations), one third in a moderate to light sedation state (RASS score, −3 to −1; n = 625 observations), and one third in an alert and calm state (RASS score, 0; n = 619 observations). Spontaneous agitation (RASS score, + 1 to + 4; n = 41 observations), which was rated prior to stimulation, was an uncommon state (<5%) found by the study nurses.

Criterion validity was tested in 411 paired observations in the validation cohort for the first 96 patients with a median of 3 observations per patient. The results of the RASS showed excellent discrimination between levels of consciousness as rated using the neuropsychiatric expert reference standard (P<.001 for all) (Figure 1). Furthermore, as the neuropsychiatric expert raters and RASS raters independently tracked level of consciousness within patients over successive days of ICU care, RASS scores continued to correlate with expert raters' evaluations despite fluctuations in consciousness (P<.001 for all) (Table 4). When comparing patients over the course of their ICU stay, the reference standard ratings of abnormal levels of consciousness (ie, delirium, stupor, and coma) compared similarly with RASS ratings regardless of intubation status: delirium (−2 median RASS score if intubated vs –2 median RASS score if extubated, P = .18), stupor (−3 vs –3, P = .92), or coma (−5 vs –5, P = .62) (Figure 1).

Five methods were used to test construct validity: (1) RASS was correlated with onset of inattention using an attention-screening examination, the pivotal criterion for delirium (r = 0.78, P<.001). (2) To compare the RASS to the GCS, 1360 paired observations (among 275 patients with a median of 3 observations per patient) showed excellent correlation and discrimination (r = 0.91, P<.001) (Figure 2). The RASS also correlated with the GCS over time (P<.001), and the odds ratio of having higher RASS scores with greater GCS scores was 1.39 (P <.001). (3) We compared RASS scores with the cumulative lorazepam, propofol, fentanyl, and morphine dose over the 8-hour and 24-hour periods prior to RASS assessments (Table 5). As described in the "Methods" section, midazolam dose was converted into lorazepam equivalents for the purposes of these analyses. As an example of the dose-response relationship between psychoactive medication and RASS scores, correlative data for lorazepam equivalents over 8 hours prior to RASS assessment are shown in Figure 3. In contrast to the benzodiazepines, correlations between RASS and fentanyl and morphine were performed and presented as separate analyses because of significantly different results for these 2 agents (Table 5). (4) Of 185 planned extubations in the 275 validation cohort patients, 19 (10.3%) patients required reintubation within 48 hours. Of these planned extubations, 137 had RASS scores available during the shift prior to extubation, of which 13 (9.5%) were unsuccessful. The median (interquartile range) RASS score for successful extubation was − 2 (−3 to 0) and for unsuccessful extubation was − 3 (−3 to − 2) (P = .07). (5) From the validation cohort, a random sampling of 124 patients was monitored over 321 days in the ICU with the BIS-XP EEG. The RASS scores correlated with the BIS-XP EEG results over the range of levels of consciousness (r = 0.64, P<.001 for all) (Figure 4). For patients in 3 different, clinically relevant states (ie, spontaneously awake and alert [RASS score, 0], arousable with verbal stimulation [RASS score, –1 to –3], and arousable only with physical stimulation or not at all [RASS score, –4 or –5]), the median (interquartile range) EEG values were 96.8 (90.0-97.6), 69.0 (57.6-87.6), and 57.4 (46.4-66.3), respectively.

Face validity was demonstrated via a survey of 26 bedside critical care nurses. According to the results of the survey, 77% of the nurses agreed or strongly agreed that the RASS levels for agitation were clinically relevant and easy to score. Regarding the construct of the sedation assessment, 92% agreed or strongly agreed with the "verbal followed by physical stimulation" scoring scheme, 69% agreed or strongly agreed that the RASS improved communication among the managing team, and 81% agreed or strongly agreed this instrument provided a consensus target for goal-directed delivery of sedative and analgesic medications.

This investigation was designed to test a very brief yet structured approach to assessment of patient sedation in the ICU for reliability and validity, using a new, broader, and more rigorous set of validation procedures than those previously studied. The RASS demonstrated strong interrater reliability and criterion, construct, and face validity. In previous work, Sessler et al²⁹ demonstrated strong interrater reliability of RASS between 5 nurse, pharmacist, and physician investigators in 192 consecutive patients who did or did not receive mechanical ventilation in surgical, neuroscience, and medical ICUs. The authors further documented strong reliability between a nurse educator and 27 bedside nurses in more than 100 patient observations. Building on these data, the present study included the use of neuropsychiatric experts as reference standard raters of patients' levels of consciousness to demonstrate criterion validity, 5 methods of confirming construct validity, and incorporating views of bedside critical care nurses for face validity.

In keeping with the stated priorities of recent critical appraisals of sedation scales,^1,20- 22,26 the present report shows the RASS to be valid over successive days in individual patients, to correlate with the administered dose of sedative and analgesic drugs as well as brain wave activity, and confirms that it is a reliable and valid measurement for patients who were and were not receiving mechanical ventilation.

This report and that by Sessler et al²⁹ are complementary and constitute an evaluation of the RASS in more than 600 patients, both having avoided selection bias present in other investigations by enrolling consecutive patients. The large sizes of these 2 investigations, combined with the strong reliability of the instrument and the scope of our validity testing, lend strong credibility for the use of the RASS in patient management.

The RASS itself has several important strengths that warrant comment. Unlike other recently validated instruments,^23,25 the RASS separates verbal from physical stimulation so that the patient's level of arousal may be graded according to the potency of the stimulus.²⁸ It has been common to consider sedation scales valid as long as they delineate levels of arousal (considered a surrogate of consciousness).^{20,23- 25,49}Consciousness, however, is classically defined as the combination of a person's level of arousal plus the content of consciousness (eg, delirium).³⁰ Only recently has the ICU community begun to focus on delirium as an essential element of patient comfort and outcome.^8,50- 51 Importantly, a key feature of delirium is the presence or absence of inattention, which can be measured in part by the ability of a person to maintain eye contact. Indeed, this study demonstrated that RASS scores correlated with the onset of inattention and delirium. These 2 main strengths of the RASS assessment procedure (ie, completely distinguishing verbal from physical stimulation, and relying heavily on duration of eye contact) complement the recently developed delirium monitoring instrument, the CAM-ICU.^31- 32 Using such a combined neurologic monitoring schema for patients receiving mechanical ventilation in future trials may offer a significant advance in our ability to measure short- and long-term cognitive outcomes of goal-directed delivery of sedative and analgesic medications or newer pharmacological agents with procognitive advantages over traditionally used agents in treating delirium.^8,52- 53

The κ values between nurses' and between physicians' paired assessments were both 0.91, and the κ values between any nurse and physician pair ranged from 0.79 to 0.88. While all very high, the subtle differences in these values likely reflect that time elapsed between the nurse and physician assessments. The fact that the correlations were high despite elapsed time between the nurses' and physicians' assessments makes the data even more compelling.

We correlated RASS scores with quantity (ie, dose) of sedative and analgesic medications and found highly significant yet moderately to low correlation coefficients. Considering the broad distribution of drug dose variables and the numerous other covariates affecting consciousness (eg, underlying illness, electrolytes, hypoxemia, and other pharmacological agents), it would be unrealistic to expect RASS correlation with actual dose to be any higher. Similar to other reports on risk factors for delirium,^47,54 we found disparate correlation coefficients between RASS levels and different drugs. For example, the highly significant correlations between RASS levels vs lorazepam or fentanyl (both P<.001) contrast sharply with that of morphine (P = .10). Future investigations should attempt to determine the relative importance of other covariates contributing to level of consciousness, such as patients' age, sex, race, disease state, body mass index, the duration and cumulative drug levels of narcotics and analgesics over days of ICU care, and pharmacological interindividual variability based on drug metabolism and transport.

Most available evidence regarding sedatives and analgesics in ICU patients indicates that it may be less important which drugs are delivered than their proper titration using goal-directed delivery to optimize patient comfort while avoiding complications, such as prolonged mechanical ventilation or reintubation.^1,3,8,55 For example, recent data showed that deeper levels of sedation at the time of extubation, measured using the RASS, were associated with a higher likelihood of reintubation.⁵⁶ Future investigations should evaluate the usefulness of this tool in single or multicenter clinical trials, local or national quality-improvement collaboratives,⁵⁷ and different management strategies based on goal-directed delivery.

Some patients are sedated but subsequently demonstrate agitation following stimulation,^5,20 which represents another limitation of this investigation. Since we only recorded a single RASS score per patient assessment, we do not know the number of times patients were initially sedated and assigned negative RASS scores, only to become agitated minutes later. It may be surprising to some that so few patients in this investigation and that by Sessler et al²⁹ were found in the vigilant or agitated state, which we believe reflects a recurring theme of oversedation in critical care.^2,10,17 On the other hand, undersedation and inadequate relief of symptoms could lead to problems such as unaddressed pain or anxiety,^7,58 and ultimately posttraumatic stress disorder.^59- 62 The fact that the RASS has an expanded set of clinically relevant scores for tracking both agitation and sedation makes it well suited for future investigators to help better understand the clinical implications of such crossover states.

While the RASS demonstrated excellent face validity among our nurses, reports of ongoing large-scale implementation projects using the RASS will aid in our understanding of how to effect sustained change in the practice patterns of ICU nurses, therapists, pharmacists, and physicians using such instruments. Preliminary work in this area has already been reported from academic settings,⁶³ but should be forthcoming from community settings as well.

Lastly, as a barometer of brain wave activity, we used BIS-XP EEG monitoring in a method comparable with that used by other investigators.^27,42,44 The BIS-XP used in this investigation was an advance over that of earlier versions of BIS⁴² because of improved screening of an electromyographic artifact.^43,64 However, the overlapping BIS values at different RASS levels (Figure 4) may be the result of the broad distribution of psychoactive drugs administered to this cohort and their interindividual effects on brain wave activity or merely a limitation in this emerging EEG technology.

The RASS, which takes less than 20 seconds to perform and requires minimal training,²⁹ has been shown to be highly reliable among multiple types of health care professionals. The RASS has an expanded set of scores at pivotal levels of sedation that are determined by patients' response to verbal vs physical stimulation, which will help the clinician in titrating medications. This extensive body of new data, with a variety of unique approaches to assess an agitation-sedation scale, expands the usefulness of such instruments for patient care. In accordance with recent recommendations, health care professionals should use valid and reliable instruments such as the RASS to implement sedative and analgesic drug delivery protocols for patients receiving mechanical ventilation. The driving unmet need for goal-directed sedation practice has been met—now an instrument has been shown to detect variations in level of consciousness over time. Taken together, advances in neurologic assessment provided by the RASS and the CAM-ICU should lead to better characterization of acute brain dysfunction as an organ failure, reductions in the random variation with which patients' sedatives are currently managed, and appropriate interventions aimed at prevention or reversal of acute brain dysfunction.