Prediction of 6-Month Survival of Nursing Home Residents With Advanced Dementia Using ADEPT vs Hospice Eligibility Guidelines FREE

The challenge of accurately estimating life expectancy in advanced dementia is a barrier to providing palliative care to the more than 5 million individuals in the United States with this condition.^1- 8 Hospice has been shown to benefit residents dying with dementia.^9- 12 Although trends indicate that hospice enrollment of patients with dementia is gradually increasing, in 2008, the National Hospice and Palliative Care Organization reported that only 11% of hospice admissions had a primary diagnosis of dementia.¹³ Hospice professionals cite prognostication as the main hindrance to enrolling patients with dementia.¹ Medicare hospice eligibility requires an estimated survival of less than 6 months and, for dementia, is guided by 2 criteria¹⁴: stage 7c on the Functional Assessment Staging (FAST) scale¹⁵ and the occurrence of least 1 of 6 specified medical conditions in the prior year. Earlier studies suggest these guidelines do not accurately predict survival, but these studies are limited by retrospective designs,^3- 4,8,16 small sample sizes,³ testing of only the FAST component,^3- 4 and simulation of hospice eligibility using the minimum data set (MDS).^4,16- 17 The prognostic accuracy of hospice guidelines for dementia has not been evaluated in a large prospective study.

To date and to our knowledge, rigorous research efforts to create prognostic models for advanced stage dementia are limited.^{4- 5,7- 8,16,18- 19} In 2004, we used MDS data, data that are collected on every US nursing home resident, to create and retrospectively validate a 6-month mortality risk score for nursing home residents with advanced dementia and found that it predicted survival with moderate accuracy (area under the receiver operating characteristic [AUROC] curve, 0.70). The mortality risk score also established the feasibility of using MDS data to create a score.⁴ However, its practical application was limited by several factors, including the inclusion of only recent nursing home admissions (most residents with advanced dementia have prolonged stays), data from only 2 states, and validation limited to retrospective analysis of secondary MDS data. In addition, although the risk score had better discrimination than the FAST stage 7c, the FAST was simulated with MDS data and the preexisting medical conditions component of hospice guidelines were not considered.

As a next step, the National Institutes of Health funded a 4-year study, the goal of which was to (1) rederive an MDS-based mortality risk score in a nationwide data set that included both recently admitted and long-stay nursing home residents with advanced dementia, and (2) prospectively validate the risk score at the bedside and compare its performance with full hospice guidelines. The rederivation of the score was completed and resulted in a 12-item Advanced Dementia Prognostic Tool (ADEPT), the details of which are reported elsewhere.¹⁶ Our study presents the prospective validation of the ADEPT score and its comparison with hospice guidelines in a cohort of 606 nursing home residents with advanced dementia followed up to 6 months.

Residents were recruited between November 1, 2007, and July 30, 2009, from 21 nursing homes with more than 60 beds located within 60 miles of Boston, Massachusetts. Study eligibility criteria included older than 65 years, a Cognitive Performance Score (CPS) of 5 or 6,^20- 21 cognitive impairment due to dementia (any type), and a health care proxy for the resident that could be identified and contacted. The CPS groups residents into 7 cognitive performance categories based on 5 MDS items (0 = intact, 1 = borderline intact, 2 = mild impairment, 3 = moderate impairment, 4 = moderately severe impairment, 5 = severe impairment, and 6 = very severe impairment with eating problems). A CPS score of 5 corresponds with a mean (SD) Mini-Mental State Examination score of 5.1 (5.3). Previously collected MDS data were not used to determine the CPS scores in this study. Research assistants provided the CPS definition to nurses on each nursing home unit during an in-person interview and asked them to identify residents with CPS scores of more than 5. A diagnosis of dementia was ascertained from the medical record.

The proxies of residents were contacted to provide oral informed consent for the residents' and their own participation. The institutional review board at Hebrew Senior Life, Institute for Aging Research, Boston, Massachusetts, approved the conduct of the study.

The ADEPT score was created using 2002 MDS data collected from all licensed nursing homes in the United States. The score consisted of 12 items and the total score ranged between 1.0 and 32.5, with higher points indicating a greater risk of death (eFigure).¹⁶ In our study, previously completed MDS assessments were not used to determine the residents' ADEPT scores. At baseline, research assistants collected primary data from residents' charts and nurse interviews to calculate the scores. Data obtained from these records included date of nursing home admission, age, sex, race, dyspnea in the prior 7 days, and a diagnosis of congestive heart failure. Race, although not an item in the risk score, was obtained to characterize the sample and facilitate comparisons with other populations. Race was obtained from an MDS item previously completed by a facility nurse with prespecified categories as follows: American Indian/Alaskan Native, Asian/Pacific Islander, black (non-Hispanic), Hispanic, and white (non-Hispanic). Weights and heights of the residents were ascertained from the chart to determine whether their body mass index (BMI, calculated as weight in kilograms divided by height in meters squared) was less than 18.5 (threshold for being underweight²²) and whether they had recent weight loss (>5% of their body weight in last 30 days or >10% in last 180 days). Total functional dependence was defined as having a score of 28 on the Activities of Daily Living scale (range, 0-28),²³ ascertained by nurse interview. Nurses also determined whether the residents were bedfast (in bed or recliner >22 hours/d) for at least 4 of the last 7 days, had bowel incontinence at least once a week during the prior 14 days, were dyspneic in the past 7 days, had poor oral intake in the past 3 days (ate <75% of food at least 2 out of 3 daily meals or did not consume all or almost all liquids), and had at least 1 pressure ulcer of more than stage 2. Tube-fed residents were considered not to have poor oral intake. Dyspnea was considered present if documented in the residents' records or reported by their nurses.

In the ADEPT score derivation, the length of stay item was based on a variable categorizing the reason for the residents' MDS assessment as being completed either for purposes of nursing home admission or annual assessment. Nursing home admission was strongly associated with worse survival; therefore, it was important to include in the score. However, the prospective study's goal was to evaluate the ADEPT score as it would be used in practice, independent of previously completed MDS assessments. Therefore, a length of stay reflecting “recent admission” needed to be determined. A cutoff of less than 90 days was chosen a priori based on research suggesting this was a reasonable period to distinguish short vs long nursing home stays.^24- 25 However, cutoffs at less than 60 days and less than 120 days were also examined as sensitivity analyses.

Medicare hospice eligibility was determined using data collected at baseline by research assistants from medical records, nurse interviews, and health care proxy interviews.¹⁴ To determine whether residents had any of the medical conditions specified in hospice eligibility guidelines in the prior year, the charts of all residents were abstracted and telephone interviews were conducted with the proxies of those residents whose nursing home stay was less than 1 year (n = 161 [26.6%]). Conditions included aspiration pneumonia, pyelonephritis or other upper urinary tract infection, septicemia, multiple decubitus ulcers of higher than stage 3, recurrent fever after antibiotics, and poor nutritional status. Poor nutritional status is defined as insufficient oral intake to sustain life (energy intake [calories] <4.184 kJ/d [<1000/d], fluids <1 L/d) or tube-feeding accompanied by a more than 10% weight loss (or serum albumin <2.5 g/dL) during the past 180 days.

The second component of hospice eligibility guidelines states that patients must be at stage 7c on the FAST, a dementia rating scale with 7 major stages and 16 substages (range 1-7f, higher scores indicate worse severity). Stage 6 consists of the following substages: 6a = inability to dress, 6b = inability to bathe, 6c = inability to toilet, 6d = urinary incontinence at least occasionally, and 6e = bowel incontinence at least occasionally. Stage 7 consists of the following substages: 7a = speech is limited to less than 5 words, 7b = all intelligible vocabulary is lost, 7c = nonambulatory, 7d = unable to sit independently, 7e = unable to smile, and 7f = unable to hold head up. At stage 7c, patients must have progressed through all previous FAST stages. Therefore, residents with all the following characteristics during the previous 14 days were considered to be at FAST stage 7c based on nurse interviews (inability to dress, bathe, and toilet; incontinence of urine and stool at least occasionally; loss of all intelligible vocabulary or inability to communicate meaningfully; and nonambulatory).

Residents with at least 1 of the aforementioned medical conditions in the year prior and at FAST stage 7c were deemed eligible for hospice.

Two research assistants independently collected baseline data within 48 hours of each other on the first 67 residents recruited into the study to assess the interrater reliability of the ADEPT score and hospice guidelines.

Whether or not residents died within 6 months of baseline was ascertained from the nursing homes' medical record departments or senior administrators.

The frequencies of all resident characteristics, ADEPT score items, and components of hospice eligibility were calculated. The total ADEPT score was calculated for each resident, and the mean (standard deviation) were determined for the cohort. For 26 residents missing recent weight data, points for BMI of less than 18.5 and recent weight loss were imputed using the mean point scores for those items from the 580 residents with recent weight data (BMI <18.5, 0.1 points; recent weight loss, 0.2 points). Categories of the ADEPT score were created using the cutoffs for the deciles of the score based on its distribution in the derivation cohort.¹⁶

The interrater reliability of the ADEPT score was computed using the concordance correlation coefficient,²⁶ and the interrater reliability of the hospice eligibility guidelines was computed using the κ statistic.²⁷

The discrimination of the ADEPT score and hospice eligibility for predicting 6-month survival were calculated and compared using the AUROC. To assess calibration of the ADEPT score, the observed and mean predicted 6-month mortality rates were calculated for each decile-based range and compared using a Hosmer-Lemeshow goodness-of-fit test.²⁸ To examine the practical application of the ADEPT score, the sensitivity, specificity, and positive and negative predictive values were calculated for cutoffs based on the upper limit of each range. Exact binomial 95% confidence intervals (CIs) were calculated for sensitivity, specificity, positive and negative predictive values, and observed mortality rates. Wald 95% CIs were calculated for the AUROCs. Bootstrap 95% CIs were computed for mean-predicted mortality rates using the percentile method.²⁹

Hospice eligibility was a binary measure and the ADEPT score was a continuous measure. Comparisons of an AUROC based on a continuous measure can be inherently biased against an AUROC calculated from a discrete measure; therefore, to make a fair comparison between the 2 measures, the specificity of the hospice guidelines in estimating 6-month survival was computed and the cutoff necessary to give the same specificity for the ADEPT score was determined.³⁰ After setting both measures to the same specificity, their sensitivities and AUROCs were compared using McNemar test,³¹ and a contrast-based nonparametric test,³² respectively. Proportional hazards regression models were used to estimate the survival in the overall cohort, as well as stratified by hospice eligibility guidelines and the ADEPT score dichotomized at the aforementioned cutoff.

Given that the ADEPT score had 12 items, the sample size was based a priori on a death-to-risk factor ratio of 10 (ie, approximately 120 deaths); commonly considered a minimum to obtain reliable estimates of regression coefficients.^28,33 The a priori level of significance was P ≤ .05 for all analyses. All analyses were conducted using SAS version 9.2 (SAS Institute Inc, Cary, North Carolina) and S-PLUS version 8.1 (Tibco Software Inc, Palo Alto, California).

Among 1425 screened residents, 830 (58.2%) met study eligibility criteria. A total of 595 residents (41.8%) were ineligible because a proxy could not be identified (n = 2), a proxy could not be contacted (n = 366), and cognitive impairment was not due to dementia (n = 227). Among those who were eligible, 606 (73.0%) residents with advanced dementia were recruited. Proxy refusal to consent to participation was the sole reason for nonrecruitment. The 224 nonparticipants did not differ significantly from participants with respect to age. However, nonparticipants were more likely to be men (25.5% vs 18.2%; P = .03) and less likely to be white (88.0% vs 94.4%; P = .002). A total of 111 of 606 residents (18.3%) died over 6 months. No residents were lost to follow-up.

Table 1 shows the characteristics of the 606 nursing home residents in the 12-item ADEPT score. A total of 307 nursing home residents (50.7%) were aged 80 to 90 years and 29 (4.8%) had resided in the nursing home for less than 90 days. A total of 256 residents (42.2%) were completely functionally dependent (Activity of Daily Living score, 28) and 59 (9.7%) were bedfast most of the day. Dyspnea was experienced by 36 residents (5.9%) and 107 (17.7%) had a diagnosis of congestive heart failure. In terms of nutritional markers, 252 residents (41.6%) had insufficient oral intake, 48 (8.3%) had a BMI of less than 18.5, and 68 (11.7%) had recent weight loss. There were 537 residents (88.6%) with bowel incontinence and 33 (5.4%) with at least 1 pressure ulcer (≥ stage 2).

The mean (SD) ADEPT score of the residents was 10.1 (3.1) points. The concordance correlation coefficient of the total ADEPT score was 0.98 (95% CI, 0.97-0.99), indicating excellent interrater reliability.

Table 2 shows the proportion of residents within specified ranges of the ADEPT score and the observed and mean predicted 6-month mortality rates in each range. The goodness-of-fit test showed no significant lack of fit (P = .69), indicating good calibration (agreement between observed and predicted mortalities). Table 2 also shows the mean mortality rates predicted in the retrospective derivation cohort for the same score ranges.¹⁶

The AUROC for ADEPT score's prediction of 6-month mortality was 0.67 (95% CI, 0.62-0.72) (Figure 1). Using less than 60 days and less than 120 days as cutoffs for the length of stay item changed the AUROC only at the level of the third decimal place (for <60 days, AUROC = 0.669; 95% CI, 0.616-0.723; and for <120 days, AUROC = 0.672; 95% CI, 0.618-0.725). Table 3 shows the operating characteristics of the ADEPT risk score at various cut points. A score of more than 7.9 points achieved a sensitivity closest to 90% (sensitivity, 91.9; 95% CI, 85.2-96.2; specificity, 28.3; 95% CI, 24.4-32.5; n = 457). A cut point of more than 11.0 achieved the highest AUROC, which had a value of 0.63 (95% CI, 0.58-0.68), sensitivity of 55.0 (95% CI, 45.2-64.4), and specificity of 71.3 (95% CI, 67.1-75.3).

A total of 215 residents (35.5%) were at a FAST stage 7c (Table 4). The number of residents experiencing the preexisting conditions in hospice guidelines included 43 (7.1%) with aspiration pneumonia; 3 (0.5%) with urinary tract infection; 8 (1.3%) with septicemia; 49 (8.1%) with fever; 6 (1.0%) with multiple stage 3 or 4 decubitus ulcers; and 59 (9.7%) with insufficient oral intake (or tube-feeding with weight loss). A total of 135 residents (22.3%) had at least 1 of these medical conditions. Taken together, 65 residents (10.7%) were both at FAST stage 7c and had at least 1 preexisting condition; therefore, they met guidelines for hospice eligibility. The κ statistic of hospice eligibility guidelines was 1.00, indicating perfect interrater reliability.

The AUROC for hospice eligibility guidelines was 0.55 (95% CI, 0.51-0.59) for 6-month survival, the sensitivity was 0.20 (95% CI, 0.13-0.28), and the specificity was 0.89 (95% CI, 0.86-0.92). Using a cutoff of 13.5 on the ADEPT score, which also had specificity of 0.89, the AUROC was 0.58 (95% CI, 0.54-0.63) and the sensitivity was 0.27 (95% CI, 0.19-0.36). Neither the AUROC nor the sensitivity of the ADEPT score using a cutoff of 13.5 was significantly different (P = .17 and P = .13, respectively) from that of hospice eligibility guidelines. Figure 2 displays the survival curves for the entire cohort, residents with ADEPT scores of more than 13.5 (n = 83 [13.7%]) and less than or equal to 13.5 (n = 523 [86.3%]), and residents meeting and not meeting hospice eligibility guidelines.

This prospective study furthers our understanding of the practical aspects of estimating prognosis in advanced dementia. When administered at the bedside, the ADEPT risk score had high interrater reliability, good calibration, and modest discrimination in predicting 6-month survival when applied as a continuous measure (AUROC = 0.67). Medicare hospice eligibility guidelines also had excellent interrater reliability, but the discrimination was poor (AUROC = 0.55). The ADEPT score's performance was not significantly different (AUROC = 0.58) when examined as a dichotomous measure using a cutoff with the same specificity as hospice guidelines. These findings underscore the challenge of prognostication in advanced dementia and suggest that determining access to hospice based on life expectancy for patients with dementia limits access to the supportive care hospice offers.

The characteristics of the residents with advanced dementia in this cohort were comparable with other studies.^{4,7- 8,16,19,34} In particular, almost all ADEPT score items ascertained using primary data collection were similarly distributed in the derivation cohort, defined, and characterized using secondary MDS data.¹⁶ One exception was that fewer residents in the prospective cohort were recent admissions compared with the derivation cohort (4.8% vs 36.2%), albeit the definition of this variable differed slightly between the 2 studies. The 6-month mortality rate was lower in the prospective cohort (18% vs 25%), possibly reflecting the inclusion of fewer recent admissions. Fewer residents in the prospective study met hospice eligibility compared with the derivation cohort (11% vs 16%), in which criteria were simulated with MDS data. Taken together, dissimilarities between the prospective validation and retrospective derivation cohorts may be attributable to both differences in resident characteristics, as well as variation in data ascertainment methods.

Despite the few cohort differences, the discrimination of the ADEPT score to predict 6-month survival in this prospective validation was comparable with its performance in the retrospective derivation data set (AUROC = 0.68).¹⁶ The ADEPT score demonstrated good calibration. However, in practice, whether to use the predicted mortality rates in this smaller prospective cohort vs those in the larger derivation cohort is debatable. Because values differed primarily in the 2 highest risk categories, it may be reasonable to consider the probability of death in these categories to be within the range of the 2 mean predicted values (eg, >16.1 points; 0.49-0.62).

Hospice eligibility guidelines for dementia are widely used, but have never been validated in a large, prospective fashion. In corroboration with prior retrospective studies, we found the discrimination of hospice guidelines to predict 6-month mortality was poor.^4,16 However, using a single cutoff to estimate 6-month prognosis, whether using existing hospice guidelines or the empirically derived ADEPT score, is problematic for determining which nursing home residents with advanced dementia should receive hospice care. For example, using a relatively low cutoff score of more than 7.9, 91.9% of residents with advanced dementia who died within 6 months would be eligible for the program (sensitivity), but only 22.3% of enrolled residents would die within that period (positive predictive value). With a high cutoff score of more than 16.1, only 9.0% of residents who died within 6 months would be eligible, but 45.5% of enrolled residents would die within that same period. That said, one potential advantage of the ADEPT score is that as a continuous measure, it offers physicians and other primary care clinicians caring for these patients (eg, nurse practitioners) flexibility to select cutoffs with different operating characteristics (ie, tradeoff between sensitivity and specificity).

There are several limitations to our study that warrant comment. First, it is possible that the ADEPT score did not capture clinical variables strongly predictive of mortality. However, given the comprehensiveness of the MDS, the rigor of our approach, and the consistency of our findings with earlier research,^{4- 5,7- 8,16,18- 19} the degree to which the accuracy of a mortality risk score for advanced dementia could be improved with additional variables is questionable. Second, the ADEPT score and hospice eligibility were ascertained at a single random time point in the residents' course. In practice, hospice referrals are often initiated when care preferences shift toward comfort following a clinical set-back. Third, our prospective cohort was predominantly white and lived in Boston-area facilities, potentially limiting the generalizabilty of our findings. In addition, the ADEPT score was derived and validated in nursing home residents. Although the majority of patients with dementia die in nursing homes,³⁵ the ADEPT score has not been validated for those patients in the community.

Dementia is a leading cause of death in the United States.³⁶ Similar to other terminally ill patients, persons with advanced dementia commonly experience burdensome symptoms (eg, pain, dyspnea).³⁷ Our study strongly suggests that delivery of palliative care to these residents should be guided by a preference for comfort as the primary goal of care and not by prognostic estimates. Therefore, the challenge for health care professionals and policy makers is to ensure that high-quality palliative care is accessible to the growing number of individuals dying with dementia in nursing homes, an effort that may necessitate both revisiting the 6-month prognosis requirement for hospice, as well as expanding comprehensive palliative care services in the nursing home.