Anticoagulation Therapy for Stroke Prevention in Atrial Fibrillation:  How Well Do Randomized Trials Translate Into Clinical Practice? FREE

Multiple randomized trials have demonstrated warfarin therapy to be highly efficacious in reducing risk of ischemic stroke and other systemic thromboembolism in patients with atrial fibrillation, with relatively low rates of bleeding.^1- 5 Aspirin has substantially less efficacy, particularly among patients at moderate to high risk of stroke.^2- 3,6 However, concerns persist about the effectiveness and safety of anticoagulation with warfarin in persons treated in usual clinical care because the randomized trials enrolled highly selected patients, included few very elderly patients, and closely monitored anticoagulation. This has important clinical implications because atrial fibrillation occurs commonly, particularly among the elderly,⁷ and because the potential benefits vs risks of warfarin therapy are dependent on good control of anticoagulation intensity within a relatively narrow international normalized ratio (INR) range.^8- 10

Previous studies of thromboembolic and hemorrhagic outcomes of antithrombotic therapy for patients with atrial fibrillation treated in clinical practice settings have been relatively small, limited in clinical detail, or lacked information on longitudinal exposure to anticoagulation.^11- 18 As a result, it remains unclear how well randomized trials of anticoagulation apply to the broader spectrum of patients with atrial fibrillation in typical clinical settings. To address these issues, we systematically examined the effects of warfarin therapy on the risk of ischemic stroke and other systemic thromboembolism, intracranial and other hemorrhage, and all-cause mortality focusing specifically on follow-up of 11 526 adults with nonvalvular atrial fibrillation who did not have contraindications to warfarin therapy at study entry within the Anticoagulation and Risk Factors In Atrial Fibrillation (ATRIA) Study¹⁹ cohort (N = 13 559).

Assembly of the ATRIA cohort has been described in detail previously.¹⁹ Briefly, we identified all patients 18 years or older with diagnosed atrial fibrillation within Kaiser Permanente of Northern California based on physician-assigned diagnoses of atrial fibrillation found in ambulatory visit (International Classification of Diseases, Ninth Revision [ICD-9] code 427.31) and electrocardiographic databases between July 1, 1996, and December 31, 1997. Patients were included if they had serial outpatient diagnoses of atrial fibrillation, with the large majority having electrocardiographic evidence of atrial fibrillation.¹⁹ We excluded patients with diagnosed mitral stenosis, previous valvular repair or replacement, transient perioperative atrial fibrillation, or recent hyperthyroidism in order to include only patients with nonvalvular atrial fibrillation that was presumably not transient or due to a reversible cause.¹⁹

We searched clinical inpatient and ambulatory visit (outpatient clinic and emergency department) databases during the 5 years before the first outpatient atrial fibrillation diagnosis during the study period (index date) to identify previously diagnosed ischemic stroke, heart failure, coronary heart disease, and hypertension using relevant ICD-9 codes.^7,19 We used a validated, comprehensive health plan diabetes registry to identify patients with diabetes mellitus.^7,19 We obtained information from clinical inpatient and ambulatory-visit databases for the following potential contraindications to warfarin therapy: previous intracranial, gastrointestinal, and other bleeding; prior cirrhosis, hepatitis, or seizure disorder; prior mechanical fall; and diagnosed dementia.¹⁹ Information from medical care received at both health plan and non–health plan facilities was obtained through these databases.

Warfarin use was determined using a combination of information from prescriptions and INR measurements found in health plan pharmacy and laboratory databases, respectively. Longitudinal warfarin exposure was based on number of days of supply per prescription and intervening INRs. For any 2 consecutive prescriptions with a gap of up to 60 days, a patient was considered continually taking warfarin. For gaps longer than 60 days, we considered the patient continually taking warfarin if there were intervening INR measurements at least every 42 days. Otherwise, the patient was considered not taking warfarin from day 31 after the end date of the first prescription until the start date of the next prescription. This grace period of 30 days at the end of each warfarin period was given since changes in warfarin dosages are common. We validated the utility of this approach by comparing the anticoagulation status from the computerized algorithm with warfarin status documented in the medical record at the time of an outcome event for 1207 patients who experienced thromboembolism or hemorrhage during follow-up using chart review (κ = 0.84).²⁰

Of note, use of non–health plan pharmacies for warfarin was low (<6%) in this cohort,¹⁹ and approximately 80% of patients receiving warfarin were managed in pharmacist- or nurse-led anticoagulation clinics. Time spent in different INR levels was calculated using interpolation methods similar to those of Rosendaal et al,²¹ with gaps between INR tests greater than 8 weeks excluded.

From the index date through August 31, 1999, we prospectively searched hospitalization and billing claims databases for discharge diagnoses indicating potential thromboembolic and outcome events. A total of 783 potential thromboembolic events were individually adjudicated by medical records review. (ICD-9 codes used for primary discharge diagnoses indicating ischemic stroke and peripheral embolism are available upon request.) We excluded nonfatal hospitalizations with ICD-9 codes 433, 434, or 436.0 lasting less than 48 hours that were accompanied by carotid endarterectomy because chart review demonstrated that the overwhelming majority of these episodes were not acute strokes.

Medical records from all potential events were reviewed by a 3-physician clinical outcomes committee. If consensus was not reached, a final decision was made by a consulting neurologist. A valid ischemic stroke was defined as a documented acute neurological deficit lasting more than 24 hours that was not explained by other etiologies (eg, primary hemorrhage, trauma, infection, or vasculitis). A valid peripheral embolism was defined as an embolus identified by radiographic imaging, intraoperative examination, or pathological findings in the absence of underlying atherosclerotic disease in the affected artery. We reviewed a random sample of 35 admissions with only a secondary discharge diagnosis of ischemic stroke; only 11.4% had a validated stroke. Thus, we did not adjudicate admissions with only a secondary discharge diagnosis of ischemic stroke because only an estimated 24 additional strokes would have been added overall.

Because data on longitudinal aspirin use were not available from pharmacy databases and because aspirin has some efficacy for stroke prevention,⁶ we further refined our estimate of the effectiveness of warfarin vs no antithrombotic therapy by conducting a nested case-control analysis of antithrombotic therapy and systemic thromboembolism.^22- 23 This analysis included a sample of 294 cohort members with incident systemic thromboembolism during follow-up and 294 control patients who were matched to case patients for follow-up time in the cohort²⁴ and who had 1 or more outpatient clinic visits within 3 months before or after the matching case's event date. None of the case or control patients had a contraindication to anticoagulation at baseline. We reviewed automated databases and inpatient medical records for cases to determine the use of antithrombotic therapy (warfarin, aspirin, or no antithrombotic therapy). Among controls, we reviewed all available outpatient clinic medical records for antithrombotic therapy status during the 3 months before and after the matching case date if no evidence of warfarin exposure was found using our automated warfarin algorithm described above. A patient with mention of aspirin use in any clinic record during this period was considered an aspirin user.

We searched hospitalization and billing claims databases for primary and secondary discharge diagnoses of intracranial hemorrhage, and primary discharge diagnoses for gastrointestinal and other nonintracranial hemorrhage. Intracranial hemorrhage associated with concomitant discharge diagnosis of major trauma (ICD-9 codes 852.1, 852.3, 852.5, and 853.1) was excluded. (ICD-9 codes for discharge dignoses indicating intracranial, gastrointestinal, and other hemorrhage are available upon request.) Hemorrhagic events not leading to hospitalization or death were excluded. We adjudicated 659 potential hemorrhagic events using a similar approach as described above for thromboembolism. Major hemorrhage other than intracranial hemorrhage was defined as fatal, requiring 2 or more units of transfused blood or occurring in a critical anatomic location (eg, ocular hemorrhage impairing vision or retroperitoneal hematoma). In addition, medical records review of a random sample of 110 hospitalizations with secondary discharge diagnoses of hemorrhage (excluding intracranial) showed that 32.7% of these events were valid hemorrhages, but only 13.6% of these were major hemorrhages, so we did not include this approach in our search strategy. We estimate these excluded events would have added an absolute 0.48% to our aggregate annual rate of nonintracranial hemorrhage with a negligible addition to the rate of major hemorrhage.

All-cause mortality through August 31, 1999, was ascertained from hospital databases, health plan member reporting, and the comprehensive California state death certificate registry.²⁵ Membership gaps of more than 90 days without evidence of interim ambulatory care was considered disenrollment from the health plan, and patients were censored at the last known membership date.

The entire ATRIA cohort included 13 559 patients. The primary focus of our report is on the 11 526 cohort members with no known contraindications to warfarin at entry to the study.

Continuous variables are provided as mean (SDs), with comparisons between baseline warfarin users and nonusers using the t test. Categorical variables are reported as proportions with comparisons between baseline warfarin users and nonusers using χ² test. Crude event rates were calculated using log-linear (Poisson regression) models with a generalized estimating equations approach to account for the same patients' contributing person-years taking and not taking warfarin and are given as events per 100 person-years with 95% confidence intervals (CIs).²⁶ We excluded 52 validated events (47 ischemic strokes and 5 other thromboembolism) that occurred on the index date because it was unclear whether the atrial fibrillation diagnosis preceded the outcome event. Event rates for thromboembolism during periods of taking and not taking warfarin are given in the presence or absence of known risk factors for stroke, as well as by the CHADS₂ score, a recently proposed stroke risk stratification scheme for atrial fibrillation. The CHADS₂ index measures stroke risk by assigning 1 point each for congestive heart failure, hypertension, age 75 years or older, and diabetes mellitus, with 2 points added for patients who have a history of stroke or transient ischemic attack.²⁷

To evaluate the effectiveness of warfarin compared with no warfarin therapy for prevention of thromboembolism, Kaplan-Meier survival curves were constructed, and multivariable Cox proportional hazard models were performed that incorporated time-dependent information on warfarin use, as well as demographic characteristics and known risk factors for stroke identified during follow-up.^28- 29

We additionally used propensity score techniques³⁰ modified for time-dependent survival analyses in an attempt to further reduce confounding by indication for warfarin. Using this method, each patient's predicted likelihood of receiving warfarin therapy was assigned daily throughout the follow-up period and incorporated into regression models as a continuous time-dependent covariate. For all analyses of the 11 526 patients without contraindications to warfarin at baseline, patients were censored if they had acquired a new relative or absolute contraindication to anticoagulation because they were no longer considered eligible to receive warfarin therapy. In analyses of the entire cohort of 13 559 patients, there was no censoring because of their having developed a contraindication to warfarin therapy during follow-up, and propensity score methods included terms both for risk factors for stroke and for contraindications. Patients in our cohort considered not taking warfarin could either be taking aspirin or no antithrombotic therapy. We examined the relative effectiveness of warfarin compared with no antithrombotic therapy through our nested case-control study using multivariable conditional logistic regression.³¹ We used similar proportional hazard model approaches as described above to evaluate the safety of anticoagulation for the outcomes of intracranial hemorrhage and nonintracranial major hemorrhage. Finally, to examine the association between warfarin therapy and all-cause mortality, we performed proportional hazard regression adjusting for age, sex, known risk factors for stroke, and time-dependent propensity to receive warfarin.

A 2-tailed P value less than .05 was considered statistically significant. All analyses were conducted using SAS statistical software version 8.2 (SAS Institute Inc, Cary, NC). This study was approved by institutional review boards of the collaborating institutions.

Among 11 526 adults with nonvalvular atrial fibrillation and no known contraindications to anticoagulation at baseline, 43% were women and were a mean age of 71 years (Table 1). More than 75% of the cohort was aged 65 years or older. Eight percent of the cohort had a prior ischemic stroke, 28.5% had diagnosed heart failure, 50.1% had diagnosed hypertension, 16.8% had diabetes mellitus, and 27.7% had known coronary disease. Overall, those receiving warfarin at baseline were more likely to be men and have had a prior ischemic stroke, diagnosed heart failure, diagnosed hypertension, diabetes mellitus, and coronary heart disease (Table 1).

We accumulated 25 341 person-years of follow-up, with mean follow-up of 2.20 years (median, 2.35; interquartile range, 1.83-2.81, years). During follow-up, 533 patients disenrolled from the health plan.

During follow-up, there were 12 958 person-years of warfarin exposure among 7445 patients. The proportion of time spent in each INR range during follow-up was INR less than 1.5 (4.2%); INR, 1.5-1.9 (22.6%); INR, 2.0-3.0 (62.5%); and INR more than 3.0 (10.7%) based on more than 210 000 INRs performed during eligible periods while taking warfarin. The INR interpolation was not performed for 18% of the period during which patients were considered to be taking warfarin because of gaps between INR tests that exceeded 8 weeks.

There were 148 thromboembolic events (141 ischemic strokes, 7 other thromboembolism) that occurred among the patients receiving warfarin therapy (1.17 per 100 person-years; 95% CI, 1.00-1.38) compared with 249 events (231 ischemic strokes, 18 other thromboembolism) among patients not receiving warfarin (2.03 per 100 person-years, 95% CI, 1.79-2.30; P<.001; Table 2, Figure 1). Of 141 strokes that occurred among those taking warfarin, 4 had no INR available at presentation, and 87 (63.5%) of the remaining 137 had INR of less than 2.0. Thromboembolic event rates among those not taking warfarin were significantly higher for patients with risk factors for stroke, including prior stroke, diabetes, hypertension, diagnosed heart failure, coronary heart disease, and age 75 years or older, but the rates were substantially lower for those taking warfarin in these subgroups (Table 3). Lower rates of thromboembolism were also observed among patients taking warfarin at each level of the CHADS₂ score (Table 3).

There were 88 patients with intracranial hemorrhagic and 267 non–intracranial hemorrhagic events leading to hospitalization (237 gastrointestinal and 30 other sites) during follow-up (Table 2). Among 267 nonintracranial hemorrhages, 139 (52.0%) met criteria for major bleeding. The crude absolute rate of intracranial hemorrhage was only moderately higher among those taking vs those not taking warfarin although there were no significant differences in the rates of gastrointestinal hemorrhage or other hemorrhage (Table 2).

Effect on Thromboembolism. Using proportional hazards models that adjusted for time-dependent confounders (demographic characteristics and stroke risk factors) and the likelihood of receiving warfarin over time, we found that warfarin therapy was associated with a reduced thromboembolic risk, adjusted hazard ratio (HR) 0.49 (95% CI, 0.40-0.61), compared with no warfarin therapy.

Using a nested case-control design, we further refined the effect of warfarin compared with no antithrombotic therapy by taking into account the use of aspirin among patients considered not taking warfarin. After adjustment for potential confounders, warfarin was associated with an adjusted relative odds of thromboembolism of 0.36 (95% CI, 0.22-0.58) compared with no antithrombotic therapy, and an adjusted relative odds of 0.42 (95% CI, 0.25-0.71) compared with aspirin.

Effect on Bleeding. Warfarin was associated with a nearly 2-fold adjusted increased risk of intracranial hemorrhage (adjusted HR, 1.97; 95% CI, 1.24-3.13) compared with no warfarin therapy although there was no significant association between warfarin use and nonintracranial major hemorrhage (adjusted HR, 0.84; 95% CI, 0.59-1.18), after adjusting for potential confounders (Table 2).

Effect on All-Cause Mortality. There were 1235 deaths during follow-up among patients without a known baseline contraindication to warfarin, with a lower rate of death among patients taking (4.46 per 100 person-years) vs patients not taking warfarin (5.33 per 100 person-years, P = .002). Warfarin use was associated with a 31% reduction in the risk of all-cause mortality (adjusted HR, 0.69; 95% CI; 0.61-0.77), after adjusting for differences in age, sex, known risk factors for stroke, and time-dependent likelihood of receiving warfarin.

Among the 2033 subjects excluded from the main analyses above because of the presence of relative or absolute contraindications to warfarin therapy, 43.6% were prescribed warfarin at index date. These patients were older with a mean (SD) age of 75.6 (10.1) years and had a higher prevalence of risk factors for stroke at baseline than the 11 526 patients without contraindications to warfarin therapy (ie, 16.8% prior stroke, 42.6% diagnosed heart failure, 55.7% diagnosed hypertension, 20.0% diabetes mellitus, and 36.1% coronary heart disease). Thirty-seven thromboembolic events (32 ischemic strokes, 5 other thromboembolism) occurred among those taking warfarin (2.20 per 100 person-years; 95% CI, 1.59-3.03) compared with 109 thromboembolic events (104 ischemic strokes, 5 other thromboembolism) among those not taking warfarin (4.39 per 100 person-years; 95% CI, 3.64-5.31; P<.001) in this subgroup. After adjusting for potential confounders and the propensity to receive warfarin, warfarin therapy was associated with a 48% (95% CI, 24%-65%) decreased rate of thromboembolism.

The results for the entire cohort of 13 559 members, regardless of the presence of contraindications to warfarin therapy at baseline, were very similar to the stratified results shown above, except for modestly higher event rates that were in part due to patients' not being censored after the development of a contraindication to anticoagulation during follow-up. There were 204 thromboembolic events (190 ischemic strokes, 14 other thromboembolism) that occurred among patients taking warfarin (1.36 per 100 person-years; 95% CI, 1.19-1.56) compared with 394 events (369 ischemic strokes, 25 other thromboembolism) among those not taking warfarin (2.54 per 100 person-years; 95% CI, 2.30-2.81; P<.001). Using proportional hazards models that adjusted for time-dependent confounders (demographic characteristics, stroke risk factors, and potential contraindications to anticoagulation) and likelihood of receiving warfarin, warfarin therapy was associated with a reduced thromboembolic risk compared with no warfarin therapy (adjusted HR, 0.51; 95% CI, 0.43-0.61). The rates of intracranial hemorrhage were 0.51 (95% CI, 0.41-0.63) per 100 person-years among those taking vs 0.33 (95% CI, 0.25-0.43) among those not taking warfarin (adjusted HR, 1.57; 95% CI, 1.10-2.26).

Within a large ambulatory cohort of patients with nonvalvular atrial fibrillation in clinical practice who appeared eligible for anticoagulation, warfarin reduced the risk of ischemic stroke and peripheral embolism by 51% compared with no warfarin therapy. Our nested case-control analysis indicated this effect was even larger when compared with those taking neither aspirin nor warfarin. Warfarin reduced the risk of thromboembolism across known stroke risk factors in atrial fibrillation. Nearly two thirds of individuals sustaining an ischemic stroke while taking warfarin had an INR of less than 2.0, suggesting even greater benefit of anticoagulation, if maintained in the recommended range (INR, 2.0-3.0).^10,32 Anticoagulation was associated with nearly a doubling in the relative rate of intracranial hemorrhage, but the additional absolute risk of intracranial hemorrhage on anticoagulation was low. We also observed no significant increase in the rate of nonintracranial major hemorrhage on warfarin, likely reflecting preferential exclusion of patients at high risk for such bleeding. Finally, we observed a favorable association between warfarin use and all-cause mortality. These effects were also observed among the 2033 cohort members excluded from our main analyses because of mostly relative contraindications to warfarin therapy at baseline.

Our results are consistent with previous clinical trials of anticoagulation for atrial fibrillation and are particularly relevant given recent results from the Atrial Fibrillation Follow-up Investigation of Rhythm Management (AFFIRM)³³ and Rate Control vs Electrical Cardioversion for Persistent Atrial Fibrillation (RACE)³⁴ trials, which highlight the importance of long-term anticoagulation in patients with atrial fibrillation even after normal sinus rhythm is reestablished. The pooled intention-to-treat analysis of the first 5 primary prevention trials and a secondary prevention trial demonstrated that warfarin reduces the risk of stroke by two thirds³⁵ while aspirin was much less efficacious.³⁶

There have been concerns that the dramatic results from these trials might not translate directly to typical clinical practice. Patients enrolled in the trials were highly selected (eg, <10% of those screened in the Stroke Prevention in Atrial Fibrillation [SPAF] study were enrolled³⁷), few very elderly patients participated, and the high quality of anticoagulation management in the trials might not be duplicated in clinical settings. Previous studies of antithrombotic therapy in atrial fibrillation outside of trials have primarily studied selected patient populations (eg, hospitalized or nursing home patients with atrial fibrillation), involved relatively modest sample sizes, had small numbers of outcome events leading to less precise estimates of effect, or other methodological limitations.^{12- 14,16- 18,38- 40}

Our results materially extend these prior findings by providing contemporary and precise estimates of thromboembolism and hemorrhage rates in a broad population of individuals with atrial fibrillation, along with more complete adjustment for potential confounders and attempts to control for the likelihood of receiving warfarin over time. Our observed stroke rates were somewhat lower than what most prior studies had reported. In part, this lower risk may have been due to the ambulatory nature of our cohort and its inclusion of more low-risk younger patients than previous studies.^41- 42 In addition, the lower stroke rate may also reflect our use of validated stroke as an end point rather than the combination of stroke and transient ischemic attack or the use of claims data diagnoses alone. Analysis of the experience of the entire ATRIA cohort, including both patients with and without contraindications to warfarin therapy at baseline or during follow-up, resulted in a modestly higher rate of thromboembolism of 2.54 per 100 person-years. Of note, a recently reported study from the Framingham Heart Study also observed somewhat lower rates of ischemic stroke among patients with presumed new-onset atrial fibrillation.⁴³ Regardless, the beneficial impact of warfarin therapy on thromboembolic risk was confirmed.

Similar to the pooled analyses of randomized trials,³⁵ we also observed a favorable association of warfarin therapy and risk of all-cause mortality although these results should be interpreted cautiously given that we were unable to adjust for all known predictors of death. In addition, our relatively low absolute rate of intracranial hemorrhage among those taking warfarin was consistent with previous trials and observational studies, suggesting that anticoagulation can be safely administered in clinical practice for the generally older patients with atrial fibrillation.

This is the largest individual prospective study of atrial fibrillation to date and included substantially more thromboembolic and hemorrhagic outcomes than reports of pooled results of the clinical trials of antithrombotic therapy or of observational studies.⁶ Our cohort is an ambulatory population of patients from a real world setting and has greater age, sex, and racial diversity⁷ than populations in the clinical trials. For example, in comparing a pooled analysis of 5 randomized controlled trials,³⁵ our study included a greater proportion of elderly patients (mean age 71 vs 69 years; 23% vs 10% age ≥80 years, respectively) and women (43% vs 27%, respectively). There was also a higher prevalence of stroke risk factors and comorbid conditions in our cohort compared with trial populations, reflecting the broader spectrum of patients with atrial fibrillation seen in clinical practice. We used several complementary methods to identify clinically significant thromboembolic and hemorrhagic events and validated these outcomes through physician-based review of medical records using standardized criteria rather than administrative claims alone or patient self-report. In addition, longitudinal warfarin exposure was obtained using both comprehensive pharmacy and laboratory databases.

Our study also had several limitations. As a study of outcomes in clinical practice, our design is necessarily observational. There may still be residual treatment selection bias in our estimates of warfarin's impact despite censoring patients with a known relative or absolute contraindication to anticoagulation at baseline or during follow-up and additional accounting for risk factors before and after index date. We previously demonstrated the utility of our use of automated databases to identify stroke risk factors compared with chart review.¹⁹ However, any residual confounding by indication would have likely led to our underestimating warfarin's effectiveness in reducing ischemic stroke.³⁰ The similar crude rate of nonintracranial bleeding observed in patients taking or not taking warfarin suggests that we may not have completely identified all contraindications to anticoagulation (ie, residual confounding by contraindication).⁴⁴ Patient features, other than age, that predispose to intracranial hemorrhage are not well established, thereby minimizing the potential for residual confounding by contraindication in our estimate of warfarin's effect on this most feared type of bleeding.

We did not have information on aspirin use among all patients not prescribed warfarin, but our nested case-control study allowed an estimate of warfarin's effect compared with those taking neither warfarin nor aspirin. We also did not attempt to further characterize stroke subtype although the majority of atrial fibrillation–related strokes are considered embolic.⁴⁵ Although our study population has previously been shown to be representative of the surrounding northern California and statewide population,⁴⁶ our results may not be completely generalizable to uninsured populations, other geographic regions, or other health care settings. For example, within our health care setting, many of the patients receiving warfarin were followed up in specialized anticoagulation clinics that likely contributed to achieving 63% of the time in the therapeutic INR range (2.0-3.0), which was similar to the level of control seen in certain trials (eg, SPAF III³).

We may have missed a small number of thromboembolic or hemorrhagic events that did not lead to hospitalization or death; but given our comprehensive approach to capturing hospitalizations and deaths, we estimate that the likelihood of missed serious events was low and not differentially distributed by treatment status.

In conclusion, we found that warfarin appears to be very effective in usual clinical practice for stroke prevention in patients with nonvalvular atrial fibrillation and only marginally increases the absolute risk of intracranial hemorrhage. Overall, our results demonstrate that findings of the randomized trials of anticoagulation for atrial fibrillation translate well into clinical practice. Our study adds further support for the routine use of anticoagulation for eligible patients with atrial fibrillation who are at moderate to high risk for stroke, particularly when well-organized management of anticoagulation can be provided.^32,47