Quality of Life, Burnout, Educational Debt, and Medical Knowledge Among Internal Medicine Residents FREE

Distress among physicians during medical training is common.^1- 3 Factors such as physician burnout, depression, job dissatisfaction, and low quality of life (QOL) have been associated with negative effects on patient care, including major medical⁴ and medication⁵ errors, suboptimal care practices,⁶ and decreased patient satisfaction with medical care.⁷ Despite the relevance of physician well-being to patient care outcomes, little is known about patterns of distress across demographic factors. To our knowledge, there has not been a nationally representative examination of factors that may be associated with well-being of internal medicine residents.

In addition, the mediators of the relationship between physician well-being and the provision of patient care are unclear. Physician distress may affect quality of care through effects on professionalism, commitment, and attention to detail.^8- 9 Physician well-being also has been hypothesized to contribute to competency in medical knowledge and to learning, which in turn contribute to competence in delivery of patient care.¹⁰ This model is consistent with limited evidence in humans suggesting that chronic or severe stress may negatively influence learning and memory.¹¹ However, recent research in internal medicine residents did not find associations between well-being and medical knowledge as determined by a standardized general medical examination,¹² calling this relationship into question.

To more fully understand the prevalence of resident distress across demographics as well as its association with the key training competency of medical knowledge, we surveyed a large national cohort of internal medicine residents. We assessed their QOL and symptoms of burnout and examined demographic variables that may be related to well-being. We also examined relationships with medical knowledge and learning as measured by change in medical knowledge on standardized testing during training. We hypothesized that distress would be associated with greater educational debt, lower test scores, and decreased learning.

This study and waiver of informed consent were approved by the Mayo Clinic institutional review board. The Internal Medicine In-Training Examination (IM-ITE) is a standardized 340-item multiple-choice self-assessment examination offered yearly in October to internal medicine residents enrolled in US programs.^13- 14 The primary goal of the IM-ITE is to evaluate progress in training by identifying areas of deficiency that require further learning. Achievement of a set level of performance is not a goal of the examination, and studying for the IM-ITE is specifically discouraged.¹⁵ Demographic data including sex, location of medical school, age, and postgraduate year are recorded, and examinees are also asked to voluntarily complete an accompanying survey at the end of the examination with questions regarding perceptions of the examination, career interests, educational debt, use of study materials, and perceptions of their training environment.

In 2008, questions concerning quality of life, satisfaction with work-life balance, and burnout were asked as detailed below (see eAppendix). Medical knowledge in this study was determined by the percentage of correct answers on the IM-ITE. Mean scores are consistently near 60% on the IM-ITE across administrations. Learning was measured by the change in IM-ITE score from 2008 to 2009.

Overall QOL was measured by a single-item linear analogue self-assessment (LASA). This instrument measured overall QOL on a scale of 1 to 5, with response anchors ranging from “As bad as it can be” (1) to “As good as it can be” (5). Low QOL was defined as response in the lowest 2 categories of this item. Linear analogue self-assessment instruments have been validated across a broad range of medical conditions and populations and are widely used in QOL research.^4,16- 18 We assessed residents' satisfaction with the balance between their personal and professional life on a similar 5-point Likert scale, with low satisfaction defined as response in the lowest 2 categories of this item. This question was adapted from a validated questionnaire that has been used in prior research.^19- 20

The Maslach Burnout Inventory²¹ (MBI) is considered the reference standard for the assessment of burnout. However, its length (22 items) limits the feasibility of its use in large surveys addressing multiple content areas within space constraints. Many evaluations of burnout have focused on the presence of high levels of either emotional exhaustion or depersonalization as the cornerstone of burnout among high-achieving medical professionals.^3,22 Therefore, in this study, symptoms of burnout were assessed using 2 single-item measures adapted from the full MBI: emotional exhaustion was assessed by the question, “How often do you feel burned out from your work?” and depersonalization by the question, “How often do you feel you’ve become more callous toward people since you started your residency?” Each question was answered on a 7-point Likert scale with response options ranging from “never” to “daily.” These single items correlated strongly with the emotional exhaustion and depersonalization domains of burnout as measured by the full MBI in a sample of more than 10 000 medical students, residents, and practicing physicians.²³

Symptoms of high emotional exhaustion were defined by a frequency of at least weekly on the single-item emotional exhaustion measure. Similarly, symptoms of high depersonalization were defined by a frequency of at least weekly on the single-item depersonalization measure. The area under the receiver operating characteristic curve for the emotional exhaustion and depersonalization items compared with their respective full MBI domain measures was 0.94 and 0.93. The positive predictive values of the single-item thresholds for high levels of emotional exhaustion and depersonalization used in this study were 88.2% and 89.6%, with positive likelihood ratios of 14.9 and 23.4, respectively.²³

We applied standard univariate statistics to characterize the sample. Because of the large sample size and multiple comparisons, 2-tailed statistical significance was set at an α level of .01. Multivariable analyses were conducted using logistic regression for binary outcomes, ordinal logistic regression analysis with generalized logits for ordinal outcomes, and multivariable generalized linear models for continuous outcomes. Because the more complex ordinal logistic regression results did not alter the summary conclusions, the simpler multivariable logistic regression model results are reported for the categorical outcomes. All 2-way interactions were evaluated for all models.

Statistical analyses were conducted using SAS version 9.1 (SAS Institute Inc, Cary, North Carolina).

The IM-ITE was completed by 21 208 internal medicine residents in October 2008, representing 95.8% of the 22 132 US internal medicine residents in training during the 2008-2009 academic year.²⁴ Of those completing the examination, 19 831 (93.5%) returned surveys. After merging with demographic data provided by the National Board of Medical Examiners, valid responses were available from 17 820 residents (84.0%) in US categorical and primary care training programs, of whom 14 958 (70.5%) responded to all variables and an additional 1436 (6.8%) responded to at least 1 variable in the current study, yielding 16 394 (77.3%) total respondents. Of these 16 394 residents, 5275 were graduating third postgraduate year (PGY-3) residents in 2008 and did not provide data in 2009. Of the remaining 11 119 residents, 8396 (75.5%) provided data in 2009 and were included in longitudinal analyses related to learning.

Demographic characteristics of the 2008 sample and the subsample of respondents to both the 2008 and 2009 IM-ITE surveys were similar (Table 1). IM-ITE scores of survey responders also did not differ from scores of nonresponders. Sex and medical school site proportions closely matched available national demographic data for internal medicine residents in the 2008-2009 academic year.²⁴

Overall QOL was classified as being “as bad as it can be” or “somewhat bad” by 14.8% (99% CI, 14.1%-15.6%) of residents (Table 2). With respect to work-life balance, 32.9% (99% CI, 31.9%-33.8%) reported being somewhat or very dissatisfied. Symptoms of emotional exhaustion at least weekly were reported by 7394 of 16 154 residents (45.8% [99% CI, 44.8%-46.8%]), and 4541 of 15 737 (28.9% [99% CI, 27.9%-29.8%]) reported symptoms of depersonalization at least weekly. Overall, at least 1 symptom of burnout was present in 8343 of 16 192 residents (51.5% [99% CI, 50.5%-52.5%]).

Quality of life was highest for PGY-3 residents and was higher among categorical and international medical graduate residents (Table 3). PGY-1 or PGY-2 residents, women, primary care residents, and US medical graduates were more likely to report dissatisfaction with their work-life balance. QOL and satisfaction with work-life balance were both lower among residents with educational debt, particularly at the highest level of debt (>$200 000) (Table 3). Quality of life and satisfaction with work-life balance were higher among residents who reported moonlighting (working at a second job outside of regular residency training duty hours). Analyses for which low QOL and low satisfaction with work-life balance were defined by the lowest 3 levels of these factors yielded similar results, except that differences between categorical and primary care residents were no longer found and international medical graduates had lower QOL (eTable 1).

Symptoms of emotional exhaustion decreased as year of training increased, while symptoms of depersonalization increased after the PGY-1 year (Table 4). Women and residents in primary care programs reported more frequent symptoms of both emotional exhaustion and depersonalization. Substantially greater emotional exhaustion and depersonalization symptoms were also seen among US medical graduates relative to international medical graduates.

Emotional exhaustion and depersonalization increased as educational debt increased (Table 4). Comparing residents having more than $200 000 of debt with residents having no debt, the odds ratios for emotional exhaustion, depersonalization, and burnout in either domain were 1.59 (99% CI, 1.38-1.84), 1.80 (99% CI, 1.54-2.11), and 1.72 (99% CI, 1.49-1.99), respectively (each P < .001).

Although international medical graduates were less likely to have debt (eTable 2), the association of debt with burnout symptoms was particularly prominent for international graduates. The degree of burnout in the highest categories of debt for international medical graduates approached or even exceeded that of US graduates (interaction P < .001 for emotional exhaustion, depersonalization, and burnout) (eFigure). Rates of high emotional exhaustion also increased with escalating debt more markedly for women than men (interaction P = .004). Among residents with no debt, the rate of high emotional exhaustion was 40.8% in women and 38.4% in men (99% CI for difference, −0.9% to 5.7%; P = .06). For those reporting debt greater than $200 000, the rate of high emotional exhaustion was 60.0% in women and 49.8% in men (99% CI for difference, 4.6%-15.9%; P < .001). Moonlighting, although slightly more common among residents with debt (eTable 2), was not statistically significantly associated with burnout at the α = .01 threshold prespecified in this study.

The IM-ITE scores increased as training progressed, by a mean of 4.1 points (99% CI, 3.9-4.3; P < .001) from PGY-1 to PGY-2 and 2.6 points (99% CI, 2.4-2.8; P < .001) from PGY-2 to PGY-3 (Table 1). Although statistically significant, differences by sex and medical school location were small. IM-ITE score decreased as educational debt increased, with a mean difference of 5.0 points (99% CI, 4.4-5.6; P < .001) between residents reporting no debt and those reporting debt exceeding $200 000.

The association of debt with IM-ITE score was stronger for international medical graduates than for US graduates (interaction P < .001) (Figure 1). Among residents with less than $50 000 of debt, test scores for US and international medical graduates did not differ. However, scores diverged at higher levels of debt, and among residents with more than $200 000 of debt, mean test scores were 58.4 for US medical graduates and 54.2 for international graduates (99% CI for difference, 3.1-5.2; P < .001). The relationship between debt and medical knowledge was also stronger in men than in women (interaction P = .002) (Figure 2). Men had higher IM-ITE scores overall, but the gap narrowed as debt increased, with no difference in test scores at the highest level of debt. Across years of training, IM-ITE scores consistently decreased as debt increased (interaction P = .22) (Figure 3).

Decreased QOL, decreased satisfaction with work-life balance, and increased frequency of burnout symptoms were associated with lower IM-ITE scores (Table 2). IM-ITE scores were most notably decreased in the 242 residents (1.5%) reporting QOL that was “as bad as it can be” (mean, 57.6 vs 60.3 for the 2469 [15.3%] residents reporting QOL “as good as it can be” [difference, 2.7 points; 99% CI, 1.2-4.3; P < .001]). IM-ITE scores also were lower for the 600 residents (3.7%) with daily feelings of emotional exhaustion (mean, 57.8 vs 62.0 for the 243 residents [1.5%] never feeling burned out from work [difference, 4.2 points; 99% CI, 2.5-5.9; P < .001]). Although statistically significant, differences in medical knowledge scores across levels of depersonalization and dissatisfaction with work-life balance were small (Table 2).

Associations of debt and well-being with learning as measured by the change in IM-ITE score from 2008 to 2009 were not seen. Scores on the IM-ITE increased by approximately 3 points at all levels of measured aspects of debt and well-being (eTable 3 and eTable 4). Thus, residents with lower QOL, more burnout, or greater debt burden who started with lower scores did not recover to the level of their colleagues during the course of training.

This study of nearly three-fourths of all internal medicine residents-in-training in the United States in the 2008-2009 academic year represents, to our knowledge, the first national cohort of residents reporting on QOL, satisfaction with work-life balance, burnout, and debt. This study is also to our knowledge the first large-scale report on the association of these factors with an objective measure of a key clinical competency, medical knowledge.

Dissatisfaction and personal distress were common across the cohort. In particular, the prevalence of burnout symptoms in this national cohort is consistent with that reported in multiple previous studies of residents, in which burnout rates have commonly exceeded 40%.³ Because all members of the study cohort began training after duty hour limits went into effect in 2003, these results suggest that distress remains common despite these regulations.

Symptoms of emotional exhaustion were more common among PGY-1 residents and decreased with each year in training, while symptoms of depersonalization (which may manifest as cynicism and callous attitudes toward patients)²¹ increased as training progressed. The factors driving these differences have not been fully elucidated, although one possible explanation is that the accumulated effects of emotional exhaustion early in the residency training process lead to longer-term erosion of physician idealism, even after emotional exhaustion improves.^3,8- 9 This may argue in favor of further protections during internship against contributors to emotional exhaustion such as fatigue, lack of resources and support, and work overload,^3,25- 26 although the optimal ways to reduce burnout have not been identified.²⁵

Higher levels of educational debt were associated with lower QOL and higher rates of burnout. These results are consistent with previously reported associations of educational debt with decreased career satisfaction,²⁷ depressive symptoms, and cynicism.² Although these results are consistent with a significant negative effect of educational debt on QOL and burnout, it is unclear if this would represent a direct effect of debt burden. It is notable, however, that moonlighting does not appear to explain these effects, because it was not associated with burnout in this study in multivariable analysis and was actually unexpectedly associated with improved QOL.

International medical graduates were markedly less likely than US graduates to report high levels of emotional exhaustion or depersonalization. This observation may suggest different experiences of burnout between these groups. One possible explanation is that international medical graduates training in US residency programs represent a subset of international graduates who are more resilient and less prone to burnout owing to their successful navigation through the complex and highly competitive selection process for US residency positions as foreign graduates.^28- 29 The lower rates of burnout symptoms among international medical graduates persisted in multivariable analyses after adjustment for debt, suggesting that lesser debt among international graduates does not explain the observed differences in burnout between international and US medical graduates. The experience of international medical graduates in US training programs is poorly understood,^28- 32 and these results should prompt renewed study of this important constituent of the physician workforce.

The frequency of low measures of well-being among residents across all demographic groups is particularly concerning, given the previously demonstrated adverse associations of distress with quality of care. Advancing understanding of these associations, the results of this study suggest that medical knowledge is lower as emotional exhaustion increases and at the lowest levels of QOL. The magnitude of these associations is illustrated by the fact that the differences in medical knowledge seen across levels of both distress and debt were as large as differences across entire years of training.

A previous study did not find a relationship between well-being and medical knowledge, likely because of small sample size and limited prevalence of the lowest levels of well-being.¹² The lowest QOL category affected only 1.5% of responding residents in our study, so the potential impact of the observed association of QOL with medical knowledge appears small. However, the most extreme emotional exhaustion category affected 3.7% of residents, and IM-ITE scores were also notably lower in the next highest category of emotional exhaustion, affecting an additional 22.1% of residents. In addition, the highest 2 categories of debt involved 13.1% and 14.3% of residents, respectively. Taken together, high levels of debt and emotional exhaustion are common among internal medicine residents nationally, and therefore their negative association with test performance warrants attention.

The association of increased debt with lower test scores was particularly strong for international medical graduates, in whom the difference in mean IM-ITE score between those with no debt and those with more than $200 000 in debt was 8 points (similar to the difference in score between PGY-1 and PGY-3 residents). Men also had steeper declines in test scores with higher debt levels than did women. It is unclear why debt should exhibit stronger associations with distress or medical knowledge in these resident groups, but the differences are sufficiently large to merit further study.

Although strong relationships between aspects of well-being, debt, and medical knowledge were identified in this study, we found no relationships with learning. There are many possible explanations for this. For example, it is possible that the stress of the examination itself might affect residents' well-being or performance at the time of the examination but not correlate with cumulative performance over the course of training. This concern is diminished by the low-stakes nature of the IM-ITE, which is not intended to affect residents' training evaluations, ability to graduate, or future certification and licensure. Furthermore, although chronic stress may lead to burnout, acute stress and fatigue differ from burnout,²¹ and such effects also would not explain the observed associations with debt.

The findings with respect to learning also may reflect an association of distress or debt with prior acquisition of medical knowledge, such as in medical school. Medical knowledge may be gained during residency in a different manner than it is during medical school. For example, stress may actually enhance learning when the content is directly related to the stressor.³³ Because much of what is learned during residency training occurs in the context of potentially stressful direct patient care activities, perhaps learning during residency training is relatively protected.

Regardless of the explanation, internal medicine residents with greater degrees of emotional exhaustion and debt, and lower QOL levels, consistently scored less well on the IM-ITE and did not recover to the level of their colleagues over the course of their training. Additional studies are needed to better understand these relationships and their effect on clinical competency. In particular, as debt relief programs are developed for physicians in training and in practice,^34- 35 their effect on both well-being and patient safety should be assessed.

This study has several additional limitations. First, although the IM-ITE is completed by nearly all US internal medicine residents and the percentage of residents completing the survey was high, it remains possible that nonresponse bias could affect the results. This concern is mitigated by the similarities of the sample to national demographic data, but it remains possible that differences in unmeasured variables were present.

Second, not all domains of well-being were assessed in this study, so these results do not provide insight into depression, job satisfaction, or other factors related to well-being. In addition, many demographic variables were not evaluated. Personal factors such as race, socioeconomic status, marital status, or parenting status, and program or institutional factors such as research emphasis or curricular structure, also may be associated with well-being, debt, or medical knowledge.^1,3,26,36 For example, debt may be a surrogate for another factor such as socioeconomic background rather than a causal factor affecting well-being or medical knowledge. However, literature suggests lower levels of burnout³⁷ and similar levels of debt³⁸ among minorities pursuing medical careers.

Third, because educational debt was evaluated by self-report, it is possible that some debt responses were misclassified. Fourth, prior performance on standardized examinations could not be accounted for in this study. However, a recent study of premedical debt among matriculating medical students showed associations of debt with failure on licensing examinations, even after adjustment for previous examination performance.³⁹

Fifth, the definition of learning in this study was restricted to an increase in medical knowledge over time. Associations of distress and debt with learning across the full range of medical competency domains have not yet been fully explored and remain an area open to further study. Sixth, it is not possible to determine causation from this study, so the observed relationships between QOL, symptoms of burnout, debt, and medical knowledge are best interpreted as associations.

In summary, suboptimal QOL and dissatisfaction with work-life balance were common in this national cohort of internal medicine residents, as were burnout symptoms of emotional exhaustion and depersonalization. Additional research is needed to investigate how the factors affecting the well-being of resident physicians interact with each other and with clinical competency.