Interventions to Improve Research Participants' Understanding in Informed Consent for Research:  A Systematic Review FREE

Valid informed consent is a key to ethical research and a requirement of federal regulation. Although US federal regulations specify 8 pieces of information that must be disclosed to research participants,¹ ethically valid informed consent demands more than just disclosure. Research participants should also understand the essential disclosed information.^2- 4

Although not conclusive, available data suggest that research participants may frequently not understand disclosed information. For example, approximately 30% of participants in a cross-section of oncology clinical trials believed that their treatment had already been proven to be the best treatment for their cancer.⁵ In a randomized trial of β-blocker drugs to prolong the lives of patients with history of myocardial infarction, 44% of research participants interviewed did not know that they were assigned to treatment or placebo by chance.⁶ Many other studies suggest that similar shortfalls in understanding are widespread.^7- 11

These apparent deficiencies have prompted calls for investigators and institutional review boards to find ways to improve research participants’ understanding.^12- 13 Interventions using techniques like multimedia technology, enhanced consent forms, and extended discussion with educators have been developed to address these concerns. What interventions work to improve understanding, and how well do they work? To answer these questions, we critically assessed trials of interventions to improve research participants’ understanding of disclosed information.

This review included randomized controlled trials, longitudinal trials, and controlled trials with nonrandom allocation. All trials compared the understanding of research participants who had undergone a standard informed consent process with the understanding of participants who had received an intervention to improve their understanding. Trials were excluded unless both the control and intervention groups evaluated a real or simulated informed consent process that provided the information required by US federal regulations.¹ Finally, trials must have gathered quantitative data on participant understanding of the information disclosed.

We identified relevant trials by conducting systematic searches of MEDLINE, using the search terms informed consent and clinical research and informed consent and (comprehension or understanding), for a total of 5400 articles. A total of 334 articles passed an initial screening by title. Review of abstracts and text from those articles yielded 22 publications that met inclusion criteria. We searched a published directory of empirical research on informed consent¹⁴ and files belonging to specialists in research ethics. We also reviewed all issues of the journals IRB and Controlled Clinical Trials, which often publish research on informed consent. These search methods identified 21 of the 22 articles found in the MEDLINE searches as well as 8 additional articles. The 30 total publications reported on 42 trials.^15- 44 One publication duplicated data from another more thorough article.³⁸ Of the trials, 6 have not yet been independently published but were summarized in a review article.⁴³

Interventions were categorized into 5 groups: (1) multimedia, (2) enhanced consent form, (3) extended discussion, (4) test/feedback, and (5) miscellaneous. Overall, 12 trials tested multimedia interventions, using computer or video technology in place of^24,39,41,43 or in addition to^{21,26- 27,43} the usual written informed consent form. A further 15 trials evaluated consent forms with modified content, writing style, format, or length.^{15- 16,19- 20,22,29,31- 33,36- 37,41- 43} The 5 trials of extended discussion evaluated interventions in which a member of the study team or a neutral educator scheduled additional time to discuss the disclosed information with research participants.^{21,23,25,34,40} These interventions ranged from a 30-minute telephone conversation with a nurse²⁵ to multiple counseling sessions lasting up to 2 hours.⁴⁰ Another 5 trials evaluated test/feedback interventions, in which research participants were quizzed about the information disclosed to them and were given a review of questions that they answered incorrectly.^{17- 18,30,36,44} The 5 trials of miscellaneous interventions were not readily comparable with any of the other interventions that were tested.^{28,35- 36,43- 44} For example, 1 trial²⁸ put research participants through a week-long tryout period for the procedures in the protocol before asking for consent.

We reported 4 quality criteria for each study: whether a trial was randomized, whether the trial evaluated real or simulated informed consent processes, the number of participants in the trial, and whether the trial was published in a peer-reviewed journal. Simulated trials, which asked volunteers to consider a hypothetical or sham decision to enroll in research, were less realistic than those that compared 2 real informed consent processes. Overall, peer-reviewed randomized trials in a real setting with relatively large enrollment have the greatest validity. We chose not to use more formal quality rating systems because the best-validated systems are not clearly relevant to this set of trials⁴⁵ and there is controversy over the merits of quality-rating.^46- 47

Trials measured understanding quantitatively with either a self-administered questionnaire or a face-to-face interview. Our primary outcome for each trial was the significance of differences (with P<.05 indicating significance) between understanding scores for the control and intervention groups, as reported in the original articles.

We also presented the average understanding scores for participants in the control and intervention groups of each trial. We normalized all of these scores to a 100-point scale in which a higher number denotes a better average understanding score, with 100 being a perfect score on that trial’s instrument. These scores only showed the direction and approximate magnitude of differences in understanding between groups of a given trial. For complete information about how understanding differed between the control and intervention groups, it is necessary to refer in detail to scores on individual questions.

In addition, the diversity of the instruments used to generate understanding scores precludes direct comparison between trials. To compare different trials, the statistical significance of the study results is more appropriate and, even so, comparisons must be made very cautiously because these trials vary greatly in their methodologies, measurement instruments, populations, and interventions.

A minority of studies attempted to assess the effect of interventions on research participants’ satisfaction and accrual. Studies that assessed satisfaction simply asked participants to rate their satisfaction. Studies in a real setting measured accrual directly; studies of simulated consent processes asked participants to either make a hypothetical decision or rate their willingness to enroll. We included these results as secondary outcomes.

Overall, 12 trials of multimedia interventions revealed that such interventions often failed to improve research participants’ understanding (Table 1).^{21,24,26- 27,39,41,43} One published trial³⁹ showed a statistically significant improvement in understanding using a computerized presentation of information. The population for this trial was primarily mentally ill patients, with a few healthy volunteers. In addition, 2 unpublished trials reportedly produced increases in understanding, one from a video presentation and the other from a computerized presentation (Dr Nancy Kass, oral communication, January 2004).⁴³ But, the significance of these results is difficult to assess prior to complete analysis, peer review, and publication of the data. None of the other 9 trials reported a significant improvement in understanding, although 2 trials of video interventions that showed no increase in understanding immediately after disclosure did show improved retention of information weeks later.^26- 27

Of the trials with enhanced consent form interventions, 6 studies showed significant gains to understanding^{15,22,31- 33,37} while 9 did not (Table 2).^{16,19- 20,29,32,36,41- 43} The 9 negative studies included 4 of 5 more rigorous randomized controlled trials of real consent processes.^{19- 20,41- 42} Of the 6 studies that showed significant gains, 5 evaluated simulated consent processes, and in each of these simulated processes there was no discussion of the information in the consent form. In these 5 studies, the consent form was the only means used to disclose information to participants.^{15,22,32- 33,37} Since in most real consent processes research participants receive some information through discussion, the effect of improvements to the consent form is likely to be larger in such a hypothetical scenario than in a real research context.

When modifying consent forms, investigators used 4 basic strategies: condensing the length of the form, revising the content of the form to make it more comprehensible and readable, improving formatting through the use of techniques like larger font size and italics, and adding graphics. The data do not indicate that any of these approaches are more successful than the other approaches. But, the 1 randomized controlled trial in a real setting to show a significant improvement used a dramatically shortened form, from 4 pages to 2.³¹ This reduction in length was accomplished by removing standard but irrelevant information on risk.

Extended discussion between study staff and research participants resulted in statistically significant increases in understanding in 3 of 5 trials (Table 3).^25,34,40 Of the 2 negativetrials,^21,23 both showed trends toward improved understanding (P = .054 and P = .08, respectively). Three of the trials in this category have questionable validity, with small sample sizes and nonrandomized designs.^21,34,40

The test/feedback approach had significant impact on understanding (all P<.05) in all 5 trials that evaluated this approach (Table 4).^{17- 18,30,36,44} Each study in this category measured the understanding outcome using exactly the same questionnaire that was used in the test/feedback intervention itself. This is a serious methodological flaw because any improvement in the test score could reflect rote memorization of the answers to questions rather than increases in real understanding.

Among the 5 miscellaneous interventions, 2 were actually combinations of more common approaches (Table 5). One trial compared a standard consent process to one enhanced by a combination of extended discussion time, additional written information, and simple teaching aids.⁴⁴ A second trial used extended discussion time and teaching aids that included computerized presentation.³⁵ Both trials simulated a consent process and resulted in significant gains to understanding. A 1-week tryout period in which research participants underwent some protocol procedures before deciding whether to give consent was also associated with a significant improvement in understanding.²⁸ Two other trials did not show a significant increase in understanding.^36,43

Actual accrual to real protocols, or stated willingness to join simulated protocols, were secondary outcomes for 12 trials. Two multimedia trials^24,27 and 1 enhanced consent form trial¹⁵ noted significantly higher willingness to join for the intervention group, although the other trials noted no statistically significant difference.^{22,25- 26,31- 32,42,44} No trial documented statistically significantly reduced accrual or willingness to join due to an intervention.

Two enhanced consent form trials that asked research participants to rate their satisfaction with the disclosure processes found that participants were more satisfied with the enhanced form than with the control form.^29,42 Two more trials, which had an enhanced consent form and a computerized presentation, found that the intervention and control groups reported comparable satisfaction.^24,31 The only other trial to measure satisfaction found that an extended discussion intervention was viewed as “worthwhile” or “very worthwhile” by 89% of patients.²⁵

Overall, 12 studies showed that research participants with higher education or reading levels had significantly higher understanding scores (all P<.05).^{16- 20,22,25,29,34,39,41- 42} The differences in understanding due to education were often large, especially compared with the improvements produced by the various interventions. For example, 1 study documented that a test/feedback intervention could increase average understanding scores by 29% (from 69% to 89%, P<.01), but there was a 95% difference between the least educated quartile of research participants and the most educated quartile (44% vs 86%, P<.01).¹⁸ Another trial found that a revised consent form had a negligible effect on understanding but the gap between research participants who read at the seventh- and eighth-grade level and those who read at higher levels was highly significant (P<.001).²⁹

Mental illness was associated with lower understanding in the 4 studies that compared mentally ill research participants with healthy or medically ill volunteers (all P<.05).^{21,35- 36,39} In addition, increased age was associated with lower understanding in 5 studies that enrolled participants with mean age of older than 50 years (all P<.05).^{16,19- 20,25,42} However, 2 other studies reported no association between understanding and age.^34,39 Finally, sex and minority status did not show any consistent effect.

Overall, these data indicate that multimedia and enhanced consent form interventions do not consistently improve research participants’ understanding. Person-to-person interactions, especially the extended discussion interventions, may be more effective in improving understanding. This review suggests several conclusions and recommendations for policy and future research.

First, multimedia interventions have potential to improve understanding; however, that potential has usually not been realized in practice. Of 12 trials, only 1 published and 2 unpublished trials have documented an improvement in understanding due to such interventions.^39,43 Most participants in the published trial were mentally ill, suggesting that multimedia intervention might be especially helpful for that population. In addition, 2 trials that used video technology, which did not find an increase in understanding, did establish an increase in retention of disclosed information.^26- 27 If this result were further verified, video technology might be useful as a tool for improving retention of information. Finally, multimedia interventions might be useful as a way of standardizing disclosure. Even if multimedia interventions do not improve understanding, at least all research participants in a study are exposed to required information.

The lack of consistent improvement in understanding due to video and computer technology may seem surprising partly because previous studies have suggested that multimedia interventions increase patients’ understanding in routine medical care.^48- 49 This observation of improvement in clinical care but not in clinical research may result from the fact that in research the informed consent process is already formalized through federal regulations that require a written consent form. Video-based and computer-based interventions may not add much to this relatively thorough disclosure process. Indeed, in previous studies, when a decision aid was compared with standard medical care, increases in understanding were quite large, but when a more elaborate decision aid was compared with a simple one, increases were small.⁵⁰ In the same way, multimedia interventions could be much better than nothing but not necessarily better than the disclosure processes already common in clinical research.

Second, the data indicate that enhanced consent forms do not typically yield significant increases in understanding. Most of the trials of enhanced consent forms that show a significant effect simulate a consent process unrealistically because there is no discussion, only a reading of the form. In such a setting, the form becomes the participant’s only source of information and this exaggerates the impact of changes to the form. The 1 realistic trial³¹ that showed an effect suggests that shortening forms by removing unnecessary standardized content improves understanding.

Third, limited evidence suggests that more person-to-person contact, rather than videos or paper forms, may be the best way to improve understanding. Any enthusiasm about the test/feedback and extended discussion interventions should be tempered by the limited quality of the evidence supporting them. Furthermore, although these methods improve understanding scores, they do not guarantee perfect understanding scores.

Despite their limitations, the 10 trials of extended discussion and test/feedback interventions provide preliminary support for a hypothesis that direct human contact tends to be more successful in improving understanding than relying on tools like consent forms and multimedia interventions.^{17- 18,21,23,25,30,34,36,40,44} Part of the rationale for this hypothesis is that an extended one-on-one interaction with another person has more potential for active engagement and responsiveness to the individual needs of the research participant. This hypothesis would support the idea that informed consent is more than just the action of reading a form and signing it. It is better thought of as a process, ideally a dialogue, that takes place over time and largely depends on interactions between human beings.

Fourth, lower educational attainment, mental illness, and perhaps advanced age are associated with lower understanding. Indeed, the differences in understanding between well-educated and less well-educated individuals outweigh any improvement in understanding from the various interventions. These results may reflect poorer test-taking skills among less-educated research participants, causing them to score lower on tests of understanding even when their understanding is actually adequate. It may also indicate that these interventions are still not effective for individuals with less education and that disclosure processes need to be more appropriate for individuals with lower cognitive skills.

Fifth, available data suggest that there is little need to be concerned that interventions to improve understanding will detract from research participants’ satisfaction and discourage them from enrolling in research.

For clinical investigators who seek to better inform participants in their clinical trials, these data suggest 5 recommendations. First, multimedia interventions are not a reliably good investment for improving understanding, being expensive to produce with little demonstrable effect. There may be ways of using multimedia interventions that are effective but the burden is currently on investigators to show that a given use of multimedia is really helping research participants. Second, investing additional effort in forms is also unlikely to have a large effect on understanding, although some evidence suggests that brevity and the elimination of irrelevant “boilerplate” information helps research participants understand. Third, using a standard consent process and adding an extra meeting with a qualified person is the most reliable approach to improving understanding, based on currently available evidence. There is no evidence that this person needs to be an investigating physician; it might be ideal to use a nurse or outside educator to reduce the risk that a physician with vested interest in the study might unduly influence the research participant’s decision. Fourth, if resources are limited, consider targeting interventions to at-risk groups: less well-educated, mentally ill, and possibly elderly populations. Fifth, there is no reason to think that participants will be dissatisfied with interventions to improve understanding or that interventions will diminish accrual.

Our data also suggest 4 recommendations for future research into improving research participants’ understanding of disclosed information. First, a randomized controlled trial of test/feedback interventions that avoids the rote-memorization methodological problem is needed. Second, further studies of extended discussion would be useful, particularly if they collected data to define how much extra time is helpful and what aspects of the interaction are most effective at improving understanding. Third, future studies should avoid simulated consent processes because such simulations are often unrealistic. For instance, in evaluating improved forms, research results would be much more persuasive if they came from a real consent process in which the form assumes its real position as only part of the consent process. Finally, multimedia interventions may have value for certain populations, such as illiterate research participants in developing countries. However, multimedia interventions often fail to improve understanding and investigators who invest in multimedia should do research to confirm that their use of multimedia is actually improving understanding.

This review has several limitations. First, although review methods were systematic and drew on multiple sources, many trials in this area are quite obscure and were not even identified by a broad search of MEDLINE. It is not possible to guarantee that we have identified every published and unpublished trial that would meet our exclusion criteria. In addition, our exclusion criteria indicated that observational and other studies without an appropriate control group were not surveyed. Second, the results of trials with complex methodologies and outcomes are presented in a brief format and it is necessary in most cases to refer to the original article to obtain a nuanced understanding of the results of the trial. Third, there was no single objective way to weigh the results of these trials based on our assessment of their quality, introducing an additional element of subjectivity into our interpretations of these results. Finally, variation in instruments and in populations from trial to trial makes quantitative comparison of understanding scores not meaningful. Hence, it was necessary to analyze these data in a review, rather than in a meta-analysis.