Usability of the Medicare Health Web Site FREE

To the Editor: The use of Internet-based applications to support health management tasks has grown enormously.¹ Studies investigating these applications have focused on prevalence of or reasons for use. We evaluated the ability of a sample of older adults to use the Medicare.gov Web site to make decisions concerning eligibility for services and prescription drug plans.

Two hundred eighteen people responded to newspaper advertisements and flyers in community and senior centers in the greater Miami area through telephone contact. Participants were required to be aged at least 50 years; be cognitively unimpaired (Mini-Mental State Examination² score >26); have had computer experience (assessed via questionnaire³ ); be English speaking; and not have depressive symptoms or severe visual or hearing impairments. Following screening, 201 were eligible for participation, 64 of whom expressed lack of interest or did not report for the study. Participants were enrolled consecutively; 112 completed the protocol and were compensated $50 (Table 1). To characterize the sample, data were collected on age, education, and race/ethnicity (based on self-report using fixed categories⁴ ). Data were collected at the University of Miami between February 2006 and May 2007. The study was approved by the university's institutional review board, and all participants provided written consent.

Participants were asked to use the Medicare.gov Web site to (1) determine eligibility for home health care services, (2) select a home health agency to meet specified needs, (3) make a decision about enrollment in Medicare Part D based on specified criteria, and (4) select a drug plan and determine associated costs based on a specified medication regime. They provided written responses. Participants also evaluated Web site usability and the value of Internet health information via questionnaire. Responses were rated by 2 investigators using a scoring sheet and through analysis of video records.⁶ Interrater reliability on 2 problems in a random sample of 20 participants was good (r = 0.96 and r = 0.97).

During data collection, changes were made to the appearance of the home page and sizing of selection buttons on the Web site; content was not altered. To assess comparability, we compared performance scores, usability ratings, and characteristics of participants who used the initial (n = 82) and modified (n = 30) versions using χ² tests, t tests, and Wilcoxon tests. No differences were found for ratings of usability or performance scores. A greater percentage of Hispanic (46.2%) and non-Hispanic white (44.9%) participants used the initial version than non-Hispanic black participants (9.0%), but more non-Hispanic black participants (42.9%) used the modified version than Hispanic (28.6%) or non-Hispanic white participants (28.6%) (P < .001). Analyses were performed using SPSS 15.0 (SPSS Inc, Chicago, Illinois), with 2-sided significance set at P < .05.

Most participants were unable to specify all eligibility criteria for home health services (n = 77 [68.8%; 95% confidence interval {CI}, 59.8%-76.8%]), choose the correct home health agency (n = 90 [80.4%; 95% CI, 72.3%-86.9%]), or execute computation procedures needed for making a plan enrollment decision (n = 94 [83.9%; 95% CI, 76.3%-89.8%]). Only about half (n = 64 [57.1%; 95% CI, 47.9%-66.0%]) were able to make an enrollment decision. Most participants (n = 81 [72.3%; 95% CI, 63.5%-80.0%]) were unable to select a drug plan or had problems navigating to the necessary Web pages, locating needed information, or executing actions required for plan selection. Participants also reported problems with usability (Table 2), but almost all indicated that the Internet was a valuable source of health information.

A sample of relatively well-educated adults with computer experience had difficulty using the Medicare.gov Web site to determine eligibility for services and enroll in a drug plan. Participants also reported problems navigating within the Web site and locating information. Although the sample was relatively small and restricted to people aged 50 years or more, it was diverse and larger than samples included in most usability studies. It is likely that persons with less computer experience would have even greater difficulty. To ensure that electronic health tools reach their full potential, broad and inclusive input from consumers should serve as the basis for design.