Statewide System of Electronic Notifiable Disease Reporting From Clinical Laboratories:  Comparing Automated Reporting With Conventional Methods FREE

Communicable disease surveillance is essential for protecting public health.^1- 2 State health departments rely on disease reports from clinicians and laboratories to rapidly identify and respond to outbreaks so that additional illness can be prevented.^3- 6 Recently, the Institute of Medicine cited "a glaring need" to strengthen disease surveillance to prepare for the threat of bioterrorism and address emerging pathogens.⁷ Previous evaluations of notifiable disease surveillance in Hawaii and elsewhere have found that reports are often submitted late and that communicable illnesses are substantially underreported.^5- 6 Most disease reports received by state health departments originate from clinical laboratories.^8- 11 Because the conventional methods of reporting via mail, facsimile, or telephone require active participation of laboratory staff, automated reporting from clinical laboratories has been proposed as a means to improve the quality and timeliness of disease notification.^12- 13 This report describes the first statewide system of electronic laboratory reporting (ELR) that uses new intercomputer communication technologies.

The Hawaii Department of Health (HDOH) has developed a prototypal laboratory-based electronic communicable disease reporting system (ECDRS) as part of a national effort to promote ELR by the Centers for Disease Control and Prevention.^14- 15 The ECDRS currently incorporates the 3 largest commercial clinical laboratories in Hawaii, which together serve all geographic areas and generate 60% to 70% of all laboratory-based notifiable disease reports received annually at HDOH.⁹ Operational since July 1998, the ECDRS remains a work-in-progress. Until it is finalized, HDOH has required the laboratories to submit notifiable disease reports simultaneously through both electronic and conventional reporting mechanisms. We present the results of an evaluation of disease reports received from laboratories participating in the ECDRS, in which we compared the timeliness and completeness of electronic and conventional reporting systems.

The HDOH requires laboratories to report positive test results for 48 communicable pathogens, called notifiable conditions. Traditionally, this reporting system has relied on laboratory staff, usually microbiologists, technicians, or infection control personnel, to recognize notifiable test results and to transcribe specific information regarding the laboratory findings, the patient, and the health care provider onto a standard form. Once completed, the forms are either mailed or sent via facsimile to HDOH. We defined this process as the conventional reporting system.

By contrast, the ECDRS uses automated data extraction and electronic communications to report notifiable conditions to HDOH. At a predetermined time each day, a computer dedicated for reporting purposes in each of the participating laboratories automatically connects to the laboratory information system in routine use at that facility. This connection launches a data extraction program stored within the laboratory information system that scans the admission-discharge-transfer file for test results posted the previous day. The extraction program then searches for records containing codes that identify particular laboratory procedures, and the accompanying "results" section of the record is compared with a data dictionary specific for that procedure. If the test results fulfill the criteria in the data dictionary, the record is flagged and written to a transfer file. The transfer file record contains the test results, details about the specimen, and the patient and health care provider identifiers available within the admission-discharge-transfer file. Following the extraction process, the transfer file is sent to the ECDRS computer, where it is encrypted using commercial software. At a set time each day, this computer connects via modem to a host computer server on standby at HDOH. The laboratory's identity is authenticated by a password before the file is transmitted to a restricted location on the host computer server. Authorized staff subsequently decrypt the file and review each record for content and accuracy.

To assess the consistency and accuracy of the data extraction and transmission process, all disease reports received through the ECDRS between July 1 and December 31, 1998, were reviewed by a single staff person to determine (1) the total number of records transmitted daily from each laboratory, (2) whether the results reported met the criteria for a notifiable condition, and (3) whether the report was unique or a duplicate. Conditions were considered not notifiable if the agent reported was not on the list of pathogens required to be reported in Hawaii, if the test results for a reportable pathogen were negative, or if the pathogen was required to be reported only under certain circumstances and the criteria for this were not met (eg, isolation from a normally sterile site). A report was considered a duplicate if it contained the same patient name, collection date, specimen number, and test result. Nonduplicate reports were termed unique.

In a separate analysis, electronic reports received via the ECDRS were compared with those received through the conventional reporting system during the same 6-month evaluation period for 5 notifiable conditions: Salmonella species, Shigella species, Giardia species, vancomycin-resistant Enterococcus, and Streptococcus pneumoniae isolated from a normally sterile site. These pathogens were selected for this evaluation because monitoring them is important for public health, but their occurrence is not so rare or alarming as to generate special attention from laboratory personnel. Moreover, the identification of these agents represents the end point of a variety of different laboratory procedures (eg, culture, microscopy) and requires varying degrees of data synthesis within the laboratory before a report is made (linkage to antimicrobial resistance and/or anatomic site information). The main outcome measures assessed were (1) the number and proportion of reports received by reporting system, laboratory, and pathogen; (2) the date reports were received at HDOH; and (3) the frequency with which a selected set of requested data elements was provided.

Since neither reporting system accounted for 100% of all the reports received, standard Chandra Sekar-Deming capture-recapture methods were used to estimate the total number of reports available for reporting at the laboratory level.¹⁶ Completeness of coverage was defined as the total number of reports actually received by each reporting mechanism, divided by the Chandra Sekar-Deming estimate of the number of reports available. The Fisher exact test and the Mantel-Haenszel test with Yates correction were used to determine the significance of the ratio of 2 proportions; confidence intervals (CIs) for mean values were calculated using Epi Info version 6.04c (Centers for Disease Control and Prevention, Atlanta, Ga).

Electronic reports were successfully transmitted from the 3 participating laboratories for an average of 133 (72%) of 184 days during the 6-month evaluation period (range, 118-154 days). A total of 15,830 distinct records were received through the ECDRS; of these, 3183 (20%) were duplicate reports and 3520 (22%) were test results not notifiable in Hawaii. The remaining 9127 records (58%) represented unique reports of test results for all notifiable conditions combined.

For the 5 notifiable conditions in the comparison study, a total of 357 unique reports of illness were identified, of which 325 (91%) were reported through the ECDRS system and 156 (44%) were reported via conventional means. As illustrated in Figure 1, 201 (56%) of the notifications were received solely through the ECDRS, 124 (35%) were received through both reporting systems, and 32 (9%) were identified only through the conventional mechanism. Dividing the total number of unique reports by the number received via the conventional system reveals that electronic reporting resulted in a 2.3-fold (95% CI, 2.0-2.6) increase in reports over those obtained through the conventional reporting system alone.

Completeness-of-coverage estimates for both reporting systems are presented by notifiable condition and laboratory in Table 1. In these analyses, electronic reporting accounted for a significantly greater proportion of reports for each of the 5 conditions and for reports originating in 2 of the laboratories (A and B); for the third laboratory (C), the number of reports received electronically equaled the number received via the conventional system. Overall, the electronic reporting system captured 80% of the estimated number of reports available at the laboratory level vs 38% for conventional reporting methods.

For the 124 reports received through both reporting mechanisms, electronic reports were received an average of 3.8 (95% CI, 2.6-5.0) days earlier than were the corresponding reports sent by mail or facsimile.

In general, electronic reports contained more complete information on the variables common to both conventional and electronic reporting formats. For 2 fields, those indicating patient last name and test result, reports from both systems had complete information. For 17 other fields, reports received electronically were more likely to be complete than were those received via conventional methods; this difference was statistically significant for 12 of the fields (Table 2). Conventional reports had more complete information for the remaining 2 fields, those indicating physician ZIP code and specimen type, although the difference was significant for the latter only.

This is the first report regarding a statewide system of automated electronic notifiable disease reporting from private clinical laboratories. For the notifiable conditions included in this evaluation, electronic reporting more than doubled the number of laboratory-based reports received. Such a substantial improvement in notifiable disease reporting could have major implications for public health, resulting in enhanced institution of disease control measures and more extensive disease outbreak investigations.

On average, electronic reports arrived several days prior to conventional reports. They were also more likely to contain key information, such as the telephone numbers for the patient and physician. These observations are important because early reports that have appropriate contact information should enable local health authorities to respond in a more timely manner. While our study was limited to 5 pathogens and, therefore, may not be representative of all laboratory-based notifiable conditions, the consistency of our finding that electronic reports outnumbered conventional reports for each of these agents suggests it may be generalizable to other laboratory results.

Along with the potential benefits of electronic reporting, this report documents challenges that remain to building dependable and efficient automated systems. During the evaluation, electronic transmissions were not received for almost 30% of the days reviewed. The consistency of reporting varied by laboratory and, in general, improved during the 6-month period (data not shown). Lapses in data transmission resulted from a variety of causes, including ongoing adjustments to the data extraction program and failure of the host computer server to reset after suboptimal modem connections. We did not adjust the percentage of all reports received electronically to account for days in which the electronic system did not transmit records, because we wanted to assess the de facto effect of the ECDRS system on reports received at HDOH during the study period. Our estimate that 80% of all notifiable laboratory reports could be captured and transmitted electronically is therefore conservative.

Reporting of extraneous records was problematic, as more than 40% of all reports received through the ECDRS were either duplicate records or results that were not notifiable in Hawaii. The majority of the nonnotifiable reports received were culture results for which reporting is not required (ie, Streptococcus pneumoniae isolated from a site that is not normally sterile, such as the throat). Improving the ability of the data extraction programs to exclude nonnotifiable results has been complicated by the diversity of coding schemes in use among the laboratories. Although each of the participating laboratories uses a major commercial laboratory information management system, these systems are highly configurable, and the test coding schemes in use at each facility have been individualized. Moreover, within each laboratory, the text placed in the test results field to identify a particular laboratory finding may vary. Therefore, the data extraction program and the data dictionary used in the ECDRS must be tailor-made for each facility. While we anticipate that the problem of nonnotifiable results can ultimately be resolved through refinements to the data extraction programs, our experience working in 3 different laboratory environments suggests that implementation of ELR systems would be greatly facilitated by widespread adoption of message standards and standardized coding schemes (eg, Health Level Seven, Laboratory Observation Identifier Names and Codes, and Systemized Nomenclature for Medicine).^15,17- 20 Without adoption of such standards, larger public health jurisdictions may find the "custom-built" approach used in Hawaii impractical.

Automating the elimination of duplicate reports will require imaginative programming solutions. Many notifiable disease reports are based on culture isolation and preliminary test results are reported at regular intervals during several days until the final test result is posted. Attempting to solve this problem by expunging all earlier test results for a given specimen is not optimal, because a single serum or stool specimen may be used for several different tests and information could be lost. In addition, while transmitting only final results would decrease the number of duplicate interim reports, earlier, albeit preliminary notification might allow for more timely public health intervention when indicated. For the immediate future, it appears that the deduplication process will require that records be individually reviewed by an experienced staff member.

Two potential concerns about the validity of this evaluation should be addressed. The first is that the relative benefits of electronic reporting cited here might actually be secondary to deterioration of the conventional reporting system. This could have occurred if the participating laboratories had relaxed their efforts to report through conventional means once implementation of the electronic reporting system had begun. To assess this retrospectively, we compared the number of laboratory reports received through the conventional system for the 5 notifiable conditions during the 6-month periods both before and after initiation of the ECDRS. During the period prior to initiation of electronic reporting, HDOH received 150 conventional reports for these 5 conditions from all laboratories in the state, of which 95 (63%) originated from the 3 laboratories participating in the ECDRS. In the period following initiation of electronic reporting, HDOH received 233 conventional reports for these conditions from all laboratories, of which 156 (67%) originated from the 3 participating laboratories. In other words, the proportion of all conventional reports that were submitted by the laboratories participating in the ECDRS remained stable, indicating that conventional reporting from these laboratories had not deteriorated once electronic reporting was initiated. This is not surprising, given that senior management and information management staff at the laboratories were aware that the legal obligation to report via conventional means would remain in effect while the ECDRS was being implemented.

An additional argument against attributing our findings to a deterioration of conventional reporting is the recognition that the conventional and electronic reporting systems operate quite independently. Historically, conventional disease reporting has been the responsibility of staff working at the bench level; the capacity to report electronically, on the other hand, has been developed by computer programmers who are separated organizationally, and often physically, from the areas of the laboratory where microbiology-related activities take place. In the laboratories participating in the ECDRS, the staff responsible for reporting diseases through the electronic and conventional systems perform their duties in relative seclusion from each other.

Another potential concern is that the electronic reporting system might have accounted for a greater number of illness reports because the data extraction and manipulation processes involved inadvertently created specious laboratory records that were then transmitted to HDOH. Although the diagnoses reported were not validated with repeat microbiologic testing, persons identified by laboratory reports as having shigellosis, salmonellosis, or giardiasis were interviewed by HDOH staff per routine procedure. We did not identify a single instance in which the interview failed to corroborate the validity of the electronic report (ie, the individual denied being evaluated for a clinically compatible illness, indicated that he or she did not provide an accordant specimen, or stated a different test result had been given to him or her by his or her health care provider). It seems highly improbable that mistakes with the automated data extraction could have resulted in erroneous yet completely plausible organism names being placed in the test results field; it is much more likely that the electronically reported results accurately reflect laboratory findings.

Other agencies considering developing ELR systems should note that the resources necessary to implement the ECDRS have been significant. Although we hope to have all laboratories in Hawaii included in the ECDRS eventually, we chose to begin with the highest-volume facilities to maximize the potential benefit for the level of effort required. In our experience, the costs for the computer hardware and commercial software involved are minimal, whereas the personnel resources required to develop this prototypal system have been substantial. At each of the participating laboratories, information management staff have had to be integrally involved in writing and refining the data extraction programs as well as solving other technical problems when they occur. This level of commitment has been undertaken without material compensation from HDOH, largely because these laboratories realize that ELR has the potential to improve the quality of disease reporting and, in the long-term, may be a cost-saving alternative to labor-intensive conventional methods. At HDOH, coordinating, monitoring, and troubleshooting the ECDRS project has required the nearly full-time attention of 1 professional staff member who has experience with computers and disease surveillance. This employee is assisted by data systems analysts under contract who make adjustments to the communications system and recommendations to the programmers at the laboratory on a regular basis. When considering costs, however, it is important to recognize that the resources required to develop an archetypal reporting system may be different than those needed to implement ELR once the requisite technologies have become better established. Incorporating the capacity to create ELR messages into commercial laboratory information system software may eventually permit wider, more economical adoption of electronic disease reporting.

At present, laboratories are required to report an average of 45 diseases or conditions under national surveillance to state public health authorities.²¹ If ultimately implemented across the nation with results similar to those reported here, ELR systems will likely have a positive impact on national morbidity figures and lead to a better understanding of communicable disease epidemiology. Given the increasing importance of laboratories in the fight against infectious diseases,^22- 23 applying neoteric communication technologies to laboratory-based reporting should enhance our efforts to identify emerging pathogens more comprehensively and respond to potential acts of bioterrorism more rapidly.^1,7,12 Finally, the value of ELR will be determined by the extent to which it results in public health action at the local level. Future work should assess whether ELR systems improve the ability of health care providers and public health departments to prevent illness through more timely and comprehensive implementation of disease control measures.