Impact of a Large-Scale Immunization Initiative in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) FREE

AFTER THE MEASLES resurgence of 1989-1991, public and private investment in immunization increased dramatically, and immunization rates have risen across the nation.^1- 3 However, approximately 80% of unvaccinated measles cases during the resurgence occurred among urban preschoolers,^4- 5 and immunization rates in inner cities have often lagged behind the rest of the country.^3,6 This raised concerns that high-risk urban children might lack the minimum level of population immunity necessary to prevent outbreaks.⁷

A 6-month randomized trial in 1991 showed that varying the frequency of food voucher issuance (voucher incentives) in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) could increase measles vaccination rates among preschoolers enrolled in 6 of New York City's more than 100 WIC sites.^8- 9 These findings have been confirmed by preliminary results from 2 randomized trials elsewhere.^10- 11 Since 44% of the US birth cohort is enrolled in WIC (US Department of Agriculture, unpublished data, 1995), with age and demographic characteristics that match immunization needs, a nationwide initiative to promote activities that raise immunization coverage in WIC began in 1995, with the endorsement of the Advisory Committee on Immunization Practices (ACIP).¹² States and localities now must spend a minimum of 10% of all federal immunization funds on WIC linkages (1996 Senate Appropriations Committee Report language). A recent survey suggested that 75% of all states have already established WIC immunization linkages.¹³

This national WIC initiative represents the most sweeping effort to link immunization to a public program since the school law campaigns of the 1970s. But can interventions, which were successful in relatively small, time-limited, and intensively monitored studies, actually raise coverage when they are applied programmatically to whole populations, particularly in inner cities where infrastructure problems are intense? Questions have also been raised as to whether some immunization interventions in WIC might have an adverse effect on enrollment.

Chicago, Ill, was a major source of cases during the measles resurgence,^4,14- 16 with data indicating that city-wide vaccination coverage was 16% to 25% over 8 years, despite the epidemic.¹⁷ A 1994 survey of children residing in public housing suggested that immunization series completion rates were 23% by 2 years of age.¹⁸ In 1995, the Chicago Department of Public Health (CDPH) began implementation of immunization interventions in WIC, a supplemental food voucher program in which approximately 70% of the Chicago birth cohort is enrolled. We present the first evaluation of the impact of a large-scale immunization initiative through WIC.

The Chicago WIC is administered through 47 sites, at which about 37000 infants are enrolled (Table 1). Approximately 45% of these clients are served by 22 sites under the direct administration of the CDPH, while the rest are served by 25 sites administered by contract agencies. In December 1995, CDPH required its sites to begin the following: (1) to ask all families with children 24 months of age or younger to bring vaccination documentation to WIC visits; (2) to assess the vaccination status of these children; and (3) to refer children in need of vaccinations to a provider. In May 1996, CDPH began the progressive implementation of voucher incentives for immunization (see below). By June 1997 (the end of the evaluation period), 14 sites were providing voucher incentives. To implement this ongoing initiative, 11 clerks were hired, under the supervision of a coordinator, at an annual cost of $271000.

Interventions.Assessment and Referral. At each WIC certification and recertification visit (which occur every 6 months), the clerk reviewed the vaccination status of each child 24 months of age or younger. Where documentation of vaccination was provided (eg, vaccination card), the vaccination dates for each antigen were entered into a software program on the WIC computer. The program determined if the child was age-appropriately vaccinated by ACIP standards. If the child was not age-appropriately vaccinated, the family was referred to a provider for vaccination.

Voucher Incentive. In Chicago, a 3-month supply of food vouchers is usually issued by WIC to enrolled families. When a child is designated as being at high risk for illness (eg, anemic), only a 1-month supply of vouchers is issued at a time, to ensure frequent contact with the family regarding the high-risk condition. In the immunization initiative, children who were not age-appropriately vaccinated were treated as high-risk clients. A 1-month supply of vouchers was issued until the child was age-appropriately vaccinated, at which time the issuance of a 3-month supply was resumed. Only voucher frequency was varied; no voucher was ever withheld from a child because of immunization status.

Evaluation Approach. We retrospectively evaluated the impact of the initiative using existing data generated for the period April 1996 (by which date all sites had been performing assessment and referral for 4 months or longer, but none had started voucher incentives) through June 1997 (when the majority of sites had been performing voucher incentives for more than 12 months). From these data, we identified groups of sites (Table 1) whose different immunization activities provided comparisons to help distinguish the effects of voucher incentives from the impact of other immunization-related activities.

Group A (+ Incentives; + Monitoring). Four CDPH sites serving about 4000 infants (8% of the Chicago birth cohort) had been prospectively selected by initiative staff as sentinel sites for intensive monitoring of the initiative from its inception (see process measures below). Selection was based on inner-city location and on the program staff's perception of the population's risk for low vaccination coverage.

Group B (+ Incentives; − Monitoring). Ten CDPH sites serving about 9000 infants (18% of the Chicago birth cohort) began voucher incentives at the same time as group A (May 1996) but were not intensively monitored. Monthly immunization rates were not collected until September 1996.

Group C (− Incentives; + Monitoring). Three CDPH sites serving about 2000 infants (4% of the Chicago birth cohort) did not begin providing voucher incentives during the evaluation period but collected immunization rate data, starting in February 1996, in preparation for the subsequent implementation of voucher incentives. Implementation did not take place at these sites during the study period because of administrative delays in hiring and assigning staff.

Group D (− Incentives; − Monitoring). Two CDPH sites serving about 1000 infants (2% of the Chicago birth cohort) did not implement voucher incentives and were not monitored during the evaluation period, although they reported having performed assessment and referral.

Data Collected.Population Characteristics. The 1996 Chicago birth cohort, the children of Chicago with reported preschool measles during the 1989-1990 epidemic, and the infants enrolled in WIC in Chicago in April 1996 were compared for population size and race/ethnicity.

Immunization Services Available at WIC Sites. One of 3 levels of immunization services was available at each WIC site: an on-site nurse, a colocated clinic, or no immunization services whatsoever (Table 1). The availability of these services at each site predated the initiative and did not change during the evaluation period. We examined the availability of these services as a potential cofactor in the impact of the initiative.

Process Measures. From the group A (+ incentives; + monitoring) sites, the following data were collected: documentation rates—the monthly proportion of children for whom vaccination status could be documented; voucher incentive eligibility rates—the monthly proportion of children seen who were eligible for a voucher incentive because they needed a vaccination; and voucher incentive delivery rates—the monthly proportion of children eligible for a voucher incentive who actually received the incentive.

Age-Appropriate Vaccination Rates. This rate is the proportion of children 24 months of age or younger who were documented to be age-appropriately vaccinated based on the series currently recommended by the ACIP (including hepatitis B and Haemophilus influenzae type b). Since the group D (− incentives; − monitoring) sites did not report immunization rates during the evaluation period, records of all children 24 months of age or younger seen at these sites for WIC certification or recertification during June 1997 (the last month of evaluation) were checked against computerized immunization files, and age-appropriate immunization rates for these sites were calculated. These results were compared to rates obtained by identical methods from the group A (+ incentives; + monitoring) sites.

WIC Enrollment Rates. Monthly enrollment data for each site were obtained from Illinois WIC to evaluate the impact of the initiative on WIC enrollment.

Statistics. Statistical tests were not applied since this is a description of a large-scale programmatic initiative rather than a report of the results of a formal scientific trial.

The WIC clients served by CDPH were comparable to WIC clients in Chicago as a whole (Table 1). Each of the 4 groups of CDPH sites served a predominantly minority population, although the racial/ethnic composition differed among groups.

Immunization Services Available Through WIC Sites. Preexisting immunization services were not comparably distributed among groups of sites, but the distribution did not appear to favor immunization at voucher incentive sites (Table 1). Among the 14 sites performing voucher incentives, 5 (36%) lacked any services, 1 (7%) had an on-site nurse, and 8 (57%) were colocated with a clinic. In contrast, all 5 sites not performing voucher incentives had available services: 2 (40%) with on-site nurses and 3 (60%) with colocated clinics.

Process Measures.Documentation Rates. The proportion of children whose immunization status could be documented at the certification visit started at a high level (range, 87%-100%), reflecting the effect of previous assessment and referral activities. Thereafter, it improved to the point where documentation was almost universal (range, 98%-100%) (Figure 1, A).

Voucher Incentive Eligibility Rates. The proportion of children seen who were eligible to receive a voucher incentive fell from a high of 51% in June 1996 to a low of 5% in April 1997. It rose to 12% in the 2 succeeding months because computer problems prevented voucher incentive delivery in May 1997 (Figure 1, B).

Voucher Incentive Delivery Rates. The proportion of incentive-eligible children who actually received the incentive began at a low level but after 4 months was consistently more than 80%, except for May 1997, during which month WIC computer problems caused a transient drop (Figure 1, C).

Age-Appropriate Vaccination Rates. Among the 4 group A (+ incentives; + monitoring) sites, immunization rates rose from 56% to 89% during the 15 months of evaluation (Figure 2). Among the 10 group B (+ incentives; − monitoring) sites, data are absent for the early phases of incentive implementation, but the final immunization rate was identical to that of the group A sites (89%). In contrast, the 3 group C (− incentives; + monitoring) sites showed no improvement, despite the monitored implementation of assessment and referral; their final vaccination rate was almost identical to the starting rate of the group A sites (57% vs 58%). The 2 group D (− incentives; − monitoring) sites, which had been performing unmonitored assessment and referral, had a coverage rate at the end of the evaluation period of 42% (42/99). Using the same computer-record survey method, coverage at the 4 group A sites for the same month was 80% (312/391).

Immunization Services and Initiative Impact. Differences in the availability of immunization services did not account for the initiative's impact. Among the sites performing voucher incentives, the final vaccination coverage at the 8 sites that were colocated with clinics was 87% (1876/2145), slightly lower than the 91% (331/362) coverage at the 5 that had no immunization delivery services available. Among the sites not performing voucher incentives, final coverage at the 3 sites that were colocated with clinics was 57% (352/622) compared to 42% (42/99) at the 2 that had an on-site nurse for vaccinations.

Race/Ethnicity and Initiative Impact. Differences in the race/ethnicity composition among sites did not account for the initiative's impact. Among the sites performing voucher incentives, the final coverage in the 8 sites with majority black populations was 93% (808/873), slightly higher than the 86% (464/538) coverage in the 2 sites serving majority Hispanic populations. Among the sites not performing voucher incentives, the final coverage in the 3 sites serving majority black populations was 48% (260/540), slightly lower than the 55% (21/38) in the 1 site serving a majority Hispanic population.

Changes in WIC Enrollment. Enrollment at the 14 sites providing voucher incentives was 13874 in April 1996 and 13066 in June 1997, while the 5 sites not providing incentives had an enrollment of 2742 in April 1996 and 2765 in June 1997 (Figure 3). Over the evaluation period, average net monthly enrollment (reflecting both attrition and addition of new clients and expressed as a fraction of starting enrollment) was the same at the incentive and nonincentive sites. Several large group B (+ incentives; − monitoring) sites instituted computerization for WIC eligibility processing, which created increased waiting periods for clients. Since WIC clients are permitted to transfer their care to a site of their choice, the increased waits apparently caused a loss of enrollment at these sites, from which some of the nearby group A (+ incentives; + monitoring) sites were beneficiaries.

This study was designed to answer a question that is often neglected in health services research: can an intervention that has been demonstrated to be efficacious in the controlled environment of relatively small, time-limited intervention trials actually prove effective in the "real world"? The population targeted by the Chicago WIC immunization initiative appears appropriate for answering such a question, since this population comprises almost one third of the birth cohort of the third largest city in the nation, all with family incomes of 185% or less of the defined poverty level, and 80% black or Hispanic. The results suggest that the programmatic implementation of voucher incentives was associated with a marked rise in immunization rates in this population and did not appear to be associated with a decline in WIC enrollment.

Prior to the evaluation period, CDPH sites had been assessing the immunization status of enrolled children and referring those in need of vaccination to a provider. This produced a high level of immunization documentation, a result that is consistent with previous studies,^8- 9 but there was no apparent impact of assessment and referral on immunization rates. In our study, 5 sites provided only assessment and referral during the evaluation period. Of these, the 3 group C (− incentives; + monitoring) sites showed no improvement in coverage during a 5-month period of monitoring (62% to 57%), and the 2 group D (− incentives; − monitoring) sites had 42% coverage at the end of the evaluation period.

In contrast, voucher incentives appeared to have a marked effect on immunization rates, raising coverage to 89%. This final high rate is remarkable, given the inclusion of all ACIP-recommended antigens. The impact did not appear to be attributable to any other identifiable factor: project monitoring (the intensively monitored group A and relatively unmonitored group B voucher incentive sites reached the same 89% final coverage); the availability of immunization services at the WIC site (final coverage at voucher incentive sites with no immunization services was 91% compared to 87% for those colocated with a clinic that administered vaccinations); or the racial/ethnic composition of the population served (final coverage for voucher incentive sites serving majority black populations was 93% compared to 86% for those serving majority Hispanic populations). At the 5 sites not performing voucher incentives, final coverage was low (42%-57%), regardless of whether the site was monitored, was colocated with a clinic or had a nurse administering vaccinations on-site, or was serving a majority black or Hispanic population.

Changes occurred in enrollment rates but were of small magnitude, and average enrollment was the same for incentive and nonincentive sites. Trends among groups were apparently attributable to WIC changes (eg, computerization delays causing clients to seek transfers) rather than immunization activities. This is consistent with the results of a recent focus group study, in which mothers enrolled in WIC did not mention voucher incentives as a reason to drop out of WIC,¹⁹ as well as with intervention trials that did not show an impact of voucher incentives on enrollment.^8- 11

This evaluation was limited by the use of existing data and by the potential confounders that may be present when subject populations have not been highly characterized. The immunization rates in this study (based on age-appropriate receipt of all recommended antigens) are lower than would be produced by a provider-verified survey of the same population based on 4:3:1 series completion by 24 months of age.

The annual labor cost of providing voucher incentives in Chicago was relatively modest ($271 000), but of the 65 federal immunization grantees (50 states plus 15 cities), only 7 are known by the Centers for Disease Control and Prevention to be providing immunization voucher incentives as a part of WIC. The rest mainly offer assessment and referral, although no study to date has established any impact for this intervention.^8- 9,11

The Chicago experience illustrates the potential value of using the results of intervention trials to formulate strategies for public health problems. But beyond science, Chicago made several management decisions that may have been responsible for the initiative's success: dedicated staff were hired; these staff were provided with dedicated supervision; the intervention was phased in site by site rather than begun abruptly at all sites; a small number of sentinel sites were intensively monitored to detect implementation problems; and vaccination rates were obtained monthly from all intervention sites to assess outcome. In addition, the Chicago WIC staff were highly cooperative with the initiative. A different impact might have resulted had the initiative's funds been used instead to reimburse a reluctant WIC for generalized but unmonitored immunization activities by existing hardworking WIC staff, with outcomes that were never measured. The applicability of Chicago's scientifically based and carefully managed approach needs to be explored for a wide range of inner-city health problems.

Since the manuscript was accepted for publication, all 19 CDPH sites have begun providing voucher incentives. By March 1998, coverage for the 5 sites that did not provide incentives during the study period had risen to 89% (694/777). Coverage in the other 14 sites, which continued providing voucher incentives, was 91% (3746/4111). WIC enrollment at all 19 sites was stable: 16001 in March 1998, compared to 16616 in April 1996, the month before the initiative began.