A Practice-Based Intervention to Enhance Quality of Care in the First 3 Years of Life:  The Healthy Steps for Young Children Program FREE

Providing quality pediatric care for young children involves matching the needs and expectations of families with the organizational structure and clinical practices of pediatric providers. There is growing evidence of deficiencies in the quality of health care for children, including low rates of preventive services,¹ persistent disparities in health status,² and lack of a usual source of care among ethnic and racial minorities and children in low-income families.³

Specific limitations have been noted in the quality of care related to developmental and behavioral services for children in the first 3 years of life,^4- 7 particularly regarding gaps between recommended and actual care received.^8- 9 In a national survey, only 23% of 2017 parents of young children discussed discipline and early learning with their child's clinician, and over half wanted more information about these topics.⁴ In a survey of 1900 Medicaid-enrolled children ages 4 years and younger, 40% of parents reported that their child's clinicians did not ask whether they had concerns about their child's development and well-being.¹⁰ Using the National Survey of Early Childhood Health, Halfon et al⁶ reported that 34% of parents of 2068 children ages 4 to 35 months did not believe their child's clinicians always took time to understand their child's needs.

The Healthy Steps for Young Children Program was designed to meet the needs of families regarding their young children's early development and behavior by enhancing relationships between parents and their children, between families and the pediatric practice, and among practice members.^11- 13 Healthy Steps aims to enhance the capacity and effectiveness of pediatric primary care by incorporating developmental specialists into pediatric practice. The program was designed for all families, not only those at risk for developmental and behavioral problems, recognizing that all parents have concerns and questions about their children's health, development, and behavior.

This article presents results from the national evaluation of Healthy Steps, the first large clinical trial of an intervention in pediatric practices to improve delivery of developmental and behavioral services. Initial findings, when the children were ages 2 to 4 months, demonstrated enhanced receipt of developmental services, increased satisfaction with care, and enhanced parent practices related to sleep position, feeding, and infant development.¹⁴ In this article, we report whether Healthy Steps increased quality of care related to developmental and behavioral services and affected parenting practices regarding discipline, perceptions of their children's behavior, and promotion of development when children were ages 30 to 33 months.

The Healthy Steps for Young Children Program was a 3-year, prospective controlled trial with 6 randomization and 9 quasi-experimental pediatric practice sites selected by program funders and located in 14 states across the United States (randomization sites: Allentown, Pa; Amarillo, Tex; Florence, SC; Iowa City, Iowa; Pittsburgh, Pa; and San Diego, Calif. Quasi-experimental sites: Boston, Mass; Chapel Hill, NC/Birmingham, Ala; Chicago, Ill; Detroit, Mich; Grand Junction, Colo/Montrose, Colo; Kansas City, Kan; Kansas City, Mo; New York, NY; and Richmond, Tex/Houston, Tex).

Consecutive newborns up to 4 weeks of age were enrolled at birth or their first office visit and followed up to age 3 years. Newborns were excluded if they were to be adopted or placed in foster care, were too ill to make an office visit by age 4 weeks, their mother did not speak English or Spanish, or the family intended to leave the practice within 6 months. Enrollment of families was staggered beginning in September 1996 and ending in November 1998.

Within each randomization site, newborns were randomized to intervention or control groups of approximately 200 newborns each. Trained staff carried out randomization in blocks of 4, using a computer-generated assignment sequence concealed in sealed envelopes. Constraints on sample size (ie, need to enroll 400 children total), space limitations, scheduling logistics, and the unwillingness of some practices to provide different services to families prevented randomization at all sites. At quasi-experimental sites, up to 200 newborns were consecutively enrolled at the intervention sites and at a comparison site matched on organizational setting (practice type and urban/rural location) and demographic mix of patients.¹²

A simulation study was conducted to evaluate the power of our design to detect treatment effects for dichotomous outcomes. Our simulation allowed for clustering of outcomes within sites and for a 33% missing outcome rate. For randomization sites for a dichotomous outcome and control-group prevalences ranging from 0.2 to 0.8, we found greater than 90% power to detect a treatment effect odds ratio (OR) of 1.5. For control group prevalences of 0.1 and 0.9, the associated power was roughly 70%. For the quasi-experimental design, the power was substantially larger.

Regardless of assignment, families received all aspects of standard pediatric care within the practice. Clinicians at the randomization sites cared for both intervention and control families. Intervention families also received the Healthy Steps program components, including contact with the Healthy Steps Specialist. Each site had 2 Healthy Steps Specialists with caseloads of approximately 100 families each. The Healthy Steps Specialists were nurses, nurse practitioners, early childhood educators, and social workers with training and experience in child development. The specialists' main focus was on developmental, behavioral, and psychosocial aspects of care.

Healthy Steps provided 7 services to intervention families: enhanced well-child care (through visits with the physician and Healthy Steps Specialist); 6 home visits in the first 3 years; Healthy Steps Specialist–staffed child developmental telephone line to address parents' developmental concerns; developmental assessments; written materials emphasizing prevention and health promotion; parent groups offering support and learning opportunities; and linkages to community resources through targeted referrals. In enhanced well-child visits, parents' questions and concerns about child development and behavior were addressed, using "teachable moments" and promotion of positive parent-child interactions. These visits also emphasized the mother's health and encouraged early learning through the Reach Out and Read program.^11,13,15

Key site personnel participated in 3 annual training sessions, stressing knowledge in child development, parenting, and practical clinical strategies; emphasizing the importance of relationships; promoting multidisciplinary team building; and helping practices implement Healthy Steps.¹⁶ Each site also received program and training manuals and technical assistance through biweekly teleconferences. Sites also received training in evaluation procedures. Implementation of written protocols was monitored by the Healthy Steps national program office. Significant changes were required in practice operations related to space, appointment scheduling (for joint Healthy Steps Specialist/physician visits), job responsibilities, and team meetings. Some sites experienced changes in practice ownership, administration, and staffing, forcing 1 site to close before completing the study. Despite these challenges, all sites made all components available from the start of the program.

The study was approved by the human subjects committees at the Johns Hopkins Bloomberg School of Public Health and all participating institutions. Families gave written informed consent at the time of enrollment and oral consent at each subsequent parent interview.

At 2 to 4 and at 30 to 33 months, families provided data for the evaluation, regardless of their continued use of care at the original Healthy Steps practice. Attempts to minimize study attrition included mailing of annual birthday cards marked "return service requested" to inform evaluators of address changes, annual queries of the National Change of Address service to update contact information, and newsletters marked "return service requested."

Computer-assisted telephone interviews in English or Spanish were conducted at 30 to 33 months by Battelle Centers for Public Health Research and Evaluation. The mother was the primary respondent (98.5%). If she was not available, the child's guardian or primary caretaker was interviewed. Respondents were asked about use of health-related services, satisfaction with care, perceptions of their children's behavior, parenting activities that promote development and safety, and the mother's health status and problems with depressive symptoms. Data on demographic characteristics came from enrollment forms and parent telephone interviews at 2 to 4 months.

Data about well-child visits and vaccinations were abstracted by trained staff from medical records when the child reached age 32 months, including type and date of visit and each vaccine received. A random sample of 5% of records was reabstracted at each site. Percentage agreement was 87% or higher for visit type, 96% or higher for visit date, 99% or higher for vaccine type, and 98% or higher for vaccine date.

Four domains of quality of care—effectiveness, patient-centeredness, timeliness, and efficiency—were operationalized using the Institute of Medicine framework (Box).^17- 18Effectiveness included receipt of Healthy Steps–related services, and patient-centeredness described satisfaction and experiences while seeking care. We created subscales to measure parents' perception of the care their family received from their child's health care providers. Internal consistency of the scales, as assessed by Cronbach α, exceeded .70 for all scales, the generally accepted level for an internally consistent scale.

Box. Quality-of-Care and Parenting Measures

QUALITY-OF-CARE MEASURES

Effectiveness

Received 4 or more Healthy Steps–related services

   Parent support groups, office visits about child's development, telephone number to discuss child's development, letter to prepare for office visits, brochures about child's development, special health booklet. Services exclude home visit (6 items; Cronbach α = .67)

Discussed more than 6 topics
   Age-appropriate anticipatory guidance topics include importance of regular routines, sleep problems, discipline, language development, toilet training, sibling rivalry, home safety, child's development, child's temperament, ways of helping child learn (10 items; Cronbach α = .92)

Received home visit
   From anyone since child was 6 months of age (1 item)

Had developmental assessment
   Given developmental assessment by someone in practice (1 item)

Received books
   Received books to read to their child from practice (1 item)

Informed about community resources
   Received information from someone in practice (1 item)

Patient-Centeredness

Someone in the practice went out of the way for them (1 item)

Disagree that pediatrician/nurse practitioner provided support*
   Clinician suggested things parent could do for child that fit into family's daily life, helped get all the information needed about child's growth and development, helped get services for child from other agencies and programs, gave advice on how to solve problems at home with child, gave new ideas about things to do with child, pointed out what parent did well (6 items, each rated as "strongly disagree," "disagree," "agree," "strongly agree"; Cronbach α = .87)

Disagree that pediatrician/nurse practitioner listened to parent*
   Clinician took time to answer questions, understood main reason for visit, didn't have other things on their mind, gave parents a chance to ask questions, thought carefully about questions, not in a rush, encouraged questions (7 items, each rated as "strongly disagree," "disagree," "agree," "strongly agree"; Cronbach α = .87)

Disagree that pediatrician/nurse practitioner respected parent*
   Clinician respected parent's knowledge, knew what was going on with child, made parents feel like they were doing a good job (3 items, each rated as "strongly disagree," "disagree," "agree," "strongly agree"; Cronbach α = .72)

Timeliness

Age-appropriate well-child care†
   1-month visit (within 41 days of birth)
   2-month visit (between 42 and 92 days)
   4-month visit (between 93 and 151 days)
   6-month visit (152 and 213 days)
   12-month visit (between 336 and 397 days)
   24-month visit (between 701 and 762 days)

Age-appropriate vaccinations†
   Diphtheria-tetanus-pertussis (DTP)1: 1 DTP given between 42 days and 92 days, inclusive
   DTP3: first DTP given on or after 42 days, second given at least 28 days after first, and third given at least 28 days after the second and before age 213 days, inclusive
   Measles-mumps-rubella (MMR)1: 1 MMR given between 365 days and 488 days, inclusive

Up-to-date vaccinations‡
   Four DTP, 3 oral polio vaccine/inactivated poliovirus vaccine, 1 MMR at 24 months (valid doses only)

Efficiency

Last visit after 20 months (1 item)
Hospitalizations in past year (1 item)
Emergency department use in past year (1 item)
Emergency department use in past year for injury-related causes (1 item)

PARENTING MEASURES

Parent Response to Child Misbehavior

Ever slap child in face/spank with an object (2 items)

Ever use harsh disclipline
   Yell in anger, threaten, slap on hand, spank with hand (4 items, each rated as "never," "seldom," "often," or "almost always"; Cronbach α = .62)

Often or almost always negotiate (1 item)

Often or almost always ignore misbehavior (1 item)

Perception of Child's Behavior§

Aggressive
   Child defiant, demands must be met immediately, disobedient, easily frustrated, easily jealous, gets into many fights, hits others, angry moods, punishment doesn't change his/her behavior, screams a lot, selfish or won't share, sudden changes in mood or feelings, temper tantrums or hot temper, unusually loud, whining (15 items, each rated as "often true," "sometimes true," or "never true"; Cronbach α = .85)

Anxious or depressed
   Child clings to adults or is too dependent; feelings are easily hurt; gets too upset when separated from parents; looks unhappy without good reason; nervous, high-strung, or tense; overtired; self-conscious or easily embarrassed; shy or timid; too fearful or anxious; unhappy, sad, or depressed; wants a lot of attention (11 items, each rated as "often true," "sometimes true," or "never true"; Cronbach α = .68)

Sleeping problems
   Child doesn't want to sleep alone; has trouble getting to sleep; has nightmares; resists going to bed at night; sleeps less than most children during day and/or night; talks or cries out in sleep; wakes up often at night (7 items, each rated as "often true," "sometimes true," or "never true"; Cronbach α = .68)

Promotion of Child Development and Safety

Discussed sadness with someone in the practice (1 item)∥
Read to child or showed child picture books every day or more often (1 item)
Played with child once a day or more (1 item)
Followed 3 routines: same mealtime, naptime, and bedtime each day (3 items)
Lowered temperature on water heater (1 item)
Used covers on electrical outlets (1 item)
Had safety latches on cabinets (1 item)

*Measure presented as dissatisfaction due to overwhelmingly positive views of care at practice.

†Children who made a visit during or after the previous age - appropriate well-child visit window were considered eligible for an age-appropriate visit or vaccination.

‡Includes children who made a visit after 20 months.

§Assessed using the Child Behavior Checklist.

∥Among those with depressive symptoms at 30-33 months, those who needed help with sadness since the child was born, and/or those who restricted their activity for 1 week or longer in the previous 6 months because of feeling anxious or depressed.

Timeliness encompassed adherence to the recommended schedule of well-child care visits⁹ and vaccinations.¹⁹ Well-child visits and vaccinations that occurred within the recommended period were considered age-appropriate. Children who received 4 doses of diphtheria-tetanus-pertussis vaccine, 3 doses of polio vaccine, and 1 dose of measles-mumps-rubella vaccine by age 2 years were considered up-to-date. Doses received before the minimum interval between doses were not counted.

Efficiency, ie, the avoidance of wasted resources, included emergency department (ED) use and hospitalizations because their costs per unit of service are high and inappropriate use of resources contributes to many of these visits.²⁰ Continued receipt of care at the same practice after age 20 months also was included since continuity of care reduces the risk of hospitalizations and ED use²¹ and increases consumer satisfaction.²²

Parenting outcomes included responses to child misbehavior, reports of child behaviors, and selected age-appropriate parenting practices that promote child health and development (Box). "Parental Response to Child Misbehavior" assessed the frequency with which parents used each of 12 responses to misbehavior in an average week in the past month.²³ The usual 7-point scale was condensed to 4 response categories ranging from 0 ("never") to 3 ("almost always") to facilitate telephone administration. Responses were dichotomized to indicate whether individual approaches to discipline were used "never" or "seldom" vs "often" or "almost always." A constructed scale included more harsh responses to misbehavior (eg, yelling in anger, threatening, slapping the child's hand, and spanking with a hand). The respondent's total score was summed and divided by the number of items and was interpreted in relation to the response categories for the item. One dichotomous variable indicated "harsh discipline" with a mean score of 1.5 or higher. A second dichotomous variable indicated whether the parent ever slapped the child in the face or spanked him or her with an object. Two nonphysical strategies (often or almost always negotiating and ignoring the misbehavior) were assessed.

The Child Behavior Checklist (CBCL) for children aged 2 to 3 years measured parents' perceptions of their children's behavioral problems.²⁴ Means of 3 CBCL subscales (aggressive behavior, anxious or depressed, sleep problems) were reported. Parents rated their child for how true ("often," "sometimes," "never") each item was at the time of the interview or within the previous 6 months.

Parenting practices included reading to the child or showing the child picture books, playing with the child, and following daily routines at naptime, mealtime, and bedtime. We also assessed parents' practices recommended by the American Academy of Pediatrics to promote child safety.⁸

A 14-item, abbreviated Center for Epidemiologic Studies–Depression scale was used to assess mothers' depressive symptoms.²⁵ It included the same response categories as the original 20-item scale. Scores of 11 or greater indicated the presence of maternal depressive symptoms. The α coefficient for the 14-item depression scale in the Healthy Steps sample was .86, similar to that for the total scale in general population samples.²⁵ Discussing sadness with someone at the practice was identified as a parenting practice that promoted child development among those respondents with depressive symptoms at 30 to 33 months, those who needed help with sadness since the child was born, or those for whom activities were restricted for 1 week or longer in the previous 6 months because of feeling anxious or depressed.

Families remained in their original group assignment regardless of the site of care or intensity of services received. Randomization and quasi-experimental sites were analyzed separately and combined. In bivariate analyses, we compared the distribution of quality-of-care outcomes and parenting practices using χ² statistics for categorical variables and t statistics for continuous variables (eg, CBCL scores).

Generalized regression models (logistic regression for dichotomous outcomes, linear regression for continuous outcomes) were used to estimate the overall adjusted effects of Healthy Steps.^26- 27 These models included site variables to account for the fact that families within sites tend to respond more similarly than those at different sites. Marginal logistic regression models were fitted for repeated-measures data (eg, well-child visits) using generalized estimating equations with working-independence covariance structures.²⁸

Baseline covariates included in regression models were site of enrollment (hospital or office), age of child at interview, and characteristics of the mother (age, education, race/ethnicity, employment), father (employment), family (marital status/father in household, number of siblings, owned home, income), and infant (low birth weight, source of payment for care). We used all available observations by introducing 3 baseline dichotomous variables to account for missing covariate data of more than 2%: unknown ethnicity (5%), unknown source of insurance (6%), and unknown data for other covariates (14%, including 3% father's employment and 6% income). The baseline covariates serve as adjustment for potential differences between intervention and control families that resulted from nonrandom assignment at quasi-experimental sites or selective reporting of outcome data.²⁹ Results of these adjusted analyses are reported as ORs for dichotomous variables and as differences in means for continuous outcomes. All statistical procedures were performed using SAS version 8.2 (SAS Institute Inc, Cary, NC); P<.05 was used to determine statistical significance.

All 5565 participating families completed enrollment forms, 4896 (88%) completed interviews at 2 to 4 months, and 3737 (67.2%) completed interviews at 30 to 33 months (Figure 1). The sample for analysis includes families providing data at 30 to 33 months. Among the 1800 families (32.3%) who did not complete interviews, 223 (4.0%) were no longer eligible (eg, deceased, foster care, withdrew from evaluation, moved away), 1317 (23.7%) could not be located, and 260 (4.7%) declined interviews. Twenty-eight (0.50%) were excluded for other reasons. Medical records data were available for 3672 (98%) of the 3737 who completed interviews at 30 to 33 months.

Compared with nonresponders, the sample providing data at 30 to 33 months tended to be somewhat more advantaged than the original cohort (data available upon request from authors).¹² More mothers in the sample were older, more educated, white, non-Hispanic, married, employed, and did not have Medicaid.

Intervention families in the sample interviewed at 30 to 33 months had fewer demographic risk factors than did control families; a greater percentage of mothers in intervention families were college graduates, white, non-Hispanic, had family incomes of $50 000 or greater, and had children not insured by Medicaid (Table 1). These differences were driven largely by differences observed at quasi-experimental sites at baseline. There was no evidence of selective attrition; at randomization and at quasi-experimental sites, the percentages of nonresponders in the intervention and control groups were similar for mothers younger than 20 years or older than 30 years, with less than high school education, and who were black, Hispanic, or employed during the last month of pregnancy (data available upon request from authors).

Overall, greater percentages of intervention than control families reported receiving effective, patient-centered, timely, and efficient care (Table 2). At randomization and at quasi-experimental sites, more than 75% of intervention families received 4 or more Healthy Steps–related services, had a home visit, discussed more than 6 topics, had developmental assessments, and received books; percentages for all 5 of these categories were less than 45% at control sites. Slightly less than 50% of intervention families were informed about community resources.

Among intervention and control families, satisfaction with care was high; more than 95% of families agreed or strongly agreed that the physicians or nurse practitioners always had time to answer their questions. Accordingly, we focused on responses at the low end of the scale, ie, those indicating dissatisfaction with care. A smaller percentage of intervention families disagreed with statements indicating that the clinician "provided support" and "listened" to them (Table 2).

For all measures of well-child care and vaccinations, a larger percentage of intervention families received timely preventive services. Significant differences were found in the percentage of children with a visit after 20 months and, at the quasi-experimental sites, in the percentage using the ED for injury-related reasons.

The odds of receiving each component of developmentally oriented care was 4 to 20 times greater among intervention than control families (Table 3). Families in the intervention group had a greater odds of receiving 4 or more Healthy Steps–related services (for randomization and quasi-experimental sites, respectively: OR, 16.90 [95% confidence interval {CI}, 12.78 to 22.34] and OR, 23.05 [95% CI, 17.38 to 30.58]), of discussing more than 6 anticipatory guidance topics (OR, 8.56 [95% CI, 6.47 to 11.32] and OR, 12.31 [95% CI, 9.35 to 16.19]), and of receiving timely well-child visits and vaccinations (eg, age-appropriate 1-month visit) (OR, 1.98 [95% CI, 1.08 to 3.62] and OR, 2.11 [95% CI, 1.16 to 3.85]). Intervention families also reported increased odds of someone in the practice going out of the way for them (OR, 2.06 [95% CI, 1.64 to 2.58] and OR, 2.11 [95% CI, 1.72 to 2.59]) and reduced odds of dissatisfaction (eg, disagree that pediatrician/nurse practitioner provided support) (OR, 0.44 [95% Ci, 0.31 to 0.63] and OR, 0.32 [95% CI, 0.24 to 0.43]).

Intervention families had increased odds of receiving timely preventive services. Where age-appropriate well-child and vaccination results were not statistically significant at randomization sites, the effect was in the same direction as for quasi-experimental sites. Children who received Healthy Steps also had an increased odds of having a visit after 20 months (OR, 2.02 [95% CI, 1.61 to 2.55] and OR, 1.75 [95% CI, 1.43 to 2.15]) and a decreased odds of having an ED visit in the past year for injury-related causes (OR, 0.94 [95% CI, 0.65 to 1.34] and OR, 0.67 [95% CI, 0.49 to 0.90]).

Parents used multiple approaches to discipline. Virtually all mothers reported often or almost always explaining rules or consequences (94.1%) and showing their child a more acceptable activity (90.9%). Smaller percentages negotiated with their child (57.1%), gave timeouts (55.5%), withdrew privileges (36.7%), or ignored their child's misbehavior (11.1%). Even smaller percentages often or almost always yelled in anger (11.8%), threatened their child (19.4%), slapped their child's hand (8.7%), or spanked with a hand (9.0%). Regarding physical discipline, 6.9% ever slapped their child in the face or spanked with an object, and 10.5% often or almost always yelled in anger, threatened, slapped the child on the hand, or spanked with a hand (Table 4).

Parents in the Healthy Steps group had a reduced odds of slapping their child in the face or spanking with an object (for randomization and quasi-experimental sites, respectively: OR, 0.82 [95% CI, 0.54 to 1.26] and OR, 0.67 [95% CI, 0.46 to 0.97]), while they had an increased odds of negotiating and ignoring, as observed in the total sample. The magnitude and direction of effect were similar although not always significant at randomization and quasi-experimental sites (Table 5).

The mean scores for mothers' responses to the 3 CBCL subscales were comparable to those for nonreferred children reported by Achenbach,²⁴ suggesting that levels of behavioral problems for the Healthy Steps children were similar to the levels for generally healthy children. Mothers in the intervention group had increased reporting of aggressive behavior and problems sleeping compared with control mothers, and comparable perceptions of anxious or depressed behaviors in their children.

For mothers with depressive symptoms, those who needed help with sadness, or those for whom activities were restricted due to anxiety or depression, a larger percentage of intervention than control mothers discussed sadness with someone in the practice. Across all families and at the quasi-experimental sites, the odds of discussing sadness with someone in the practice was 2 times greater for intervention mothers (OR, 0.95 [95% CI, 0.56 to 1.63] and OR, 2.82 [95% CI, 1.57 to 5.08]).

Mothers' use of safety practices was high: 95.1% always or almost always used a car seat, 90.2% used covers on electrical outlets, 97.3% had working smoke detectors in their homes, and 90.3% knew a number to call if their child swallowed something harmful. Families were less likely to use other age-appropriate safety measures: 62.6% used safety latches on cabinets and 31.7% placed stickers on bottles of poisonous liquids. We found no significant differences in safety practices between intervention and control families.

At 30 to 33 months, participation in activities that promote development also was high. There were no differences between groups in reading to or playing with their children or in following routines.

This article documents findings from a large-scale intervention implemented in response to concerns about developmental needs of young children and the ability of parents to meet these needs. Healthy Steps is unique among early child interventions in that it was based in pediatric practices, took a "universal" approach to addressing families' needs, and introduced a new developmental specialist into pediatric practices.

Participating pediatric practices showed significant improvements in effectiveness, patient-centeredness, timeliness, and efficiency of care. These improvements included marked parental satisfaction with the program; more timely preventive care, including on-time immunizations; and the receipt of more developmentally-oriented services. We observed similar increases at 2 to 4 months.

At 30 to 33 months, intervention families reported increased use of favorable discipline techniques. Although the use of negative discipline strategies was low, we believe that reductions in yelling in anger, threatening, slapping in the face, and spanking with an object are meaningful given the associations of early discipline strategies with later socioemotional development, mental health, and parent-child relationships.³⁰ These treatment effects were observed in families who participated in a universal intervention broadly focused on development and behavior. Prior evaluations of early childhood interventions with home visits have focused on high-risk populations.

The program also appears to have affected mothers' perceptions of their children's behaviors. Intervention mothers were more likely to report aggressive behaviors and sleep problems. They also were more likely to talk to someone at the practice about their own depressive symptoms. We suspect that participating mothers became more sensitive in reading their child's behavior and more comfortable discussing their child's behavior and their own sadness with someone at the practice. Full implementation of Healthy Steps at each of the sites likely contributed to the positive findings.³¹

Although Healthy Steps was not conceived as a "quality improvement" intervention, our findings indicate that Healthy Steps made its impact through improvements in quality of care and selected effects on parenting practices. Relative to other early childhood interventions, however, the program was modest in intensity. For example, Early Head Start provides daily contact with low-income families through center-based programs or weekly contact through home-based programs.³² The Infant Health and Development Program provided an average of 67 home visits in children's first 3 years of life.³³ In contrast, the average Healthy Steps family made 11 well-child visits and received less than 2 home visits within their child's first 2.5 years. Corresponding costs for the Healthy Steps program ($402-$933 per year)¹⁵ were substantially less than those for Head Start ($4500 per year) or the Infant Health and Development Program ($10 000 per year).³⁴

Contrary to expectations and despite increased receipt of timely preventive services, Healthy Steps did not influence hospitalizations or overall ED use. Reducing receipt of these acute services may require further systems and practice-level changes (eg, making primary care available during weekends and evenings). In addition, use of these services is less discretionary in younger children.

The design of the evaluation is unique in that it drew upon the strengths and weaknesses of both the randomization and the quasi-experimental designs. The randomization design reduces bias in the selection of families in the program, but has the weakness of possible spillover effects. The quasi-experimental design reduces spillover effects and makes it easier to implement the program, but does not eliminate the possibility of selection bias.^35- 36 The use of prospectively defined controls at quasi-experimental sites likely contributed to minimized discrepancies in outcomes between randomization and quasi-experimental groups.³⁷ For several parenting outcomes, such as discipline practices, findings were of similar magnitude and direction at randomization and quasi-experimental sites, but statistically significant at only quasi-experimental sites, where the sample size was larger; they were significant in the pooled sample, as well.

A number of limitations should be noted. The site selection process meant that Healthy Steps was evaluated against a high standard of performance among practices already oriented toward providing developmental and behavioral services. Many evaluation sites already provided one or more program components. During the evaluation, some comparison practices adopted national programs, such as Reach Out and Read,³⁸ that were components of Healthy Steps from its inception.

Overall high levels among intervention and comparison families of baseline practices to promote toddler development and safety likely contributed to lack of treatment effects in relation to injury prevention and parents' practices to promote development. Parents' engagement in these activities was higher than that reported in the National Survey of Early Childhood Health (NSECH)⁶ and the Commonwealth Fund Survey of Parents with Young Children (CWF).⁴ For example, more Healthy Steps parents read to their children daily or showed their children picture books (Healthy Steps, 68%; NSECH, 52%; CWF, 48%) and had the same bedtime (Healthy Steps, 77%; NSECH, 68%). Similar percentages of parents played with their child daily (Healthy Steps, 87%; CWF, 84%) and covered electrical outlets (Healthy Steps, 90%; NSECH, 94%). Our findings at 30 to 33 months contrast with our findings at 2 to 4 months showing enhanced feeding practices, decreases in use of the prone sleep position, increased daily use of picture books, and increased daily playing.^14- 15 At 2 to 4 months, however, we similarly observed no treatment effect with regard to infant routines.

Parents who completed the interview at 30 to 33 months were more socially advantaged than the total enrolled population. Medical records data for nearly all enrolled children (5351 of 5565 [96%]) showed that nonsampled children had fewer age-appropriate well-child visits and vaccinations (data available from authors upon request). Comparable treatment effects, however, were observed when analyses were conducted on the full sample with medical records (n = 5351). Nevertheless, for some parenting practices, our findings may be overestimates if families lost to follow-up would have required more intensive interventions to change behavior. Findings also may be underestimates because responding families may have reported increased use of favorable parenting behaviors at baseline. Finally, baseline differences between quasi-experimental intervention and control families are a limitation to the extent that covariates did not account for these differences. We nonetheless believe that we adequately captured baseline differences. Moreover, the consistent findings for randomization and quasi-experimental sites appear to support this conclusion.

Given nonresponse, generalizability of our findings to the original cohort is possible to the extent that (1) within levels of baseline covariates, treatment assignment is independent of the outcome; (2) within levels of baseline covariates, missingness of an outcome is independent of the outcome; and (3) our regression models are correctly specified.³⁹ While the first and second assumptions cannot be empirically validated, we did not have differential attrition of families, and we believe that we have included a set of covariates to increase the plausibility of these assumptions.

Pediatric practices provide unique opportunities to favorably influence health care for young children, increase parents' satisfaction, and influence parental practices to improve child development and behavior. This evaluation of Healthy Steps demonstrates that improvement is possible and that all families benefit, not only those at high risk. Eleven of the 15 participating practices continued to provide all or some of the program components 18 months after demonstration funding ended. In addition, variants of the Healthy Steps model have been or are being implemented in additional sites across the country.¹⁵ Finally, the evaluation is ongoing as we assess whether there are sustained and emerging effects on parenting as children reach 5.5 years of age.