CFRP Policy Brief | B.015.0415
What Should be Expected when Taking Home Visiting Programs to Scale?
|April 2015||Click for PDF version|
In 2010, as a part of the Patient Protection and Affordable Care Act, Congress established the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV), which offered states, jurisdictions, and American Indian tribes $1.5 billion in annual formula grant funding over five years to support the expansion and development of home visiting programs. Importantly, states were required to spend at least three-quarters of the federal funds on home visiting models that met federal standards of evidence-based effectiveness. As many policy scholars have noted, that a national initiative brought the importance of evidence-based practice to the forefront of public policy is a triumph for social science and demonstrates the importance of rigorous program evaluation. With that triumph, however, comes a responsibility to ensure that the public’s expectations for success of these programs are consistent with what researchers understand about the empirical evidence – will the same positive outcomes found in programs’ randomized controlled trials emerge when those programs are taken to scale? Concomitant with increased investment is an increased concern for the return on investment – policymakers and the public want to know if the programs being invested in produce the outcomes expected of them.
The primary purpose of this brief is to guide expectations about the extent to which home visiting programs can demonstrate significant and meaningful change when implemented at the community level
To that end, the primary purpose of this brief is to guide expectations about the extent to which home visiting programs can demonstrate significant and meaningful change when implemented at the community level. This brief presents the findings of a larger research paper, currently under review, in which we review the evidence for four home visiting program being implemented with MIECHV funding and compare these effects to population averages and professional recommendations to provide a context in which to interpret those effects. Here, we highlight our review and synthesis of the evidence to illustrate the best possible outcomes that can be expected from home visiting programs given what we know about the prior research.
Data and Sample
We consulted the federal Home Visiting Evidence of Effectiveness (HomVEE) project to identify the relevant research and the Design Options for Home Visiting Evaluation (DOHVE) project to identify the parenting outcomes prioritized by MIECHV. Our review focused on four widely-used home visiting program models that met the Department of Health and Human Services “evidence-based” criteria and that are included in the legislatively mandated, large-scale evaluation of the effectiveness of MIECHV-funded home visiting programs: Early Head Start-Home Based (EHS-HB), Healthy Families America (HFA), Nurse-Family Partnership (NFP), and Parents as Teachers (PAT).
For each of the selected home visiting program models, this study examined the evidence base for six parenting outcomes.
For each of the selected home visiting program models, this study examined the evidence base for six parenting outcomes (prenatal care, breastfeeding, well-child visits/immunizations, learning support behaviors, child maltreatment, and harsh discipline). Home visiting programs should have the greatest and most immediate impact on parenting outcomes relative to child outcomes because home visiting programs target parenting directly as a mechanism of change in children.
An exhaustive search of the HomVEE website, program model websites, and government agency websites produced a combined total of approximately 60 research studies for review: 11 articles were reviewed for EHS-HB, 18 for HFA, 19 for NFP, and 10 for PAT. For each program model, we identified which, if any, of the parenting outcomes had been examined. Studies that did not include an examination of at least one of the six parenting outcomes of focus were excluded from our review. For each of the remaining studies, we identified how the outcome(s) was measured, whether there was a significant effect or not, and the size of the effect if evident.
The results of our review are summarized for each parenting outcome below. In general, the research shows that home visiting programs have the greatest, albeit modest, effect on parents’ support for children’s learning and in reducing the prevalence of child maltreatment, but that these effects are strongest for the most disadvantaged program participants. The research provides less support for the effect of home visiting programs on early health behaviors including prenatal care, breastfeeding, or well-child visits, or on reducing the use of harsh parenting.
Across the four home visiting programs, the evidence linking home visiting programs to prenatal care is thin. The strongest, albeit still modest, evidence for a link comes from an evaluation of the NFP program, which shows impacts on attendance at child birth classes and knowledge of how to access prenatal care, but does not show an impact on actual prenatal care visits. More limited, but still promising, evidence comes from studies of EHS-HB and HFA. , Evaluations of both programs suggest participation in the program is associated with accessing prenatal care, but neither provide enough information or rigor to draw valid conclusions about impacts. No evaluation of the PAT program has examined prenatal care.
The evidence for an impact of home visiting programs on breastfeeding is driven almost entirely by the modest findings from an evaluation of NFP, in which nurse-visited mothers were significantly more likely to have attempted breastfeeding (though, as the authors report, there were no differences in duration of breastfeeding.)
A study of HFA showed no significant impact on breastfeeding and the evidence provided in the EHS-HB evaluation does not lend itself to causal conclusions. , Breastfeeding has not been examined as an outcome in evaluations of the PAT program.
Well-Child Visits and Immunizations
For the most part, home visiting programs have largely been shown to be ineffective at increasing rates of well-child visits and immunizations (Table 2). It is important to note, however, that the comparison group in many studies showed high levels of achieving adequate well-child visits or meeting immunization standards. This makes it harder to detect statistically significant differences, even if positive changes are seen among program participants. Studies of NFP and EHS-HB showed no statistically significant impact of the programs at increasing immunizations; nor is there any evidence that NFP has a statistically significant effect on well-child visits. , There is some evidence that HFA may increase the number of well-child visits at age three, but there is little empirical support for the effectiveness of HFA on well-child visits or immunizations within the first two years. Two evaluations of PAT produced mixed findings, although one study did find a statistically significantly higher share of children who were fully-immunized for his/her age as compared to children in the comparison group.
Many home visiting programs aim to educate parents about the importance of supporting children’s early learning through frequent reading and a stimulating home environment and provide parents with the tools to support their children’s early learning. In general, evaluations of home visiting programs show fairly positive impacts on parents’ support for children’s learning, though the evidence is strongest for the most disadvantaged program participants (e.g., poor, unmarried teens; very-low income participants).
The results from several evaluations and randomized control trials show that home visiting programs may be an effective approach to reducing the prevalence of child maltreatment, but generally only for the most disadvantaged or at-risk families (e.g., first-time mothers, families who previously interacted with child protective services, low-income families, and poor, unmarried teens). Studies of HFA, NFP, and PAT find effects for certain subgroups of their samples, but show no overall impact of program participation on reductions in child maltreatment. Mixed evidence emerges from the single EHS-HB study that has looked at child maltreatment.
In contrast to the support for reductions in child maltreatment, there is little support for a link between home visiting programs and a reduction in the frequency with which parents use spanking as a disciplinary technique. The research linking participation in EHS-HB and HFA with discipline practices is mixed. One small study of EHS-HB found that low-income families reported lower frequencies of spanking at age three, however, these results contradicted findings in other studies. Studies of HFA found that a larger share of mothers reported never slapping their children’s hand, fewer acts of minor physical aggression, and fewer acts of harsh parenting in the past week. There is little evidence to support an effect of NFP on discipline nor is there a research base for the link between PAT and harsh discipline.
The research suggests that these programs may positively influence parents’ support of their children’s learning and reduce rates of child maltreatment
What, then, can be realistically expected from home visiting programs? The findings from the present review provide a basis to be optimistic about the improvement home visiting programs can have in some areas of child and family well-being. The findings also highlight, however there are other areas of child and family well-being where the evidence is not very strong, suggesting that expectations for success in these areas may need to be tempered.
If home visiting programs target the most at-risk families (i.e., young and poor parents, parents with a history of child maltreatment), the research suggests that these programs may positively influence parents’ support of their children’s learning and reduce rates of child maltreatment. Because the most at-risk families may also be the most difficult to reach in center-based settings, home visiting programs are at an advantage in that the very same high-risk families they need to target are the same families they may be better suited to reach.
To be the most effective, home visiting programs should be provided in conjunction with other services and support. The findings from this review indicate that these programs have the greatest potential to influence the processes that happen within the home environment, but are considerably more limited in their ability to influence outcomes that rely on resources and factors external to the home. Delivering home visiting programs within an organized system of early childhood services and support would likely allow home visiting programs to be more effective and allow families to be better served by them.
- Astuto, J. & Allen, L. (2009). Home visitation and young children: An approach worth investing in? SRCD Social Policy Report, 23, 1-24.
- Astuto, J. & Allen, L. (2009). Home visitation and young children: An approach worth investing in? SRCD Social Policy Report, 23, 1-24.
- Boller, K., Daro, D., Del Grosso, P., Cole, R., Paulsell, D., Hart, B., Coffee-Borden, B., Strong, D., Zaveri, H., & Hargreaves, M. (2014). Making replication work: Building infrastructure to implement, scale-up, and sustain evidence-based early childhood home visiting programs with fidelity. Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services.
- Caldera, D., Burrell, L., Rodriguez, K., Crowne, S. S., Rohde, C., & Duggan, A. (2007). Impact of a statewide home visiting program on parenting and on child health and development. Child Abuse & Neglect, 31(8), 829–852; Duggan, A. K., McFarlane, E. C., Windham, A. M., Rohde, C. A., Salkever, D. S., Fuddy, L., et al. (1999). Evaluation of Hawaii’s Healthy Start program. Future of Children, 9(1), 66–90; discussion 177–178; Mitchell-Herzfeld, S., Izzo, C., Greene, R., Lee, E., & Lowenfels, A. (2005). Evaluation of Healthy Families New York (HFNY): First year program impacts. Albany, NY: University at Albany, Center for Human Services Research; Landsverk, J., Carrilio, T., Connelly, C. D., Ganger, W., Slymen, D., Newton, R., et al. (2002). Healthy Families San Diego clinical trial: Technical report. San Diego, CA: The Stuart Foundation, California Wellness Foundation, State of California Department of Social Services: Office of Child Abuse Prevention.
- Chazan-Cohen et al. (2013).
- Chazan‐Cohen, R., Raikes, H. H., & Vogel, C. (2013). Program subgroups: Patterns of impacts for home-based, center-based, and mixed-approach programs. In J. M. Love, R. Chazan-Cohen, H. Raikes, & J. Brooks-Gunn (Eds.), What makes a difference: Early Head Start Evaluation Findings in a Developmental Context. Monographs of the Society for Research in Child Development, 78, 93-109.
- Daro, D. A., & Harding, K. A. (1999). Healthy Families America: Using research to enhance practice. The Future of Children, 9(1), 152-176.
- Green, B.L., Ayoub, C., Bartlett, J.D., Von Ende, A., Furrer, C., Chazan-Cohen, R., Vallotton, C. & Klevens, J., (2014). The Effect of Early Head Start on Child Welfare System Involvement: A First Look at Longitudinal Child Maltreatment Outcomes, Children and Youth Services Review 42: 127-135.
- Jones Harden, B., Chazan‐Cohen, R., Raikes, H., & Vogel, C. (2012) Early Head Start Home Visitation: The role of implementation in bolstering program benefits. Journal of Community Psychology, 40(4), 438-455.
- Kisker & Kuhns (2004).
- Kisker, E., & Kuhns, C. (2004). Health and Disabilities Services in Early Head Start: Are Families Getting Needed Health Care Services? (No. 3983). Mathematica Policy Research
- Kitzman et al. (1997)
- Kitzman et al. (1997).
- Kitzman, H., Olds, D. L., Henderson, C. R., Hanks, C., Cole, R., Tatelbaum, R., et al. (1997). Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing. A randomized controlled trial. JAMA: The Journal of the American Medical Association, 278, 644–652.
- LeCroy, C. W., & Krysik, J. (2011). Randomized trial of the Healthy Families Arizona home visiting program. Children and Youth Services Review, 33(10), 1761–1766.; DuMont, K., Mitchell-Herzfeld, S., Greene, R., Lee, E., Lowenfels, A., Rodriguez, M., et al. (2008). Healthy Families New York (HFNY) randomized trial: Effects on early child abuse and neglect. Child Abuse & Neglect, 32(3), 295–315.
- Mitchell-Herzfeld, Izzo, Greene, Lee, & Lowenfels, 2005
- Olds, Henderson, Tatelbaum, & Chamberlin, 1986
- Roggman, L. A., & Cook, G. A. (2010). Attachment, aggression, and family risk in a low-income sample. Family Science, 1, 191-204.
- U.S. Department of Health and Human Services, Administration for Children and Families, Administration on Children, Youth and Families. (2013). Home Visiting Evidence of Effectiveness. About Us: Executive Summary. Retrieved from http://homvee.acf.hhs.gov/document.aspx?rid=5&sid=20&mid=2.
- Wagner, M., Clayton, S., Gerlach-Downie, S., & McElroy, M. (1999). An evaluation of the northern California Parents as Teachers demonstration. Menlo Park, CA: SRI International
- Wagner, M., Lida, E., Spiker, D., Hernandez, F., & Song, J. (2001) The multisite evaluation of Parents as Teachers home visiting program. Menlo Park, CA: SRI International.
- Wagner, M., Spiker, D., & Linn, M. I. (2002). The effectiveness of the Parents as Teachers program with low-income parents and children. Topics in Early Childhood Special Education, 22(2), 67–81.
Click for the PDF version of What Should be Expected when Taking Home Visiting Programs to Scale?.
Click for more on CFRP’s Home Visiting research.
© April 2015, Child and Family Research Partnership, All Rights Reserved.