Meeting Topic

Background and Context

The Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF) funds rigorous research and evaluation projects to help improve the lives of children and families with low incomes. To date, these projects have largely focused on understanding the implementation and impact of social policy programs. Many questions remain about the cost effectiveness of these programs and their potential return on investment. This information is important for policymakers to make informed decisions regarding how best to allocate limited resources for social policy programs.

Economic evaluation is a method to “identify, measure, evaluate, and compare the costs and results of programs and policies.”[1] It broadly refers to methods that estimate the resources required to implement a program (“costs”) and the economic implications of the corresponding outcomes of the program (“benefits”). Specific approaches include:

• cost analysis to calculate the total cost of administering a program,
• cost-effectiveness analysis to estimate the costs of producing effects in a way that allows comparison across alternative programs that achieve a common goal, and
• benefit-cost analysis to examine the economic benefits of a program’s effects relative to its cost.

These methodologies are frequently used in the fields of medicine and environmental science. For example, the U.S. Public Health Service first commissioned a panel in 1996 to generate guidance on how best to conduct cost-effectiveness research in the field of health and medicine.[2] They later updated this guidance in 2016.[3] Additionally, the U.S. Environmental Protection Agency has funded research to explore the costs and benefits of the Clean Air Act.[4] This work has helped policymakers understand a wide range of health benefits associated with the Clean Air Act, including reductions in school loss and early deaths, among others. Recently, researchers have advocated for increased use of such forms of economic evidence in programs for investments in children, youth, and families, including a 2016 panel by the National Academies of Sciences, Engineering, and Medicine. [5]

As economic evaluation expands into the social policy arena, it is critical that researchers and policymakers understand how to rigorously produce, use, and interpret this research. It is also critical that researchers and policymakers understand the unique challenges of using economic evaluation for social policy research, including challenges with converting outcomes of interest into monetary units. This requires that funders and researchers understand which methodologies of economic evaluations (e.g., cost analysis, cost effectiveness analysis, benefit-cost analysis) are best suited for different scenarios. It also requires that researchers and evaluators are familiar with the best practices in estimating costs, incorporating outcomes, and valuing benefits.

The 2024 OPRE Methods Meeting will convene representatives from the federal government, research firms, and academia to discuss these topics. The overarching goal will be to introduce key components of economic evaluation for assessing social policy programs.

Meeting Topics and Goals

The meeting has the following goals:

• Define economic evaluation, including its goals and methods, and how incorporating these methods can strengthen implementation and impact evaluations.
• Present economic evaluation methods and how these can be integrated into evaluation and project life cycles.
• Introduce attendees to the main methods and measurement approaches used, the differences between the approaches, and how and when to use one over another.
• Discuss the unique considerations of applying economic evaluation methods to social policy programs, including challenges with converting outcomes of interest into monetary units, and implications for participants, evaluators, funders, and decisionmakers.

The meeting will include presentations and discussions on the following questions:

• What are the advantages of conducting economic evaluations of social service programs? What are the challenges to conducting this type of research?
• What steps can researchers and practitioners take to estimate the costs of implementing programs and when in a project lifecycle should teams consider measuring costs?
• What methods are most appropriate when considering the desired outcomes and the context of a particular program? Under what circumstances is it suitable to keep program outcomes in their natural units, and when should these be converted into economic valuations?
• In what ways do the methods of economic evaluation fall short and what are the most pressing challenges?
• How can researchers best relay and disseminate findings of economic evaluation to decisionmakers?
• How can OPRE and other federal agencies support the use of economic evaluation in their own work and the research that they sponsor?

Meeting Attendees and Logistics

The 2024 OPRE Methods Meeting will be held virtually October 22 and 23, 2024. Participants and speakers will include representatives from the federal government, research firms, and academia.

[1] Lane, R., Soyemi, A., & Corso, P. S. (2016.). Part I: Introduction to Economic Evaluation. National Center for Chronic Disease Prevention and Health Promotion [PowerPoint slides]. National Center for Chronic Disease Prevention and Health Promotion, Center for Disease Control. https://www.cdc.gov/cardiovascular-resources/media/pdfs/Economic-Evaluation-Part1.pdf

[2] Weinstein, M. C., Siegel, J. E., Gold, M. R., Kamlet, M. S., & Russell, L. B. (1996). Recommendations of the panel on cost-effectiveness in health and medicine. Jama, 276(15), 1253-1258.

[3] Sanders, G. D., Neumann, P. J., Basu, A., Brock, D. W., Feeny, D., Krahn, M., … & Ganiats, T. G. (2016). Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. Jama, 316(10), 1093-1103.

[4] DeMocker, M. J. (2003). Benefits and Costs of the Clean Air Act 1990–2020: Revised Analytical Plan For EPA’s Second Prospective Analysis. Industrial Economics Incorporated, Cambridge, MA.

[5] National Academies of Sciences, Engineering, and Medicine, Eugene Steuerle and Leigh Miles Jackson, eds. (2016). Advancing the Power of Economic Evidence to Inform Investments in Children, Youth, and Families, Washington, DC: The National Academies Press.