Meeting Topic

Policy research focuses on finding out whether programs or policies have effects on the conditions they are intended to address, with the end goal of guiding the effective allocation of public dollars. Providing that guidance requires more than just an estimate of the net effects of a program or policy; it is also necessary to understand variation in those effects, the circumstances under which a program or policy has effects, and how and why it works (or does not work). Decades of experimental and non-experimental research has produced many techniques for addressing such questions, and especially promising innovations have been developed more recently as attention has turned to exploring these issues.

This meeting will include presentations and discussions of innovative applications of methods and analytic techniques that can be used to inform practice and policy by addressing questions such as:

• How much variation is there in program effects and study findings; what are the sources of that variation; and how can we find out which of these sources of variation drive program impacts?
• What methods and techniques do we have to answer questions about what treatment features or components drive program impacts?
  • How can we use natural variation in treatment across sites and studies to identify common program elements that consistently produce positive impacts?
  • How can we design studies to experimentally test program components and thereby help to maximize impacts?
  • How can programs use evaluation data to quickly answer questions about what is working well and what can be done to improve existing program operations?
• What are the characteristics of individuals, sites, and contexts that explain why interventions work better, worse, or differently for particular subgroups?
  • How can we identify these subgroups, based on observed or latent characteristics of their members?
  • How does an individual’s participation in a program, such as compliance, dosage, and path through program services predict outcomes?
  • How do contextual features, such as site-level characteristics or neighborhood setting, affect program impacts?
• What methods and techniques do we have to answer questions about steps in the causal pathway to participant outcomes?
  • How can we design future studies to learn about causal processes, and what are the challenges in establishing causality?
  • What analytic techniques can be used to explore questions about causal pathways in studies that are already complete?
• How can these methods inform the work of policymakers, researchers, and practitioners?
  • What can we do to balance questions about what works, under what circumstance, and how, without compromising our ability to demonstrate overall average program impacts?

We will focus on how experimental design and statistical analysis can be used to address the preceding questions in research that involves one site or multiple sites, or that synthesizes findings across multiple studies.

The meeting will convene federal staff, researchers, and practitioners with an interest in ways to expand the evidence base about drivers of program effects. The ultimate goals of the meeting are to 1) better understand the state of knowledge and how it is best communicated and applied in the field, 2) identify important gaps in knowledge, and 3) help to build a research agenda that will fill those gaps.