Improving Implementation Research Methods for Behavioral and Social Science

Meeting Topic

A cornerstone of a successful intervention is the effective translation of evidence-based practices and research into actionable and sustainable practices. Yet, fidelity in the conversion of theory to practice is often difficult. Implementation research is an emerging, but not well understood field of research into the factors that affect the implementation and sustainability of programs. This field of research can yield important insights into the efficacy, quality, scalability, and adaptability of initiatives. Implementation research offers great promise, but the variety and fluidity of frameworks, definitions, and methodologies employed create unique analytical challenges. Presenters at this meeting will discuss what we can learn from implementation research, the role of community context and cultural diversity in implementation, measures and data collection issues in implementation research, study designs and analytic techniques, and next steps for policy and practice and for research.

The meeting will convene federal staff and researchers with an interest in improving the theoretical frameworks and methodological approaches utilized in implementation research. The ultimate goals of the meeting are to 1) provide a better understanding of the capacity of implementation research; 2) identify the promises and challenges involved in using such research; and 3) promote utilization and bolster support for implementation research.

Agenda and Presentations

Monday, September 20th

Welcome and Opening Remarks

9:00-9:30

Mark Greenberg, Administration for Children and Families
Naomi Goldstein, Administration for Children and Families

Overview and Setting the Context

9:30 – 10:30
Moderator
Kimberley Freire, CDC Division of Violence Prevention

Slide Deck: The Context and Case for Implementation Science
Karen Blase, National Implementation Research Network, UNC-Chapel Hill

Slide Deck: Community, Culture, and Service Contexts in Implementation Research 
Luis Zayas, Center for Latino Family Research, Washington University in St. Louis

Frameworks for Studying Implementation

10:45 – 12:30
Moderator
Carol O’Donnell, Institute of Education Sciences, Department of Education

Slide Deck: Advancing Implementation Science: Process & Outcome Conceptual Framework 
Enola K. Proctor, Washington University

Slide Deck: Measuring Enactment of Innovations and the Factors that Affect Implementation and Sustainability 
Jeanne Century, University of Chicago

Slide Deck: A Multi-Level Framework to Understand Factors Influencing Program Implementation in Schools 
Celene Domitrovich, The Pennsylvania State University

Implementation Measures and Data Collection

1:30 – 3:30
Moderator
Melissa Brodowski, Children’s Bureau, Administration for Children and Families

Slide Deck: Models, Methods, & Measures: Examining the Adoption, Implementation and Sustained Use of Innovations in the Ohio Mental Health System 
Phyllis C. Panzano, University of South Florida

Slide Deck: Assessing Organizational Context for Implementation Research 
Gregory A. Aarons, University of California, San Diego

Slide Deck: Conceptualizing and Measuring the Effects of Curriculum in Classroom Settings: A Focus on Quality of Delivery 
Bridget Hamre, University of Virginia

Okhee Lee, University of Miami

Break Out Session – Small Group Discussions with Representatives from Federal Agencies Regarding Federal Implementation Research Agendas

3:45 – 4:30

Administration for Children and Families & Assistant Secretary for Planning and Evaluation (Grant I)

Centers for Disease Control and Prevention (Grant II)

Institute of Education Sciences (Potomac)

National Institutes of Health (NIDA & NIMH) (Ballroom)

Moderated Panel Discussion with Federal Agency Members

4:30 – 5:00

Members will report out on small group discussion followed by questions and answers

Tuesday, September 21st

Study Design and Analytic Technique

9:00 – 11:30
Moderator
Brendan Kelly, OPRE, Administration for Children and Families

Slide Deck: Conceptualizing Intervention Fidelity: Implications for Measurement, Design, and Analysis 
Chris Hulleman, James Madison University

Slide Deck: Learning from Variation in Program Effectiveness and Implementation 
Howard Bloom, MDRC

C. Hendricks Brown, University of Miami Miller School of Medicine

Catherine Bradshaw, Johns Hopkins Bloomberg School of Public Health

Summary and Implication for Policy and Practice

12:00 – 1:15
Moderator
Molly Irwin, OPRE, Administration for Children and Families

Lauren Supplee, OPRE, Administration for Children and Families

Tammy Mann, Frederick D. Patterson Research Institute, UNCF

Naomi Goldstein, OPRE, Administration for Children and Families

Meeting Products

Journal of Behavioral Health Services & Research (JBHS&R)
Issue 4, October 2012

Abstracts available on http://jbhsr.fmhi.usf.edu/toc/39.html#39_4 

Directions in Implementation Research Methods for Behavioral and Social Science
Molly Irwin PhD, MPH, Lauren H. Supplee PhD
Pages 339-342

Measuring Enactment of Innovations and the Factors that Affect Implementation and Sustainability: Moving Toward Common Language and Shared Conceptual Understanding 
Jeanne Century EdD, Amy Cassata PhD, Mollie Rudnick MS Ed, Cassie Freeman MA
Pages 343-361

A Concept Mapping Approach to Guide and Understand Dissemination and Implementation
Amy E. Green PhD, Danielle L. Fettes PhD, Gregory A. Aarons PhD
Pages 362-373

A Procedure for Assessing Intervention Fidelity in Experiments Testing Educational and Behavioral Interventions
Michael C. Nelson BS, David S. Cordray PhD, Chris S. Hulleman PhD, Catherine L. Darrow PhD, Evan C. Sommer BS, BA
Pages 37-396

The Assimilation of Evidence-Based Healthcare Innovations: A Management-Based Perspective
Phyllis C. Panzano PhD, Helen Anne Sweeney MS, Beverly Seffrin PhD, Richard Massatti MSW, Kraig J. Knudsen PhD
Pages 397-416

Examining the Association Between Implementation and Outcomes
Elise T. Pas PhD, Catherine P. Bradshaw PhD
Pages 417–433