Enhancing Rigor, Relevance, and Equity in Research and Evaluation through Community Engagement

Meeting Topic

Background and context. Community leaders, members, and researchers increasingly recognize that engaging communities in research and evaluation is critical for implementing effective programs and building trust. These groups have also recognized the value of community-engaged research to improve the rigor and relevance of research and evaluation and better address historical and ongoing systemic inequities.

President Biden’s Executive Order on Advancing Racial Equity and Support for Underserved Communities through the Federal Government (EO 13985) underscores the Federal Government’s commitment to advancing racial equity and support for underserved communities. It includes a directive for agencies to “consult with members of communities that have been historically underrepresented in the Federal Government and underserved by, or subject to discrimination in, Federal policies and programs.”1 This presents an opportunity for those who conduct Federally supported research to identify opportunities to increase engagement with underserved communities. EO 13985 names multiple, often intersecting, dimensions of inequity. Given the short length of the 2021 OPRE Methods Meeting (2 half-days), this meeting will broadly focus on how community-engaged research methods can be used to address inequity by collaborating with communities impacted by our programs to identify promising practices and co-create programs. The presentations in this meeting will acknowledge and explore the multiple, complex dimensions of inequity that ACF program participants and service providers must contend with, such as racial inequity.

Community-engaged research is defined “not by the methods used, but the principles that guide the research and the relationships between researchers and the community.”2 Such research is best understood as a spectrum of varying degrees of community involvement; some researchers consult community advisory boards at key points in a project, while others design studies with a community-based participatory research approach that centers community perspectives throughout the research and evaluation process. Similarly, while the term “community” has many definitions, this meeting will focus on communities directly impacted by the human service programs we seek to understand (as opposed to a community of youth or a community of technical experts, for example).

Conducting community-engaged research requires intentionality. It is critical that researchers who engage communities work to prevent tokenizing and ignoring power dynamics between research entities and communities. Researchers should approach community-engaged research with a full understanding of the level of effort required to involve community members at each step of the process and potentially build a long-term partnership. To effectively leverage the strengths and navigate challenges of the Federally funded research and evaluation context, additional considerations are required.

When implemented well, community-engaged approaches offer several significant advantages to research and evaluation. Engaging communities in research and evaluation— especially when identifying research questions, designing and implementing the intervention, recruiting study participants, and interpreting findings—may lead to more apt research questions, greater recruitment success, increased external validity, greater retention of participants, and enhanced understanding of findings.3 Community-engaged research also can benefit the people participating in the research or evaluation. It can provide an opportunity to “establish and exercise trust, [and] balance historically rooted racial, ethnic, gender, and other power differentials”.4 This is particularly true when community members are engaged from the initial phase of the research process. Community-engaged research is more than a project or study; it is a process that requires ongoing input from researchers and communities to sustain long-term relationships and collaboration.

Meeting topics and goals. OPRE’s 2021 research and evaluation methods meeting convened researchers, evaluators, Federal staff, community members, and others working in different human services areas to (1) explore research approaches that engage communities throughout the research process; (2) discuss the potential value of doing so for program design, improvement, and evaluation; (3) articulate the motivation for using community-engaged approaches (4) offer practical considerations for applying these approaches in Federal research and evaluation projects (5) present examples of how community-engaged approaches have been applied in the context of Federally-sponsored research and evaluation.

The meeting included presentations and discussions on the following questions:

  • What is community-engaged research broadly defined, being mindful that these approaches fall across a spectrum of involvement depending on time, budget, research type, and other factors?
  • How can Federally funded community-engaged research better address historical and ongoing systemic inequities?
  • How do researchers and evaluators meaningfully engage communities in each stage of the research and evaluation process, especially in a Federal research context?
  • How can engaging communities help to enhance racial equity in research and evaluation?
  • How can researchers engaging with communities of color acknowledge their power and position? How can they acknowledge the ways research and researchers have created harm and/or perpetuated community mistrust in the past, especially in a Federal research context?
  • Who are the primary stakeholders for a particular research project? How can researchers prioritize equity when determining whose voices are incorporated in the research process?
  • What can we learn from how community-engaged research has worked in fields other than human services evaluation?
    What are the potential benefits and considerations of engaging communities in research in various social services settings? Are there challenges to conducting this type of research?
  • When, for what program types, and under what circumstances are these methods most suitable? What are some examples of a community-engaged approach that has proven successful?
  • How does and can the Federal Government support community-engaged research?
  • How can we mitigate potential time and resource challenges associated with community-engaged research, especially in the context of Federally funded research?

The goals of the meeting were to:

  1. Help attendees develop a broad understanding of community-engaged research and methods for undertaking community-engaged research with social services programs.
  2. Explore considerations for successfully applying community-engaged research methods and understand the settings where they are most appropriate, in a Federal research context
  3. Help attendees consider ways to address racial equity when using community-engaged research.
  4. Explore working on Federally led or sponsored research from the perspective of community-based organizations or members.
  5. Discuss ways the Federal Government does or could support and use community-engaged research approaches when appropriate.
  6. Equip attendees with tangible tools and resources to incorporate community-engaged research methods in their work.

The meeting convened Federal staff and researchers interested in exploring community engaged approaches to research and evaluation. It was held virtually October 27 and 28, 2021. More than 300 attendees and 25 speakers participated. Participants and speakers included representatives from the Federal Government, community members who have participated in research, research firms, and academia.

Footnotes

1. Executive Order No. 13985, 3 C.F.R. 7009–7013. (2021). Advancing racial equity and support for underserved communities through the Federal Government. https://www.federalregister.gov/documents/2021/01/25/2021-01753/advancing-racial-equity-and-support-for-underserved-communities-through-the-federal-government

2. McDonald, M. A. (2009). Practicing community-engaged research. Duke Center for Community Research. https://ccts.osu.edu/sites/default/
files/inline-files/Practicing%20Community-engaged%20Research_Training%20Module.pdf

3. Viswanathan, M., Ammerman, A., Eng, E., Garlehner, G., Lohr, K. N., Griffith, D., Rhodes, S., Samuel-Hodge, C., Maty, S., Lux, L., Webb, L., Sutton, S. F., Swinson, T., Jackman, A., & Whitener, L. (2004). Community‐based participatory research: Assessing the evidence, summary. AHRQ Evidence Report Summaries.

4. Isler, M. R., & Corbie‐Smith, G. (2012). Practical steps to community engaged research: From inputs to outcomes. The Journal of Law, Medicine & Ethics40(4), 904–914.

Agenda and Presentations

A note-taking worksheet for the meeting is available here to download. You can use this tool to set an intention for the meeting, note key terms and definitions, and capture speaker comments and cited resources.

Wednesday, October 27

Welcome and Opening

1:00-1:10 pm EDT

Gabrielle Newell, Social Science Research Analyst, OPRE 

Naomi Goldstein, Deputy Assistant Secretary for Planning, Research, and Evaluation, OPRE 

Motivation and Level-setting

1:10-2:00 pm EDT – Presentations and emcee remarks

Reimagining Our Approach to Research to Advance Racial Equity

Amanda Coleman, Deputy Division Director, Division of Child and Family DevelopmentOPRE 

Video: Reimagining Our Approach to Research to Advance Racial Equity

Video with audio descriptions: Reimagining Our Approach to Research to Advance Racial Equity

Integrating Participatory Approaches Into OPRE-Funded Evidence Generation to Support the Lives of Children and Families

Emily OzerProfessor of Community Health Sciences, University of California, Berkeley 

Break

2:00-2:10 pm EDT

Case Study 1: Community Participatory Action Research

2:10-3:00 pm EDT – Presentation and Q&A

Community Participatory Action Research Case Study #1: Broward County, Florida

Sue Gallagher, Chief Innovation Officer, Children’s Services Council of Broward County 

Adamma Ducille, Director of Equity and Organizational Development

Tiffany Csonka, Parent Co-Researcher & Youth System Organizing Consultant

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

3:00-3:20 pm EDT

Case Study 2: YPAR In Action

3:20-4:00 pm EDT – Presentation and Q&A

Youth Participatory Action Research (YPAR)

Brian Villa, Researcher, University of California, Berkeley, Innovations for Youth (i4Y) 

Leili Lyman, Community Liaison, University of California, Berkeley

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

4:00-4:10 pm EDT

Case Study 3: Federally-Sponsored Example

4:10-4:55 pm EDT – Presentation & Q&A

Community Engagement in a Federally-Sponsored Center: The TRC

Jessica V. Barnes-Najor, Director for Community Partnerships, Michigan State University and The Tribal Early Childhood Research Center

Deana Around Him, Senior Research Scientist, Child Trends 

Ann Cameron, Head Start Director, Inter-Tribal Council of Michigan

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE 

Day 1 Closing Remarks

4:55-5:00 pm EDT

Gabrielle Newell, Social Science Research Analyst, OPRE 

Thursday, October 28

Discussion: Reflections from Community Members

1:00-2:00 pm EDT

Tiffany Csonka, Parent Co-Researcher & Youth System Organizing Consultant

Brian Villa, Researcher, University of California, Berkeley, Innovations for Youth (i4Y) 

Leili Lyman, Community Liaison, University of California, Berkeley

Ann Cameron, Head Start Director, Inter-Tribal Council of Michigan 

Q&A Moderator: Shariece Evans, Senior Social Scientist, OPRE 

Break

2:00-2:10 pm EDT

Discussion: How Can Funders Support Community-Engaged Research?

2:10-3:00 pm EDT – Panel Discussion and Q&A

Aleta Lynn Meyer, Lead for Primary Prevention and Resilience, OPRE 

Laura Erickson, Social Science Analyst, ASPE

Nadra Tyus, Program Director/Health Scientist Administrator, NIMHD  

Shruti Jayaraman, Chief Investments Officer, Chicago Beyond 

Vivian Tseng, Senior Vice President of Programs, William T. Grant Foundation 

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

3:00-3:15 pm EDT

Part 1 of 2: Tools and Supports for Implementing Community-Engaged Research and Evaluation Approaches

3:15-4:00 pm EDT – Presentation and Q&A

Community Engagement Methods

Elsa Falkenburger, Principal Research Associate, Urban Institute 

Eona Harrison, Senior Research Associate, Urban Institute  

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

4:00-4:05 pm EDT

Part 2 of 2: Tools and Supports for Implementing Community-Engaged Research and Evaluation Approaches

4:05-4:50 pm EDT – Presentation and Q&A

Advancing Racial Equity Through Research & Community Engaged Methods

Jenita Parekh, Senior Research Scientist, Child Trends 

Chrishana Lloyd, Senior Research Associate, Child Trends 

Esther Gross, Technical Assistance Specialist, Child Trends 

Kristine Andrews, Senior DirectorIdeas to Impact 

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Closing Remarks

4:50-5:00 pm EDT

Gabrielle Newell, Social Science Research Analyst, OPRE 

Meeting Products

Check back soon for additional meeting products.

 

Session Video: Reimagining Our Approach to Research to Advance Racial Equity

Session Video: Reimagining Our Approach to Research to Advance Racial Equity (audio descriptions)

Engaging Communities in Each Research and Evaluation Step: A Note-Taking Worksheet for the OPRE 2021 Methods Meeting

(If clicking the link to the worksheet does not automatically download the Word document, paste https://opremethodsmeeting.org/wp-content/uploads/2021/08/2021-Methods-Meeting-Notetaking-Tool.docx into your browser to download the worksheet).