Enhancing Rigor, Relevance, and Equity in Research and Evaluation through Community Engagement

Register Now For The 2021 Virtual Meeting

(Must use Chrome, Edge, or Firefox browser)

Meeting Topic

Background and context. Community leaders, members, and researchers increasingly recognize that engaging communities in research and evaluation is critical for implementing effective programs and building trust. These groups have also recognized the value of community-engaged research to improve the rigor and relevance of research and evaluation and better address historical and ongoing systemic inequities.

President Biden’s Executive Order on Advancing Racial Equity and Support for Underserved Communities through the Federal Government (EO 13985) underscores the Federal Government’s commitment to advancing racial equity and support for underserved communities. It includes a directive for agencies to “consult with members of communities that have been historically underrepresented in the Federal Government and underserved by, or subject to discrimination in, Federal policies and programs.”1 This presents an opportunity for those who conduct Federally supported research to identify opportunities to increase engagement with underserved communities. EO 13985 names multiple, often intersecting, dimensions of inequity. Given the short length of the 2021 OPRE Methods Meeting (2 half-days), this meeting will broadly focus on how community-engaged research methods can be used to address inequity by collaborating with communities impacted by our programs to identify promising practices and co-create programs. The presentations in this meeting will acknowledge and explore the multiple, complex dimensions of inequity that ACF program participants and service providers must contend with, such as racial inequity.

Community-engaged research is defined “not by the methods used, but the principles that guide the research and the relationships between researchers and the community.”2 Such research is best understood as a spectrum of varying degrees of community involvement; some researchers consult community advisory boards at key points in a project, while others design studies with a community-based participatory research approach that centers community perspectives throughout the research and evaluation process. Similarly, while the term “community” has many definitions, this meeting will focus on communities directly impacted by the human service programs we seek to understand (as opposed to a community of youth or a community of technical experts, for example).

Conducting community-engaged research requires intentionality. It is critical that researchers who engage communities work to prevent tokenizing and ignoring power dynamics between research entities and communities. Researchers should approach community-engaged research with a full understanding of the level of effort required to involve community members at each step of the process and potentially build a long-term partnership. To effectively leverage the strengths and navigate challenges of the Federally funded research and evaluation context, additional considerations are required.

When implemented well, community-engaged approaches offer several significant advantages to research and evaluation. Engaging communities in research and evaluation— especially when identifying research questions, designing and implementing the intervention, recruiting study participants, and interpreting findings—may lead to more apt research questions, greater recruitment success, increased external validity, greater retention of participants, and enhanced understanding of findings.3 Community-engaged research also can benefit the people participating in the research or evaluation. It can provide an opportunity to “establish and exercise trust, [and] balance historically rooted racial, ethnic, gender, and other power differentials”.4 This is particularly true when community members are engaged from the initial phase of the research process. Community-engaged research is more than a project or study; it is a process that requires ongoing input from researchers and communities to sustain long-term relationships and collaboration.

Meeting topics and goals. OPRE’s 2021 research and evaluation methods meeting will convene researchers, evaluators, Federal staff, community members, and others working in different human services areas to (1) explore research approaches that engage communities throughout the research process; (2) discuss the potential value of doing so for program design, improvement, and evaluation; (3) articulate the motivation for using community-engaged approaches (4) offer practical considerations for applying these approaches in Federal research and evaluation projects (5) present examples of how community-engaged approaches have been applied in the context of Federally-sponsored research and evaluation.

The meeting will include presentations and discussions on the following questions:

  • What is community-engaged research broadly defined, being mindful that these approaches fall across a spectrum of involvement depending on time, budget, research type, and other factors?
  • How can Federally funded community-engaged research better address historical and ongoing systemic inequities?
  • How do researchers and evaluators meaningfully engage communities in each stage of the research and evaluation process, especially in a Federal research context?
  • How can engaging communities help to enhance racial equity in research and evaluation?
  • How can researchers engaging with communities of color acknowledge their power and position? How can they acknowledge the ways research and researchers have created harm and/or perpetuated community mistrust in the past, especially in a Federal research context?
  • Who are the primary stakeholders for a particular research project? How can researchers prioritize equity when determining whose voices are incorporated in the research process?
  • What can we learn from how community-engaged research has worked in fields other than human services evaluation?
    What are the potential benefits and considerations of engaging communities in research in various social services settings? Are there challenges to conducting this type of research?
  • When, for what program types, and under what circumstances are these methods most suitable? What are some examples of a community-engaged approach that has proven successful?
  • How does and can the Federal Government support community-engaged research?
  • How can we mitigate potential time and resource challenges associated with community-engaged research, especially in the context of Federally funded research?

The goals of the meeting are to:

  1. Help attendees develop a broad understanding of community-engaged research and methods for undertaking community-engaged research with social services programs.
  2. Explore considerations for successfully applying community-engaged research methods and understand the settings where they are most appropriate, in a Federal research context
  3. Help attendees consider ways to address racial equity when using community-engaged research.
  4. Explore working on Federally led or sponsored research from the perspective of community-based organizations or members.
  5. Discuss ways the Federal Government does or could support and use community-engaged research approaches when appropriate.
  6. Equip attendees with tangible tools and resources to incorporate community-engaged research methods in their work.

The meeting will convene Federal staff and researchers interested in exploring community engaged approaches to research and evaluation. It will be held virtually October 27 and 28, 2021. At least 250 participants and 15 speakers are anticipated. Participants and speakers will include representatives from the Federal Government, community members who have participated in research, research firms, and academia.

Footnotes

1. Executive Order No. 13985, 3 C.F.R. 7009–7013. (2021). Advancing racial equity and support for underserved communities through the Federal Government. https://www.federalregister.gov/documents/2021/01/25/2021-01753/advancing-racial-equity-and-support-for-underserved-communities-through-the-federal-government

2. McDonald, M. A. (2009). Practicing community-engaged research. Duke Center for Community Research. https://ccts.osu.edu/sites/default/
files/inline-files/Practicing%20Community-engaged%20Research_Training%20Module.pdf

3. Viswanathan, M., Ammerman, A., Eng, E., Garlehner, G., Lohr, K. N., Griffith, D., Rhodes, S., Samuel-Hodge, C., Maty, S., Lux, L., Webb, L., Sutton, S. F., Swinson, T., Jackman, A., & Whitener, L. (2004). Community‐based participatory research: Assessing the evidence, summary. AHRQ Evidence Report Summaries.

4. Isler, M. R., & Corbie‐Smith, G. (2012). Practical steps to community engaged research: From inputs to outcomes. The Journal of Law, Medicine & Ethics40(4), 904–914.

Agenda and Presentations

Session times are approximate. Please check back for updates.

A note-taking worksheet for the meeting is available here to download. You can use this tool to set an intention for the meeting, note key terms and definitions, and capture speaker comments and cited resources.

Wednesday, October 27

Welcome and Opening

1:00-1:10 pm EDT

Gabrielle Newell, Social Science Research Analyst, OPRE 

Naomi Goldstein, Deputy Assistant Secretary for Planning, Research, and Evaluation, OPRE 

Motivation and Level-setting

1:10-2:00 pm EDT – Presentations and Q&A

Reimagining Our Approach to Research to Advance Racial Equity

Amanda Coleman, Deputy Division Director, Division of Child and Family DevelopmentOPRE 

Integrating Participatory Approaches Into OPRE-Funded Evidence Generation to Support the Lives of Children and Families

Emily OzerProfessor of Community Health Sciences, University of California, Berkeley 

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

2:00-2:10 pm EDT

Case Study 1: Community Participatory Action Research

2:10-3:00 pm EDT – Presentation and Q&A

Community Participatory Action Research Case Study #1: Broward County Florida

Sue Gallagher, Chief Innovation Officer, Children’s Services Council of Broward County 

Adamma Ducille, Assistant Director of Training and Capacity Building, Children’s Services Council of Broward County 

Tiffany Csonka, Parent Co-Researcher & Youth System Organizing Consultant

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

3:00-3:20 pm EDT

Case Study 2: YPAR In Action

3:20-4:00 pm EDT – Presentation and Q&A

Youth Participatory Action Research (YPAR)

Brian Villa, Researcher, University of California, Berkeley, Innovations for Youth (i4Y) 

Leili Lyman, Community Liaison, University of California, Berkeley

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

4:00-4:10 pm EDT

Case Study 3: Federally-Sponsored Example

4:10-4:55 pm EDT – Presentation & Q&A

Community Engagement in a Federally-Sponsored Center: The TRC

Jessica V. Barnes-Najor, Director for Community Partnerships, Michigan State University and The Tribal Early Childhood Research Center

Deana Around Him, Senior Research Scientist, Child Trends 

Ann Cameron, Head Start Director, Inter-Tribal Council of Michigan

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE 

Day 1 Closing Remarks

4:55-5:00 pm EDT

Gabrielle Newell, Social Science Research Analyst, OPRE 

Thursday, October 28

Discussion: Reflections from Community Members

1:00-2:00 pm EDT

Tiffany Csonka, Parent Co-Researcher & Youth System Organizing Consultant

Brian Villa, Researcher, University of California, Berkeley, Innovations for Youth (i4Y) 

Leili Lyman, Community Liaison, University of California, Berkeley

Ann Cameron, Head Start Director, Inter-Tribal Council of Michigan 

Tia Newman, Community Liaison

Break

2:00-2:10 pm EDT

Discussion: How Can Funders Support Community-Engaged Research?

2:10-3:00 pm EDT – Panel Discussion and Q&A

Aleta Lynn Meyer, Lead for Primary Prevention and Resilience, OPRE 

Laura Erickson, Social Science Analyst, ASPE

Nadra Tyus, Program Director/Health Scientist Administrator, NIMHD  

Shruti Jayaraman, Chief Investments Officer, Chicago Beyond 

Vivian Tseng, Senior Vice President of Programs, William T. Grant Foundation 

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

3:00-3:15 pm EDT

Part 1 of 2: Tools and Supports for Implementing Community-Engaged Research and Evaluation Approaches

3:15-4:00 pm EDT – Presentation and Q&A

Community Engagement Methods

Elsa Falkenburger, Principal Research Associate, Urban Institute 

Eona Harrison, Senior Research Associate, Urban Institute  

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Break

4:00-4:05 pm EDT

Part 2 of 2: Tools and Supports for Implementing Community-Engaged Research and Evaluation Approaches

4:05-4:50 pm EDT – Presentation and Q&A

Advancing Racial Equity Through Research & Community Engaged Methods

Jenita Parekh, Senior Research Scientist, Child Trends 

Chrishana Lloyd, Senior Research Associate, Child Trends 

Esther Gross, Technical Assistance Specialist, Child Trends 

Kristine Andrews, Senior DirectorIdeas to Impact 

Q&A Moderator: Kriti Jain, Senior Social Science Research Analyst, OPRE

Closing Remarks

4:50-5:00 pm EDT

Gabrielle Newell, Social Science Research Analyst, OPRE 

Attendee Logistics

Join the discussion by tagging @OPRE_ACF and #OPREmethods on Twitter! 

You can also sign-up for the OPRE Newsletter, like OPRE on Facebook, and follow OPRE on Instagram.

Virtual Meeting Access for Attendees

The virtual meeting will take place on Wednesday, October 27 and Thursday, October 28, 2020 from 1:00 p.m. to 5:00 p.m. EDT each day.  Please see below for details about accessing and participating in the event. If you have additional questions, please contact OPREmeeting@insightpolicyresearch.com.

How will I access the OPRE methods meeting event?

After registering, you will receive an email from the OPRE Methods Meeting Team with information about accessing the meeting livestream website, along with the event password. You may need to check your ‘Other’ rather than ‘Focused’ tab of Outlook, or your junk box to find the email.

Once on the livestream website, you will be prompted to enter the event password to enter. The event password was included in the email confirmation from the OPRE Methods Meeting Team when you registered. If you registered but did not receive an email with the livestream website information, please contact OPREmeeting@insightpolicyresearch.com for assistance.

Will I be expected to turn on my video camera?

No, meeting participants will not be visible or audible and will not need to use their webcams. Instead, the presentations will be livestreamed in Hubilo, a platform that will allow you to engage in various ways with the presentations (for example, by typing in questions for speakers or answering poll questions). We also encourage you to engage via social media using the information at the top of this page!

Which web browser will I need to access the event?

Google Chrome is recommended for optimal viewing. The livestream may work on other browsers (including Microsoft Edge, Safari, Firefox and Internet Explorer) but accessibility issues are more common using those platforms.

Are there any other technology requirements to view the event?

To ensure you can see the event, you should have an up-to-date version of your preferred media player (e.g., Windows Media Player, Flash).

Additionally, for the best experience, you should disconnect from VPN before the meeting and ensure you have a strong internet connection. You can go to speedtest.net to check your connection (10-12 Mbps download for HD streaming and 6 Mbps for SD streaming is recommended).

Speaker Logistics

Virtual Meeting Access for Speakers

The planning team will email meeting speakers customized links to join the live portions of the event (e.g., Q&A discussions) a few days in advance. Note you will be asked to join your live session 30 minutes before it is scheduled to begin. Our team will send you a customized invitation for your live session with the time to join and other important details. Please email the logistics team at OPREmeeting@insightpolicyresearch.com should you have any questions.

When you’re not signed in for your live session, you can view the meeting as a participant using the details on the Attendee Logistics tab.

Meeting Products

Engaging Communities in Each Research and Evaluation Step: A Note-Taking Worksheet for the OPRE 2021 Methods Meeting

(If clicking the link to the worksheet does not automatically download the Word document, paste https://opremethodsmeeting.org/wp-content/uploads/2021/08/2021-Methods-Meeting-Notetaking-Tool.docx into your browser to download the worksheet).