Applying Mixed Methods and Qualitative Approaches to Social Policy Questions

Register Now For The 2022 Meeting

Meeting Topic

Background and context

The Office of Planning, Research, and Evaluation (OPRE) is tasked with using rigorous research to evaluate Administration for Children and Families’ programs and build evidence to inform policy and practice. To fully describe how, when, and why a program works, OPRE periodically employs mixed methods designs, which integrate qualitative and quantitative approaches and data, drawing on the strengths of each to ask and answer research questions.  

When implemented well, a mixed methods approach to program evaluation offers several significant advantages. Integrating qualitative and quantitative methods can answer research questions that one approach alone cannot. For example, analysis of survey or aggregated administrative data may reveal a trend in program participation; combining these data with interviews with program participants can illuminate motivations for seeking program services and experiences receiving them.1 The context revealed through incorporating qualitative methods can help explain trends in the quantitative findings. Qualitative findings can also reveal options to modify the program design, delivery, outreach, or other important factors which may not be discovered through quantitative methods alone. Mixed methods can also add value beyond providing context by increasing the validity of findings—for example, by supplementing survey data with interviews or focus groups to expand the reach to other study groups or enhancing available quantitative data. Qualitative research can illustrate different perspectives, creating space to learn more about program participant motivations and experiences: this factor is critical for promoting equity in research. Some qualitative methods, such as Photovoice and arts-based approaches, broaden the ways in which people can express themselves and promote a participatory approach. Importantly, using qualitative approaches independently or mixing methods can provide opportunities for wider involvement from communities that are the focus of research.  

While more systematic integration of mixed methods approaches into OPRE’s work holds promise, it is critical that standards of rigor and reproducibility apply to each phase of the research process if the research will inform policy and program decision making.2Researchers and evaluators should understand how to develop a mixed methods study design that is appropriate for the type of questions that decision-makers need answered. Similar to quantitative data, qualitative data analysis requires rigor. It is also important for study teams to understand how to effectively interpret findings from mixed methods research. In addition, qualitative research may present an opportunity to disseminate findings in a different way than quantitative findings, for example, with images and corresponding descriptions used from a Photovoice data collection. Researchers should be transparent about how all of the components of a study, including qualitative methods, are conceptualized and executed to make it easier for other teams to replicate the study with the same data.  

OPRE’s 2022 methods meeting will convene researchers, evaluators, Federal staff, community members, and others working in different human services areas. This meeting will broadly focus on how mixed method research approaches, along with qualitative methods within those approaches, can be applied to social policy and program evaluation. 

Meeting topics and goals

The meeting has the following goals: 

  • Describe what mixed methods research and mixed methods evaluations are. Explore how researchers can integrate qualitative and quantitative methods to increase the rigor of results. 
  • Define qualitative methods and understand how to write qualitative research questions, compared with the types of questions best answered by quantitative methods and mixed methods research designs. 
  • Expose attendees to examples of innovative mixed methods and qualitative research questions and designs. 
  • Help attendees consider how to integrate mixed methods and qualitative methods into their work from the start of a project. 
  • Provide project officers and contractors with an understanding of how mixed methods designs, including qualitative approaches, fit into project life cycles. 
  • Explore examples of high-quality, mixed methods approaches that have been applied in the context of federally-sponsored research and evaluation, particularly in the field of social policy. 

The meeting will include presentations and discussions on the following questions: 

  • What role do mixed and qualitative methods play in policymaking? What role could these methods play? 
  • How can researchers and evaluators ensure rigor, credibility, and other considerations for high-quality mixed and qualitative methods in their work? 
  • How can mixed and qualitative methods promote equity in research and evaluation by incorporating participant voices from a diversity of groups? 
  • What are the potential benefits of using mixed and qualitative methods in research in various human services settings? What are the challenges to conducting this type of research? 
  • What study designs are appropriate for the research questions that decision-makers need answered? What are some examples of mixed methods approaches that have proven successful? 
  • How can researchers ensure that appropriate human subjects protections are applied with the expanded use of qualitative methods? 
  • How does the terminology used in qualitative approaches differ from that of quantitative approaches? How can these terminology differences be reconciled? 
  • What are examples of mixed methods approaches that researchers and evaluators have successfully applied in the context of human services evaluation (i.e., how can researchers use qualitative data to inform the appropriateness or applicability of a quantitative measure for a given population)? 
  • How can we integrate qualitative and quantitative data in the context of evidence reviews? 
  • How can we effectively and accurately interpret and communicate the results of qualitative and mixed methods research? 
  • What are considerations when implementing mixed methods designs in federally-sponsored research? 

Meeting attendees and logistics

The meeting will convene Federal staff and researchers interested in exploring mixed methods and qualitative approaches to research and evaluation. It will be held virtually October 19 and 20, 2022. At least 500 participants and 20 speakers are expected to attend. Participants and speakers will include representatives from the Federal Government, community members who have participated in research, research firms, and academia. 

Footnotes

1. Halpern-Meekin, S., Hoiting, J. (2022, February 18). Abdul Latif Jameel Poverty Action Lab. Strengthening randomized evaluations with qualitative research: Baby’s First Years household measurement. https://www.povertyactionlab.org/blog/2-18-22/strengthening-randomized-evaluations-qualitative-research-babys-first-years-household

2. Brannen, J., & Moss, G. (2012). Critical issues in designing mixed methods policy research. American Behavioral Scientist, 56(6), 789–801.

Attendee Logistics

Join the discussion by tagging @OPRE_ACF and #OPREmethods on Twitter! 

You can also sign-up for the OPRE Newsletter, like OPRE on Facebook, and follow OPRE on Instagram.

Virtual Meeting Access for Attendees

The virtual meeting will take place on Wednesday, October 19 and Thursday, October 20, 2022 from 1:00 p.m. to 5:00 p.m. EDT each day.  Please see below for details about accessing and participating in the event. If you have additional questions, please contact OPREmeeting@insightpolicyresearch.com.

How will I access the OPRE methods meeting event?

The livestream site for the Methods Meeting is https://OPREmethodsmeeting2022.com. Only people who have registered for the meeting can enter the site. If you have registered, enter your email address into the login screen where prompted. You will then be asked to enter the passcode that was automatically emailed to you when you registered. You may need to check your ‘Other’ rather than ‘Focused’ tab of Outlook, or your junk box to find the email.

If you have issues logging in to the livestream site,  please contact OPREmeeting@insightpolicyresearch.com for assistance.

Are there any other technology requirements to view the event?

To ensure you can see the event, you should have an up-to-date version of your preferred media player (e.g., Windows Media Player, Flash).

Additionally, for the best experience, you should disconnect from VPN before the meeting and ensure you have a strong internet connection. You can go to speedtest.net to check your connection (10-12 Mbps download for HD streaming and 6 Mbps for SD streaming is recommended).

Will I be expected to turn on my video camera?

No, meeting participants will not be visible or audible and will not need to use their webcams. Instead, the presentations will be livestreamed on the meeting website. You will be able to interact with the speakers and other attendees by typing your questions, upvoting other comments and questions, and participating in polls. We also encourage you to engage via social media using the information at the top of this page!

Speaker Logistics

Virtual Meeting Access for Speakers

The planning team will send you a calendar invitation for your live session (e.g., Q&A discussions) with the time to join and other important details.  Note you will be asked to join your live session 30 minutes before it is scheduled to begin. Please email the meeting planning team at OPREmeeting@insightpolicyresearch.com should you have any questions.

When you’re not in your live session, you can view the meeting as a participant using the details on the Attendee Logistics tab. You must be registered as an attendee to view the meeting. You can register at https://opremethodsmeeting2022.com.

Meeting Products

Check back soon for meeting products.