Addressing Unit Missingness in Social Policy Survey Research

Register Now For The 2023 Meeting

Meeting Topic

Background and context

The Office of Planning, Research, and Evaluation (OPRE) routinely administers surveys to evaluate Administration for Children and Families’ programs and better understand the communities in which programs operate. Surveys enable OPRE to collect and summarize perspectives from large numbers of program staff, partners, and participants with the goal of improving the implementation and delivery of human services. However, declining survey participation rates over the past decade,[1] exacerbated by the COVID-19 pandemic, threaten to undermine the validity of survey data to inform important program improvements. When participants do not respond to a survey request, uncertainty about the accuracy or bias of estimates increases, leading to the possibility that survey findings may not adequately represent the needs, perspectives, or experiences of the population of focus. Such nonresponse bias can occur if the attitudes, characteristics, and experiences of respondents are systematically different from those of nonrespondents.

Of particular concern is that marginalized populations or communities with specific communication considerations (e.g., communities where English is not the primary language or Internet access is limited) may be less likely to participate in surveys than other groups. For instance, in 2020, researchers found that households with high incomes were more likely to respond to the U.S. Census Bureau’s Current Population Survey Annual Social and Economic Supplement than those with lower incomes, upwardly skewing reported income data.[2] When survey respondents do not represent the full population of focus, data collection is inequitable and the data can be biased, limiting the survey’s utility for informing equitable programmatic and policy decisions. Therefore, it is important for OPRE to understand how to minimize unit missingness to ensure evaluations are rigorous and resulting data are high-quality and representative.

Researchers can, however, act at each step of the study design process to avoid, mitigate, and address the impact of nonresponse. Practices to improve outreach and recruitment of high priority populations—such as engaging community members to build trust and rapport, addressing privacy considerations, and using a range of methods and tools to reach participants—can significantly reduce nonresponse. Innovative questionnaire design and administration techniques, combined with usability and cognitive testing to design more respondent-centered and accessible surveys, can also encourage response.

Data from other sources can also be used to supplement or fill gaps left by missing survey data. For example, researchers can use observational methods to collect data, such as monitoring behaviors, observing interpersonal interactions, collecting passive data using sensors and geolocation, and collecting social media and digital trace data. Researchers may also link administrative data from private, federal, state, and program sources into their studies to augment missing data. Finally, many analytic techniques are available to measure and mitigate the impact of missing data on survey estimates, assess nonresponse bias, and adjust estimates for nonresponse. However, each of these strategies is associated with specific underlying assumptions, trade-offs, and limitations that researchers should carefully consider.

The 2023 OPRE Methods Meeting will convene researchers, evaluators, federal staff, and others working in human services research to address nonresponse in survey data. Experts in the field will discuss the reasons behind declining survey response rates and the potential for increased nonresponse bias before discussing the implications for human services research, programmatic decisions, and policy decision-making. Meeting presenters will also address research design strategies to reduce nonresponse and mitigate the impact of missing data on resulting estimates. They will also explore the use of other data sources to supplement and minimize the risk of missing data on surveys.

Meeting Topics and Goals

The meeting has the following goals:

  • Define unit missing data and discuss potential implications for human services research and evaluation.
  • Introduce strategies attendees can use to reduce survey nonresponse and mitigate the impact of missing data in their work.
  • Identify data collection methods and non-survey data sources that can be used to supplement or replace survey data.
  • Describe projects that successfully implemented strategies to mitigate survey nonresponse.

The meeting will include presentations and discussions on the following questions:

  • Why is nonresponse an important issue in survey research?
  • How can researchers identify patterns of missing data, including which populations have disproportionate nonresponse rates?
  • How do missing data affect the quality of survey estimates and what implications does this have for policymaking and programmatic decisions?
  • How can study and questionnaire designs reduce survey nonresponse?
  • What data analysis techniques and strategies can researchers use to address missing data in their projects?
  • What strategies can researchers use to improve outreach to populations that experience barriers to completing surveys.


[1] Office of Survey Methods Research. (2022). Household and establishment survey response rates. U.S. Bureau of Labor Statistics.

[2] Rothbaum, J., & Bee, A. (2021). Coronavirus infects surveys, too: Survey nonresponse bias and the Coronavirus pandemic.

Agenda and Presentations

Speaker slides will be posted soon.

Wednesday, October 18


1:00-1:10 pm EDT

Lauren Supplee, Deputy Assistant Secretary for Planning, Research, and Evaluation, OPRE

Statement of the Problem

1:10 -1:35 pm EDT

Brad Edwards, Vice President & Lead Scientific/Methodology Advisor, Westat

Emilia Peytcheva, Research Survey Methodologist, RTI International

Sampling and Recruitment

1:40-2:40 pm EDT

Mike Brick, Senior Vice President, Westat

Courtney Kennedy, Vice President of Methods and Innovation, Pew Research Center

Brady West, Research Professor, University of Michigan

Measuring Nonresponse

2:50-3:20 pm EDT

Andy Peytchev, Fellow, Survey Methodology, RTI International

Design and Administration

3:30-4:50 pm EDT

Ting Yan, Vice President and Associate Director, Westat

James Wagner, Research Professor, University of Michigan


4:50-5:00 pm EDT

Li Wang, Social Science Research Analyst, Office of Planning, Research and Evaluation

Thursday, October 19


1:00-1:15 pm EDT

Caitlin Lowery, National Poverty Fellow, Office of Planning, Research and Evaluation

Nonresponse Reduction and Adjustment Techniques

1:15-2:30 pm EDT

Raphael Nishimura, Director of Sampling Operations at the Survey Research Operations, University of Michigan

Uses of Administrative Data to Address Unit Missingness: Experiences from Past OPRE Projects

2:40-3:40 pm EDT

Rupa Datta, Distinguished Senior Fellow, NORC at the University of Chicago

David Judkins, Principal Associate, Social & Economic Policy, Abt Associates

Reflection: Where Do We Go From Here?

3:50-4:50 pm EDT

Lauren Supplee, Deputy Assistant Secretary for Planning, Research, and Evaluation, OPRE
Brad Edwards, Vice President & Lead Scientific/Methodology Advisor, Westat
Emilia Peytcheva, Research Survey Methodologist, RTI International
Raphael Nishimura, Director of Sampling Operations at the Survey Research Operations, University of Michigan

Kristyn Wong Vandahm, Social Science Research Analyst, Office of Planning, Research and Evaluation


4:50-5:00 pm EDT

Kelly McGowan, Office of Planning, Research and Evaluation

Attendee Logistics

Join the discussion by tagging @OPRE_ACF and #OPREmethods on Twitter! 

You can also sign-up for the OPRE Newsletter, like OPRE on Facebook, and follow OPRE on Instagram.

Virtual Meeting Access for Attendees

The virtual meeting will take place on Wednesday, October 18 and Thursday, October 19, 2023 from 1:00 p.m. to 5:00 p.m. EDT each day.  Please see below for details about accessing and participating in the event. If you have additional questions, please contact

How will I access the OPRE methods meeting event?

The livestream site for the Methods Meeting is  Only people who have registered for the meeting can enter the site. If you have registered, click the Login button, which takes you to the Login page, then enter the email address you registered with and the passcode that was sent to you in the event reminder emails from You may need to check your ‘Other’ rather than ‘Focused’ tab of Outlook, or your junk box to find the emails.

If you have issues logging in to the livestream site, please contact for assistance.

Are there any other technology requirements to view the event?

To ensure you can see the event, you should have an up-to-date version of your preferred internet browser (preferably Chrome).

Additionally, for the best experience, you should disconnect from VPN before the meeting and ensure you have a strong internet connection. You can go to to check your connection (10-12 Mbps download for HD streaming and 6 Mbps for SD streaming is recommended).

Will I be expected to turn on my video camera?

No, meeting participants will not be visible or audible and will not need to use their webcams. Instead, the presentations will be livestreamed on the meeting website. You will be able to interact with the speakers and other attendees by typing your questions, upvoting other comments and questions, and participating in polls. We also encourage you to engage via social media using the information at the top of this page!

Meeting Products

Check back soon for 2023 Methods Meeting products.