The Promises and Challenges of Administrative Data in Social Policy Research

Meeting Topic

Accountability in federal and state programs is becoming increasingly important, and these programs are often required to collect administrative data to track services and demonstrate their progress or effectiveness. There is a growing understanding of the promise of these data, as improvements in technology and statistical methods make it possible to use administrative data for research purposes. Whether the data are collected by federal agencies, state governments, private industry, or local programs, attention has turned to the potential for using these data for policy and program evaluation. As outlined in The Office of Management and Budget (OMB) memo M-14-06, Guidance for Providing and Using Administrative Data for Statistical Purposes, there are high-quality and reliable data that can provide the foundation for research and evaluation to help understand how public needs are changing, how well policy and programs are addressing those needs, and where greater progress could be made.

The primary advantage of using administrative data for research is that agencies collect the data as part of their regular procedures. This means that data can be obtained from large populations over time without fielding a survey, which is expensive and burdensome for respondents. In addition, agencies stand to benefit from research that can inform their decisions about policies and programs. There are many innovative ways to capitalize on administrative data, including longitudinal follow-ups and analyses, conducting experiments by using existing lottery systems to randomly assign individuals to services, and linking across datasets from different agencies to understand service utilization and outcomes across a variety of domains. However, there are many challenges in using administrative data for research purposes, including difficulty gaining access to the data, concerns about privacy and confidentiality, linking across data sources, data quality, and unique analytic issues.

This meeting will focus on understanding what kinds of research question can be addressed using administrative data. Speakers will share their experience and knowledge around access and use of various types of administrative data for research purposes. It may include presentations and discussions on the following questions:

  • What can we gain and what do we lose by using administrative data, in addition to or in lieu of data from surveys?
  • What are the challenges around gaining access to administrative data, including governance and concerns about privacy, and what are some strategies for addressing them?
  • What are some strategies for creating successful partnerships between data owners and data users?
  • How can we get buy-in at the state and local levels and ensure that administrative data they collect are useful for both service providers and researchers, and is it reasonable to expect that the same data can serve both purposes?
  • What are the challenges in linking and matching administrative datasets, and what are some strategies for addressing these challenges?
  • What research questions are appropriate to address using aggregated data, and kinds of conclusions can be drawn from these analyses?
  • How can we use administrative data for longitudinal follow-ups and cost analyses?
  • How can we conduct opportunistic experiments using administrative data, for example in cases where agencies are overburdened and services are provided via a lottery?

The meeting will convene federal staff and researchers with an interest in expanding and improving utilization of administrative data. The ultimate goals of the meeting are to 1) better understand the availability and application of administrative data; 2) identify the promises and challenges involved in using such data; and 3) promote capacity, utilization, and innovative uses of administrative data.

Agenda and Presentations

Day 1: Thursday, October 1

Welcome and Opening Remarks

8:30 – 9:00

Maria Cancian, Deputy Assistant Secretary for Policy, Administration for Children and Families

Roundtable Discussion – Gaining Access and Maintaining Confidentiality

10:15 – 12:00
Moderator
Molly Irwin, Chief Evaluation Office, Department of Labor
Discussant
Kelly Maxwell, Child Trends

Panelists:
Beth Green, Portland State University
Charles Michalopoulos, MDRC
Maya Bernstein, Department of Health and Human Services
Jennifer Noyes, University of Wisconsin

Cool Applications - Part I

1:30 – 3:15
Moderator
Nicole Constance, Office of Planning, Research and Evaluation

Slide Deck:The long term impacts of cash transfers to poor families 
Anna Aizer, Brown University

Slide Deck: Evaluating a large-scale high school reform using administrative data from a naturally-occurring randomized trial 
Howard Bloom, MDRC

Slide Deck: Treatment costs among adults with serious mental illness: Influence of criminal justice involvement and psychiatric diagnoses 
Allison G. Robertson, Duke University

Slide Deck: The role of administrative data within economic evaluation 
Max Crowley, Penn State University

The Nuts and Bolts of Working with Administrative Data

3:30 – 5:00
Moderator
Anupa Bir, RTI International

Slide Deck: Building data sharing infrastructure at the state level 
Aaron Schroeder, Virginia Tech

Slide Deck: Linking data across multiple states and multiple data sources 
Julia Henly, University of Chicago

Slide Deck: Challenges in linking state and federal datasets 
Robert Goerge, Chapin Hall

Slide Deck: The role of the policy context in using and understanding administrative data 
Elizabeth Davis, University of Minnesota

Day 2: Friday, October 2

Cool Applications - Part II

8:45 – 10:15
Moderator
Anna Solmeyer, Office of Planning, Research and Evaluation

Slide Deck: Quick turnaround with administrative health data 
Katherine Giuriceo, Center for Medicare and Medicaid Innovation

Slide Deck: Using random assignment to test “nudge” messaging on SNAP applications 
Kinsey Dinan, NYC Human Resources Administration

Slide Deck: Using aggregate state assessment data to assess the impact of school-based interventions 
Robin Jacob, University of Michigan

Slide Deck: Estimating the impact of reserve activation on earnings: Survey vs. administrative data 
Jacob Klerman, Abt Associates

Federal Efforts and Future Directions

10:30 – 12:00
Moderator
Christine Fortunato, Office of Planning, Research and Evaluation

Census Bureau efforts to utilize and share data
Amy O’Hara, Census Bureau

Slide Deck: Building state capacity to use longitudinal data systems 
Missy Cochenour, AEM Corporation

Slide Deck: Improving service delivery through administrative data integration and analytics 
David Mancuso, Washington State Department of Social and Health Services

Slide Deck: Integrated data systems and their utility for policy research and evaluation 
John Fantuzzo, University of Pennsylvania

Meeting Products

Roundtable Discussion: The Promises and Challenges of Administrative Data in Social Policy Research