Falling Response Rates to Social Surveys

Ho Hum.  Americans are less willing to do surveys these days.  Ho hum…

Unfortunately, this is a trend that is not positive; and we need to care.  If we want to understand and monitor the well-being of children, we need good data!

Fortunately, a Capitol Hill briefing on April 26, “Falling Response Rates to Social Surveys:  Challenges, Implications and Solutions for Policy and Business,”brought the issue to policymakers.  It was sponsored by the American Academy of Policy and Social Science, the Annie E. Casey Foundation, the National Academies’ Committee on National Statistics, Sage Publications, and the Russell Sage Foundation. Moderated by Chicago Tribune Columnist Clarence Page, panelists highlighted why survey response rates matter and what needs to be done to avoid having biased data.

At Child Trends, we know that accurate, reliable data about the development and well-being of children are bedrock to developing programs and policies that improve child well-being.  Dr. Paul Emrath, with the National Association of Home Builders, described at the briefing how the housing field also relies on federal data to inform business decisions, such as construction plans.

Dr. Roger Tourangeau, vice president of Westat, noted that getting good data has become increasingly costly due to declining response rates.  The difficulty of recruiting respondents has raised data collection costs, as interviewers have to make repeated call-backs, send letters of introduction, provide incentives, etc. Dr. Doug Massey, president, American Academy of Political and Social Science, and a professor of sociology at Princeton University, emphasized that the focus on response rates reflects concern that low survey response rates increase bias, as well as the cost of collecting data.

Dr. Kenneth Prewitt, former director of the U.S. Census Bureau, commented on “Big Data”, noting that these huge data bases derived from search engines and smart phones are not representative of the population and don’t provide the kind of information needed to inform policy and programs.

David McMillan, formerly of the Census Bureau, called on Congress to support R&D on data quality.  Indeed, all of the panelists emphasized the need for federal agencies and other researchers to focus on ways to improve data quality while holding down costs.  Adding administrative data to survey data is one way to do this.  Another approach is to do multi-mode surveys, which may, for example, draw an address-based sample and seek responses on the web (which are very low-cost), but follow up with phone or in-person interviews for people who don’t respond.

All panelists agreed that reliable data are a public good.  Private organizations don’t have the resources or the interest to conduct major surveys like the American Communities Survey, nor do they have any reason to make the data available as a public good.  I guess that leaves us with a need to invest in getting better response rates and unbiased data.

In the information age, a nation cannot function well without good information!