Thomas Guterbock summed things up nicely at the recent “New Technologies and Survey Research” conference held by the Program on Survey Research at Harvard University.

In his presentation “Strategies and Standards for Reaching Respondents in a New Age of Technology,” Guterbock – the Director of the University of Virginia’s Center for Survey Research – described both sides of the new technology coin: it’s easier than ever for pollsters and survey researchers to get in touch with people – via cellphone, email, instant message, etc.; at the same time, other technologies are making it easier for those same people to avoid surveys—by using caller id, answering machines, etc.

How to take advantage of technology in survey research while minimizing the drawbacks was the question at the center of this year’s PSR conference – which brought some of the foremost experts in survey research to Harvard.

“The point was to discuss the challenges and opportunities that new technologies pose for survey research methods,” said D. Sunshine Hillygus, Frederick S. Danziger Associate Professor of Government and Director of the Program on Survey Research. “By bringing together experts from academia and the commercial world offered very different perspectives about how best to incorporate new technologies into survey research.”

For instance, Scott Keeter, Director of Survey Research at the Pew Research Center, presented new research on the differences between surveys which include cell-phones in the sample compared to more traditional telephone surveys. While more and more people are switching to cell phone-only households, and while those people are usually younger, Keeter presented data showing that the difference in political attitudes resulting from a standard poll and a poll that includes cell phone users aren’t yet large enough to matter in most cases. He also noted how cell-phone surveys - while inevitable – are simply too expensive to be worth conducting right now. It costs about three times as much to conduct a cell phone survey as a traditional land line survey. LaToya Lang, an attorney and State Legislative Director for the Council of Market and Opinion Research (CMOR), outlined some of these reasons, such as government restrictions on the use of auto-dialers for cell phones.

One of the larger differences Keeter found was in media use, and media use matters a lot to Michael Link, who is Chief Methodologist for the Nielson company. Link explained that they could not ignore cell-phone only households. Nielson’s solution is to use an address-based sample to mail out letters, which give people the option of responding in a number of different ways (phone, mail, internet).

“These multi-mode surveys are increasingly being used to solve all sorts of survey research problems, both at Harvard and elsewhere,” explained Chase Harrison, Preceptor in Survey Research in the Department of Government. One of the recurring lessons from the conference was that there was not one-size-fits-all solution to the challenges created by new technologies – the solution should be dictated by the subject studied.

If there were two sides to the cell phone debate, there were at least that many in the debate over the use of the internet in survey research.

One problem with web surveys is that many Americans still lack web access, and there is currently no easy way to draw a random sample of Internet users. Instead, many conduct web surveys using opt-in or convenience samples.
One of the panel participants was Doug Rivers, a Stanford professor and CEO of Polimetrix, a net-based, opt-in survey firm. He argued that proper statistical corrections can adjust Internet data to make it equivalent to current telephone samples for some purposes. In contrast, Stanford University Professor Jon Krosnick presented research that found significant differences between Internet- and telephone-based survey results.

Mick Couper looked at a different aspect of Internet research, discussing the possibilities of Web 2.0 for survey research. Couper, who holds appointments at the universities of Michigan and Maryland, made his Harvard presentation just one week before receiving two prestigious awards from the American Association of Public Opinion Research, including the Warren Mitofsky Innovators Award. Fittingly, Couper looked at new research innovations on the web. For instance, in the online alternate-universe world of Second Life, there are already survey and market research applications. Not just in Second Life, but all over the web, there’s the chance for greater, more fluid, interactions between survey researchers and their respondents.

“If you’re a market researcher interested in testing a new product brand, your survey can include a virtual grocery story shelf,” Hillygus said. “You can enable the respondent to inspect—virtually--the items on the grocery story shelf, picking them up and turning it over before answering questions about the item. Web 2.0 applications can make surveys dynamic and interactive.”

Just like the conference. Not only was the lineup of experts diverse and impressive, the attendees knew their stuff, too.

“Some of the best contributions came from the audience,” Hillygus said. “I hope they all come back next year.” The program on survey research is already planning a conference next spring on the topic of evaluating survey quality.