The death of the survey?

I recently got a survey in the mail and I actually did something I haven't done in years: I filled it out. It was the first "traditional" marketing survey I'd gotten in what felt like a long time, and there was a crisp, flat dollar bill attached to the letter. So, whether I threw the survey away or sent it back (I honestly don't remember), I had a dollar. Or at least, I did until one of my daughters needed money while we were at an ice cream stand.

Surveys and polls used to be the mainstay of social science and consumer research. We know more about ourselves as a society than perhaps we should at this point. And honestly, there are a lot of really ridiculous and ultimately meaningless uses of survey data out there. And then, of course, it's piggybacked onto data that isn't meant to be used in the way it's being used.

Of course, maybe I'm the only one who finds the whole "freakonomics" approach to social knowledge is snake oil -- I mean, I must be, since it's spreading like crazy. Amazon recently used their sales data to create a "blue and red readers" map of the United States, suggesting that the purchase of certain political books over others would peg the individual (and eventually the state) as a Republican or Democratic. But based on my recent book recommendations, Amazon also thinks I'm a serious dieter, a heavy wine drinker, an evangelical Christian who's also interested in Jewish mysticism, a science fiction freak who also wants to learn how to draw birds, and possibly going through puberty. It might be possible to draw some inferences about why people in Texas bought more "red" books and people in Massachusetts bought more "blue," and yes that does coincide with their voting patterns -- but only by a five percent margin of difference within each state.

Moving back to marketing data from surveys, even the retail industry is becoming less certain about their usefulness. Proctor and Gamble and Unilever are working with the Advertising Research Foundation to focus more on different ways of gathering consumer information, such as blogs, social networks, and consumer feedback on websites. The emphasis is on "mining consumer insights online." That may be an interesting new source of information, but again, it's still data that can't really tell enough about how or why people use products. And while surveys get tossed in the trash, recycled, or filled out incorrectly for a lark (yes, we've all done this), there is at least a small chance that the researcher can exercise a bit of control over the sample. The Advertising Research Foundation is pitching its approach as "learning how to listen," an insight that research methods in sociology and anthropology have insisted upon for many, many years. Despite the ascendency of survey research, for its "feel" of scientific certainty and the immediacy of results, even quantitative social scientists know that good research design is both inductive and deductive.

We're not at the end of survey data, by far, despite predictions based on the ARF's push for new insights into consumer behavior. Even so, the deathknoll might push more companies towards good solid ethnographic research, where researchers can see the difference between what folks are willing to say on social media and what they actually do. And then maybe someone can create a red and blue map that actually tells us something useful!