Tuesday, February 20, 2007

New Mueller Survey

So we should be grateful to Catellus for letting us give input into the retail landscape at Mueller? Well, how about this: you can start by collecting my input accurately.

The survey that was announced the other day will not reflect people's actual preferences, and it forces them to rank items for which no ranking may be necessary. That's a problem.

If you haven't taken the survey yet, let me explain. On one page, you're given a list of types of businesses that you'd like to see at Mueller. A coffeehouse? A drafthouse? An ethnic eatery? A family-style restaurant? Now, I care more about a coffeehouse first (check the name of this blog), and ethnic eateries ranks second, but I care about drafthouses and family-style places just about equally. (All of this critique extends to the rest of the survey; I'm just illustrating with these items.) However, the survey won't let me indicate this preference, which is my actual preference. Instead, I can only indicate one level of preference for each type of business. So I'm forced to create a ranking where I don't hold one. Why should I rank drafthouse over family-style place? That hierarchy doesn't exist in my mind.

Eventually, I just stopped answering questions. How can I know whether I want a yoga club over a sporting goods store? Ahh, maybe it's a thinking exercise. But give me a thinking exercise that's labeled as such; don't call it participatory neighborhood planning. Item over item, survey-taker over survey-taker, what will be generated is an inaccurate picture of the retail that people are interested in.

You may say, it's easy to criticize. What alternative would you propose? Well, a better mechanism would have been to put each item on a Likert scale (e.g., 0=not interested, 5=very interested). I'd bet money that Survey Monkey has this functionality. You say, what if everyone says they're very interested in all of the options? I guess that means you better get cracking providing people the options they want, instead of creating an environment where you can doctor the results with your methodology.

Look, I operate from the perspective that if people's input is actually important, then survey design and other data collection methodologies matter, too. I'm new to the Mueller Redevelopment listserv, and new to getting involved in planning. So can anyone tell me, is there any way to influence the survey methodology? Or is this the way it's always been done?

I sent a note to "Lkunz," who sent the note about the Survey Monkey survey, with this same critique. I'll post here when I hear from him or her about why a Likert scale (or some other design) wasn't chosen.


Stephanie said...

I had similar thoughts as I was completing the survey, and like you I eventually stopped ranking the items I was equally indifferent to.

I'm not clear, though, if the survey was composed with the intention of skewing my feedback in some manner, or it was just bad/thoughtless data design. I'll be interested to hear what you find out.

D'Amico said...

same thoughts here...and yes, you can do pretty much anything in surveymonkey.