1. Not knowing what you don’t know
Its easy to do online surveys these days. Too easy. It may be so cheap and easy that you do it without understanding the basics and end up with misleading answers that send your business down the wrong path. This is worse than never doing any research in the first place. Spend a little time and get to know what you don’t know about market research. A basic review of the following topics is a great start.
- Sampling and sampling error
- Quantitative vs. qualitative research
- Question bias / question design
- Response rates / confidence levels
- Questionnaire coding
- Why people take surveys (social contract)
Some great books on these subjects are:
“Mail and Internet Surveys: The Tailored Design Method” by Don A. Dillman
“Asking Questions: A Definitive Guide to Questionnaire Design” by Norman Bradburn, Seymour Sudman, Brian Wansink
2. Not eliminating sampling errors
Now that you know what sampling error is you can understand why it is critical to conducting meaningful market research. Many of the online surveys you see today are full of potential sampling errors. Don’t be one of them. Take the time to develop a good sample and then make sure you get as many of those people as possible to your survey. This is probably the biggest difference between professional market research and your do-it-yourselfers. The pros take the time and money to develop good samples and then make sure that they get good response rates. You can to if you put in the effort.
- Always use a true random sample
- Tracking your respondents (PINs)
- Program the survey to eliminate duplicates and respondents with bad intentions
- Check the data for oddities (clean the data of illegitimate records)
- Use incentives (does not have to be monetary, see social contract)
3. Making decisions with inaccurate information
If you never understood any of # 1 and # 2 it is a good bet your survey is useless. Worse than that you may think it is telling you what to do with your important business decisions. Making decisions with inaccurate information is worse than taking a guess.
4. Writing bad questionnaires
You might get everything else right and then go and write a bad questionnaire. Lots of online surveys have at least one bad question. What is a bad question? It’s any of the following:
- Biased questions
- Unanswerable questions (impossible to know the answer)
- Questions with two meanings
- Hard to understand questions (way to long, strange use of words)
- Dumb questions (asking about something the researcher should already know, or has already asked)
5. Programming a hard to take survey
After you have spent all that time creating a good sample and writing good questions don’t ruin it by programming a hard to use survey. One of my top gripes is forcing respondents to complete every answer. Too much of this is going to get you either a contrived answer or the respondent leaving. Neither is good.
- Don’t force non-critical questions
- Don’t have non-standard buttons
- Don’t use non-standard technologies (java applets, etc.)
6. Going cheap
Both the good and bad thing about online market research is that it can be much less expensive than in the past. The bad of this is that it is just too easy to conduct flawed market research. Many of the above items cost time and money (sampling, questionnaire design, etc.) Spend the time and money to do it right. Even better hire a quality market research firm to do it for you. Either way you will save money in the long run by conducting quality market research.
7. Confusing social networking with quantitative market research
Talking with lots of people (social networking) might gain you valuable qualitative information but it is not quantitative market research. The difference is qualitative information rarely represents all of your audience and gives you individual opinions and ideas. Quantitative research on the other hand is designed to represent all of your audience and gives you answers that you can know reflects all of your customers. Don’t confuse the two. Social networking can be useful but understand its limitations.
8. Being overly “cute” with the survey tool
Your market research is supposed to gather meaningful information about your target audience. It is not supposed to impress them with all the high technology you can master. Keep your survey technology as simple as possible to reduce excluding respondents that are not up to speed with the latest and greatest.
- Use tried and true web technologies
9. Relying on only one source of information
Market research is a snapshot of opinions at a certain time. If your research results in wildly different answers than you were anticipating it is wise to confirm these conclusions with more data.
- Conduct another survey
- Look for corroborating data
10. Ignoring your market research
If you go to all the trouble to conduct a good study then have a plan to do something with that information. Too many organizations will conduct market research for one reason or another and when they get information back just sit on it. Don’t be the one who ends up saying “Wow, if we had just done what our market research told us we wouldn’t be in this bad position”. Before you conduct any online research have a plan as to what you will do with it.
Zachary Wilson is Vice President and Web Manager of Wilson Research Group (a small Silicon Valley market research firm) and has been conducting online surveys for over 10 years. This article targets those who are not market research professionals but want to conduct professional surveys.