| ||||
It's called the Information Age for a good reason: information is the lifeblood of business today, and companies live and die by the stuff.
Surveys are one of the primary vehicles for collecting the information businesses need. Done right, surveys can reduce new product and other risk; generate insights about employees, customers, and markets; and align PR, advertising, and other communications programs with target constituencies. Done poorly, they can derail strategy and generate misguided marketing, customer service, and communications plans.
Your business—and your business strategy—is only as good as the information you have. So how do you ensure that a survey will give you the information you need?
First, you have to identify what you are looking for and understand just what surveys can—and cannot—do.
Powerful tools
Advanced statistical analysis makes surveys enormously powerful and insightful. Once, conducting a survey was so complex and time-consuming that few companies could afford to do it. But increased processing power, new technologies like computer-aided telephone interviewing (CATI), affordable analytical software, and lower communications costs have put the capability to conduct meaningful surveys within the reach of the smallest company or department.
| Many make the mistake of starting to write questions without spelling out objectives. |
|
— Roberta L. Sangster, American Statistical Association |
Generally, corporate surveys seek to understand markets, relationships, or transactions, says Frederick C. Van Bennekom, author of Customer Surveying: A Guidebook for Service Managers. Market surveys seek opportunities and requirements for offerings. Vincent Vaccarelli, director of the Xerox Business Research Group, a market research service based in El Segundo, California, that conducts research for Xerox and other firms, notes that for most companies it takes 120 key decisions, each with two or more alternatives, to launch a product. That's a lot of opportunities for things to go wrong; surveys can help reduce the risk.
Customer and employee surveys typically explore relationships, identifying both strengths and areas for improvement. Transaction surveys can operate as a quality control check, revealing the perceived effectiveness of a purchase or service call.
Surveys can be conducted over time to demonstrate the impact of communications and other programs. For example, New York City-based software leader Information Builders conducts surveys every six to twelve months to measure awareness of the company among target markets.
Surveys demand well-defined objectives, a good match of the intended communication to the medium, and clear presentation of results. Here's how to get a survey to give you information you can use.
Define objectives precisely. What problem will the results solve, or how will the results be incorporated into decisions or operations? "Many make the mistake of starting to write questions without spelling out objectives," says Roberta L. Sangster, chair of the survey review committee for the American Statistical Association, based in Alexandria, Virginia. Without an objective as a beacon, questions can lack relevancy, surveys can turn into swamps of complexity, and results can fail to illuminate issues.
What's more, when there's not a clearly defined objective, nice-to-know questions proliferate, making questionnaires too long, raising costs, complicating analysis, and lowering response rates. Ideally, surveys should take no more than twelve to fifteen minutes to complete. To limit survey length, Sangster recommends you ask yourself the following before composing questions:
- How well will the requested information help me meet my objectives?
- How much detail do I need?
- Can I get the desired information elsewhere?
Avoid committee-driven surveys with multiple objectives, and break up multi-part surveys into discrete sections. For instance, if you're seeking participants' feedback on an all-day event, divide the survey into several parts and distribute them in context: during registration, at lunch, etc.
A lack of objectives can also complicate analysis. Even a modest survey can generate an overwhelming volume of results, since each question can be examined alone or in combination with others. Without an objective to add focus, it's difficult to separate the informational gold from the dross.
Write the questionnaire properly. Effective questionnaire development involves three steps—running a focus group to reveal key issues and analytical requirements, question writing, and testing—and is an iterative process that can take weeks or even months.
| Open-ended questions can generate the most insights but are the hardest to categorize for analysis. |
| — Nick Wreden |
Made up of five to ten participants representing the survey population, focus groups can unveil new perspectives, uncover attitudes or responses that might need to be explored more deeply in the survey, and determine whether survey terms are commonly understood. For example, a focus group for Information Builders revealed confusion about the term business intelligence. As a result, the term was revised to information delivery in the actual survey, according to Jerry Inman, marketing director.
In writing the questions, use simple, familiar words whenever possible. Define technical or unfamiliar terms. Strike hot-button words like conservative or liberal as well as abbreviations and slang. Avoid question structures that can confuse respondents or lead to multiple answers. For example, break a question like "Should spending be reduced for R&D and marketing?" into two parts. Multiple-choice answers should have a neutral zone (e.g., "no change").
Since writing for the eye is different than writing for the ear, mail and telephone surveys differ. While asking if a respondent is "very satisfied, somewhat satisfied, neither satisfied nor dissatisfied, slightly dissatisfied, or very dissatisfied" works on the page, it does not work on the phone. When such detail is required, segment answers with either/or choices, then follow up the answer with more detailed choices.
Group questions by category or objective whenever possible. Within subject areas, group questions that are similar in structure. For example, put yes/no questions together, followed by those using scales or rankings. To boost response rates, put the most interesting questions at the beginning, and save personal or potentially embarrassing questions for last.
Open-ended questions can generate the most insights but are the hardest to categorize for analysis. Yes/no questions produce clarity, but can miss the shadings that can signal growing problems. Scales or rankings require the most work in analysis.
Finally, questionnaires need to be tested. Observe testees as they take a survey. Do they spend a lot more time on one question than on others? Do they seem confused by a particular section of the questionnaire?
Choose the right survey instrument. Surveys have traditionally been given via mail, on the telephone, or in person, but Web-based and e-mail surveys are gaining in popularity. In general, mail surveys are the least expensive to conduct and direct interviews are the most. Methodologies can of course be combined. For example, Xerox Business Research Group asks respondents to log on to a Web site during a telephone interview, especially if advertisements or other visuals are involved.
If you decide to run an e-mail survey, be careful; if your respondents haven't granted you permission to contact them, you could be accused of spamming them.
Manage the project carefully. Developing and testing the questionnaire isn't enough. It must be administered so as to minimize errors and maximize response; a response rate of 50% to 60% is considered acceptable by professional researchers. For mail surveys, experts recommend following a four-step process: first you send an introductory notice, then a questionnaire with personalized letter, and finally a follow-up postcard with either a thank-you or a reminder; the questionnaire is remailed after a few weeks to non-respondents.
Effective project management also includes data checking; a respondent sample should be recontacted to ensure survey integrity.
Present results effectively. The point of conducting a survey is to make sure that results are translated into action that benefits customers, employees, or constituencies. That won't happen if key results remain hidden in thick reports that land with a thud on executive desks. Extract key points that relate to the objective and ensure that they are clearly communicated—ideally, with easy-to-read graphs and charts—in terms that relate to the objectives. Avoid the survey jargon beloved by academics.
Surveys can reduce decision-making risk and improve customer service, employee relations, marketing, and communications. But your job isn't over when the results come in: once a survey has told you what the respondents want, it's up to you to decide how you're going to satisfy them.
· · · ·