Survey FAQ

What is a statistical survey? What does this mean? When should it be used vs. a take the pulse survey?

Most surveys conducted in the business world are convenience samples or self-selected samples, rather than scientifically designed statistical samples.  Examples of convenience samples are those that are done at a mall or other setting where groups of people are asked their opinion on a product or service. 

Self-selected samples are growing more popular with the typical examples being surveys conducted on web sites.  These types of surveys can give an organization valuable feedback at a reasonable cost.  

If an association wants to know whether a new service will be valuable to their members, an on-line survey can determine whether large numbers would find it useful or not.    With these types of surveys, there is some virtue in having large numbers of respondents.  If a relatively large number of members are in favor of a new service, it makes sense to offer it whether those respondents really represent the majority of members or not.

This representation issue is the crux of the difference between a self-selected respondent survey and a statistical survey.  When members are free to respond to the survey or not, you have no real assurance that that only people who have strong, negative, or positive have responded to the survey. You do not know whether a bias is being introduced into the responses. 

A statistical survey is designed to represent the all members of the association or a particular target population.  Surveys can be designed in various ways to achieve specialized goals, but a simple random survey of members can reliably reflect the views of the general population.     

A statistical survey is more reliable than a self-selected survey because you know it represents your target population.  The accuracy of a statistical survey is determined by the number of respondents in the sample. The validity of the survey is determined by the proportion of people in the original sample who respond to the survey, also know as the response rate.  

A survey with about 900 respondents will have a margin of error of about +-3 percentage points.   For a statistical survey to be valid, a response rate of approximately 60 percent is required.  A statistical survey that ends up with a low response rate suffers from the same unknown representation as in the self-selected survey noted above.

The chief obstacle in doing statistical surveys is the cost.  To get a response rate of 60 percent typically requires an inducement, plus 2-3 follow up contacts to persuade people to respond to boost the response rate.  Also, there are front-end costs in survey design and the final analysis will be more complex. 

There may be cases were statistically valid data are worth the cost. Examples of these are situations where decisions involving large amounts of money or other high risk outcomes will be made on the basis of the survey results.  If the information is going to be presented to other organizations or legislative bodies, more reliable might be more effective in influencing policy outcomes.

Regardless of whether you choose our standard package or a custom designed statistical sample, our surveys  include special mailers to help boost the proportion of your members responding to the survey, making you are getting  the best results for your budget.  For more information on response rates and survey validity, please click here.

How do you write survey questions that are not misleading? Tips for writing questions that get the appropriate response.

No one individual can write the survey questions.  As the client, you should identify the goals and general framework of the questions.  The framework should be determined by your organization's policy makers, or client organizations.  

Track Marketing Group will work with you to help refine the questions, and we will suggest appropriate demographic or general questions from a list of extensively tests questionnaire items.  After the team has completed the questionnaire, it should be tested by a group of potential respondents for clarity, utility, and neutrality.  The more people involved in the design process, the better.

Questions should be easy to understand, and flow in a consistent basis.  People tend to like to run through the surveys quickly, so sentences should be short and direct.   Scales should be done in a consistent format.   Questionnaires should be short to minimize "respondent burnout."  Use of loaded terms or extensive jargon should be avoided.   Avoid combining 2 questions into one response.  

Return to TakeSurvey home page

©2002 Track Marketing Group. All Rights Reserved.