Using Questionnaires: The Do’s and the Don’ts
August 2018
Perhaps one of the least well-understood and most used forms of tourism research is the “questionnaire”. Tourism professionals and marketers utilize this scientific method on a regular basis and statistics play an important role in tourism decision-making and marketing.
Classically tourism should be divided into quantitative and qualitative research. Both forms of research are important and both provide important insights into the market. Statistical data are also divided along a second fault line, that of descriptive data (data that provides a picture of current reality) and inferential data (information from which an inference or a prediction may be made)
Many tourist offices maintain good descriptive data in the form of demographic profiles. These data provide an accurate picture of the visitor population. How many visitors come from area x of the world, what is the age breakdown, or what is the income grouping into which the visitor belongs?
Often tourism professionals assume that such “simple” statistics are easy to obtain. In reality accurate data are much more difficult to obtain than many people realize. Data must be collected through some measurement instrument, usually a questionnaire. Despite the popular misconception that questionnaires are easy to write, but questionnaire writing is an art with a whole host of challenges. For example, a good questionnaire must be developed in such a manner that each question means the same thing to each and every person who responds to it. In areas where there is a great deal of international travel that means that there are both language and cultural issues involved in the writing of a questionnaire. Among some of the other more common mistakes are the following.
- Keep the questions short, simple and to the point. Long written questionnaires either are not answered or answered in a haphazard manner.
- The longer the researcher waits the less accurate the data may be. As time passes people tend to forget or idealize a place. Once people have returned from a vacation they are more willing to speak about it or fill out longer questionnaires, but short-term memory loss begins within thirty days of the traveler’s return home.
- Decide what type of questioning best fits your goals. Do you want to use a direct face-to-face approach (often called an intercept study) or a self-administered questionnaires. Both forms have problems and biases. For example, in the intercept study, people often reflect what they think the interview wants to hear, and may positively or negatively react to the “personal chemistry” between the interviewer and the interviewee. Other problems that you should consider are: were these data corrupted because the questions were asked at the beginning of end of a trip? Did respondents answer differently due to the questionnaire being administered during the morning, afternoon or evening? In self-administered questionnaires there is no way to be sure that the person filling out the questionnaire is an adult or child or even a tourist.
- Decide on what is your definition of a tourist and a visitor. Much of the tourism industry is still arguing over who is a tourist. When reading tourism data be sure to seek the definition of a tourist. Furthermore, take the time to question how respondents were located and how were respondents enticed into answering the questionnaire.
- Despite what many marketers might like to believe, there are very few real random samples. Most samples are large “convenience samples.” It is often impossible to isolate the tourism population from others who may not be locals.
To help you construct proper and accurate: questionnaires Tourism Tidbits suggests the following:
- Always connect your questionnaire to a social model. In other words you need to determine what it is you want to know, and how you are going to utilize that information before you begin to collect it.
- Have an expert statistician examine the questionnaire before it is distributed. A questionnaire written in such a way that the data cannot be either tabulated or analyzed will be nothing more than a costly exercise in frustration. The most common errors in questionnaires are: the use of “compound questions” (a question that contains (or implies) an “and” or an “or”), words that have more than one meaning, confusing a convenience with a random sample, and poor questionnaire standardization.
- Select the style of questionnaire that best suits your needs. Information may be gathered in person, by telephone, or through the mail. Each style of information gathering has its own pluses and minuses. Beware of hidden costs that make what appears to be the least expensive option in reality the most costly.
- Test your questionnaire prior to distribution. Make your questionnaire say what you mean, not what you assume you mean. If people misinterpret what you are saying you have collected a lot of worthless data. This misinformation is not only useless it is harmful. To avoid such pitfalls run a pre-test to see if ambiguities have crept into to your questionnaire.
- Obtain the cooperation of the people who are going to distribute the questionnaire before you develop it. A non-distributed questionnaire is nothing more than a very expensive academic exercise! Bring the idea of a questionnaire before the next meeting of your hotel and restaurant association or local chamber of commerce. Take a straw poll to see if people will cooperate in its distribution before spending the money to develop the questionnaire.
- Develop a reward system that encourages the desire to respond. The further someone is in both place and time from his vacation spot the less likely he/she is to fill out the questionnaire, and the less accurate the given information will be. Some form of a reward system, such as a coupon for a discount at a local establishment may be the incentive needed to increase the response rate as well as the usefulness of your data.
- Develop a distribution technique that is the most scientifically representative of your locale. Statistics when applied correctly can often give highly accurate and useful information for the tourist professional. To obtain this information, however, professionals have to gather data with some form of a probability sample. As noted, it is impossible to obtain a probability sample without prior definitions and a determination of what “population universe” is to be examined. Often tourist information is obtained from what statisticians call a “convenience sample.” For example, a convenience sample is when we simply stop people on the street without each visitor having an equal chance of being selected. Information collected from that technique might be highly biased and misleading.
- Start simple. Always start simple and work up the ladder of sophistication. For example, before seeking inferential statistics, be sure that you have correct descriptive statistics. Make sure that you are collecting the data that meets your needs. Finally never forget that statistics can give the tourism researcher a great deal of information, but statistics never tell the whole story; they tell only part of it.
