Questionnaire Design

A Manual for Preparing a Questionnaire

Questionnaire Design

Perhaps the designing of questionnaires is the most important part of a survey process, since the created questionnaires should be able to accurately measure the experiences, opinions and behaviours of the population under research. This is also based on the fact that accurate random sampling in combination with high response rates is just but a mere waste of resources if the gathered information is based on a shaky foundation of biased or ambiguous questions. Therefore, creation of good measures in a research involves writing good and well organised questions to form the questionnaire. This is mainly because a questionnaire acts as a formalized set of questions which are used to obtain information from respondents in a particular study.  Thus, it is important to note that the overriding objective of designing questionnaire is to ensure the researcher’s information needs are translated into a set of specific questions that the selected respondents of the study are willing and capable to answer.

A questionnaire is undoubtedly one of the most important instruments used in the collection of data during a study.  This is mainly because a questionnaire is the main means through which quantitative and qualitative primary data can be collected.  Therefore, questionnaires are essential in enabling collection of both qualitative and quantitative data in a standardized way in order to make sure that the collected data are coherent and internally consistent for analysis.  This is very crucial in making sure that a questionnaire ensures comparability as well as standardization of the data across interviewers, which in turn leads to increased recording speed and accuracy, in addition to facilitating data processing. The significance of a questionnaire in a research as a data collection instrument makes the questionnaire design process equally significant.

The process of designing a questionnaire is multiple-staged, requiring attention to many details at once in order to make sure the questionnaire which is eventually designed can be utilised to gather the required data. However, the process of questionnaire design may be complicated by the fact that surveys can ask about various topics in varying details degrees, different ways can be used to ask questions, and earlier asked questions in a survey questionnaire may influence responses to questions that follow later.

Questionnaire design is undoubtedly one of the daunting  and also one of the most crucial steps of any research process putting into consideration that irrespective of two researcher having same objectives, it would probably be very rare to design the same questionnaire. However, there are no scientific principles that guarantee an ideal or optimal questionnaire since the process of questionnaire design is regarded to be as much an art as it is a science and for many years, the process of questionnaire design had been treated as an art, but substantial research over the years has indicated that science is inherently involved in the process of designing and developing good questionnaires hence it is essential to adopt the best practices and procedures of designing questionnaires. Therefore, the most crucial things in this process that play a major role in the resultant questionnaire design are the experience, creativity, and skills of the researcher.  However, there are several guidelines which are available and very essential in assisting researchers to design and develop their questionnaire and to help them in avoiding repetition of major mistakes. The guidelines which can be used in supporting the process of questionnaire design are shown below as a series of six steps that follow each other in a chronological manner:

Step 1: Decide what information is required

The first step in the process of designing a questionnaire is to specify the needed information. Thus, in order to design a focused questionnaire there is need to undertake a continual review of the research earlier stages, particularly the specific components of the research questions, the problem, and the hypotheses.    In addition, the design of the questionnaires should also be done putting into consideration the target respondents, as well as considering their experience and educational level.  This is essential in making sure that the language used as well as the context of the asked questions is familiar to the target respondents.  This is due to the fact that questions that are appropriate for people with college education may not be inappropriate for people with only a high school education. Hence, in designing a questionnaire, it is necessary to consider the respondent’s level of education because failure to considering this factor leads to a high incidence of no opinion or uncertain responses.

Step 2: Make a rough listing of the questions

After determining the information to be contained in the questionnaire, the next step is to make a list of all the questions to be included in the questionnaire. This step is aimed at ensuring that a comprehensive listing of questions in a questionnaire without worrying about the questions phrasing. In this step it is also necessary to make sure that only good questions are included in the questionnaire. This is mainly because a good question is the one producing reliable answers that are valid measures of the variables to be described whilst a bad question obscures, distorts or prohibits the communication from researcher to respondent, and vice versa.

In addition, it is advisable to use closed instead of -ended questions wherever possible. This is helpful in making sure that the context for all respondents is the same.  However, the closed question responses pattern is critically dependent on the presented answer set presented, and caution should be taken to ensure that the inclusion of other as an option in the answer set does not facilitate omission of another important answer. Questions that are more specific are highly preferred because they make it easier for the respondents to understand the information the researcher is looking for. It is also advisable to ensure that you avoid asking questions in a context that is may play a role in strongly deviating from the expected context. Longer questions should also be avoided in the questionnaire because they may produce longer responses which make the work of analysis difficult. In personal interviews, the context effects may be increased, reduced or eliminated by the introduction given by the interviewer to a particular question or series of questions, as per the desire of the researcher.  However, in questionnaires that are self-administrated, the same context effects are obtainable by the inclusion of printed instructions.

Step 3: Refine the question phrasing and wording

The determination of the right questions to be included in the questionnaire should be followed by making sure that the wording or phrasing of the questions makes sense in order to generate the right answers. Variations in the question wording per se generally do not have a great impact on the survey results stability.  However, they usually become significant upon introducing or tapping a different reality or concept or emotional level surrounding the issue under research. Therefore, it is generally advisable to keep your questions short, simple and concrete.

In addition, in order to ensure that the appropriate responses are generated it is crucial to avoid using unfamiliar or difficult words, two questions in one, a lot of information-carrying words in a single question, broad concepts (e.g., the government, children, etc.), words that does not have succinct specificity and sound like something else (impartial/partial), suggestions, big names as well as questions that call for a lot of effort from the respondent. In the wording or phrasing of questions it is important to note that specific questions are prone to contrast effects while general questions to assimilation effects.  In either of the above cases, context effects are likely to be much smaller, or vanish in case the judgement that was previously-formed t or substantial amounts of information that is relevant is chronically assembled in memory.  If there is necessity for response alternatives to a question, then their order may be put in a random manner unless a natural ordering exists. It is also important to strongly avoid stimuli or items that receive ratings that are extremely positive or negative because if you are asking for judgement about items that are in a series, the items that follow the ones rated in the extremities will be strongly affected. In addition, there is need to always offer a no opinion option, and whenever measuring attitudes it is essential to always omit the measure intensity and middle alternative.

Step 4: Develop the response format and put the questions into an appropriate sequence

Every question included in the questionnaire needs a response, and this could be included in a pre-coded list of responses or it sometimes the questions could be ended for the purpose of collecting verbatim comments. However, considering of the questions responses is essential in helping to get the questions right. Moreover, the way in which your questions are ordered is important since it brings about logic as well as flow to the interview, if the questionnaire is to be administered in a personal interview. Normally the questionnaire should be prepared in order to ease the respondent into the task with questions that are relatively easy and straightforward while questions that are more difficult or sensitive should be left until they are warmed up, and should be asked lastly. This is refereed to as the downward funnel where general questions are asked first followed by questions that are more sensitive and personal.

However, the designing of questionnaires should be carried out in a way which makes the task of questions reading, following instructions and answering as easy as possible for both the respondents and interviewers. Therefore, whenever the questions are being formatted they should meet the needs of the respondent first, the needs of the interviewer second while those of the data processor or researcher last. In self-administered questionnaires formatting and graphic design are particularly essential in order to guide or motivate respondents through the questionnaire with the aim of achieving good information organisation. The questions should be dominantly featured over any other additional explanatory information. It is also necessary to make sure that each question is included with all of the relevant information which is necessary for respondents to provide an answer instead of specifying this information in instructions. Always ensure that all the questions are aligned vertically together with the response categories.

When formatting the questionnaire it is important to utilize single-formats instead of multi-task formats while at the same time single-question formats should also be utilised instead of the matrix-question formats. There also the need to make instructions and headings at the top of the page more prominent compared to the ones that are provided in the page middle. Additionally, graphical design techniques should be utilised in order to establish a clear path through the questionnaire for ease following by the respondent. It is also important to remember that information the respondent needs to see is required to be visually emphasized while at the same time de-emphasizing on the information which the respondent does not necessarily need to see. Questions graphical layout can be utilised on the page with a purpose of distinguishing between different question structures while at the same time maintaining consistency. Whenever answers are requested in numbers it is critically essential to provide descriptive captions either beneath, above, or to the right of the provided answer spaces in addition to utilising the appropriate symbols or signs. It is crucial to avoid separating questions using rectangles and lines, in favour of an format. Thus, the questionnaire should be structured and organized in such a way that it, firstly, it is making sense to the respondents and, secondly, it does not give the respondents a choice of the order in which to answer the questions.

Step 5: Finalise the layout of the questionnaire

Upon putting into consideration all the steps outlined above the questionnaire now needs to be fully formatted while at the same time providing clear instructions to the respondents and the interviewer, including an essential introduction, probes, and routings. There is also the need to always make sure that enough space is provided for writing in response codes and answers needed.  This step is also aimed at verification of the spellings and grammar, and to ensure that the overall format of the questionnaire is appropriate.

Step 6:  Pre-test and revise the questionnaire

This is the final step in the questionnaire design process. The pre-test is also referred to as the pilot study whereby the questionnaires are administered to a relatively small number of respondents purposely to ensure that it works, but not towards obtaining pilot results. In theory the pilot study of the questionnaire should be done using the interview method that will be used in the real study such as (telephone interviews; self completed or personal interviews). After the pre-test or pilot study the necessary revisions are made in order to make sure that the appropriate answers or responses are obtained from the real research survey. This is also aimed at improving the validity and reliability of the questionnaire. Finally, the effectiveness of all the questionnaire design that is selected must be assessed in a pre-test or pilot study whereby the real picture of the envisaged outcome is determined, followed by thorough revisions whenever necessary.  The process of designing a questionnaire is a very important aspect of any research, and any extent of effort devoted to questionnaire design can payoff handsomely in terms of the completeness and quality of the collected data.

References

Bartels, L. M. (2002). Question order and declining faith in elections. Public Opinion Quarterly, 66(1), 67€“79.

Belson WA (1981).   The Design and Understanding of Survey Questions. Aldershot, England: Gower.

Bickart, B. A. (1993). Carryover and backfire effects in marketing research. Journal of Marketing Research, 30(2), 52€“62.

Converse, J.M. & Presser, S. (1986).   Survey Questions: Handcrafting the Standardised Questionnaire.  Newbury Park, CA: Sage.

Diamantopoulos, A., Schlegelmilch, B. B., & Reynolds, N. (1994). Pretesting in questionnaire design: The impact of respondent characteristics on error detection. Journal of the Market Research Society, 36(3), 295€“314.

Jenkins, C.R. & Dillman, D.A. (1995).  Towards a Theory of Self-administered Questionnaire Design.   In Survey Measurement and Process Quality, edited by Lyberg, L; P. Biemer; M. Collins; E. D. Leeuw; C Dippo; N Schwarz and D Trewin.  New York: Wiley-Interscience.

Labaw, P.J. (1980). Advanced Questionnaire Design.  Cambridge, MA: Abt Books.

Malhotra, N. K. (2004). Marketing research: An applied orientation (4^th ed.). Upper Saddle River, NJ: Prentice Hall.

Martin, E. & Polivka, A. E. (1995). Diagnostics for redesigning survey questionnaires. Public Opinion Quarterly, 59(4), 547€“567.

Sudman, S. & Bradburn, N.M. (1983).  Asking Questions: A Practical Guide to Questionnaire Design.  San Francisco, CA: Jossey-Bass.