Reliability and validity in survey questionnaire development

Reliability and Validity in Research

Reliability

Reliability is a term used to talk about the "test-retest" quality of a measurement. Reliability refers the capacity of a measurement to obtain the same results over many repetitions when all the underlying conditions are stable (Chisnall, 2005; Saunders, Lewis and Thornhill, 2009). Many research variables can change over time, and these variables can impact reliability. However, statistical procedures exist to correct for most of those changes and to allow reliable data to be collected.

Reliability of the questionnaires

Of the three options for establishing reliability (equivalent forms, internal consistency, parallel forms, and test-retest), internal consistency offers the most practical application for a survey to be used for concept testing or a product launch (Mitchell, 1996). To implement a strategy for determining internal consistency, a market researcher designs a number of survey questionnaire items that measure the same construct (idea, phenomenon, thing) (Chisnall, 2005). These similar question items will provide insight into the reliability of that particular survey instrument. Internal consistency is reflected by the degree to which the people taking the survey respond in the same way to questions that are designed to measure the same thing (Chisnall, 2005). For that matter, a pilot test of the survey questionnaire can be used to support internal validity in much the same way. The advantage of the pilot test is that it affords an opportunity to correct ambiguous questions or question items that do not demonstrate internal consistency when it was thought that they would do so.

2.2 Validity of the questionnaires

Content validity demonstrates that questionnaire covers an adequate span of topics and is generally determined by having two or more judges independently rate each item on the questionnaire. Criterion-related validity enables a researcher to be confident that a questionnaire is effective in predicting the indicators or criterion of a construct. Two different types of criterion validity are used: 1) Concurrent validity, and 2) predictive validity.

Concurrent validity reflects the capability of the questionnaire instrument to obtain both the criterion measures and the test scores at the same time (McDaniel & Gates, 2013). In a manner of saying, the current status or situation of the individual taking the test would be evident in the test results: a person responding to a questionnaire about how a brand makes them feel would report how they are currently feeling when looking at the brand logo, or some such (McDaniel & Gates, 2013). Predictive validity must be determined over time since the criterion measures are judged against some change or attainment that occurs generally after a considerable amount of time has passed following the time when responses to the questionnaire items were provided. That is to say, that when asked how likely the survey respondent is to buy a product, predictive validity would mean that responses to this question would also reflect the consumer's behavior later on (Saunders, Lewis & Thornhill, 2009). The construct validity of an instrument is based on the association between the instrument scores or outcome and the prediction of a characteristic (often a theoretically established trait) being measured.

With a sufficiently large sample size, data analysis with SPSS, and a pilot study conducted beforehand, there is reason for confidence that the overall validity of the questionnaire will be sufficiently high.

Reference

1. Chrisnall, P. (2005) Marketing Research. (7th ed.) Maidenhead: McGraw-Hill Education.

2. Saunders, M., Lewis, P., Thornhill, A. (2007) Research Methods for Business Students. (4th ed.) Harlow: Prentice Hall.

3. Cooper, D.R., Schindler, P.S. (2008) Business Research Methods. (10th ed.) New York: McGraw-Hill.

4. Bradley, N. (2010) marketing research -- tools and techniques. (2nd ed.) New York: Oxford University Press

5. McDaniel, C., Gates, R. (2008) Marketing Research Essentials. (6th ed.) New York, NY: John Wiley & Sons, Inc.

[Type text]