Market Research for a Store Launch

Reliability and Validity in Research

Reliability

Reliability is a term used to talk about the "test-retest" quality of a measurement. Reliability refers the capacity of a measurement to obtain the same results over many repetitions when all the underlying conditions are stable (Chisnall, 2005; Saunders, Lewis and Thornhill, 2009). Many research variables can change over time, and these variables can impact reliability. However, statistical procedures exist to correct for most of those changes and to allow reliable data to be collected.

Reliability of the questionnaires

Of the three options for establishing reliability (equivalent forms, internal consistency, parallel forms, and test-retest), internal consistency offers the most practical application for a survey to be used for concept testing or a product launch (Mitchell, 1996). To implement a strategy for determining internal consistency, a market researcher designs a number of survey questionnaire items that measure the same construct (idea, phenomenon, thing) (Chisnall, 2005). These similar question items will provide insight into the reliability of that particular survey instrument. Internal consistency is reflected by the degree to which the people taking the survey respond in the same way to questions that are designed to measure the same thing (Chisnall, 2005). For that matter, a pilot test of the survey questionnaire can be used to support internal validity in much the same way. The advantage of the pilot test is that it affords an opportunity to correct ambiguous questions or question items that do not demonstrate internal consistency when it was thought that they would do so.

2.0 Validity

Validity is the extent to which a test measures what it claims to measure (Cooper & Schindler, 2008; Brandly, 2010; McDaniel and Gates, 2013; Saunders, Lewis & Thornhill, 2007). It is vital for a test to be valid in order for the results to be accurately applied and interpreted. There are three different types of validity: 1) Content validity, 2) criterion-related validity, and 3) construct validity. Each of these validity types is addressed below in reference to survey questionnaire development and administration.

2.2 Validity of the questionnaires

Content validity demonstrates that questionnaire covers an adequate span of topics and is generally determined by having two or more judges independently rate each item on the questionnaire. Criterion-related validity enables a researcher to be confident that a questionnaire is effective in predicting the indicators or criterion of a construct. Two different types of criterion validity are used: 1) Concurrent validity, and 2) predictive validity.

Concurrent validity reflects the capability of the questionnaire instrument to obtain both the criterion measures and the test scores at the same time (McDaniel & Gates, 2013). In a manner of saying, the current status or situation of the individual taking the test would be evident in the test results: a person responding to a questionnaire about how a brand makes them feel would report how they are currently feeling when looking at the brand logo, or some such (McDaniel & Gates, 2013). Predictive validity must be determined over time since the criterion measures are judged against some change or attainment that occurs generally after a considerable amount of time has passed following the time when responses to the questionnaire items were provided. That is to say, that when asked how likely the survey respondent is to buy a product, predictive validity would mean that responses to this question would also reflect the consumer's behavior later on (Saunders, Lewis & Thornhill, 2009). The construct validity of an instrument is based on the association between the instrument scores or outcome and the prediction of a characteristic (often a theoretically established trait) being measured.