Quantification and measurement in research methodology

¶ … Value of Open-Ended Questions in Quantitative Research

Study example: To what degree do American consumers try to eat a healthy diet?

The value of open-ended questions, even in a study presented and assessed in a quantitative manner, is that they allow greater flexibility of responses than narrow questions scaled, for example, on a 1-10 basis, or with a predetermined vocabulary of responses. The majority of respondents may even defy the expectations and preliminary research of the constructors of the study and give additional valuable information in terms of what they say about a particular issue. Take the complex question of 'healthy eating' and the degree to which people have been changing or not changing their eating habits in the face of increasingly negative media coverage of the fast food industry and the American obesity epidemic. Eliciting personalized responses that are then quantified can yield more unexpected, spontaneous, and therefore more useful results than a narrow 'check the box' type of study, with prefabricated and predetermined answers.

The use of such an open-ended study might be particularly valuable when grappling with such a question as "to what degree are American consumers trying to eat a more healthy diet," given that individuals notoriously underreport their daily calorie totals in general, and underreporting of so-called 'bad' foods is particularly epidemic in close-questioned studies in which respondents can 'fudge' (no pun intended) there answers (Lafay, et al. 2000). While a question like 'how often do you eat fast food' might seem to be an acceptable quantitatively- assessed question, an open-ended question might be better, for example, asking: "What restaurants do you patronize? How often?" Someone might not consider getting, for example, a burger at Ruby Tuesday's or even Wendy's to be 'fast food,' even though he or she consumes a highly caloric burger at such a chain sit-down restaurant. Different types of convenience food establishments could also be assessed, as subjects would volunteer their responses, rather than simply look to a list, and the relative difference between eating at a sit-down burger chain vs. A drive-through might yield interesting comparative results. Likewise, a correlation between eating at different fast food chain restaurants and different independently-run establishments and respondent's BMIs could be examined.

Asking for a list of restaurants also may provoke a more honest answer than asking about frequency of dining at fast food establishments alone. Simply agreeing that 'I eat at fast food restaurants X times a week' may encourage underreporting but thinking and listing the restaurants actually requires more thinking. The open-ended response of 'what restaurants do you frequent and how often' also teases out the respondent's underlying beliefs about what constitutes 'fast food dining' unlike a directed question like 'how many times a week do you eat fast food,' which might be answered with "never," in the sense that he or she does not eat at McDonald's or a restaurant with a drive-through. Then responses could be grouped into fast food, sit-down, independent, and chain restaurants, according to the needs of the study's designers.

The next question could be: "What is your typical order at a restaurant?" Asking the typical menu items consumed at the restaurants he or she regularly patronizes also provides clues as to how different foods are construed by the respondent and their perception of the health of their food choices. A respondent might feel that he or she is a healthy eater, for example, because he or she does not go to get a 250 calorie cheese burger at McDonald's and goes to Subway, a restaurant that advertises itself as a kind of diet fast food establishment. However, asking specific, open-ended questions about what the individual actually eats at Subway yields more interesting data than simply asking him or her if health is a question when ordering -- perhaps he gets a highly caloric meatball sub, and feels comforted that Subway is a 'diet restaurant.' In contrast, an individual with small children who frequents fast food restaurants might be marked as an unhealthy eater in a closed-question quantitative study if he or she is merely asked 'do you go to fast food restaurants that sell burgers more than three times a week?' That individual might order a salad at McDonald's while her children get happy meals, while someone who goes to Subway and orders the unhealthy food options will not have his or her food choices adequately recorded. A more accurate portrait of different ways of patronizing restaurants and what was consumed on the premises would be to ask for a list of restaurants and typical order, as well as the type of restaurant so the nutritional breakdown of the order to be 'done' on the part of the researcher, rather than on the part of the more subjective, and perhaps inaccurate part of the respondent. Again, specificity through open-ended questions, and only then quantifying the data, for example, estimating the calorie count or nutritional breakdown of the menu items after they are reported, yields a more telling result of the profile of the American eater by allowing the respondent to reflect his or her unconscious biases and to be more honest about his or her habits.