The Hawthorne Effect and validity threats in qualitative research

¶ … Difficulties

Empirical research is necessarily designed to provide a workable framework through which a researcher may test a hypothesized explanation for observable phenomena, but the two primary branches of scientific inquiry differ greatly in terms of the analytical scope and style employed throughout an experiment. While quantitative research is capable of recording, sorting and analyzing voluminous amounts of numerical data, from credit card usage rates for various tax brackets to the pace of population acceleration within a given demographic, this methodology is left lacking when researchers seek to explain the trends and configurations they have identified. In order to develop informed explanations of behavioral patterns, emotional capacity, artistic inclination, and any number of similarly intangible phenomena, the use of qualitative research must be employed to ascertain the motivational processes used to determine basic decision making. Although the traditional quantitative method of research is more widely known by laymen, with surveys, questionnaires and tests becoming ubiquitous in today's modern informational age, qualitative methodologies are most often applied to explain shifts in cultural attitude, collective experiences such as childrearing or aging, and other aspects of human or animal behavior which must be firmly comprehended before they can ever be improved upon.

The "Hawthorne Effect"

Although qualitative research is the preferred mode of study within the social sciences (anthropology, political science, psychology, etc.), researchers who employ this methodology often encounter a series of difficulties which must be overcome if valid conclusions are to be drawn from the data gathered. Although a wide variety of methods can be used within a qualitative study to collect and sort data, including ethnography, action research, and narratology, each is limited by the same set of inherent threats to validity. Among the most infamous of these methodological hurdles is known as the "Hawthorne Effect," which describes a curious but confirmed phenomenon whereby the participants of a qualitative research study alter -- on either the conscious or subconscious level -- the very behavior being subjected to scientific scrutiny. A simple qualitative study conducted by the National Research Council in 1924 became the impetus for this theory, after researchers attempted to determine whether improved ambient lighting in the workplace -- in this case Western Electric's Hawthorne Plant in Cicero, IL -- enhances the productivity of workers therein. Although increases in visibility and ambient lighting did result in the expected rise in worker productivity, the research team stumbled upon an apparent aberration when they elected to return the work environment to its previously dim and dingy atmosphere. When the workers under observation continued to produce demonstrable gains in productivity, despite the lack of light previously thought to be the limiting factor of their output, researchers realized the mere act of observing the subjects was enough to perceptibly influence their behavior, and "from these experiments, emerged the concept of the "Hawthorne effect," which is defined & #8230; as an improvement in the performance of workers resulting from a change in their working conditions, and caused either by their response to innovation or by the feeling that they are being accorded some attention" (Levitt & List, 2011).

Impact of the Researcher's Perspective

When expanded to the entirety of qualitative research, the "Hawthorne Effect" can be more widely defined as "the confounding that occurs if experimenters fail to realize how the consequences of subjects' performance affect what subjects do & #8230; (because) performance is impacted -- possibly unconsciously -- by possible positive or negative personal consequences unconsidered by the experimenter" (Parsons, 1974). Also known as the "observer effect," the phenomenon whereby test subjects inevitably alter their behavior based on awareness that they are being observed has limited the acceptance of theories generated from qualitative research studies, simply because the validity of any conclusions is entirely dependent on the independence and authenticity of the data being collected and analyzed. Although the specter of the "Hawthorne Effect" must be recognized and respected by researchers engaged in qualitative study, the problem actually derives from a broader limiting factor affecting the manner in which qualitative research is performed.

Qualitative Research in Practice

In their jointly authored research paper titled "Adoption of Cloud Computing Technologies in Supply Chains: An Organizational Information Processing Theory Approach," which was published by the International Journal of Logistics Management in 2012, researchers Casey G. Cegielski, L. Allison Jones-Farmer, Yun Wu and Benjamin T. Hazen study the correlation between a firm's distinct information processing structure and its willingness to consider the conversion to cloud computing to facilitate supply chain management systems. The research team begins by defining cloud computing as "a connectivity-facilitated virtualized resource (e.g. software, infrastructure, or platforms) that is dynamically reconfigurable to support various degrees of organizational need, which allows for optimized systems utilization," before stating flatly that "cloud computing technologies may be especially useful for managing the supply chain" (Cegielski, Jones-Farmer, Wu & Hazen, 2012). In order to gain entry to tech firms, which are notoriously secretive regarding internal policymaking processes, the research team submitted written requests to the largest such organizations within their chosen demographic, while informing each firm that their competitors were also contacted. In doing so, the researchers used the inverse of the "Hawthorne Effect," in that knowing that their rivals would possibly be participating greatly increased each firm's likelihood of participating as well. The authors also choose to conduct their analysis of supply chain management through the specified lens of organizational information processing theory, a theoretical framework first proposed by J.R. Galbraith in 1974, and this qualitative approach necessarily informs their subsequent findings. One of the fundamental arguments made by the authors throughout the paper holds that "task uncertainty, environmental uncertainty, and inter-organizational uncertainty effects intention to adopt cloud computing technology and information processing capability may moderate these relationships" (Cegielski et al., 2012), and it this connection between the systemic uncertainty of cloud computing models and their rate of implementation which forms the article's primary thrust.

The authors present a compelling case for the meaning and significance of this qualitative study by demonstrating on multiple occasions that "a review of the published research on cloud computing reveals that most studies either focus on exploring the architectures and applications of the cloud environment or propose lists of opportunities and obstacles for firms considering cloud computing" (Cegielski et al., 2012). When the relative dearth of research focused on the phenomenon of cloud computing within an expanded organizational context from a theoretical perspective is fully considered, it becomes readily apparent that the authors' study is both novel and necessary. With many complex organizations seeking viable methods through which to streamline their supply chain management systems, the authors' envision cloud computing technology as the most suitable tool to address the inherent uncertainty that tends to disrupt continuously operational multi-tiered supply chains. The empirical evidence reviewed throughout the article, which "uses a multiple method approach, thus examining the hypothesized model with both quantitative and qualitative methods" (Cegielski et al., 2012), is extremely relevant within the real-world confluence of industry and information technology, because it demonstrates that artificial barriers have been mistakenly erected between many firms and the increasingly integral resource of cloud computing.

Researching Unfamiliar Demographic Segments

As the preceding review of this study on cloud computing implementation clearly demonstrates, the ability of qualitative research to codify seemingly intangible qualities like organizational commitment to technological innovation is unparalleled. The study also exposes another fundamental flaw within this methodology, in that the unique perspective of the scientist performing the research necessarily influences their own interpretation of objective data, and the conclusions which are subsequently extrapolated. Of all the qualitative methods typically employed by researchers in the social sciences, the practice of ethnography subjects scientists to the often daunting task of ingratiating themselves to a group of individuals with which they have little or nothing in common. Although researchers working across the spectrum of social science fields differ in their definition of ethnography, "the important point about the concept of ethnography, regardless of one's language and terminological preference, is that the practice places researchers in the midst of whatever it is they study," (Berg & Lune, 2011) and it is this emphasis on immersion that differentiates it from other qualitative methodologies. In the early era of empirical inquiry, ethnography typically referred to the anthropologist's custom of travelling to far flung locales and living among the native population for an extended period to observer and participate in unrecorded social customs. An example of contemporary ethnography occurs, however, when a clinical psychologist visits a drug and alcohol rehabilitation center to conduct fieldwork, as the researcher is immersing themselves in a group with norms, values and behaviors which are foreign to the status quo. Indeed, ethnography is the preferred mode of study for research into addictive qualities, and "some early ethnographic authorities, such as Agar (1973), Johnson and colleagues (1985), Preble and Casey (1969, and Weppner (1977) describe ethnography as an extremely effective method for studying illicit drug use and users" (Berg & Lune, 2011). Like all forms of qualitative research, however, ethnography is entirely dependent on the researcher's ability to separate their own biases and perspectives from the interpretations they base on personal observation.