Observational and experimental approaches in Sandman's drought management study

Observational vs. Experimental Studies: A Comparison of Two Dissertations

Observational Study
Unlike experimental studies, which involve the manipulation of variables to determine causation or at the very least, correlation, an observational study is one in which the researcher does simply observe a population or phenomenon without interfering or assigning conditions. In “City of Fresno Business Leaders’ Management Style versus China’s Business Leaders’ Management Style for Handling California’s Drought and its Impact on the Agricultural Industry,” Sandman (2017) conducts interviews with business owners in California and in China, with the purpose of exploring—observing—the differences in approach to managing drought conditions. The author explores several different variables including economic policy and management styles and organizational cultures, through the in-depth interviews. Results of the study are interpretive and analytical based on the perspectives shared by the stakeholders in both countries.
However, the purpose of the study is more to help the Fresno agricultural industry to implement best practices for drought and water management. Extending from the purpose of the study, the primary research question is whether a continuation of current business practices in Fresno would lead to improved land management and agricultural practices in drought conditions versus implementing the Chinese approaches to the same. Because the researcher does not apply an experimental condition or examine the effect of a variable on specific outcomes, this is clearly an observational study that uses qualitative methods.
The qualitative research design used combines case study with interviews. If the researcher had been interested or able to use quantitative methods and experimental designs, it might have been possible to measure the agricultural outputs after a drought prevention policy or program, or to measure the water table after a similar intervention. As it stands to be an observational study, there is no independent or dependent variable, and nor is there a directional research hypothesis. Instead, the author aims to better understand the full gamut of the situation, offering in depth insight and analysis of the factors involved in agricultural management and environmental policy. As Sandman (2017) points out, using a case study approach is especially helpful in business research as it highlights the contextual variables that impact business practices and outcomes.
Yet researchers also need to be aware of the potential drawbacks with using qualitative methods in business research and throughout the social sciences. Some of the pitfalls in qualitative research like observational studies are similar to those in quantitative studies such as selection bias (Ellenberg, 1994) and validity concerns (Gile, 1998). For example, Sandman (2017) purposefully selects both the case of Fresno and the observational analysis of Chinese business practices without necessarily distinguishing between these two populations or detailing why each was chosen other than personal interest. In observational studies that use a case study design, though, the selection biases are often more apparent than they would be with a quantitative design, which relies on the persuasive power of numbers and statistical analysis to convey credibility, reliability, and validity (Ellenberg, 1994). Another drawback with observational studies like this one is that causality cannot be determined, but even in quantitative research studies that purport to establish causality through their experimental design, it is possible to mistakenly assume a causal relationship between the independent and dependent variables (Imai, Keele, Tingley & Yamamoto, 2011). While quantitative studies do offer the bonus of numerical data that can be used to influence public policy in a definitive way, and also enable the replication of the research under different conditions, a qualitative study offers the hows and whys of business practice that are fundamental to organizational practices. Therefore, choosing a qualitative, observational research design is appropriate for studying the research question of what business practices would work best to manage the drought in Fresno. The methods used included the solicitation of agricultural industry leaders, with a total of forty participants being selected for the current observational study.
An observational study is not without numerical data, though. In “City of Fresno Business Leaders’ Management Style versus China’s Business Leaders’ Management Style for Handling California’s Drought and its Impact on the Agricultural Industry,” Sandman (2017) presents an abundance of numerical and statistical data related to the business environment and agricultural outputs in Fresno, including mention of the number of farms in operation, the market statistics applicable to the agricultural industry in Fresno, total acreage and farm sizes, credit ratings, the quantitative, operationalized definition of drought, the features of the economic cycle, gross domestic product, and market forecasting. This data is used to inform best practices based on the subjective observations and assessments of business leaders in California and in China. Water delivery methods, political climate, and a host of other variables are also taken into consideration in this observational study, to illuminate the complexity of the issue and the inability of any researcher to distill a drought into a set of statistics. Also, Sandman (2017) does quantify some of the research, by tabulating responses to specific questions used in the survey instrument. For example, almost three-quarters (73%) of the participants disagreed with water distillation plants as a solution but almost half (45%) agreed that water recycling would offer a possible solution to the threat of drought.
Key findings of the Sandman (2017) research included an observation of differences in managerial styles between China and California, with corresponding contextual constraints like economic recessions, water shortages, and drought. While no definitive conclusions can be drawn, the observational design does allow for the future formation of hypotheses that are related to specific policy interventions that may help boost economic output and promote sustainable agriculture in Fresno and elsewhere.
Experimental Study
Experimental research designs are defined by their use of random assignment to an experimental and control group, and the application of some treatment or intervention on the experimental group. Because of the methods used in experimental research, the researcher can assess either correlation or causation between the independent and the dependent variables. In business research like “An Experimental Investigation of Motivational Crowding,” experimental methods and binary choice regression modelling show whether some factor causes a specific outcome. Specifically, Roubecas (2015) uses experimental research methods to examine the correlation between equity-based compensation models and malfeasant behavior. Moreover, the experimental methods allow the researcher to examine whether crowding-out motivation or rent seeking cause the correlative relationship. The research questions are grounded in concrete theoretical orientations related to motivation theory and agency theory. Roubecas (2015) also clarifies the purpose and implications of the research on business practices, noting that the experimental design reveals the predictive relationship between compensation strategies and leader motivation. Because scandalous, malfeasant behaviors and white collar crimes are costly as well as unethical, the implications of the research on business are tremendous and meaningful. Agency theory and motivation theories can account for individual behaviors within the context of organizational cultures and structures. Prior research has failed to explain adequately the contradictory relationship between compensation and motivation to act in benevolent ways towards the organization. Roubecas (2015) also clarifies that the contradiction has been defined in the literature as motivation crowding.
The current experimental research is designed to explore whether motivational crowding is what accounts for fraud and other malfeasant behaviors in spite of extrinsic motivation factors like compensation. Roubecas (2015) expresses the research question in terms of whether internalizing motivation would impact agency malfeasance. The methods include the use of a survey that measures compensation timing on completion rates. Furthermore, the research determines whether offering a reward after a task has been completed (as a form of extrinsic motivation) leads to malfeasant behavior due to motivational crowding. Fifty participants were solicited, randomly assigned to the experimental and control groups. The method of sampling used was a nonprobability cluster sample. Roubecas (2015) admits that one of the weaknesses of the sample selection methods is that it is a convenience sample, which can lead to problems like selection bias (Ellenberg, 1994). However, the researcher also shows how the sample is representative, allowing the results to be generalizable.
The experimental group received immediate compensation for their completing the survey, whereas the control group would only receive a financial reward after completing the survey. The instrument used was a standardized personality test, but the results of the actual test were not instrumental to the results of the experiment because the experiment was more about the completion rates and whether offers of compensation impacted completion rates. Therefore, the independent variable was the structure of the compensation strategy, and the dependent variable was the completion of the task.
The methods used for data collection and analysis are appropriate for the experimental research design in the Roubecas (2015) study. A binary choice model (yes or no) was used because there were only two groups with two potential outcomes. Furthermore, the binary choice model is appropriate because as the author points out, one of the variables is a “dummy,” in that completing the survey was more important than the respondent’s answers (p. 65). The research only relies on posttest data, too, which is another reason why the binary choice model is appropriate for data analysis. Roubecas (2015) also ensured double blind random assignment to the treatment versus no-treatment conditions. Random assignment and double blindness are crucial components of an effective experimental research design (Imai, Keele, Tingley, et al., 2011). Also, the current study conforms to the fundamental tenets of experimental research in that it distinguishes between causation and correlation, showing how the treatment variable influences a specific outcome. In this case, the use of a different type of compensation strategy (money offered beforehand as good faith incentive versus money offered afterwards as a reward) impacted behaviors.
The main variables like motivational crowding was measured via assessing differences in survey completion rates, which was designed to mimic real world business scenarios in which individuals would be asked to complete some menial task. Using a straightforward binary approach does not encapsulate all the complexities of real-world variables such as organizational culture, climate, and social norms. Yet the binary approach does offer future researchers some guidance as to whether or not agency theory and motivation theory like motivation crowding are more important than previously thought. An experimental design was essential in revealing the causative factors in malfeasant behaviors. The implications of the results show that agency theory alone does not adequately account for the relationship between compensation and malfeasance. Opponents of agency theory may also suggest that increased pay scales including offering equity actually increases the tendency for individuals to commit fraud given their ability to get away with it or to actually capitalize on their behaviors. Motivational crowding was shown to be a factor in determining behavioral outcomes, using a simple yet effective experimental research design in this study.

References Ellenberg, J. H. (1994). Selection bias in observational and experimental studies. Statistics in medicine, 13(5?7), 557-567. Gile, D. (1998). Observational studies and experimental studies in the investigation of conference interpreting. Target Journal of Translation Studies 10(1): 69-93. Imai, K., Keele, L., Tingley, D., & Yamamoto, T. (2011). Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies. American Political Science Review, 105(4), 765-789. Roubecas, C. (2015). An experimental investigation of motivational crowding. Dissertation. UMI Number: 3684971. Sandman, B.R. (2017). City of Fresno’s business leaders’ management style versus China’s business leaders’ management style for handling California’s drought and its impact on the agricultural industry. Dissertation. ProQuest Number:10257565