The philosophical impossibility of natural experiments in educational research

Education Experiment

The Impossibility of "Natural" or "Real" Experiments in the Filed of Education: A Philosophical Examination

Science, it is generally thought, is built on empirical investigation and is ultimately dependent on experimentation -- the control of various elements in the observed event, process or phenomenon. In many of the "hard" or "natural" sciences, this is exactly the formula by which knowledge progresses; initial observations lead to the development of hypothetical explanations for these observations, and experiments are designed that isolate the defined causes and effects of the hypothesis in order to determine whether or not the hypothesis was correct. If the hypothesis is deemed valid, a new scientific theory exists that increases knowledge, and knowledge is also increased by a failure of the hypothesis that leads to new assumptions, new experiments, and so on. This is thought of as scientific progress.

In the "social" or "human" sciences, however, knowledge cannot necessarily progress in this fashion for both practical and ethical reasons. There are many causal factors in human affairs that simply cannot be manipulated by researchers, and others that it would be hugely unethical to attempt to manipulate (Shadish, Cook, & Campbell 2002). This makes direct experimentation in many different social scenarios either impossible or, in the case of potential ethical violations, either highly unreliable or unrepeatable, both of which are antithetical to the scientific method. The problems with social research do not end here, however.

The philosophical and intellectual frameworks in which social research must be conducted, though they are diverse and highly varied, do not themselves truly permit the conducting of real or natural experiments in many different fields. Education is once such field; not only are there practical and ethical limitations on the types and degrees of research that can be conducted in regards to determining educational differences and efficacies, as well as the effects of education or non-education, but the experimental mindset simply cannot adequately engage with or describe the qualitative issues in this field. There is an inherent failure of the process and an unreliability of the results in any experiment proposed to examine education even before the experiment is conducted. This paper will explore the philosophical theories and underpinnings that have led to this conclusion.

Issues of Cause and Effect

First and foremost, there are the inherent issues of cause and effect that are an automatic part of the experimental mindset in regards to natural and "real" experiments. It is the determination of causal relationships that science and especially scientific experiments is primarily concerned with, and this has been the case since the Enlightenment beginning in the seventeenth century (Shadish et al. 2002, pp. 1-3). Put most simply, "the key feature common to all experiments is still to deliberately vary something so as to discover what happens to something else later" (Shadish et al. 2002, p. 3). This framework is replete with specific definitions of both causes and effects (Shadish et al. 2002, pp. 4-6).

The issues of cause and effect do not automatically preclude the possibility or the effectiveness of experimentation in the educational field -- there are certainly effects caused by education, and factors that cause changes in the delivery and potency of education. But these factors are ultimately uncontrollable, and experiment is precisely about control. Natural experiments do not include conscious and purposeful manipulations of the causal features of a situation, but they do require that these causes be finite and definable, and this is simply impossible in the field of education. There exists a complete lack of manipulability in many different factors affecting education, making it impossible in a directly practical sense for observation of any educational scenario in the natural experimentation methodology to be decisive and conclusive. The cause and effect philosophy of traditional scientific inquiry is simply inadequate when it comes to investigating education.

A broader and more profound contextual issue exists that limits experimentation in the educational sphere as well as in other domains of social science in the determination of what is considered "real" in scientific inquiry. The philosopher Charles Sanders Pierce defined the real as, "that which, sooner or later, information and reasoning would finally result in, and which is therefore independent of the vagaries of me and you" (Hacking 1983, p. 58). In qualitative research, such independence is not wholly possible, and it is even less so when there are broad categories of factors that cannot be manipulated in a way the significantly limits subjectivity -- the "vagaries of me and you," as Pierce so succinctly put it.

In order for experimentation to be effective in the educational sphere, according to this definition of the real (which must be accepted as the definition of the real in traditional scientific investigation, or else such investigations and there many conclusions are ultimately meaningless), the complete framework of influences on education and of education's effects must be established and greed upon as an objective reality. Only after this framework has been established would investigation as to the specific interactions of various causes and effects, and the degrees of influence that these various causes have on their identified effects, be possible. Not only are many factors that have been proposed as influential in educational efforts highly debated amongst scholars and researchers, precluding this notion of an agreed-upon framework, but the act of education is itself highly subjective and thus would subvert any attempt to define the "real" -- education depends on individual vagaries that are inherently "unreal" in the framework of experimentation as defined above.

Conclusion

Natural and real experiments depend upon the existence of a rigid framework of previously agreed upon realities and truths, for which further casual relationships can be inferred. Not only does the lack of theoretical agreement in the study of education preclude the use of these experimental methodologies due to the non-existence of these realities and truths but the act of education itself is so highly subjective that it will continue to resist the development of any such truths. Other research philosophies are necessary to lead to any effective understanding of education and the factors that influence it.