Bayes Confirmation Theory and Scientific Theory Testing

Introduction to Bayes' Theorem and Scientific Theory

Theorizing in science is a complex and time-consuming undertaking. The theorist uses collected evidence from some means of scientific inquiry to project a generalized case. However, there is a difficulty with this process: there is some probability that the theory will be wrong. Even if this is not a harmful outcome, it is difficult for the theorist to overcome in their professional lives. Researchers therefore want to understand the probabilities involved in the success of their theories.

Bayes' theorem addresses the probability that an event will occur — in the context of this paper, whether a theory is correct or not. Bayes considered two different events, one of which can be used to increase the probability that the other is correct. For example, suppose that a statement has a fifty percent chance of being true; that is not enough to build a theory on. To increase the probability that the statement is true, something else is added — a known quantity that relates to the previous statement — that raises the probability that the first statement was true.

As an illustration, one person (A) tells another (B) that they have just met with a friend and had lunch together. B hypothesizes that the friend was a woman, but there is only a fifty percent chance that this is so. More information is needed to confirm this supposition. In the course of the conversation, person A mentions that the friend left to have a spa day. The probability that the friend was a woman increases because B knows that seventy-five percent of people who go to spas are women. Using Bayes' theorem, the second piece of evidence further confirms the first supposition. This does not mean that B is now completely convinced that the person was a woman — twenty-five percent of spa-goers are men — but there is now approximately a sixty-seven percent chance that the individual was female.

This logic can also be applied to the probability that a hypothesis is correct, because the same pattern appears in both cases. Every theory begins with a hypothesis that has some probability of being correct and some probability of being false. A theorist who is not satisfied with the uncertainty implied by this initial probability seeks to ensure the correctness of their findings by using another piece of evidence to confirm or disprove the assumption. That additional evidence has this power because it can increase the probability that the hypothesis will be supported, or indicate that it should be abandoned. This paper examines Bayes' theorem in detail with regard to its ability to help predict the correctness of a proposed theory.

Examples of Bayes' theorem are easy to find in the literature because it is one of the primary methods used to determine the probability of a statement being true or false. Simple probabilities — such as whether a coin will turn up heads or tails — are not affected by Bayes' theorem because no data can be added to better predict the next outcome (assuming the coin is fair). Since there are only two equally likely possibilities, it is impossible to assign one a higher probability than the other. Both have a value of one-half in the Bayesian context. Most real-world events, however, are not so simple.

A useful example is determining the probability that a test subject belongs to a particular category within a set — such as whether a participant is male or female. This is a good illustration because, as Meehl notes, "physical theories typically predict numerical values."¹ Assuming that half of the population is male and the other half female, the prior probability that a randomly chosen subject is male is 0.50. However, newly obtained data reveals that the participant is also a cigar smoker. Given that 9% of male participants admittedly smoke cigars while only 2% of female participants do, it becomes possible to determine that the probability the chosen participant is male rises to approximately 82%.² This shows how Bayes' theorem works in practice, but it does not yet answer whether the theorem can adequately confirm a scientific theory.

Example of Bayes' Theorem in Practice

Several challenges to the fundamental utility of Bayes' theorem have been identified in the philosophical literature. These include the values of the priors, the problem of the zero prior required for the equation, the problem of old evidence, and the observation that scientists do not actually attempt to confirm theories in a strictly probabilistic way. Bayes' theorem is considered too simplistic in certain respects because it requires specific foundational numbers to function, and those numbers are not always available when a theorist is developing a hypothesis.³

The requirement for prior evidence both helps and hurts the predictive utility of the theorem. The problem of the priors is particularly significant because it can undermine the theory entirely: there is often no logical explanation for where the prior probability value originated. As for zero priors, most probabilities have an initial value associated with them, but there is a possibility that the value of H or H/E in the theorem could be zero. If this is the case, no updated probability can be obtained. It is also difficult to know to which quantities a zero should be assigned.

The issue of old evidence appears to be the most puzzling challenge for scientists. As Rosenberg explains, "scientists who construct hypotheses by intentional 'curve fitting' are rightly criticized and their hypotheses are often denied explanatory power on the grounds that they are ad hoc."⁴ This means that the effectiveness of a theory is called into question when old — or ad hoc — evidence is used as its proof. However, Rosenberg also points out that scientists encounter a further problem when attempting to "distinguish cases like the confirmation of Newton's and Einstein's theories by old evidence from cases in which old evidence does not confirm a hypothesis because it was accommodated to the old evidence."⁵ It is therefore difficult for critics to distinguish between a theory that is genuinely confirmed by existing evidence and one that was constructed around that evidence — a distinction that is crucial before dismissing a theory outright.

Challenges to Bayes' Confirmation Theory

The final challenge is that probabilistic theory confirmation sits uneasily with the method generally used to prove or disprove a hypothesis. In empirical theory-building, the researcher tries to gather as much evidence as possible regarding the question at hand, rather than seeking confirmatory evidence that strengthens the theory from the outside. Confirmatory evidence, in the empirical tradition, arises from the veracity of the initial hypothesis as tested against a large body of observational data.

Neither the positive nor the negative aspects of Bayes' theorem yield a definitive conclusion about whether it should be used to confirm scientific theories. The only thing that can be conclusively established from the existence of pros and cons is that some researchers find it efficacious and others do not. In scientific inquiry, if a tool is unreliable in some cases, there is an argument for setting it aside altogether, since this reduces the risk of error in its application. Nevertheless, there are clear instances in which the theorem has proven useful, owing to the simplicity of its principles. In particular, Bayes' theorem helps theorists avoid the threat of underdetermination — a significant problem for many hypotheses in fields such as theoretical physics.⁶ The fact that Bayes' theorem yields a concrete probability value, in many if not all cases, is one factor in its favor.

In an article for the journal Philosophy of Science, Meehl examined when and for what types of study Bayes' theorem is best suited.⁷ He was investigating the paradox that arises from applying Bayes' theorem to phenomena in the field of psychology. He argued that:

"In the physical sciences [physics, and others], the usual result of an improvement in experimental design, instrumentation, or numerical mass of data, is to increase the difficulty of the 'observational hurdle' which the physical theory of interest must successfully surmount; whereas, in psychology and some of the allied behavioral sciences, the usual effect of such improvement in experimental precision is to provide an easier hurdle for the theory to surmount."⁸

This distinction comes down to the exactness of the science and the ability of researchers to generate actual numerical values. In physics, it is possible to produce a definite numerical value as long as the data set is large enough, and mathematics provides the means to determine an approximate number even when large amounts of data are unavailable. If a hypothetical data set were small, every corresponding piece of data that confirmed the original finding would strengthen the case. People, however, are not exact, and a precise numerical value is rarely achievable when dealing with sentient subjects. When a hypothesis tested on human subjects is confirmed by research, it is in fact weakened by additional data because the tests themselves are inherently suspect.

Conclusion

Rosenberg, A. Philosophy of Science: A Contemporary Introduction, 2nd edn, Routledge, New York, 2005.

Steel, D. Bayesian Confirmation Theory and the Likelihood Principle, 2007. Retrieved 25 March 2012 from https://www.msu.edu/user/steel/Bayes_and_LP.pdf

Triola, M. F. Bayes Theorem, 1997. Retrieved 24 March 2012 from http://faculty.washington.edu/tamre/BayesTheorem.pdf