Theory based research: foundational approaches and applications

¶ … Testing

A critical element of research is determining whether a set of observations are the product of chance or the result of the action on the independent variable on the dependent variable. This approach to knowledge creation and verification engages deductive reasoning in the decision making process. Thus, the essential feature of this process is making the correct decision with reference to what the data are pointing to. To make this process more reliable and valid researcher engage a type of thinking and analysis involving statistical testing called hypothesis testing.

A hypothesis is a conjectured statement or predicted statement of relationship between two variables or more variables. The researcher uses the hypothesis to represent the relationship he anticipates will explain some aspect of the variance in the dependent variable. For the purposes of hypothesis, testing the researcher will usually have a null hypothesis and an alternate hypothesis. The null hypothesis is the hypothesis that is actually tested. It is either accepted or rejected by the researcher based on the test results (Ryan 2004). To accomplish this successfully requires that some decisions be made as to when the relationship exists or does not.

The result of the hypothesis test is considered significant if it is highly unlikely that it could be the product of chance. This is determined using a specific threshold for the rejection of the null hypothesis based on a specific significance level. The significance level for the rejection of the null hypothesis is determined before the test in undertaken.

There are two errors that can be made in hypothesis testing, and they center on the rejection of the null hypothesis. The researcher can reject the null hypothesis when the null hypothesis is true. This type of error is called a type I error and the probability of making this type of error called the alpha level. It therefore stands to reason that the lower the probability that is set for the alpha level, then there is a smaller chance of making a type I error. This also means that a more extreme test value is required to have the result be considered as significant. The other type of error that can be made in hypothesis testing is called the type II error. This is the reverse of the Type I error. As the researcher seeks to ensure that the result is not the product of chance, you increase the possibility of not rejecting the null when the null is actually false. So that the type II error leads, the researcher to determine that there is no effect when there actually was an effect.

Aron, Coups & Aron (2011) identify a five-step procedure for successful hypothesis testing. The first step involves restating the research question as a "research hypothesis and a null hypothesis about the populations" in general the null hypothesis is the opposite of the research hypothesis and states that there is no change, no difference or no effect. Secondly, the characteristics of the comparison distribution is identified; "the comparison distribution is the distribution that represents the population situation if the null hypothesis is true" (149).

The third step in hypothesis testing involves the determination of the value at which point the null hypothesis will be rejected. The idea being that once the test statistic falls beyond this value and into the region of rejection the researcher will reject the null hypothesis. The fourth step is to perform the relevant statistical test and acquire a test statistic along with its p or probability value (Iyanaga & Kawada, 1980). The fifth and final step involves making a decision to reject or not reject the null hypothesis based on the test statistic. If the p value is less than the pre-determined level of significance, you would reject the null hypothesis (Voelz 2006).

The role of confidence intervals in hypothesis testing is such that if the confidence interval (which is an interval within which the researcher has a specific degree of confidence that the population parameter of concern exists) does not contain the mean of the null hypothesis distribution then the result is significant (Aron, Coups & Aron 2011 p. 191). The confidence interval then provides limits within which one expects to find the mean or statistical value that is related to the null hypothesis.

The student T-test is used to assess whether the means of two groups are in fact statistically different. Many different types of T-test are employed in hypothesis testing. There is the single sample T-test, where a sample mean is compared to a known population mean with the intent of determining whether the sample has come from that population. There is also the T-test for independent and dependent means. These all test a basic hypothesis that the means of the two groups are different.

The inherent weakness associated with engaging in multiple T-test's for many groups, is that the error associated with each round of testing increases the likelihood of getting a false positive. To compensate for that weakness statisticians utilize Analysis of Variance or ANOVA. ANOVA assesses the hypothesis that there is a difference between groups and unlike the T-test requires only on round of testing for three or more groups. When ANOVA is used for hypothesis testing, the researcher is able to determine that there is a difference between the groups but does not know which groups are different.