Internal Validity of Any Research Design Refers

Internal validity of any research design refers to the design's ability to make causal inferences from the data collected and the results of the study (Shadish, Cook, & Campbell, 2002). The research designs that offer the highest levels of internal validity are true experiments where the participants are randomly assigned to the conditions in the study. Quasi-experimental designs attempt to improve on the poor internal validity of correlational designs by allowing the researcher to control the assignment of participants to the treatment condition such as using some type of eligibility criteria, but due to the nature of the study the researcher cannot use random assignment. Most quasi-experimental designs are performed with intact groups that cannot be the subject of any random assignment (Shadish et al., 2002).

Metheny, Davis-Jackson, and Stewart (2010) performed a two-group quasi-experimental design to determine the effectiveness of an Aspiration Risk Reduction Protocol (ARRP) in critically ill, mechanically ventilated tube -- fed patients on an intensive care unit. Aside from the aforementioned problems with internal validity in quasi-experimental designs the Metheny et al. study has several threats to its internal validity:

1. Probably the most glaring threat to the internal validity of this study is that the researchers do not appear to have been blind to the conditions of the participants. The data was collected prospectively and data collection for the study was a permitted only on participants gave informed consent. It appears that the researchers knew which participants belong to which group. This could be a major threat to internal validity and to any generalizations of the study. A more useful way to collect data would be to not have the participant groups identified and to run the analysis without knowing which group was the treatment or control group.

2. Although there may be no way around this in some studies only getting data from a particular subset of the sample can threaten the internal validity of the study. In the current study, only those participants who gave informed consent had data collected on them. This implies that larger groups of patients were exposed to both conditions. It might be interesting to look at any differences in outcomes for patients not in the study (although one could not publish this) in order to determine if there were differences in those that gave informed consent and those participants that did not. This is often a problem in sampling methods as well and can affect both internal and external validity. For example, studies that apply survey data with poor return rates cannot guarantee that there is no difference between people to complete the survey and people that did not. Likewise, here we do not know if there is a difference between the participants that completed the informed consent forms and those that did not. Any differences could threaten the internal and external validity of the study.