Evidence, Inference, and Policy Decision

Evidence, Inference, And Policy Decision

Criminology is a fascinating field of study. Every aspect of it is subject to scholarly inquiry. Research scientists investigate criminal behavior and conduct evaluations of programs designed to curb different kinds of criminal behavior. Much of this research is relevant to the duties and responsibilities of this organization. We can use the findings of such research and program evaluations to help us create policy, but we must make sure that we are focusing our information gathering on the best research. Not all research is conducted equally. Some research methods are sounder than others, and some findings and evaluations are more usefully reported than others. When selecting research and program evaluations on which to base our policy decisions, we need to apply standards to ensure we reference only those that use sound research methodologies and detailed report writing. David P. Farrington offers some examples of straightforward standards for measuring the quality of research studies and program evaluations (Farrington, 2003).

Edward Tufte quotes social scientist David T. Campbell who wrote in 1969 that developed countries "should be ready for an experimental approach to social reform… in which we try out new programs designed to cure specific social problems, in which we learn whether or not these programs are effective, and in which we retain, imitate, modify or discard them on the basis of apparent effectiveness." Campbell went on to observe that "most ameliorative programs end up with no interpretable evaluation" (Tufte, 1974, p. 6). By analyzing the research and program evaluations in criminology, we can learn to avoid the mistakes of the failures and build on the gains of the successful. To select those most successful, we need to employ standards to discern the useful from the unhelpful.

Farrington's standards of quality are clear and hierarchical in nature. Most important is that the cause or intervention produces the expected effect or outcome, which is a direct measure of the success of the intervention. Next is that the research report provides all the details necessary to effectively assess the quality of the research method, including such revealing information as the sample size and composition, what control group or groups were used, the reliability of measurements of the intervention impact, and so on. Third is that relationship between the intervention and the outcome is real and the impact of the intervention is statistically significant. Fourth is that target behavior of the research is clearly defined and that this behavior is what was measured. Last is the ability to generalize the results across situations beyond those presented in the study. A quality rating system, either simple or weighted, can be devised to score studies according to these standards.