This paper examines the distinction between statistical significance and effect size in research methodology, then applies both concepts to two forensic psychology studies. Statistical significance indicates that observed results are unlikely due to chance, but it does not convey practical importance. Effect size addresses this limitation by quantifying the magnitude of a finding on a standardized scale. The paper illustrates these concepts through Douglas, Ogloff, and Hart's (2003) evaluation of the HCR-20 violence risk assessment scheme and Lalumière and Quinsey's (1994) meta-analytic study using phallometric assessments to discriminate rapists from non-sex offenders. Together, these examples demonstrate why researchers must consider both measures when interpreting and applying study results.
Statistical significance refers to a property of statistical data used to establish that changes observed in a participant's behavior, reactions, attitudes, or any other surveyed aspect are due to the effects of the study rather than to chance. A critical probability value (alpha value) is determined in advance — for example, 0.05. If the p-value obtained from the test is less than the alpha value, one can conclude that the observed changes did not occur randomly and are therefore statistically significant.
Statistical significance, however, does not indicate that results are useful, meaningful, or helpful in decision-making. All one can conclude is that a difference was found that did not occur by chance. A clear illustration is evaluating the effect of a particular activity on student knowledge using pre- and post-tests. Suppose the mean score on the pre-test was 48 out of 100, and the mean score on the post-test was 45 out of 100. The difference in scores might reach statistical significance — especially when a large sample is used — yet the difference between the two scores is so small that the effect is practically unimportant.
In order to determine whether a study has any practically important effects, effect size is calculated alongside statistical significance. All effect sizes are computed on a common, standardized scale, which makes it easier to compare different programs or interventions according to the same outcome measure.
Douglas, Ogloff, and Hart (2003) tested the interrater reliability and validity of violence risk judgments made using the HCR-20 violence risk assessment scheme. The HCR-20 assesses 20 key risk factors across three domains: historical, clinical, and risk management. The sample consisted of 100 forensic psychiatric patients who had been released after being found not criminally responsible due to a mental disorder. Community violence following release was operationalized from case file records.
Interrater reliability for low, moderate, and high violence risk categories showed acceptable levels of agreement. The final risk judgments were not only significantly predictive of community violence after release, but effect sizes also showed moderate to large results. This indicates that the study was not merely statistically significant but also weighty — its results were worthy of practical consideration. The researchers therefore suggested that the HCR-20 could contribute meaningfully to violence risk assessment, provided that certain conditions were in place.
"Meta-analysis discriminating rapists via phallometry"
Together, these two forensic studies illustrate why statistical significance alone is an insufficient measure of a study's real-world value. Effect size provides the additional context needed to judge whether findings are worth acting upon. In the HCR-20 study, both significance and effect size converged, lending strong support to the instrument's practical utility. In the phallometric study, statistical significance was widespread across variables, but effect size analysis revealed that only one variable — stimulus set — carried meaningful discriminative power. Researchers and practitioners in forensic psychology therefore benefit from routinely reporting and interpreting both measures.
You’re 69% through this paper. Sign up to read the remaining 1 section.
Sign Up Now — Instant Access Already a member? Log inAlways verify citation format against your institution’s current style guide requirements.