Statistical significance of mental health and functioning measures between independent groups

¶ … groups in this study are independent. Despite the fact that they were not randomly assigned to groups but were assigned based on physical sex/gender is immaterial; inclusion or exclusion in one group did not influence inclusion or exclusion in the other group in any way. As there was no matching/pairing and no influence on the inclusion/exclusion or group assignment of any participant based on the status of another participant, the groups are independent.

is the t-statistic for the Mental Health (MH) questionnaire completed by the participants. This corresponds with a p-value of 0.002. According to the Bonferroni principle, significance in this study would be indicated by a p-value of 0.0033 or lower given the alpha level of 0.05 and the fact that 15 individual measures were taken/t-tests performed (0.05/15=0.0033). As 0.002 < 0.0033, this result is significant -- the groups can be said to have a significant difference in Mental Health.

3)

A t-value of -1.99, which was the calculated statistic for the Health Functioning measure in this study and corresponds to a study-bound p-value of 0.049, does not indicate a significant difference in these populations. Though the p-value is slightly lower than the study-wide alpha value of 0.05, the division of this value amongst the 15 measures according to the Bonerroni principle that works to reduce Type I errors means each test-specific alpha value is 0.0033. 0.049 > 0.0033, therefore the difference for this measure is not specific.

4)

The Mental Health (MH) measure showed the largest difference between the two groups at -3.15 (absolute value = 3.15), and with a p value of 0.002 this finding is significant. Not other t-statistic has a larger absolute value nor is any other finding deemed statistically significant.

5)

Of the two t-values -2.50 and -2.54, -2.54 would have the smaller p value. |-2.54| > |-2.50|, therefore the difference indicated by a t-statistic of -2.54 is greater than the difference indicated by a t-statistic of -2.50. The greater the difference, the smaller the p value, and the higher the probability that the observed difference is significant (roughly speaking).

6)

A Type I error is an error in which the null hypothesis -- in t-test scenarios, that there is no significant difference between the two tested groups on the tested metric -- is falsely rejected for the alternative hypothesis -- in t-test scenarios, that there is a significant difference. This study certainly has the potential for Type I errors; any t-test runs some risk of finding a difference to be significant when it isn't, and multiple t-tests increase that risk.

7)

The Bonferroni procedure should absolutely be conducted in this study to minimize the exposure to Type I error risk. The many t-tests conducted on the different measures increase this risk 15-fold over a single t-test, thus lowering the alpha level for each individual test is necessary to control for this increased risk.