Using Analysis of Variance to Study Student Achievement

¶ … populations, factors, or variables.

The objective of carrying out this statistical analysis is to learn more about the relations between variables that may influence student performance in the Lincoln County schools. The specific variables of interest in this analysis are the region of the county and the curriculum. One relationship of interest is whether a difference in student performance can be attributed to the region of the county in which the schools are located and the curriculum that is being used at the various schools. There is also interest in exploring if student performance seem to change based on the combination of curriculum used at the school the students attend and the region in which the school is located.

(a) Data Samples

A randomization generator accessible on the Web was used to select the sample. Twenty individual students were selected with each student identified only by the number indicating their order in the complete data set, which includes all schools participating in the study.

(b) Properly Expressed, Mathematical Hypotheses

The hypotheses that are being tested in this statistical procedure are as follows:

Ho = = ?1 = ?2… k (No differences in student performance can be accounted for by the region in which the school the student attends is located.)

Ho = ?1 = ?2… k (No differences in student performance can be accounted for by the curriculum being used at the school the student attends.)

Ha ? ui uj (The curricula used at a school account for some of the differences measured in student performance.)

Ha ? ui uj (The regions in which the schools are located account for some of the differences measured in student performance.)

(c) The ANOVA Tool

The Excel ANOVA tool was accessed through the add-in tool for Mac (StatPlus). The results are shown at the bottom of this paper. The ANOVA test results are generated to populate an Analysis of Variance (ANOVA) table. The test statistic used in an ANOVA is an F test with k-1 and N-k degrees of freedom, where N. is the total number of subjects, and k is the number of groups. When a low p-value is seen in this test, it generally indicates that there is reason to reject the null hypothesis in favor of the alternative hypothesis. That is to say that, there is evidence that at least one pair of means are not equal. A two-way analysis of variance enables the investigation of two factors at the same time in one single experiment. Because of this capacity, a researcher is able to avoid performing two independent one-way ANOVA's and then trying to cobble the data together. A two-way ANOVA provides more information and is more economical and efficient as the effects of two-factors are explored simultaneously.

(d) Results of Statistical Analysis

The first calculation in an ANOVA is for the grand mean, of the mean of all the observations, regardless of which group they are in. The grand mean for this analysis is 1,502. Following, the sum of squares is calculated. The sum of squares is essentially an estimate of the variation among the groups. That is to say that, it is an estimate of the variation among the groups. This is another way of saying that the sum of squares is an estimate of the deviation of group means from the grand mean. The sum of squares (SS) is 223,222 for factor 1 (RE) and 65,040 for factor 2 (CU), 1,830 for factor 1 + 2 (RE x CU), and the within groups SS is 128,839. RE stands for the regional area and CU stands for curriculum.

The test statistic for an ANOVA is called an F-value. The F-value is a ratio of among groups to within groups variation. A large F-value indicates that the variation among groups is large compared to the variation within groups; and this means that some given variable or experimental manipulation accounts for the variation rather than simply chance. Alternately, when the variation among groups and within groups is similar, the F-value is small, which means that the difference among groups is not likely to be due to a given variable or manipulation, but rather is largely accounted for by measurement error or chance natural error. The F-values and critical F-values are as follows: Factor 1 (RE) is 1.2617 @ 5.2407 f crit; factor 2 (CU) is 7.068 @ 6.8879 f crit; and Factor 1 + 2 (RE +CU) is 0.09945 or 0.10 rounded @ 5.2407 f crit.

(d) Summary of Findings and Conclusions

If the F-value is equal to or larger than the critical F-value (f crit, which is found by references a distribution table), then the results at that level of probability are significant. Thus, it can be seen that the factor 2 (curriculum) is significant at the 0.01872 level, (or p = 0.02). Therefore, we can reject the null hypothesis that no differences in student performance can be accounted for by the curriculum being used at the school the student attends, and accept the hypothesis that the curricula used at a school account for some of the differences measured in student performance. Of the main effects, the region where a school is located does not appear to influence student performance to an important degree. Moreover, when taken together, the curriculum and the region in which the school is located do not appear to account for the variation in student performance.

B. Test hypotheses about two separate populations, factors, or variables by applying ANOVA using a sample of the data set.

Two-way ANOVA

Summary

Response

Factor #1

RE

Fixed

Factor #2

CU

Fixed

Descriptive Statistics

Factor

Group

Sample size

Mean

Variance

Standard Deviation

RE x CU

1 x 5

6

1,502.33333

7,442.66667

86.27089

RE x CU

1 x 6

3

1,449.

21,316.

RE x CU

2 x 5

5

1,579.6

10,966.8

RE x CU

2 x 6

1

1,437.

#N/A

#N/A

RE x CU

3 x 5

2

1,592.

28.28427

RE x CU

3 x 6

3

1,400.

2,163.

46.50806

RE

1

9

1,484.55556

10,691.77778

RE

2

6

1,555.83333

12,162.56667

RE

3

5

1,476.8

12,340.7

CU

5

13

1,545.84615

8,599.80769

92.73515

CU

6

7

1,426.28571

8,448.90476

91.91792

ANOVA

Source of Variation

SS

d.f.

MS

F

p-level

F crit

Omega Sqr.

Factor #1 (RE)

23,222.14444

2

11,611.07222

1.2617

0.31347

5.24075

0.02111

Factor #2 (CU)

65,040.87912

1

65,040.87912

7.06755

0.01872

6.88797

0.24476

Factor #1 + #2 (RE x CU)

1,830.4431

2

0.09945

0.90597

5.24075

0.E+0

Within Groups

128,838.53333

14

9,202.75238

Total

218,932.

19

11,522.73684

Omega squared for combined effect

0.19322

Comparisons among groups of Factor 1 (RE) within each Factor 2 (CU) level

Factor 1 (RE) group '1' comparisons

Scheffe contrasts among pairs of means

Group vs. Group (Contrast)

Difference

98% Confidence Interval

Test Statistics

p-level

5 vs. 6

53.33333

-119.80337

0.61817

0.44196

Bonferroni Test for Differences Between Means

Alpha/N

0.02

Group vs. Group (Contrast)

Difference

98% Confidence Interval

Test Statistics

p-level

5 vs. 6

53.33333

-119.80337

0.78624

0.44196

Fisher LSD

Group vs. Group (Contrast)

Difference

Test Statistics