Improving research design validity in reading instruction method studies

¶ … Accept This Conclusion?

This conclusion is spurious because there are too many potential intervening variables. For one, the students are enrolled and being taught at two different schools. There is no mention of their ages, grade levels, background, or any other pertinent data that could affect reading habits or scores on reading tests. Any number of factors could influence their literacy levels, including demographic issues and the reading resources available at their respective schools.

Second, there is no definition of terms or operational definitions that would be critical for clarifying issues related to literacy. Simply noting that the word method and phonics method were being used is not specific enough. The researchers need to indicate what tools and techniques are being used, in what manner, and in which classrooms, in order to classify one group as "word" and one as "phonics." Finally, the participants were not given a pre-assessment of their reading ability, so there is no way of knowing whether the students in the phonics group were already more advanced readers versus their counterparts. A pre-test and a post-test would be necessary for a more reliable and valid study.

Designing a Better Method

The students need to be randomly assigned to each group in order to make the research more robust and reliable. The research would also be better if the students were (a) all assessed before the interventions so that their baseline reading scores could be tabulated; and (b) were subjected to at least a year of instruction using one or the other method. Only by knowing the baseline scores and administering the differential methods would it be possible to know that either words or phonics had any affect on reading ability. It is also critical to clarify which specific type of word-based or phonics-based instruction is being used, and which reading tests are being used.

Materials

The types of reading pre-assessments and post-tests need to be mentioned. These instruments need to be standardized and generally accepted in the educational community particularly for literacy instruction.

Measures

The results of the tests need to be measured and tabulated, and then ran through an SPSS system to determine the statistical differences between the two groups while also being able to control for extraneous variables that might have impacted student performance.

Conclusion

The researchers should base their conclusions on a larger data set with more robust controls and stronger experimental design, rather than jumping to the conclusion that phonics was a superior method.