Assessment of the Validity of a Research Design

¶ … Accept This Conclusion?

This conclusion is spurious because there are too many potential intervening variables. For one, the students are enrolled and being taught at two different schools. There is no mention of their ages, grade levels, background, or any other pertinent data that could affect reading habits or scores on reading tests. Any number of factors could influence their literacy levels, including demographic issues and the reading resources available at their respective schools.

Second, there is no definition of terms or operational definitions that would be critical for clarifying issues related to literacy. Simply noting that the word method and phonics method were being used is not specific enough. The researchers need to indicate what tools and techniques are being used, in what manner, and in which classrooms, in order to classify one group as "word" and one as "phonics." Finally, the participants were not given a pre-assessment of their reading ability, so there is no way of knowing whether the students in the phonics group were already more advanced readers versus their counterparts. A pre-test and a post-test would be necessary for a more reliable and valid study.

Designing a Better Method

The students need to be randomly assigned to each group in order to make the research more robust and reliable. The research would also be better if the students were (a) all assessed before the interventions so that their baseline reading scores could be tabulated; and (b) were subjected to at least a year of instruction using one or the other method. Only by knowing the baseline scores and administering the differential methods would it be possible to know that either words or phonics had any affect on reading ability. It is also critical to clarify which specific type of word-based or phonics-based instruction is being used, and which reading tests are being used.

Participants and Procedure

Ideally, schools would be randomly selected from schools in different districts around the country. Only students from the same grade levels should be compared with one another. It would be helpful if the researchers could design a study that tested the difference of words versus phonics on different age/grade groups. Informed consent forms need to be offered to the teachers and to the parents. Pre-testing would assess a baseline reading level that would qualify students for participation. Moreover, the classrooms should be evaluated to ensure standards of reading exercises in the words and phonics settings.

Materials