5. Is there anything in the procedures for collecting the information or in the instruments themselves that could bias the results or weaken the study?
The author does not describe the source of his schools merely stating the inclusive and exclusive criteria that they satisfied. The schools, all in Milwaukee, had to satisfy three main criteria: firstly that the program under study was introduced during a period when rating were available, secondly, that the number of schools introducing the program must be sufficient for statistical results, and thirdly, that there should be sufficient and adequate comparison groups. His research seems immune to bias.
The author does, however, mention the possibility of bias in the appearance of marks and the schools that were chosen and concludes that the most likely bias that he could see occurring in the producing of the data was selection bias, namely that the schools may have been specially selected due to particularly low of high test scores that would have either regressed to the mean even without intervention or would have been too challenging to maintain.
6. Is the magnitude of the correlation or difference among the groups large enough to suggest practical significance or importance?
Although questions exist -- and the author discusses these at length -- the Target Teach program does show statistical significance. Correlations amongst the groups showed that amongst the schools using 'Target Teach' curriculum, and schools employing Direct Instruction average rating...
However, schools implementing the SAGE class- size reduction program did not. The author took this to indicate a direct correlation between Direct instruction and student achievement and recommends that the Direct Instruction intervention programs be used as control group in future studies in order to evaluate the effectiveness of programs for enriched learning.
7. Do graphic presentations of the data distort the findings?
Graphic illustrations in the article -- all Tables -- are clear and support the text.
8. Do the conclusions and interpretations follow logically from the results? Are unwarranted causal conclusions made from correlations or comparisons? Are limitations indicated?
The conclusions and interpretations follow logically from the study, although the author fails to discuss results of the Target Teach curriculum, focusing only on the Direct Instruction program. His brevity, apparent throughout the article, omits many potentially valuable factors, whilst obstructing other data. No unwarranted causal connections seem to have been made. The study is admirable in its scientific tautness and accuracy. Possible limitations have been addressed throughout with the researcher making all attempts to meet them and being honest when he was unable to.
Reference
Thompson, B. (2006). Evaluating Three Programs Using a School Effectiveness Model: Direct Instruction, Target Teach, and Class Size Reduction, Third Education Group Review, 2, 1-10.
