Schools Use Data for Implementing

¶ … schools use data for implementing strategies in order to prove their accountability to the No Child Left Behind legislation. Almost 300 of the articles that I browsed through dealt with qualitative research (primarily surveys and focus groups), until I happened on Thompson's (2006) experimental study that deals with the search for programs that would help raise student achievement so that instruction throughout the school, in this case for elementary students, would be improved.

As Thompson observes, true randomized experiments are rare in education due to the multiplicity of factors involved. Non-experimental studies are rare, too, for the same reason. Nonetheless, as his literature review on adopted programs cursorily shows, an experimental study on methods that can prove helpful to raising student achievement whilst holding all other factors (such as poverty, family, environment, teachers, and school) fixed can prove invaluable to the field of data assessment in specific and to school education (including elementary education) in general. Therein, lies the contribution of his research.

2. Is the review of literature establish the relationship between previous studies and the current one? Is the review well-organized and up-to-date?

The literature review, whilst not thorough, is up-to-date and proceeds in a commonsensical sturdy manner delineating the need for such research and showing the deficiencies caused to the educational system by omission of a true experimental study. Thompson explains why this has been the case, and concludes with limitations of quasi-experiemental research in terms of reliability of evidence.

No literature review is, however, presented on the previous programs that had been adopted by various educational institutions and then replaced due to disillusionment, nor have the reasons for their disillusionment been described. Likewise, former studies (whether qualitative or quasi-experimental) have not been described. Thompson merely plunge directly into his description of the need for a true-experimental study on the subject.

3. Is the specific research hypothesis or question clearly and concisely stated?

The research hypothesis is stated in an indirect manner with the author stating the problem with the current state of affairs and, rather than stating his objective, proceeds to describe the ramifications of the omission of a true experimental study: that too many involved factors have been overlooked and thus outcome of studies canto be trusted, hence many programs formerly accepted have failed, and that quasi-experiments are more subjective than true experiments and, therefore, their reliability is in question.

4. Is the method of sampling clearly presented? Could the way the sample was obtained influence the results?

The author does not elaborate on his data source merely stating that he uses existent data of 112 elementary schools in the Milwaukee Public Schools (MPS) starting in the 1996-97 school year that were selected on the basis of their meeting certain conditions. Here, he then uses statistical tools (specifically regression coefficients) to evaluate whether school rating changed when three new programs were introduced in these schools. His intention is to use an experimental approach by using statistical tools to quantify and assess program effectiveness by comparing school effectiveness ratings before implementation of the program with schools effectiveness ratings following the implementation of the program.

5. Is there anything in the procedures for collecting the information or in the instruments themselves that could bias the results or weaken the study?

The author does not describe the source of his schools merely stating the inclusive and exclusive criteria that they satisfied. The schools, all in Milwaukee, had to satisfy three main criteria: firstly that the program under study was introduced during a period when rating were available, secondly, that the number of schools introducing the program must be sufficient for statistical results, and thirdly, that there should be sufficient and adequate comparison groups. His research seems immune to bias.

The author does, however, mention the possibility of bias in the appearance of marks and the schools that were chosen and concludes that the most likely bias that he could see occurring in the producing of the data was selection bias, namely that the schools may have been specially selected due to particularly low of high test scores that would have either regressed to the mean even without intervention or would have been too challenging to maintain.

6. Is the magnitude of the correlation or difference among the groups large enough to suggest practical significance or importance?

Although questions exist -- and the author discusses these at length -- the Target Teach program does show statistical significance. Correlations amongst the groups showed that amongst the schools using 'Target Teach' curriculum, and schools employing Direct Instruction average rating underwent significant statistical increase. However, schools implementing the SAGE class- size reduction program did not. The author took this to indicate a direct correlation between Direct instruction and student achievement and recommends that the Direct Instruction intervention programs be used as control group in future studies in order to evaluate the effectiveness of programs for enriched learning.