¶ … Toolkit for Commenting on Two Peer-Reviewed Research Articles Evaluation and Improvement of an Advisory Program

By John Walter Van Ornum

A rural Hawaiian high school is faced with a problem -- it cannot be clearly stated whether or not the school's student advisory initiative's goals are being met. Statistical techniques utilized in the study (paper 1) for computing the results were not clear, and no evidence exists regarding the current survey's validity and reliability. Further, these qualities were not evaluated.

The study's purpose was to evaluate Molokai High School's advisory program. The research questions seek responses from the advisors and advisees to evaluate the efficacy of Advisory program. The researcher believes that this study will help those

Who want to improve such programs, in their schools?

Who want to implement such programs effectively, having learnt the drawbacks and lacunae, and;

Who seek a feedback about the programs, to improve it through insights from actual users?

Research Question

This research paper aimed to evaluate the impact of advisory program on the academic performance of the students. The research objective is very vast. The research lacked specific questions to seek what, if at all, plagued the program. Stating that advisors seek adequate training seems to give a generalized opinion. It lacks answering the specific issues, such as disparities between the background of the students and advisors, and possible solutions/training required.

However, the research has been systematically designed to reveal both, the opinion of students about the advisors and those of the advisors, about the advisory program. The paper also explains research methods; thus, study designs can be understood. In as much, other researchers can repeat the study by adopting identical methodologies. The paper includes a list of data obtained from advisors and students. Data appears to support study outcomes. Key principles are summarized and the fact that researchers developed a protocol has been reinforced. I have personally examined data reports, and can safely state that the answers do influence the topic I have chosen, in that, they can give answers to research questions that have been derived from my topic.

Conclusions

The following main conclusions were arrived at in the paper:

1. Data analysis from this study indicates that the advisory process was supported by advisors as well as students.

1. Information gathered with regard to advisors' content knowledge indicates that students feel their advisors have sufficient knowledge pertaining to the advisory initiative. However, advisors feel inadequate, and desire more training.

The researcher could have connected the issues of "support" of advisory program and academic "outcomes" of the students. Secondly, the researcher should have evaluated the gap in the perspective of the students and the advisors; the research questionnaire lacks that aspect.

Conclusions must always be considered from the perspective of readers (Coughlan, 2007). After reading any paper, readers wish to understand the benefits to them from whatever has been accomplished in a study (Hall, 1992).

Paper Conformation to Trochim and Donnelly's (2008) Definition of Validity

The paper follows Trochim and Donnelly's (2008) definition of validity "the best possible approximation of the truth of a given proposition, inference, or conclusion" (p. 14). This point can be noted especially clearly in the paper. Qualitative, descriptive, cross-sectional study design was employed. A non-experimental survey, which maintains information as collected was employed as well; hence, the research work is value-free and objective. Response coding was performed using numerical values that aid with identification of program weaknesses and strengths. Population sampling was utilized for advisors, while advisees were chosen randomly. The sample was representative of every grade. According to Creswell (2008), this kind of sampling represents the most rigorous probability sampling technique. Hypothesis was also tested by researcher.

Theoretical and Observational Levels of Study

This article helps me to understand that such a research needs to be carried out in continuum, if it is to provide real insight. Specifically, the advisors could not help the students in completing the Personal Transition Plan. I understand that a proper method would be to track the development of the student in succeeding classes/grades to gain a better understanding of the Advisory program. A mental examination of the study's observational and theoretical levels does reveal some potential errors and confusions. For instance, the researchers have confused what they know as something that everybody else ought to know. Casual readers will firstly have a look at the paper's abstract; if the abstract is capable of retaining the interest of readers, they will go on to examine the study's conclusions. And only when conclusions present an element of value, which is of significance to readers, do they go on to read the entire paper.

Paper Objectivity

The research is, indeed, objective. Researchers have provided...

Also, use of statistics and facts has allowed the researcher to maintain maximum possible neutrality throughout the paper.
Rigor

Rigor was guaranteed by the employment of a qualitative descriptive, cross-sectional study design. A non-experimental survey, which maintains information as collected was employed as well; hence, the research work is value-free and objective. Credibility was assured by establishing results as believable from participant perspective (Creswell, 2008). The researcher asked specific questions about cultural background of, both the advisors and the advisees to provide valuable, comprehensive feedback that any future research (such my own) can rely and build upon to probe further if advisors attended to cultural disparities and attempted to resolve issues that emanated from them and affected academic outcomes. Thus, the article reaches out to issues other than the school environment that affect the Advisory Program.

Recommendations

Researcher believes repeatability can only be addressed by researching the very same class all through the four high school grades. Though students will be transferred to and from that class, changes ought to be negligible, as most members remain the same. I believe that this study has pointed out the general gap in the perception carried by the students and the advisors. In my work, I will try to address this issue further by following the students into each succeeding year of their schooling and provide further, tangible insights.

Methodology

Methods used are of good quality. The list of questions that was previously designed was employed and its application explicitly stated in the paper. The paper's question about the frequency of advisory programs and that of discussing advisory programs at faculty meetings were founded on basic presumptions of the research. The researcher also considered sources that did not agree with the argument he presented, ("Faculty advisors may see this program as another time constraint further limiting their preparation time for their regular education classes (Myrick, 1990)" (p. 26) in addition to some that were similar to his arguments. The author states that "Reasons for use of the prescriptive model range from lack of training, more use of e-mail communication venues, to an increase of advisor part-time positions (McClellan, 2010; Smith, 2002)" (p.32).

Opposing Arguments and Rationale for Findings

The paper presented no opposing and alternate perspectives.

Evidence of Common and Diverging Themes

One important themes arising in the discussion is that a healthy rapport between teachers and students lead to better employment prospects for the students in future. However, literature points out that lack of sufficient personal contact at higher classes leads to problems of depression, withdrawal and angst towards teachers. This article has attempted this issue through a recommendation that suggests continued training (for advisors) and evaluating students on a yearly basis.

Overarching Concepts

The main concept discussed and addressed in the paper is the extent of effectiveness of school advisory programs, and means for evaluating them properly.

Sample Representative of Entire Population

The researcher has used a single class as the study sample to represent all the four years. Therefore, the entire population has been clearly represented and thus utilized.

Discussion

The discussion fails to critique as well as discuss methodology applied. This should be performed. Firstly, researchers need to elucidate (in the methodology section of their paper) the precise methodology used. Subsequently, they need to critique it, i.e., outline its weaknesses and strong points (Eaton, 2009).

Results

The data brings out the perception of the students and advisors regarding each other. However, it fails to specify whether the outcomes of the programs meet initial objectives of the advisory programs adopted by the school (creating a healthy rapport between faculty and students and empowering students to seek worthwhile livelihood). Secondly, the chosen school for the project did not have enough resources to evaluate the Advisory Program, because of its small size. However, the results obtained in the study agree with some themes (as stated earlier) and are backed by responses (tables 23-24, 28, 33, 39, 40, 41, 42).

Paper 2: "Can School structures improve teacher-student relationships? The relationship between advisory programs, personalization and students' academic achievement"

By Larry McClure, Susan Yonezawa and Makeba Jones, University of California, San Diego

Research Problem

The paper's focus was the association between student advisory initiatives, academic achievement of students, and personalization.

Research Purpose

The study presented the findings of a research spanning three years, which addressed perceptions of students with regard to personalization and, in specific, advisory as a strategy for improvement, together with its relationships with…