Hypothetical Designs Three Hypothetical Study

Hypothetical Designs

Three hypothetical study designs: Post-tenure review

Quantitative hypothetical design

To assess the ability of the post-tenure review process to enhance student learning, a quantitative study could compare students' satisfaction levels with professors at institutions that used post-tenure review with students' levels of satisfaction at institutions that did not utilize the post-tenure review process. Surveys of students of tenured professors at institutions that used the post-tenure review process could be compared with the results of student satisfaction surveys at other institutions where post-tenure reviews were not deployed. The professors of the students being surveyed in the control group should be demographically similar to those in the experimental group. For example, if at the institution with post-tenure review, students of history, chemistry, and English professors were surveyed, the control group should consist of students with the same type of professors from an institution without post-tenure review.

Quantitative designs are problematic for this subject, given that the study is trying to measure something subjective, namely the effects of post-tenure review on teaching quality and student satisfaction. Also, post-tenure review processes will likely differ widely between departments, thus categorizing the experimental group as the 'post-tenure review' grouping might conceal important differences that make one department's review process more effective than the other department's proceedings. Student's commitment to their academics and ability to objectively measure their professor's competency would also likewise vary and affect the students' levels of satisfaction with teaching quality.

Qualitative research design

In a qualitative design, the researcher could observe faculty meetings at a department implementing a post-tenure review process. This could present some ethical problems, regarding consent. All faculty members would have to consent to the observation for this type of case study format to be feasible. The case study would also have to be supplemented with individual interviews of the faculty members, and ideally of students who had the professors under review. These interviews should be structured rather than unstructured, to allow for an easier comparison of results of different student groupings and demographics. For example one question might be if female students more satisfied than male students, after the professor's classroom management regarding calling on students of both genders was assessed?

Problems with type of qualitative research might arise when selecting a department willing to allow such access to a researcher. Faculty members given a poor post- tenure review might be embarrassed, if their identity was discovered. Students might also be less than candid when interviewed, for fear of getting a bad grade, if they criticized the department. By necessity, only a few researchers could be present at the post-tenure review meetings: otherwise the level of candor of faculty members in the observed discussions would be low and participant reactivity is always a concern in any observation (Eysenck, 2004, p.19). Of course, transcripts of department meetings could be used, as in past studies, but their use would limit the ability of voice inflection and other variables to shape the case study.

There is also the perpetual problem with qualitative research: namely because it is a single sample size, the study might only be representative of one particular population. However, some qualitative researchers argue that any type of research is subjective: they believe that even the approach of quantitative research disguises idiosyncratic aspects of reality and even the most rigorously empirical construct says more about a researcher's biases than an external reality. The many subjective beliefs about post-tenure review itself within a single department are manifest in the varied responses of faculty members as to the impact of such reviews in Wood & Johnson's study (2005, p. 425). Thus, it may actually be more valuable to study one department, and let readers decide for themselves if the process is appropriate for their educational environment.

Selecting several representative departments (one at a small liberal arts school, one at a large university and selecting departments from different subject areas) might be one way to address some of the concerns regarding 'generalizability' of results but would require more observers to take part in the research (Trochim, 2008, p.148). Still, this could be a valuable approach: while previous studies have focused on pre-existing beliefs about post-tenure review more research is needed on the results (O'Meara, 2004, p.183). Also student input is essential -- while some faculty have reported increased congeniality as a result of the process, ultimately it is the students' perspective where teaching quality receives its true test (O'Meara, 2004, p.193)

Action Research