341). In this regard, Neumann (2005) reports that, "The reasons for length of time to completion and non-completion rates are important considerations for universities and governments," and notes that: "Concerns about slow time to completion relate to the commitment of staff and other resources for extended periods, the desirability of concentrating graduate studies in a reasonable time period, the possibility that information obtained in the research may become obsolete before completion of the thesis, and delays in researchers moving on to other projects" (Neumann, 2005, p. 16). The growing body of evidence to date suggests that the reasons for degree noncompletions fall within three broad areas: (a) personal or family factors, (b) changes in career aspirations and development, - issues of quality of supervision and to these a fourth should probably be added: "Growing pressure for the PhD to be everything to everybody" (Neumann, 2005, p. 16). Furthermore, almost all data in higher education are inherently multilevel, even though this feature of the data is typically not taken into account; depending on the application, the different levels of analyses could include countries, geographic regions or states, different universities, faculties or departments within universities, and individual students or academic staff. As illustrated in the instant study, research, policy questions, data, and statistical analyses that are appropriate at one level of analysis may be inappropriate or even misleading when evaluated at another level of analysis. Overview of Study

This study used a five-chapter format to accomplish the above-stated research purpose. The first chapter introduced the topics under consideration and provided the purpose, importance, scope and rationale of the study. Chapter two below provides a critical review of the relevant peer-reviewed, scholarly and organizational literature and chapter three describes more fully the study's methodology. Chapter four presents an analysis of the statistical data and the concluding chapter provides a summary of the research, salient conclusions and recommendations for policymakers and doctoral students alike.

Chapter 2: Review of Related Literature

Background and Overview.

The relationship between doctoral students and the supervisory faculty at graduate universities is inextricably related to how the research experience will likely unfold for the students involved. According to Delamont, Atkinson and Parry (2004), "Academic disciplines and their subject-matter are mutually constitutive. A discipline furnishes its members with definitions-often tacit-of what is 'thinkable', with appropriate assumptions as to what 'counts' as research problems, suitable research methods, definitions of research programmes and the approved modes of graduate student research" (p. 173). Given the importance of this relationship, it is reasonable to posit that the better the supervision provided, the more favorable the doctoral students will perceive their university experience and the support they received during their research experiences. According to Walfish and Hess (2001), "Graduate school course work emphasizes the preparation of graduate students for research" (p. 167). In this regard, Greene, Hardy and Smith (1995) report that, "Research training plays the central role in the present definition of doctoral education. But as new graduates are discovering, research inside the ivory tower often has little in common with what happens outside. In the past, university research has been a two-for-one bargain: cutting-edge research paid for with the same dollars that educated the next generation of researchers" (p. 59).

Doctoral Students' Perception of Their University and Research Experiences.

While there is a growing body of research concerning undergraduate students' evaluations of classroom teaching effectiveness and some research on the quality of supervision of research and PhD students, there remains a paucity of current research concerning the reliability and validity of PhD students' evaluations (Marsh et al., 2002). According to the most recent the Report of the Postgraduate Research Experience Questionnaire, "Simultaneous analysis of data from the 2003, 2004 and 2005 surveys showed that there is quite a substantial amount of variation among the annual PREQ scores for many if not most institutions. While such variation was most closely associated with those institutions that provided only very few responses, this was not always the case and it is apparent that most institutions' scores vary over the years. The composite institution-level estimates show that there are generally only statistically significant differences between institutions at the top and bottom of each distribution, and that there are only a few institutions which are routinely located in the top five for all facets of the experience measured by the PREQ" (PREQ, 2006, p. viii).

One intended purpose of PhD students' evaluations is to provide informative feedback...

There is clear evidence that feedback from students' evaluations of teaching, coupled with appropriate consultation, can lead to improved teaching effectiveness. For example, in a study by Marsh and Roche (1993), randomly assigned intervention- and control-group teachers completed self-evaluations and were evaluated by students before and after the intervention. An essential component of the intervention was a set of teaching strategy booklets -- one for each factor on the student evaluation instrument. Teachers selected the factor to be targeted in their individually structured intervention and then selected the most appropriate strategies from a book of strategies relevant to that factor. The intervention teachers improved significantly more than control group teachers. Furthermore, for the intervention group (compared to control group), targeted dimensions improved substantially more than nontargeted dimensions. The study demonstrated that feedback from students' evaluations of teaching and consultation are an effective means of improving teaching effectiveness.
These authors also note that it is important to keep in mind that this intervention can only be conducted with a well-designed, multidimensional instrument and that the specificity of the intervention effects to the targeted dimensions further supports the construct validity of multidimensional students' evaluations of teachings. The lessons from this research that may be useful for improving the quality of postgraduate supervision based on PhD students' evaluations are that the feedback needs to be specific to each supervisor; supervisors may need concrete strategies about how to improve their supervision; and this feedback may need to be complemented by a trained consultant. Even when supervisors are motivated to improve their supervision and have feedback about their strengths and weaknesses, they still require professional assistance on how actually to improve their supervision (Marsh et al., 2002).

The PREQ represents a contentious policy issue in higher education in Australia for three reasons:

It is used to develop performance indicators with comparisons between universities;

There is the threat of funding being granted or reduced as a result of performance indicators; and,

There is some disagreement about the validity of the PREQ, especially since many supervisors have an insufficient number of postgraduates to satisfy a reliability criterion (and they cannot be identified for ethical reasons) and the intended unit of analysis is broad 'fields of study' within a university (rather than supervisors) (Waugh, 2001).

A concomitant analysis of data from the 2003, 2004 and 2005 surveys demonstrated a significant amount of variation among the annual PREQ scores for many, if not the majority of Australian institutions; although this level of variation was most closely associated with those institutions that provided only very few responses, this was not always the case and it is apparent that most institutions' scores vary over the years (PREQ, 2007). The composite institution-level estimates show that there are generally only statistically significant differences between institutions at the top and bottom of each distribution, and that there are only a few institutions which are routinely located in the top five for all facets of the experience measured by the PREQ (PREQ, 2007).

Review of Recent Australian Government Publications Concerning Doctoral Programmes and the Research Experience of Doctoral Students.

In their publication, "Researching training doctoral programmes: what can be learned from professional doctorates?," McWilliam and her associates discuss the relatively recent experience of offering doctoral education through professional doctorate programmes as a contribution to the improvement of doctoral education in Australian universities. The evaluation focused on the extent to which such programmes had developed practices for sustaining closer collaboration between universities and industry. According to this authors, "Doctoral education in Australia is currently under pressure to become more industry focused. When set against the 800-year history of the PhD, the professional doctorate is a young doctorate, the first being set up in Australia within the last two decades. The nature and status of professional doctorates remains unclear to many, including a number of university administrators of research training, as well as government and industry personnel. The fact that 61 per cent of professional doctorate programs fall under the classification of 'research' higher degrees is not widely understood. Moreover, the 131 programs we found to exist in 35 of the 38 Australian public universities, exhibit a wide range of structures and features.

While there is strong evidence of an increase in the number of professional doctorates being offered in Australian universities, and there is some evidence of…