Most were unwilling to invest in classes or even seminars regarding proper use of library reference materials.

The library "information desk" was perceived as unhelpful and further, unwilling to be helpful. The "barrier" between the patron and the desk disallows the librarian to visit the computer with the client and do one-on-one tutoring.

Desk staff refer less than 10% of the questions to reference librarians, even when clearing dealing with questions far beyond their purview (Ibid 210-12).

Using the standards suggested by Kuruppu, one is able to glean a broad critique of the Massey-Burzio study:

Issue

Massey-Burzio Use

Comments

Methodology

Focus Group

Good as a primary identifier of questions and direction, inappropriate as a data source; responses are too broad, lack the specificity necessary to tabulate, and only "suggest" data use or further research, does not define (Kuruppu 375).

Sample Size

38 Individuals over time

Admitted small sample size, too small for statistical validation. However, as tool suggested above, the first step in the research, the materials is valuable and suggests some issues with the library, as well as numerous questions for additional research.

Longitudinal Rate

1-Year, new focus group participants over time

Lacks robust veracity -- see above.

Detailed Questions

Limited

Questions limited, does not really reflect the actual need of the initial thesis of the study; may simply be too limited, and study author should have indicated that this was simply an overview prior to actual data collection (Ibid).

Moderator

Knowledgeable about reference materials; not a trained FG moderator

Probably immaterial in this case, but if longer group, a trained moderator who would have probed more and used incoming data to suggest further research; study as Part 1 of a larger, and longer systemized approach, might have proven helpful (Ibid 376).

Assumptions

Generalities of behavior

Conclusions reached indicated that these students are a) representative of the general population of undergraduate and graduate students at Johns Hopkins and b) that the participants are indicative of college age/bound individuals globally. Interpretation...

Essentially, it accentuates the negatives of qualitative research that, when used properly, can provide a wealth of information for use in a more robust setting. This might have been easily corrected simply by introducing the bias of the study, and positing for further research, giving more specifics for data collection, sourcing, demographic and pscyhographic modifiers, and a questionnaire based, qualitative element.
REFERENCES CONSULTED

Kuruppu, P., "Evaluation of Reference Services -- a Review." Journal of Academic

Librarianship. 33 no. 3 (2007): 368-81.

Pierce, S. "In Pursuit of the Possible: Evaluating Reference Services." In Evaluating

Reference Services, the Reference Librarian, ed. B., Katz and R. Fraley. (Haworth

Press, 1984).

Whitlach, J. "Evaluating Reference Services in the Electronic Age." Library Trends.

50 no 2 (Fall 2001): 207-17.