Population Identified and Described? Are Eligibility Criteria

¶ … population identified and described? Are eligibility criteria specified? Are the sample selection procedures clearly delineated? Yes. The sample consisted of 350 college students at a Midwestern University. All the students were enrolled in a personal health class as a social science elective.

Do the sample and population specifications support an inference of construct validity with regard to the population construct? Of n=350, 86% were White, 5% African-American, 4% Asian-American, 3% Latino, and 2% Other. This is not representative of the collegiate population in general, nor is it representative of the baseline population breakdown for most of America. However, because the classes are a social science elective, the sample does serve as an adequate representation of a cross-section of this particular Midwestern University.

What type of sampling plan was used? Would an alternative sampling plan have been preferable? Was the sampling plan one that could be expected to yield a representative sample? The sampling plan was based on the students who were enrolled within health courses, 63% female and 27% male with a mean age of 21 years. If the researchers had wished to develop a more representative sample, the would have needed to include additional criteria with which to base their observations.

4. If sampling was stratified, was a useful stratification variable selected? If a consecutive sample was used, was the time period long enough to address seasonal or temporal variation? Sampling was not stratified; it was demographically based on those students who were involved in the course and/or who volunteered for the project. We know that the research was done during one academic period, but do not know which quarter or time of year it took place.

5. How were people recruited into the sample? Does the method suggest potential biases? Recruitment was done based on personal health classes -- students had a variety of majors and ethnic backgrounds, as well as a variety of attitudes about physical activity. In the bounds of this study, there was no apparent bias based on recruitment, but to project data from this to a larger population would require additional sampling.

6. Did some factor other than the sampling plan (e.g. A low response rate) affect the representativeness of the sample? Sample was consistent with research values -- the researchers had no control over who would enroll in these classes, or what the makeup would be. There is an assumption that regardless of the school quarter or season, the make-up of the classes would be similar in demographics.

7. Are possible sample biases or weaknesses identified by the researcher themselves? There is a sample bias inherent in the study, but it is part of the overall design. This sample is of only college students, only taking a certain set of classes, and only within this one particular Midwestern University.

8. Are key characteristics of the sample described (e.g., mean age, percent female?) Key characteristics are described, but no cross-tabulation based on sampling bias or issues was included in the results. The sample population used a self-regulated set of instruments that took 10-15 minutes to complete. During the 4 weeks of the study, the subjects completed a 7-day recall of physical activity. Measurement was done based on a 43-item instrument which addressed behavioral strategies (goal setting, self-monitoring, behavioral issues, etc.)

9. Is the sample size sufficiently large to support statistical conclusion validity? Was the sample size justified on the basis of a power analysis or other rational? As noted, the results of this study do not support a great deal of external validity. The data could be representative of certain college campuses in the Midwest, for instance, but would not be representative in other geographic areas

10. Does the sample support inferences about external validity? To whom can the study results reasonably be generalized? The study, however, could be generalized with a caveat -- as a representative sample of undergraduate college students in a college or university (as opposed to community or technical school) setting.

Guidelines for Critiquing Data Collection Plans in Quantitative Studies

1. Was the collection of data using structured methods (in contrast with unstructured methods) consistent with study aims? Unstructured due to an inability to predict or stratify; sample had to use the students who were available.