Reliability and Validity in Point-In-Time Counts of Homeless Populations

¶ … substance abuse, PTSD, domestic violence, family functioning, juvenile delinquency or adult criminality, parenting skills, self-esteem, depression, OCD, child well-being, mental status, adoption stability, anxiety, and wellness. If there is a variable of interest to you that is not included on this list, please check with your instructor to determine if it is an appropriate substitution.

Description: Provide the name and a brief description of the instrument including how it is administered, the length (number of items), general scoring information and other relevant information. Be sure to cite the sources for the information you use.

The instrument I am exploring is the Family Assessment Measure III, which can be completed by family members from age 10 to adult. There are six scales in the FAM-III, each of which takes approximately 20 minutes to complete. The self-report instrument has three forms: 1) a General Scale for assessing overall family well-being (50 items); 2) a Dyadic Relationship Scale that examines how family members view their relationships with one another (42 items); and 3) the Self-Rating Scale that enables each individual to rate her or his own functioning within the family (42 items). The test developers are Harvey A. Skinner, PhD, Paul D. Steinhauer, MD, and Jack Santa-Barbara, PhD. Information about the Family Assessment Measure III may be found on the publisher's website (http://www.mhs.com/product.aspx?gr=edu&id=overview&prod=famiii).

2. Norms: When scales are developed and tested, they need to utilize samples of subjects to establish "norms." It is important for social workers to understand the samples on which a scale was normed in order to assess the degree to which the scale or measure is useful for different populations. Describe how the norms were developed and reported for this scale or instrument. This section is not to describe the scoring rubric -- it is to describe the population on which the scale was developed and tested. Cite the reference materials you use. Discuss any potential cross-cultural, gender, sexual orientation or age issues that should be considered in terms of others using this measurement procedure.

Details and descriptive statistics regarding the clinical samples used for norming the test are included in the instrument guide and manual. The normative data for the FAM-III came from the analysis of 247 normal adults, of whom 106 were males and 141 were females, and 65 normal adolescents, of whom 33 were males and 32 were females. A preliminary analysis was also conducted on the FAM-III with a heterogeneous sample of 475 families: 429 were adult males, 504 were adult females, 226 were adolescent males, and 276 were adolescent females. No information was provided in the online accessible brochure and instrument description about the sexual orientation of members of the sample, nor was there any discussion about cross-cultural issues. Considering the limited region from which the participants were drawn, it is reasonable to question the adequacy of norming with regard to ethnicity, race, and cultural differences.

3. Reliability: Discuss the reliability of measurement for the variable. Make sure to cite all reliability information you get from other sources. It is not appropriate to state, "this measure has been found to be reliable." You need details and citations. Your description should indicate that you have a clear understanding of the meaning of the concept of reliability by providing a relevant discussion in your own words about methods used for establishing reliability and about the reliability coefficients reported in the material you cite. Simply cutting and pasting and then citing what was written in reviews is not adequate.

Detailed data about the internal consistency reliability and test-retest reliability are provided in the FAM-III Technical Manual. The product description notes that the reliability measures included factor analysis and inter-correlations among the subscales. Fortunately, since there is not much available information on reliability readily available from the instrument publisher, a summary of relevant research and a rating (a score of "A") by The California Evidence-Based Clearinghouse for Child Welfare (CEBC) were located.

Skinner, H.A., Steinhauer, P.D., & Santa-Barbara, J. (1983). The Family Assessment Measure. Canadian Journal of Community Mental Health, 2(2), 91-105.

In Skinner, et al. (1983), 475 families who participated in health and social service settings in Toronto were divided into "Problem" and "Non-Problem" families. To be categorized as a problem family, one or more members had to be receiving treatment for psychiatric problems, substance abuse, school issues, or major legal problems. Family members aged 10 years and older completed the FAM. All three subscales ere found to have good internal reliability for respondents of all ages. The FAM effectively distinguished between the two family type groups, and the "Problem families" showed significantly different scores on Involvement and Role Performance; the "Non-Problem families" scored higher in the areas of Denial and Social Desirability. Note that race and ethnicity were not mentioned in the study description.

Jacob, T. (1995). The role of time frame in the assessment of family functioning. Journal of Marital and Family Therapy, 21(3), 281-288.

In Jacob (1995), families with at least one adolescent aged 12 to 18 were recruited through newspaper advertisements. The race and ethnicity reported that 90% of the participants were white. The participating families completed two rounds of the Family Environment Scale (FES), the FAM, and the Family Adaptability and Cohesion Evaluation Scales (FACES-II). The time frames addressed by the rounds were varied, such that the participants might be asked to respond with regard to their family in general, or according to some time certain, say, two weeks ago. All three assessments showed good test-retest reliability for both parents and children. High positive correlations were found between the different time-frame instructional conditions. One exception was found in that the FAM did not show significant time-frame instruction effects. The authors believe this is due to the emphasis on general characteristics in the FAM compared to the other two instruments.