Understanding Scales and Reference Standards for Counselors

Norm- Versus Criterion-Referenced Tests The difference between norm- and criterion-referenced tests is that the former compares test scores to a reference group, while the latter compares test scores to a performance standard. Norm-referenced tests are quite common. For example, student reading performance in primary schools may be compared to the mean score for all children of the same age. The norm comparison group would likely consist of all students within a school district, state, or nation who took the same test at the same age.

Students who scored lower or higher than the mean for the norm reference group would be ranked as low or high achievers. Imagine, however, if someone wishing to qualify for a motor vehicle license was only required to achieve a score close to the mean score for all drivers? Using a norm-referenced driver test would likely be a bad public safety choice, especially if there are a lot of bad drivers on the road.

By comparison, public safety would be better served if all licensed drivers were required to understand 90% of road signs, be able to parallel park, and could navigate a complex and busy intersection without any problems. These represent standards of performance and therefore driver's tests are typically criterion-referenced tests. When using a criterion-referenced test it does not matter whether the majority of the population performs more poorly or better than the reference standard because the standard is not tied to population performance statistics.

This is probably the most important difference between norm- and criterion-referenced tests, because the performance of the norm-referenced group may change over time, thereby altering the performance standards of the test. By comparison, the reference standards on a criterion-referenced test will not change, regardless of changes in the sampled population.

Question 2: Qualities of norming groups There are no standards that can be used to describe what a good norming group would be; however, the selection of an adequate reference group will depend on the demographic being assessed, the goal of the assessment, and in what manner the testing results will be used.

Although selection of the norming group depends largely on who is conducting the testing, all norming groups should be adequately described to facilitate the performance testing being done and to provide enough information for other researchers interested in using the norming group for their own needs. Other considerations include a group size sufficient to create enough power for statistical comparisons. Norming groups are often minimally described using the demographic variables of age, gender, ethnicity, education, and income.

When children are administered the Wechsler Intelligence Scale for Children (WISC) the scores obtained are compared to mean test scores of children at the same age (School Psychologist Files, n.d.). The means were obtained by having thousands of children take the test, which implies the intelligence measured by the WISC is in comparison to norming groups stratified by age. When Yang and colleagues (2013) administered the WISC, version IV, to Taiwanese school children suffering from attention deficit hyperactivity disorder (ADHD) they were comparing the scores to norming groups from China.

Adequate norming groups for this study would have been Chinese children stratified by age; however, the authors expressed some concern about the validity of the comparison between WISC scores obtained by Taiwanese children and those obtained by children living in mainland China, due to cultural differences. This example does illustrate, however, how complex the qualities of a norming group can be and how important it is to select an appropriate norming group for a specific comparison.

Question 3: Scales There are four basic scales that can be used in assessments: (1) nominal, (2) ordinal, (3) interval, and (4) ratio. These scales are important because their use allow meaningful comparisons between variables. Nominal variables represent defined categories of objects or persons, but a nominal scale is limited to counting the number of objects or persons and representing the counts as percentages. Examples of nominal variables would be fruits and vegetables, cars and trucks, and liquid drinks and solid foods. Another example would be the patient distributions for a psychiatric ward by diagnosis.

If there are 230 patients and only 26 suffer from delusional disorder, then the count for this nominal category is 26 patients and this count represents 11.3% of the entire patient population. Another qualitative scale is the ordinal scale, which offers more information than the nominal scale because items belonging to an ordinal variable can be ranked according to some measure. For example, the patients belonging to the ordinal variable of delusional disorder might be ranked in terms of disease severity, from mild to severe.

One of the more useful variables used in counseling assessments would be ordinal variables. Interval variables will have all the characteristics of nominal and ordinal variables, but with the added complexity of equal intervals; therefore, counts, percentages, and rankings are possible, in addition to being able to conduct meaningful comparisons of values. For example, the WISC verbal comprehension scores differed between children diagnosed as with ADHD predominantly inattentive (ADHD-I) or combined (ADHD-C) (Yang et al., 2013).

These differences were statistically meaningful, in the same way that the difference between 20°, 40° and 60° F. is meaningful. Interval variables do not have a meaningful zero point and ratios make no sense. The interval between an IQ of 90 and 100 is equal to the interval between 100 and 110, but a person with an IQ of 120 is not twice as smart (200%) as a person with an IQ of 60. By comparison, the ratio scale has a meaningful zero point and ratios are possible.

Temperature measured using Kelvin has a zero point and a temperature of 30° K. is twice as hot as 15° K. The same mathematics cannot be done using the non-linear temperature scale Fahrenheit. One of the most common measures used in ratio scales is time, such as response time or processing speed. The scale with the least utility in counseling would probably be ratio scales because there would be few variables that have a meaningful zero.

Question 4: Application example The Montreal Cognitive Assessment (MoCA) is reportedly better able to detect mild cognitive impairment than.