Analyzing Tests and Scales

¶ … tests (CRTs) and scales vs. norm-Referenced Criterion-referenced tests (CRTs) are often the preferred method of assessing the performance of many practitioners in the healthcare and 'helping' professions such as nursing. An example of a criterion-based objective is that a student mastered 90% of the terms on a particular test (McDonald 2002). The NCLEX (National Council Licensure Examination) for nurses is an example of such a test: all nurses that pass the test can obtain licensure. The test is deemed to be both reliable and valid.

"The reliability of the NCLEX examination is assessed via a decision consistency statistic. This statistic is used instead of a traditional reliability statistic such as Cronbach's alpha because it captures the reliability of dichotomous pass/fail decisions rather than the reliability of continuous scores or ability estimates" (Reliability of NCLEX, 2013, NCSBN: 2). In terms of the NCLE, the exam attempts to ensure content validity; face validity; construct validity; predictive validity; and scoring (passing standard) validity.

A variety of item writers are contracted to ensure the content will "cover the entire domain of entry-level nursing practice;" sampling validity is ensured by "on going evaluation of the scope of entry-level nursing practice;" face validity is ensured through "real and simulated examinations...read by experienced test developers to ensure that the balance and juxtaposition of content is on face, representative of the domain of nursing;" scoring validity is determined by the process whereby "each examinee receives at least 15 'tryout' items…not counted towards an examinee's score…performance on these items is tracked for all examinees" to assess what types of examinees do well on the test; and "the minimum level of competency that an examinee must attain in order to pass the NCLEX examination is investigated thoroughly on a triennial basis" (Reliability of NCLEX, 2013, NCSBN: 3-4).

The NCLEX test is a criterion-referenced test unlike a norm-referenced test whereby the score is determined by the performance of the other test-takers. Partially because the NCLEX is criterion-based, it would not be valid to test on other populations since it is specifically designed to measure nursing competency. As well as actual tests, criterion-referenced nursing scales can also measure critical nursing traits. For example, the Nursing Student Self-Efficacy Scale measures nursing students' self-efficacy beliefs, psychomotor skill performance and communication skills.

The test was normed upon 421 nursing students and "40% of the items provided high information about self-efficacy and 20% provided moderately high information" (Stump, Husman, & Brem 2013). Similarly, this scale is intended solely to measure skills in relationship to nursing and is not suitable for other populations. References McDonald, M. (2002). Systematic assessment of learning outcomes: Developing multiple-choice exams. Jones & Bartlett Learning Reliability of NCLEX. (2013). NCSBN. Retrieved: https://www.ncsbn.org/Reliability.pdf Stump G.S., Husman J., Brem S.K. (2013). The Nursing Student Self-Efficacy Scale: Development using item response theory.

Nursing Research, 61(3):149-58. doi: 10.1097/NNR.0b013e318253a750. Did they report Cronbach's alpha reliability for the measures? "Cronbach's alpha is a measure of internal consistency, that is, how closely related a set of items are as a group. A 'high' value of alpha is often used (along with substantive arguments and possibly other statistical measures) as evidence that the items measure an underlying (or latent) construct" (What does Cronbach's alpha mean, n.d., SPSS FAQ).

It is a 'test' of reliability but not technically a statistical test and is frequently used to test the reliability of a variety of types of standardized tests (What does Cronbach's alpha mean, n.d., SPSS FAQ). Standardized tests must be reliable as well as valid to ensure fairness for all test-takers, regardless of what version of the test students take and when in the year they take it.

According to reliability testing of the NCLEX according to Cronbach's alpha "the average of this probability over all candidates is the estimated decision consistency of the NCLEX examination. The decision consistency of the NCLEX examination is psychometrically sound normally running between .87 and .92," indicating its reliability (Reliability of NCLEX, 2013, NCSBN).

It is essential that a test such as the NCLEX be reliable, given that it is given multiple times a year as a standardized test and all candidates who pass the test will be given the right to practice as nurses in the field of healthcare and have control over people's health. Also, taking the NCLEX is a tremendous financial investment so it is only fair that candidates who deserve to pass receive their license. Which is more important validity or reliability? Both validity and reliability are essential for.