Determining Reliability and Validity

¶ … Reliability and Validity in Psychological Testing

In any kind of academic and professional testing, it is important to obtain at least some degree of reliability and validity. Failing this, the tests cannot be applied for results that are consistent or usable in an academic setting, since they cannot be verified in terms of repeatability or in comparison to other results. In psychology, which is more often than not studied by qualitative rather than quantitative means, it is often difficult to establish reliability and validity, since the specific numbers to do so are lacking. However, there are means to ensure an optimal level of validity and reliability in this kind of testing

Reliability

According to Kline (2013, p. 7), reliability comes in two distinct manifestations: Reliability in terms of consistency over time, and reliability in terms of internal consistency. Overtime, reliability is determined by administering tests to the same individuals on more than one occasion. When the results are the same or similar, the reliability is at a high level. This is also known as an instrument's test-retest reliability. Internal consistency reliability, on the other hand, concerns the relation of the test items to each other. The danger with this type of reliability is that the test items may simply be paraphrases of each other, which negatively affects the validity of the instrument (Kline, 2013, p. 13).

The importance of the reliability factor in test instruments is shown in studies such as the one conducted by Narrow et al. (2013). In this document, the level or reliability for several populations were measured when administering test items for the cross-cutting symptom in terms of the DSM-5 measurement document. The authors found the test-retest reliability of instrument items good to excellent. In terms of the populations being interviewed, parents proved to be reliable in terms of reporting symptoms in their children. For child respondents, however, responses were less uniformly reliable, while clinicians were able to rate psychosis reliably but not clinical domains when it comes to psychosis for children or suicide across the age groups.

Validity

The validity of a psychometric test is high when the test is shown to measure what it intended to measure. According to Kline (2013, p. 17), many psychometric tests have surprisingly low validity, which demonstrates the complicated nature of the matter and its measurement. In psychological testing, validity is often difficult to measure because of the fluid nature of the concept of validity; there is no singular measure or statistic against which it can be measured for validity. For those attempting to determine the validity level of a psychometric test, there are various types of validity that can be used to determine the level of this component of testing.

Face validity, for example, refers to the appearance of a test in terms of measuring what it intends (Kline, 2013, p. 18). Concurrent validity, on the other hand, refers to the correlation of the test with others measuring the same variable at the same time. Construct validity is very similar to this, in that it refers to the correlation of the test results with other established tests that are being administered on a regular basis.

The Importance of Reliability and Validity

The importance of reliability and validity is demonstrated by a study conducted on online continuous performance tests (Raz et al., 2014). Here, a high level of both reliability and validity was found. Measures of split-half reliability, test-retest reliability, and construct validity all showed a high reliability and validity for the instrument, with tests taken in laboratory settings and those taking in different settings on an online platform showing highly correlated results.

The main implication here is that the high reliability and validity of the instrument holds advantages in terms of cost savings and ease of administration, as well as high compliance among test participants in the future. Although the authors admit that more control would need to be exercised over test environments in the online setting, there are clear advantages of these early indications of highly reliable and valid results. One suggestion is the use of web cameras to determine consistency in test environment conditions.

This study shows the importance of high validity and reliability in order to determine future types of test administration that can be beneficial to both test participants and those administering the instrument. New technology is constantly changing the academic world. With the ability to determine reliability and validity, extremely valuable developments can be identified as the future of psychometric testing.