Explaining Instances of Reliability and Validity

Part 1
Inter-rater reliability
Inter-rater reliability is considered the degree to which two or more evaluators of a study agree on its results and outcomes for reliability (Lange, 2011). When a rating system is implemented in a study, its consistency is addressed by this parameter.
It was tested for the reliability of a new clinical examination method called Cognitive Assessment and Reference Diagnosis System (CARDS) for early detection and management of dementia (Suh & Lee, 2003). The method was applied to 70 dementia patients and 34 healthy and control individuals. 22 of the 104 subjects were interviewed separately for checking the inter-rater reliability based on the fact that their responses were blind to each other and were not shared between them. The results showed that CARDS was a reliable instrument for detecting early dementia and its management in patients.
Test-retest reliability/ repeated measures reliability
Test-retest reliability is used for testing the extent to which test scores remain unaffected when measuring a steady distinct attribute on several occasions (Vilagut, 2014). The measurement error can be estimated by intra-individual responses when a retest assessment is conducted.
An example of this test is taken from the same study conducted by Suh & Lee (2003) for checking whether CARDS is an effective system for early detection of dementia in patients and its management. Out of 104 subjects, 52 were tested again for this purpose after two weeks. The results showed that CARDS was a reliable instrument for the early detection of the mentioned disease.
Face validity
Face validity measures the degree to which the test can measure what it aims to measure (Johnson, 2013). If people agree that it determines the items properly, which are part of the study, then the face validity would be higher.
An example of a face validity test was used in Mousazadeh, Rakhshan & Mohammadi (2017) to investigate the psychometric properties of the sociocultural approach towards appearance questionnaire among female adolescents. To check face validity, people who received the questionnaire were 25 female adolescents, a psychologist, and three nurses. The results indicated that the questionnaire was a valid instrument to check the social viewpoints of female adolescents.
Predictive validity
Predictive validity is the way of checking how much a measure can predict its outcome and validate its test's correlation scores (Criteria, n.a.). For the results to be following their validity, the sample size must be fairly large, such as that of schools or colleges.
An example of the test is seen in the study that was conducted by Kennet-Cohen et al. (2016) for examining whether a two-step process for getting admission in the medical schools is valid in predictive terms or not. The students' academic performance would be checked with the predictive validity of the admission tests and s its two-step process. This test result was high, which showed that the two-step process was valid for predicting the students' academic achievement, being high in the first academic year.
Concurrent validity
Concurrent validity is the measure to which one construct can produce valid results with other already validated measures of the same construct (Lin & Yao, 2014). It is similar to predictive validity as they both are parts of criterion validity. This very measure measures the correlations between the tests.
An example of the test is seen in the study that was conducted by Godwin et al. (2013) to check the concurrent and convergent validity of the Simple Lifestyle Indicator Questionnaire (SLIQ) for assessing lifestyle matters like physical activity, diet, and smoking, etc. conducive to a higher risk of cardiovascular diseases. The concurrent validity was checked by comparing physical activity in the questionnaire and the steps on the pedometer. For diet, it was assessed in comparisons with diet score on Diet History Questionnaire (DHQ), and for other similar factors, it was evaluated after making comparisons. In this study, the concurrent validity was expected to be higher if the correlation measures were 0.4 or higher; however, the results showed it to be 0.303.
Part 2
From the annotated bibliography, the first selected study that was conducted by Tomini, Tomini & Groot (2016) had reliable and valid measures since validity was checked for a large data where the quality of life (QOL) was checked for population aging 50 years and above from 16 European countries with the help of the fourth wave of SHARE, an instrument used for collecting large data about aspects of life. Reliability was checked since the different versions of the same measure, which was quality of life or life satisfaction in this study, were frequently used and measured for the individuals' wellness.
Another study was conducted by Kemperman et al. (2019) to assess the impact of social networks and neighborhoods' living environment on their loneliness in old age. Validly was evident in this study since a survey was conducted within a large sample consisting of 182 adults aging 65 years or older all over in the Netherlands. Inclusion of socio-demographic factors such as income level, gender, and whether a person has a partner and his health status was comprehensively noted. The Bayesian belief network (BBN) was used for estimating the relation between the selected variables, which were socio-demographics, mobility, social participation, living environment, and loneliness. Reliability is evident since the BBN involved three-phase dependency for checking the relationship among the variables. It means that variables and the measures were checked repeatedly to verify that the results were reliable.
References
Criteria. (n.a.). Definition of predictive validity. Retrieved from https://www.criteriacorp.com/resources/glossary/predictive-validity
Godwin, M., Pike, A., Bethune, C., Kirby, A. & Pike, A. (2013). Concurrent and convergent validity of the simple lifestyle indicator questionnaire. International Scholarly Research Notices, 2013. https://doi.org/10.5402/2013/529645
Johnson, E. (2013). Face validity. In Volkmar, F.R. (Ed) Encyclopedia of autism spectrum disorders. Springer, New York. https://doi.org/10.1007/978-1-4419-1698-3
Kemperman, A., Berg, P., Weijs-Perree, M., & Uijtdewillegen, K. (2019). Loneliness of older adults: Social network and the living environment. International Journal of Environmental Research and Public Health, 16(3), 406. DOI: 10.3390/ijerph16030406
Kennet-Cohen, T., Turvall, E., Saar, Y. & Oren, C. (2016). The predictive validity of a two-step process to medical schools. Journal of Biomedical Education, 2016. https://doi.org/10.1155/2016/8910471
Lange, R.T. (2011). Inter-rater reliability. In Kreutzer, J.S., DeLuca, J. & Caplan, B. (Eds) Encyclopedia of Clinical Neuropsychology. Springer, New York. https://doi.org/10.1007/978-0-387-79948-3
Lin, W.L. & Yao, G. (2014). Concurrent validity. In Michalos A.C. (Ed) Encyclopedia of quality of life and well-being research. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0753-5
Mousazadeh, S., Rakhshan, M. & Mohammadi, F. (2017). Investigation of content and face validity and reliability of sociocultural attitude towards appearance questionnaire-3 (SATAQ-3) among female adolescents. Iranian Journal of Psychiatry, 12(1), 15-20.
Suh, G. & Lee, M.H. (2003). Cognitive assessment and reference diagnosis system: Development of a neuropsychological and clinical examination for clinic and population use. Psychogeriatrics, 3(2). https://doi.org/10.1046/j.1479-8301.2003.00010.x
Tomini, F., Tomini, S., & Groot, W. (2016). Understanding the value of social networks in life satisfaction of elderly people: a comparative study of 16 European countries using SHARE data. BMC Geriatrics, 16. Retrieved from https://bmcgeriatr.biomedcentral.com/articles/10.1186/s12877-016-0362-7
Vilagut, G. (2014). Test-retest reliability. In Micholas, A.C. (Ed) Encyclopedia of quality of life and well-being research. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0753-5