Reliability and Validity Baum Et

Reliability and Validity

Baum et al. (2008). Reliability, validity, and clinical utility of the Executive Function

Performance Test: A measure of executive function in a sample of people with stroke

The American Journal of Occupational Therapy, 62 (4): 446.

Baum et al. (2008). Reliability, validity, and clinical utility of the Executive Function

Performance Test: A measure of executive function in a sample of people with stroke

The American Journal of Occupational Therapy, 62 (4): 446.

What is the rationale of the study?

Assessing the capacity of stroke victims to function in a reasonably independent fashion is an important but often difficult task for an occupational therapist. Executive functioning is essential to the type of multitasking demanded of everyday life, including activities as simple as cooking, self-administration of medication, using the telephone and paying bills. Deriving a systematic method to assess the ability to perform such basic life tasks is the goal of the Executive Function Performance Test. The authors of the article "Reliability, validity, and clinical utility of the Executive Function Performance Test" specifically examined the test's reliability and validity for stroke victims.

Q2. What is the research study design that is implemented in this study? Explain the characteristics in the study that support your choice of study design.

73 participants with mild to moderate stroke symptoms and 22 unaffected individuals as a control group were matched for their age- and education levels. All were assessed for their functioning on four basic EFPT tasks, including cooking, using the telephone, administering medications in a correct fashion, and paying bills. "Significant differences were found between participants with mild and moderate stroke and healthy control participants" (Baum et al. 2008: 446).

Q3. What are the distinguishing characteristics of the EFPT compared to current/existing performance based assessments?

Existing tests did not measure a person's level of functioning with increasing levels of support (as might by provided in an assisted living facility) or only did so with a single task, such as the Kitchen Task Assignment. Given that the purpose of the test was to see how much additional support the stroke victim required, the EFPT's gradation of levels of assistance was deemed to be superior compared with other test models, especially combined with its ease of administration and its focused, occupational emphasis.

The Executive Function Performance Test also "has several advantages over existing performance-based assessments," most notably its ability to evaluate individual cognitive functions. This is particularly useful when evaluating a stroke victim, as often the impairments vary widely from individual to individual. The EFPT's isolation of four functions that are activity-based and occupational in nature makes it ideal to prescribe treatment plans, and as it is focus on instrumental activities of daily living (IADLs). "It assesses actual performance" rather than relies "on proxy or self-report" in a subjective fashion on the part of the assessor (Baum et al. 2008: 447).

Q4. List some of the variables in this study. How would these variables be classified? State how the variables were operationalized.

Independent variables in the study included the relative degree the stroke had impacted the life of the person (mild vs. moderate impairment), while dependent variables included the levels of measured performance on the test. Operationalization involved demonstrating the ability to perform the tasks of daily life. Simple cooking was tested by asking the test subject to cook oatmeal; using a telephone was tested by requiring the subject to inquire about grocery delivery on the phone; and the test subject was required to select and administer medications correctly and select and pay bills from a pile of mail.

Q5. Define reliability. Discuss the means by which the reliability of the instrument/measure was assessed in this study.

Reliability is defined as reproducibility, or the ability of a test to yield reliable or similar results with similar target population groups and under the same circumstances. "We determined interrater reliability with three trained raters who simultaneously rated 10 participants, 5 with mild stroke and 5 healthy controls. The total score and each of four subtest scores were analyzed, and ICCs were computed," and this indicated a high level of reliability between different raters of subjects' responses to the four tasks (Baum et al. 2008: 451). Subjectivity in terms of rating subject's competence thus did not vary widely between assessors.

Q6. Define validity. How was the validity of the instruments assessed in this study?

Validity is defined as the test's ability to test what it purports to assess. Validity was confirmed when "significant moderate correlations were found between the EFPT total score" and other neuropsychological measurements that assessed cognitive functioning on a more technical, biological and psychological level than the EFPT test, including the widely-accepted Wechsler Memory Scale assessing working memory, verbal fluency, and attention (Baum et al. 2008: 451).

Q7. Summarize Table 1

Table one is a comparison of the performance of the control group, the moderately impaired group, and the mildly impaired group on the EFPT. It shows that the moderately impaired individuals had significantly more difficulty performing basic life tasks than the mildly impaired stroke victims, while the control group had the least trouble overall.