Healthcare Master Case Study Baum, C.M., Et

Healthcare Master Case Study

Baum, C.M., et al. (2008). Reliability, Validity, and Clinical Utility of the Executive Function Performance Test: A Measure of Executive Function in a Sample of People With Stroke The American Journal of Occupational Therapy 62 (4); pg 446.

Study rationale. The research study is designed to assess the validity and reliability of a test for executive function in post-stroke occupational therapy patients. Clinical tests of executive function may not be good predictors of a patient's ability to function in day-to-day life. The Executive Function Performance Test (EFPT) employs ordinary daily living skills in which the post-stroke patients are likely to have engaged in the past, and are reasonable target behaviors for adaptation to independent or supported living arrangements. The test is particularly valuable in that it offers a convenient test for executive function using real-world tasks.

Research design. An experimental design is employed in this study. A control group is used to assess the utility and sensitivity of the EFPT to degrees of stroke-related dysfunction. Data collected in the study was quantitative and was analyzed using statistical procedures. Three hypotheses were identified prior to the commencement of the actual research.

Distinguishing characteristics of EFPT. The EFPT administration protocol seems easy to learn. A manual is provided that articulates how test administrators are to present the four tasks, cue the patients, and score their behavior. The four distinct tasks are designed to elicit behaviors that will isolate cognitive components of the executive functions activated during task completion -- these findings will later help to determine a course of therapy or treatment. The EFPT promotes top-down objective assessment of executive function as demonstrated through the performance of real-world tasks. This aspect of the test avoids the pitfalls of predictive behavior based on contrived or proxy task performance. Also, on a very practical level, the test provides specific information about support or assistance needs of the patients with regard to their near future performance of the four tasks.

Research variables. The independent variables were age, gender, race, and educational level of the study participants. Level of impairment at 6 months post-stroke as measured by their admission score on the National Institutes of Health Stroke Scale was another important independent variable. The level of dysfunction post-stroke was operationalized by creating three groups of participants: A control group, a mild stroke group, and a moderate stroke group. The dependent variables were subjects' scores on the EFPT, scores on the neuropsychological tests, and the scores on the functional outcome tests.

Reliability measures. Reliability refers to consistency across test (measurement) repetitions. For a test to be reliable, it must good test-retest results and good internal consistency. Interrater reliability was assessed by having three trained raters simultaneously score the task performance of a number of study participants, and calculating intraclass correlation coefficients (ICC). To test for internal consistency, Cronbach's alpha correlation was used to check all possible combinations for correlation. A Pearson's correlation coefficients were calculated for the relationship between each of the test domains and the total test score. In all instances, reliability measures were in the high acceptable range.

Validity measures. Validity is the estimate of the strength of research with regard to the conclusions drawn. There are four types of validity: Conclusion, external, internal and construct. Construct validity refers to the accuracy with which a method actually measures what it is designed to measure. Significant differences between the post-stroke patients and an age-matched control group are expected to indicate that the test demonstrates the discriminative validity (within the larger category of construct validity), in that, scores that are anticipated to not be related do not, in fact, show correlation.

Significant differences were found among the three groups for the total scores, tasks, and executive functions, except for initiation. Only the cooking subtest was found not to discriminate between the mild and moderate participants. Concurrent validity was assessed by comparing the scores on the neuropsychological tests -- an established measure of the phenomenon under study -- and the total EFPT scores. Significant moderate correlations confirmed the concurrent validity of the EFPT. Similarly, correlation between the scores on the functional outcomes tests and the scores on the EFPT were calculated and the result again suggested concurrent validity of the EFPT was supported for the daily living skills.

Summarization of Table 1. The demographic information about study participants and the participants scores for all domains of the EFPT are presented in Table 1. No significant differences were found among groups for age, race, or educational level. The study participants were predominantly white and predominantly women. Scores for the mild and moderate groups on the National Institutes of Health Stroke Scale were significantly different. The mean score for participants in the mild group was 2.00 (SD = 1.50). The mean score for the participants in the moderate group was 10.64 (SD = 2.99).

The mean EFPT scores are shown for all tasks (cooking a meal, using the telephone, taking medications, and paying bills) and all components (initiation, organization, sequencing, safety and judgment, completion). A mean total EFPT score is also shown for each group. Because the provision of assistance is part of the assessment process, the highest level of cuing support needed by a participant is recorded. This is reflected in the scores, which consist of three parts: 1) the executive function component score; 2) the task score, and 3) a total score. The executive function component score is a composite that is calculated by adding the task scores for initiation, organization, safety and judgment, and completion. The executive function component cores can range from 0 to 5 for each task, with a composite range of 0 to 20. The F scores from the ANOVA describe the differences between the groups in comparison to the within-group variability. To make a prior adjustment, the p value was divided by 9 (the ANOVA computed) and the adjusted criterion was p < .005. Differences between the groups on EFPT scores are significant, confirming that the test is sensitive to degrees of diminished executive function. Exceptions are described above in the section on validity.