Program evaluation methods and applications

Program Evaluation

The importance of accuracy in program reviews

"Accuracy: information is accurate and justifies the conclusions drawn, and is focused on accurate standards" Accuracy is perhaps the most important attribute of any program evaluation. Even if an evaluation method is useful, legal and feasible, spending money upon a potentially inaccurate evaluation is wasteful -- and also can result in faulty decision-making and program planning when redesigning or refining the curriculum in the future. In ESL program reviews, a combination of both qualitative and quantitative measures would yield the most accurate results, and still be cost-effective.

All ESL students completing the program should receive an annual, year-end evaluation form in their email in-box. To increase compliance, a small prize could be given to several randomly selected respondents, such as a $100 gift certificate at Barnes & Noble. Demographic information and information about program satisfaction could be solicited in the questionnaire. All students would be guaranteed that their responses would be anonymous, and that their input would not affect their grades or standing at the university. Students would be asked to grade the effectiveness of the program overall, as well as specific components they liked and disliked. They would also be able to grade professors and administrators regarding their helpfulness for ESL students, and to 'write-in' suggestions for improving the program in a final 'field' on the computer program.

If financing allows, having students take a survey before the program, to prioritize their expectations in a list format (for example, numbering statements in order of importance such as "I want to improve my English to get a better job," or "I want to improve my ability to understand English in technical subjects) as well as to self-rate their own English proficiency, would be helpful. This would also allow students to state their goals, and then compare their goals against their final achievements. Also, as part of the program, students are assessed for placement purposes in their English proficiency. After completing their courses annually, students should have to be assessed once again, to see if their English had showed improvement, based upon their first set of scores. This would reduce the impact of bias of professor's grading, and place the emphasis on improvement, rather than overall proficiency, which is a better measure of the program's success.

These quantitative measures would not be enough to fully assess the program. Throughout the program's duration observers trained in ESL education should sit in on various classes, to assess the efficacy and suitability of supplementary educational efforts, as well as the performance of various types of ESL students in mainstream classes. ESL students should be classified into various sub-groupings, for the purposes of the study (such as traditional undergraduates, mid-career shifters, international students, and naturalized citizens with parents from foreign nations). During the interview processes, the academic records of various subgroups could be compared, to see if the program was more effective for one type of student than another.