Program evaluation methods and applications

L.E.A.D. Evaluation

Legal Enrichment and Decision-Making Program Evaluation

This study was designed to evaluate both process and outcome. The researchers state their purpose as being to assess if the program was administered in a consistent manor, as well as evaluate the outcomes for students.

The goals of the Legal Enrichment and Decision-making Program (L.E.A.D.) are to help students understand the social, personal, and legal consequences of common juvenile offences, to build positive relationships between students and legal authorities, to reduce prejudice and increase appreciation of diversity, and to help students develop positive aspirations as alternatives to criminal activities and knowledge of the choices that will help them achieve those aspirations (Chi & Middaugh, 2005 p. 4).

The program evaluation was initiated to explore the following four questions: Is the program being implemented in a consistent manner?, Are the students better able to understand the reasons for laws and the consequences of engaging in illegal behavior as a result of participation in the Project L.E.A.D. program?, Do students' attitudes about laws and legal authorities change as a result of participation in the Project L.E.A.D. program?, and Do students' attitudes about positive, educational and career aspirations (protective behaviors such as staying in school) change as a result of participation in the Project L.E.A.D. program?, (Chi & Middaugh, 2005 p. 5 & 7).

The L.E.A.D. Program was implemented in elementary schools at the fifth grade level. Student attitudinal surveys, content assessment, student focus groups and teacher interviews were administered, where possible, to paired classrooms in the same school; one classroom that participated in Project L.E.A.D. And a similar one that did not participated in Project L.E.A.D. Volunteer "facilitators" (prosecutors and staff from the District Attorney's office) worked in classrooms in communities with high rates of juvenile crime, gang activity and high school drop-outs. The facilitators worked with classroom teachers for approximately 20 weeks to implement the curriculum using active learning strategies such as role-play scenarios, field trips and a mock trial to orient students to issues and procedures common to the justice system (Chi & Middaugh, 2005 p. 2 & 5).

The study took place over a two-year period. The first year of the evaluation focused on developing and piloting student attitudinal surveys, a content assessment, student focus group protocols and teacher interview protocols. These instruments were revised in the second year and administered in the test and control classrooms. The student surveys and content assessments were administered at the beginning and end of the school year. The student focus groups and teacher interviews took place near the end of the school year. Student survey data was analyzed for pre-post changes as well as for comparisons between the Project L.E.A.D. And non-Project L.E.A.D. classrooms. The student focus groups provided qualitative data that enhanced understanding of the survey data (Chi & Middaugh, 2005 p. 2).

The design of the evaluation, with a control and experimental group, follows standard experimental design. A potential problem with the validity of the findings of the comparison component of the evaluation may have occurred due to the fact that students and teachers in both control and experimental classrooms, assumedly, had contact with each other (lunch, recess, before and after school, etc.) during the evaluation. This interaction could have plausibly influence responses and skewed the results. The comparison was used to demonstrate student growth in knowledge and attitude during the course.

The authors' position is that, "overall," the four goals of program were met. In support they report that 79% of the time students who participated in the program correctly identifying illegal acts, reasons for laws, and components of the legal system. They assert that information from the student focus groups and interviews indicated that many students seemed to understand the main "message" of the Project L.E.A.D. program: it was important to do the right thing and that there are consequences to bad decisions. They further reported that the project survey items suggest that tolerance and appreciation of diversity was an area with great need for intervention as many students were likely to report being teased by peers and the program again appeared to have a protective effect. The program also appeared to have protected against a decline in students positive attitudes over the course of the year. Project participants showed no decline in their belief in the importance of school and in their plans for further schooling. Comparison students, however, were much less likely to report a belief in the importance of staying in school or an understanding of the implications of the school for their future employment. The research questions posed in the evaluation were addressed in the findings and the conclusions and recommendation sections of the evaluation (Chi & Middaugh, 2005 p. 11-19).