This paper provides a comprehensive overview of program evaluation theory and practice as applied to health education settings. It examines the distinctions between basic evaluation and formal research procedures, explores qualitative data-collection techniques including focus group interviews, nominal group process, and the Delphi technique, and outlines steps for developing valid and reliable questionnaires. The paper also defines formative evaluation and its four phases, describes five research designs used in outcome evaluation, compares community-focused evaluation with generic evaluation, and applies evaluation guidelines to school health education programs. Together, these discussions demonstrate how systematic evaluation frameworks can improve program planning, decision-making, and health education outcomes.
M. E. Gredler explains that evaluation is the structured accumulation of information and knowledge that helps decision-makers arrive at informed and productive choices. M. B. Dignan adds that all basic evaluations are procedural assessments of the results and overall functionality of programs. P. D. Sarvela and R. J. McDermott, in Health Education Evaluation and Measurement, offer a more detailed explanation, describing basic evaluation as the utilization of processes designed to determine whether a program has been implemented in accordance with the aim for which it was created. In essence, basic program assessment examines whether a program was able to practically attain the objectives it originally set out to achieve.
Research procedures in program evaluation, by contrast, encompass a different set of activities. When research procedures are implemented, questions about sample size, control group structure, and statistical analysis all become relevant and must be answered. Basic evaluations, on the other hand, focus on procuring input from and assessing participants, sponsors, and collaborators. While research procedures are built around filling a gap in knowledge or adding to existing knowledge on a topic, evaluation helps attain community input, recognition, and increased funding for programs assessed as either popular or necessary.
Evaluation can also be applied in normal daily routines. It helps individuals understand the purpose behind everyday actions such as paying bills or identifying the shortcomings of a process. In a corporate setting, it can help employees improve customer service. It also helps recognize daily functional glitches, conduct comparisons across cultural norms, assess behavioral changes before and after program implementation, determine whether programs are reaching the right target audience, and evaluate the cost-benefit ratio of program investments.
As a health education professional, evaluation can improve job performance in several ways: identifying the average cost of a program per individual; assessing the impact a process has on individual, group, or one-on-one activity behavior; recognizing overall savings generated by program implementation; addressing which program components work most effectively; and ensuring that cost-effective programs with favorable benefit ratios are identified so that managed care organizations can implement them within national health education structures. Additional approaches include fostering mutual respect among participants and collaborators, designing a planned and systematic inquiry process before evaluation begins, and ensuring that evaluations examine the extent to which programs fulfill social and global public welfare responsibilities.
A focus group interview is a process designed to identify and collect comprehensive data on numerous smaller issues that form part of a larger evaluation. Data are collected through controlled focus groups in which the interactions among group members serve as vital input alongside individual responses. Focus group interviews can gather general data and statistics for use in later survey questionnaires, obtain perceptions and opinions about a specific topic or event, interpret data collected through other formats, and serve as a human test run for a program or product before large-scale implementation.
The nominal group process is primarily a preparatory step before the actual evaluation begins. It covers the aims to be achieved, the nature and design of the groups to be interviewed, the hiring and training of collaborators and volunteers, the construction of primary and supplementary questions, and the setup of logistical and physical facilities. The nominal process is formal and controlled; final results are compiled and prioritized through a voting consensus conducted after each question and at the close of each session.
The Delphi technique is another preparatory process with a similar purpose to the nominal group process, but it places greater emphasis on ensuring that supporting logistics — facilities, volunteers, timelines, and staff training — are properly arranged. It also focuses on identifying essential participants who add depth to the topic and on simplifying evaluation aims. The key distinguishing feature of the Delphi technique is that it conducts pre- and post-interview rounds: questions are modified based on results from the first round and then redistributed for a second round. A third round is occasionally required. All answers from each session are recorded, listed, and assessed.
Of the three approaches, the Delphi technique is perhaps the most valuable for program evaluation in a modern organizational context. It helps collaborators establish simplified and specific goals, focuses attention on the quality of supporting staff and participants, and uses the pre- and post-interview structure to analyze changes in behavior, public perception, or social structure before a program is implemented on a broad scale.
Valid and reliable questionnaires are complementary: a questionnaire that is valid will generally also be reliable, and the ideal instrument achieves a strong balance of both qualities. Validity and reliability can be built into a questionnaire through attention to four key factors: the ability to repeat the questions across long-term and similar studies; the consistency of results when the same instrument is used in different contexts; the consistent tallying of scores across the coefficient range; and structural consistency between two parallel sets of questionnaires.
When designing a questionnaire for a large survey, these four factors must be considered alongside a clear identification of the aims the instrument is intended to achieve. Once those aims are established, the next step is to distribute them across the questionnaire in a way that interlinks them while maintaining logical flow. The choice of venue and facilities also matters, as it sets a standard and creates a first impression for both staff and participants before questionnaires are even distributed.
Questionnaires must not come across as biased or leading. All questions should be straightforward and framed in a way that participants will find engaging and are likely to answer honestly. Participants must be given full assurance that their responses will remain confidential. Perhaps most importantly, mutual respect between participants and supporting staff should be established from the outset, as this promotes integrity and honesty in responses while minimizing bias on both sides.
Formative evaluation is fundamentally an assessment of the cultural, behavioral, and social patterns of a community. This includes defining the evaluation process first from the community's perspective, so that evaluators understand what is actually being demanded. It is equally important that all data collected be analyzed through the community's lens — not all information gathered will be deemed relevant or credible by the community, and information released must be formatted in ways the community finds acceptable. Both the reporting style and the specific facts reported carry equal importance.
The four phases of formative evaluation are:
1. Identify the restrictions of the evaluation process. Recognizing constraints early allows the data collected on community or social structure to be efficiently sorted across available program choices, making the selection of an appropriate evaluation program less time-consuming and more effective.
2. Recognize and implement the most suitable evaluation plan. Selecting the right plan is not enough on its own — correct implementation is equally critical. An appropriate program that is poorly applied will be viewed negatively. This phase must also account for time and resource constraints while remaining acceptable to the existing social order.
3. Accumulate and assess all data collected. Data collection and analysis directly determine whether the evaluation process will be accepted by the broader community. The entire health education enterprise depends on whether individuals who use healthcare and health education services perceive the evaluation and its results as applicable to their lives.
4. Accurately report the results. Reporting must reflect the community's values and expectations. If the community does not perceive the results as relevant to their social structure, the entire evaluation effort is wasted and the credibility of the evaluators is diminished.
"Five outcome evaluation research design types described"
"Survey, physical measurement, and observer methods compared"
"Community evaluation defined and contrasted with generic evaluation"
"Applying evaluation guidelines to school health settings"
You’re 43% through this paper. Sign up to read the remaining 4 sections.
Sign Up Now — Instant Access Already a member? Log inAlways verify citation format against your institution’s current style guide requirements.